A message queue is a service that allows producers to place serialized messages onto a highly reliable queue from which one or more consumers can read those items. Message queues are an essential aspect of decoupled, event-driven architectures and in this post I cover a few of the potential use cases and tradeoffs.
When writing a Medium.com post you may notice the drafts list page will show a word count for each post. Upon closer inspection you may also notice it often takes several minutes for the word count to display accurately. …
In more than a decade in a variety of settings I’ve observed a few dos and don’ts reappear in several domains. In this post I share the anti-patterns and best practices I feel are most valuable to consider.
It’s not uncommon for product development to be partitioned by department or even by company as is commonplace in software consulting. There may be a somewhat discrete separation between the what and the how for software projects.
Requirements and designs that are conceived by one group and handed off to another can have some major drawbacks. Consider all the conversations that go into devising individual features. The weighing of pros and cons. The cost vs value-add tradeoffs. …
This post explores another sufficiently complex example, this time leveraging Docker in the cloud. Similar to the last post, let’s start with a seemingly trivial ffmpeg command and see what it takes to build a similar solution in the cloud.
# Splits audio from input.mp4 into a file called output.mp3
-i input.mp4 \
-f mp3 -ab 192000 \
The above command uses ffmpeg to read in a video file called
input.mp4 and split out the audio into a file called
output.mp3. It seems easy enough. …
At NoRedInk we recently conducted a fire drill with Team Zamboni, a team responsible for “smoothing over the ice” for internal departments by providing engineering tooling and automation solutions.
A fire drill is a deliberately introduced problem in production caused by a couple members of the team. The idea is a couple team members temporarily break something that will cause an alert to trigger and prompt the rest of the team to swarm on it, resolve the issue, and result in an overall learning experience.
There are several reasons I wanted to lead the team through a fire drill.
Finding a concrete definition for story points is no easy task. Different sources tend to have somewhat nebulous definitions.
One post at Mountain Goat Software describes story points as “a unit of measure for expressing an estimate of the overall effort that will be required to fully implement a … piece of work.” Another definition from scrum.org defines a story point as “a relative unit of measure… to provide relative estimates of effort for completing requirements.”
Many other definitions orbit around relative complexity, difficulty, risk, or some combination, and story points are not equatable to time.
I think it’s important to take a moment to reflect on why we provide software estimates in the first place. Here is what I see as the primary reason for providing estimates. …
GANs are an incredibly interesting evolution of neural networks. Before getting into what a GAN is let’s start with the basics and get an understanding of what a neural network is.
Neural networks can be very intimidating. Below is a representation of a neural network and it demonstrates only a fraction of the complexity they encapsulate.
Don’t worry about trying to understand that!
For the purposes of this post there isn’t a need to drill down to that level of detail. Instead, let’s keep things simple. Let us consider a neural network as a black box.
With* functions are functions that take the shape of
withProperty : Prop -> CustomType -> CustomType. When several are chained together using the pipeline operator they tend to produce terse, yet legible, code and a pleasant API.
Let’s consider some code that is a good candidate to refactor to use this pattern. Here is a module that exposes a configurable button.
This example shows several permutations of a button by leveraging an API that is expecting the following configurable record to be passed in.
Here we see a handful of optional properties which leads to somewhat verbose code at the call site. …
One of the best ways to learn something is to build a sufficiently complex example and teach it to others. In this post we’re going to cover nearly everything it takes to build a video stabilization service. It will allow users to upload a video file and ffmpeg will do the heavy lifting for performing video stabilization.
# Calculate transform vectors
ffmpeg -i shaky-video.mp4 -vf vidstabdetect=stepsize=6:shakiness=5:accuracy=15:result=transform_vectors.trf -f null -# Stabilize the video
ffmpeg -i shaky-video.mp4 -vf vidstabtransform=input=transform_vectors.trf:zoom=0:smoothing=30:crop=black,unsharp=5:5:0.8:3:3:0.4 -vcodec libx264 -preset slow -tune film -crf 18 -acodec copy stabilized-video.mp4
This may seem too trivial, even the Hello World variety, but once we wire together various AWS services and apply some cloud best practices it’ll quickly become sufficiently complex to learn valuable lessons. …
A fuzz test, also called an invariant test, is a test where random inputs are provided to a test and the developer asserts that some property, some invariant, holds true.
This is best understood by example. Consider a function that accepts a list as input and returns a reversed list.
Given a list of type
a this function will return a reversed copy of that list.
The traditional way to test this function is for the unit test author to make up a mock array or two and make some assertions. …
What’s true of every bug found in the field? … It passed the type checker [and it] passed all of the tests. - Rich Hickey
All bugs pass the type checker and all of the tests. More importantly all bugs represent a guarantee that was intended to be present, but for some reason it wasn’t.
Perhaps you intended to display the first name of a person but firstName was mistyped as firstname and undefined was displayed instead.
Perhaps you intended the user experience of all data entry forms to be the same. Users should see validation errors as they tab through each form. …