A message queue is a service that allows producers to place serialized messages onto a highly reliable queue from which one or more consumers can read those items. Message queues are an essential aspect of decoupled, event-driven architectures and in this post I cover a few of the potential use cases and tradeoffs.

Message Queue Use Cases

When writing a Medium.com post you may notice the drafts list page will show a word count for each post. Upon closer inspection you may also notice it often takes several minutes for the word count to display accurately. …


Photo by Sven Mieke on Unsplash

In more than a decade in a variety of settings I’ve observed a few dos and don’ts reappear in several domains. In this post I share the anti-patterns and best practices I feel are most valuable to consider.

It’s not uncommon for product development to be partitioned by department or even by company as is commonplace in software consulting. There may be a somewhat discrete separation between the what and the how for software projects.

Requirements and designs that are conceived by one group and handed off to another can have some major drawbacks. Consider all the conversations that go…


How to Leverage Docker in the Cloud

This post explores another sufficiently complex example, this time leveraging Docker in the cloud. Similar to the last post, let’s start with a seemingly trivial ffmpeg command and see what it takes to build a similar solution in the cloud.

# Splits audio from input.mp4 into a file called output.mp3
ffmpeg \
-i input.mp4 \
-f mp3 -ab 192000 \
-vn output.mp3

The above command uses ffmpeg to read in a video file called input.mp4 and split out the audio into a file called output.mp3. It seems easy enough. …


Photo by Ricardo Gomez Angel on Unsplash

What is a Fire Drill?

At NoRedInk we recently conducted a fire drill with Team Zamboni, a team responsible for “smoothing over the ice” for internal departments by providing engineering tooling and automation solutions.

A fire drill is a deliberately introduced problem in production caused by a couple members of the team. The idea is a couple team members temporarily break something that will cause an alert to trigger and prompt the rest of the team to swarm on it, resolve the issue, and result in an overall learning experience.

There are several reasons I wanted to lead the team through a fire drill.

  • Test…


How to Provide Reliable Estimates to Stakeholders

What is a Story Point?

Finding a concrete definition for story points is no easy task. Different sources tend to have somewhat nebulous definitions.

One post at Mountain Goat Software describes story points as “a unit of measure for expressing an estimate of the overall effort that will be required to fully implement a … piece of work.” Another definition from scrum.org defines a story point as “a relative unit of measure… to provide relative estimates of effort for completing requirements.”

Many other definitions orbit around relative complexity, difficulty, risk, or some combination, and story points are not equatable to time.

Why Points?

I think it’s important…


How Neural Networks Creatively Generate Data

GANs are an incredibly interesting evolution of neural networks. Before getting into what a GAN is let’s start with the basics and get an understanding of what a neural network is.

A Conceptual Look at Neural Networks

Neural networks can be very intimidating. Below is a representation of a neural network and it demonstrates only a fraction of the complexity they encapsulate.

https://www.researchgate.net/figure/Diagram-of-time-delay-neural-network-focused-TDNN_fig6_324015913

Don’t worry about trying to understand that!

For the purposes of this post there isn’t a need to drill down to that level of detail. Instead, let’s keep things simple. Let us consider a neural network as a black box.


What are with* functions?

With* functions are functions that take the shape of
withProperty : Prop -> CustomType -> CustomType. When several are chained together using the pipeline operator they tend to produce terse, yet legible, code and a pleasant API.

Let’s consider some code that is a good candidate to refactor to use this pattern. Here is a module that exposes a configurable button.

This example shows several permutations of a button by leveraging an API that is expecting the following configurable record to be passed in.

Here we see a handful of optional properties which leads to somewhat verbose code at…


Building a Video Stabilization Service Backed By AWS

Sufficient Complexity

One of the best ways to learn something is to build a sufficiently complex example and teach it to others. In this post we’re going to cover nearly everything it takes to build a video stabilization service. It will allow users to upload a video file and ffmpeg will do the heavy lifting for performing video stabilization.

# Calculate transform vectors
ffmpeg -i shaky-video.mp4 -vf vidstabdetect=stepsize=6:shakiness=5:accuracy=15:result=transform_vectors.trf -f null -
# Stabilize the video
ffmpeg -i shaky-video.mp4 -vf vidstabtransform=input=transform_vectors.trf:zoom=0:smoothing=30:crop=black,unsharp=5:5:0.8:3:3:0.4 -vcodec libx264 -preset slow -tune film -crf 18 -acodec copy stabilized-video.mp4

This may seem too trivial, even the Hello World variety, but…


Moving Beyond Unit Tests

What is a fuzz test?

A fuzz test, also called an invariant test, is a test where random inputs are provided to a test and the developer asserts that some property, some invariant, holds true.

This is best understood by example. Consider a function that accepts a list as input and returns a reversed list.

Contrived.. but this an intro after all

Given a list of type a this function will return a reversed copy of that list.

The traditional way to test this function is for the unit test author to make up a mock array or two and make some assertions. …


What’s true of every bug found in the field? … It passed the type checker [and it] passed all of the tests. - Rich Hickey

All bugs pass the type checker and all of the tests. More importantly all bugs represent a guarantee that was intended to be present, but for some reason it wasn’t.

Perhaps you intended to display the first name of a person but firstName was mistyped as firstname and undefined was displayed instead.

Perhaps you intended the user experience of all data entry forms to be the same. Users should see validation errors as they tab…

Charlie Koster

Solutions Architect | Conference Speaker | Blog Author

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store