Product Engineering Best Practices

Photo by Sven Mieke on Unsplash

In more than a decade in a variety of settings I’ve observed a few dos and don’ts reappear in several domains. In this post I share the anti-patterns and best practices I feel are most valuable to consider.

Anti-pattern: Sending Requirements Over the Fence

It’s not uncommon for product development to be partitioned by department or even by company as is commonplace in software consulting. There may be a somewhat discrete separation between the what and the how for software projects.

Requirements and designs that are conceived by one group and handed off to another can have some major drawbacks. Consider all the conversations that go into devising individual features. The weighing of pros and cons. The cost vs value-add tradeoffs. The research, the user interviews, the raw ideas that are debated and iterated on and molded into great ideas.

This context is extremely valuable when developers build the feature because very often there are dozens of micro-implementation tradeoffs that will be made autonomously by the Engineering team. Having that context in mind ensures those tradeoffs align with the overall vision of the feature.

On the flip side of this, when Engineers aren’t involved with specifications this creates an empathetic vacuum. Engineers will find it harder to relate to the end result making it easy to fall into a “devs for hire” mentality and become divorced from the mission of the product. Involving Engineering early in the product design lifecycle is a great way to combat this mentality and stoke the team’s intrinsic motivation. Everyone wants to feel a strong sense of fulfillment. Feeding into that as part of product development will quickly produce compounding benefits.

✔️ Best practice: Ownership Over a Slice of the Mission

In recent years the idea behind ownership in the tech industry shifted from teams owning a technology silo, such as the frontend or backend, to teams owning the full vertical stack of a monolithic slice or microservice. This shift towards vertical ownership is a huge leap in the right direction.

Pressing forward in that same direction means extending ownership beyond the technology stack and additionally embracing ownership over solving the end users’ needs. Phrased differently, it’s ownership over a thin slice of the product’s mission.

Extend ownership to encompass a slice of the mission in addition to the vertical stack.

For example, rather than a team owning only the full stack behind a dashboard page the team would take on ownership over ensuring users can quickly find relevant information and efficiently administer common tasks. It’s ownership over the problem that page is attempting to solve from the database to the UI to how well the feature is meeting users’ needs.

It’s a subtle, yet important, distinction because ownership defines the boundaries of accountability and this boundary crosses the Product-Engineering line. Extending accountability across this line serves as a positive forcing function for Product, Design, and Engineering to empathetically work as a singular unit. Everyone is in the same boat and all must work together to serve the product’s mission.

Anti-pattern: Happy Path Engineering

One skill many end users have is the ability to find ways of using your product in ways you did not anticipate. Even though we have all observed this firsthand it’s a fact that is easy to breeze over when planning a project.

Consider what it takes to build a page which shows items in a virtual shopping cart. The happy path certainly seems simple. Users can review their carts items, add and remove them, and proceed to checkout. However, the paths users will encounter is something else.

• What should users see while the page is asynchronously fetching cart items from the database? Are there placeholder rows and images?
• What happens if one of multiple API requests fail? A failure to retrieve a product’s thumbnail image may need to be handled differently than an API call that determines if an item is still in stock.
• What does the user see if the cart is empty?
• If they have many items in the cart which part of the viewport is scrollable? Can the shopping cart be so large there needs to be a “Back to top” button or pagination?
• Do long cart item names get clipped or line wrapped?
• What about accessibility, different devices, and different browsers?

These less-than-happy paths will be handled eventually one way or another. Note that it is orders of magnitude cheaper to work through those paths earlier in the feature’s lifecycle as compared to waiting for users to discover these paths on their own.

✔️ Best practice: Encapsulate Desired Behavior

This best practice breaks down into two categories. The first is covered in another post and relates to bettering technical implementation of a project by using an invariant first approach.

The second aspect more closely tied to product engineering is to use a Design System.

A Design System describes the tooling which encapsulates and enforces aspects of product design. At its heart, a team effectively using a Design System will start with the product’s atomic components. The typography, layout, palette, and other fundamental concepts. These atoms compose together to make higher level components such as headings, menus, and forms. We’re talking about the rigidly constructed legos that snap into place with other components and give the product a sense of unity and holism. We’re also talking about never creating ad hoc components and styles. Everything on the site is a Design System lego and nothing is duct taped together.

Design Systems can be taken even further. I once worked on a team that implemented every new page disconnected from any backend within our Design System and QA’d that page with the Designers prior to putting it in the actual product.

This Design QA step included a simple yet incredibly powerful plugin which allows one to manipulate the Design System component in real time to understand how it deals with edge cases. Is it responsive? Does it handle long names? Is there a fixed header when scrolling? It allowed us to test many paths users are likely to encounter.

Storybook + React Knobs

Using a Design System to its fullest ability did wonders for delivering a quality and holistic user experience.

Anti-pattern: Mythical Person-Months

There are a variety of reasons why giving an FTE-Month estimate is problematic:

• Scope will grow. Requirements will change. These are immutable facts of projects of any size and paradoxically are the facts most likely to go unacknowledged at the start of a project.
• Overly optimistic estimates are incentivized. This statement is unfortunately particularly true in software consulting, although, outside of consulting other forces such as quarterly metrics, upcoming investor meetings, or even the natural desire to report good news upward can lead to optimistic leaning estimates.
• The tech industry tends to use unreliable methods for estimating. Providing estimates for anything that spans multiple weeks or months is already a difficult undertaking. With software and tech generally being less tangible and more complex than other industries it is certainly no easier to confidently anticipate how long it will take to implement a large feature or project.

Repeating advice from a previous post, estimates are a means to an end and one way to achieve that end is to frequently and empathetically communicate with stakeholders. Strangely enough, frequent face-to-face communication is the most effective alternative to estimates I’ve ever employed. It achieves the same intent of providing stakeholders with information to make business decisions without presuming the effectiveness of software estimates.

Anti-pattern: Viewing Project Completion as a Goal Line

On paper a project has fairly well-defined boundaries. It’ll start around this date, staffed with these individuals, journey through specific milestones, and lastly it will roll out to production. Project compete.

Though the project may be done on paper there are two important ongoing activities to consider. Firstly and most importantly, if users are using new features in unanticipated ways, or their needs aren’t being met, then it is time to iterate (refer to the ownership best practice above). The project is only tentatively done so long as their needs aren’t met.

Additionally, with the release of new features comes with it growth in what that team is accountable to monitor, maintain, and fire fight. This is analogous to the purchase of a vehicle. As that vehicle is driven by end users there may be problems along the way and general upkeep required. Purchasing additional vehicles (completing subsequent major features) only adds to the overall ongoing maintenance required.

✔️ Best practice: Cut Scope Until It Hurts — Build & Iterate

The most successful teams I’ve worked with are the ones who took the cut scope “until it hurts” aspect of this best practice to heart. When you’ve cut project scope until it’s literally too painful to cut anymore, that’s when you know you’ve found your minimally viable product. That’s when you know you’re exercising lowercase “a” agility.

When you’ve cut scope until it’s literally too painful to cut anymore, that’s when you know you’ve found your minimally viable product.

Remember, scope will grow. Unexpected situations will come up. Overly optimistic estimates will be confronted by the real world. One way or another reality will take the plan that looked great on paper, turn it on its side, and you’ll find yourself pressed for time.

Tangibly, this best practice breaks down into the following:

• Start with a project scoped seemingly too small (Don’t worry! It’ll grow again). Break it down into small value-driven releases with bias toward the riskier and critical path items first.
• Get something of value into production soon and repeatedly
• Observe. What did you learn? What technical hurdles didn’t you anticipate? How are users using what’s released so far?
• Iterate. Continually take what you’ve learned to reshape what you’ve already built or prioritize what to build next, build it, and release it to production. Do this again and again until the next most valuable thing the team can work on is something else.
• Keep technical upkeep in mind. Make time to take a breath and tune up the clockwork that keeps those features ticking.

✔️ Best practice: Measure What Matters

I am an avid subscriber of a You Are What You Measure philosophy. In brief we all tend to, either purposefully or subconsciously, shape our decisions and behavior to optimize what we’re measured against.

As an example, a team measuring number of bugs may intend for that metric to lead to a higher quality product. However, with the metric aimed at bug count and nothing to do with quality the team may optimize on the bug count specifically. Similar bugs will tend to get grouped together. Would-be new bugs may get rolled into specifications for upcoming work. They may even make claims of “that’s a feature not a bug”. All of these resulting actions attempt to minimize bug counts while none optimize on quality, the intent behind the metric.

The opposite effect of this philosophy is that which is not measured is not optimized and often ignored until it reaches critical levels. Quality assurance, operating costs, time spent context switching, system wide complexity, sustainable work hours. When important metrics aren’t explicitly measured their remedies will naturally be superseded by actions that will optimize metrics that are explicitly measured.

You are what you measure. Ask yourself whether the optimization of your metrics will reliably lead to a desired outcome. Or can it be gamified and accidentally lead to an unexpected outcome? Carefully designed metrics are extremely beneficial when the natural gamification of those metrics reliably leads to the intent of that metric.