How to Test Your Assumptions

For a new business to succeed, many assumptions have to prove true. Testing assumptions in a logical order gives the team the best chance of making course corrections early — and not wasting time and money. In this essay, I outline a method for (1) identifying the assumptions or unknowns and (2) resolving these assumptions on the basis of three parameters: severity, probability, and cost of resolution.

Between stints in academia, I spent more than 20 years in marketing, engineering, and general management in both startups and public companies. Most pertinent to this essay, I served as vice president of engineering for Align Technology Inc. from December 2000 to July 2004. It was at Align — now a 6,800-employee company based in San Jose, California — where I learned firsthand the importance of testing assumptions. Since leaving Align, I have worked with more than 100 startup teams and developed a systematic approach for testing assumptions. Below, I’ll explain my method, illustrating how testing assumptions helped Rent the Runway Inc., a 505-employee company based in New York City, become a fast-growing success. I’ll also explain how my method might have helped Align — prosperous though it has now become — bleed less cash during its early days.

The enthusiasm surrounding the “lean startup methodology” and its many offshoots has created a mindset that entrepreneurs should just launch, failing early and often — iterating, to use startup parlance. But failure alone does not teach. If there are an infinite number of bad ideas, eliminating one gets us no closer to a good idea. Rather, the businessperson contemplating a new venture must begin by evaluating factors that have to be true for the venture to succeed. He or she also must model these factors in a way that allows for reasonable testing. For example, the assumption that people will buy a product for the asking price is a big one; it would take a full launch to completely validate this. Therefore, the entrepreneur must split big assumptions into discrete, manageable assumptions that can be tested at a level of detail allowing for efficient learning.

Identifying the assumptions is the first step. The second is determining a sequence for testing them. Each step should resolve a critical unknown. And each resolution should spur the entrepreneur to continue, change direction (in other words, “pivot”), or, in the worst case, abandon the enterprise. The core of this method is prioritizing assumptions according to three factors: severity, probability, and cost of resolution.

Severity is the impact on the venture if an assumption is not true. The most severe assumption is that there is a customer need at all. Rent the Runway is a case in point. Launched in 2009 by two Harvard Business School graduates, Rent the Runway allows consumers to rent ultrafashionable, luxury-brand designer clothing. The business model depended on the assumption that consumers would want to rent dresses over the internet. Accordingly, the founders tested this assumption before proceeding. The business model also depended on whether the founders could acquire dresses at wholesale prices, so before proceeding, the founders tested whether such partnerships were feasible.

But assumptions about partnerships or technology do not have the same severity for all ventures. For example, a medical device startup’s viability often hinges on a particular technology rather than any partnerships. In short, severity can be difficult to quantify, but the intuition behind it is clear. The simple rule is: If all else is equal and A has higher severity than B, then test A before B.

The second factor is the probability of an assumption being true. What is counterintuitive to many is that assumptions that have a lower probability of being true should be tested first. It seems natural to validate likelier assumptions; positive feedback feels good. But this is the wrong approach. If the goal is to minimize the time and money expended before a key pivot or the decision to abandon the venture, then the assumptions that are least likely to be true should be addressed first. That is: If all else is equal and assumption A is less probable than B, then attempt to validate A before B.

The third factor is the cost of resolving uncertainty. What constitutes resolution? What evidence is sufficient to declare the assumption validated? In Rent the Runway’s case, the founders resolved uncertainty by mailing PDF photos of dresses to 1,000 prospective customers. The overwhelmingly positive response validated their assumption that customers would rent dresses from a website. Tellingly, the cost for validating this key assumption was merely the low price of a bulk mailing — not the potentially high cost of building a full-fledged website.

In estimating the cost of resolution for an assumption, entrepreneurs should (1) assess what evidence they need for resolution and (2) find the simplest, lowest-cost way of gathering that evidence. Hence, the third simple rule of my method: If all else is equal and resolving assumption A costs less than resolving B, then address A before B.

Having considered these three factors individually, we can now put them together. If we can judge both the potential severity of unknowns and their probability, we can construct a ranking, using an index that is something like severity of potential impact times probability of the negative occurrence. (This quantity is what we mean by “risk,” in the colloquial sense.) If we add the estimated cost in money and time to resolve the unknown, we can construct a ratio of risk to the cost of resolution:

This ratio creates a ranking of assumptions. Addressing the assumptions according to this order — wherein the highest-risk, fastest/cheapest-to-resolve assumptions are ranked first — provides a path to removing the greatest risk for the lowest cost.

I believe that this method could have saved Align Technology significant time and money. Founded in 1997, Align raised venture capital to develop a device for correcting malocclusion (crooked teeth). The company’s flagship product, Invisalign, debuted in 1999. The concept was a line of custom-made, clear plastic retainers with the ability to move teeth. Today, Align is a dominant supplier to the orthodontic market, with 2016 revenues of $1.1 billion. But in its first six years, Align, despite having raised more than $250 million, at one point almost ran out of cash. Therefore, I believe it’s worth asking: Could Align have avoided those major early losses if it had taken a smarter approach to identifying and testing assumptions?

Align’s founders believed they could create an automated, technology-driven manufacturing platform that produced Invisalign at a fraction of the market’s willingness to pay. Years later, they’ve achieved this goal. But it took a number of costly generations of technology — and much trial and error. Exacerbating the problems was a focus on pricey marketing programs targeting consumers.

Admittedly, hindsight is 20/20 — and Align’s present-day success casts an optimistic “all’s well that ends well” sheen on its early struggles. However, I believe that testing certain assumptions could have helped Align have a smoother path in its early years. For example, with some regional testing, Align could have discovered that it was not patient demand alone — but rather a combination of patients’ awareness and orthodontists’ acceptance — that was the true driver of widespread adoption. Of course, testing the market in this manner would have involved creating a manufacturing system capable of producing the Invisalign product. But this could have happened on a scale that was smaller and more controlled than that of the company’s actual operations.

Practical realities will influence how entrepreneurs put these assumption-testing principles into action. For example, investors may have different opinions than entrepreneurs about which assumptions are most severe. And time to market may require testing multiple assumptions at the same time — instead of following the sequential path in the guidelines I’ve described. Founders will, to be sure, always work in an environment of unknowns and insufficient information. But to the extent it is possible, I hope I have offered a practical method for managing the uncertainty that lies at the heart of new-venture creation.