Lean Startup, day 10: Build-Measure-Learn explained

As we saw in the previous post in this series, after problem interviews and competitive benchmark, there is not much more we can learn unless we start building something. But what should we build? And in which order? The Lean Startup method helps answer these questions.

Experiments

Lean Startup is all about learning. There are a lot of assumptions in a Business Model Canvas. Lean Startup encourages you to test each of these assumptions independently, using a time-honored principle: scientific experimentation.

Let's talk about medecine for a minute. Before the 20th century, if your leg was cut, the physician would put ointment on the cut. You had a certain percentage to heal, and a certain percentage to die in horrible suffering. Basically, it was lottery, backed up by a bit of experience. At the end of the 19th century, a bunch of physicians decided that the current approach wasn't that great, and used experiments and statistical analysis tools to prove the efficacy of medicines. They took 2 cohorts of people, all with a cut on the leg. To the first cohort, they gave an ointment. To the second cohort, they gave a placebo. Then they compared the differences in healing between the two groups. If the group treated with the ointment had a significantly better healing, that meant that the ointment was efficient. That was the invention of modern medicine, and since then, life expectancy has increased 65%.

This scientific approach is exactly what Lean Startup advises to use in order to verify the assumptions of a Business Model. Take each assumption, find a metric that may show whether the assumption is right or wrong, design an experiment to move the metric, and run the experiment.

If you don't know where to start, take the most risky assumption, and test it first. If you are wrong, it's better to discover it sooner than later. Now you see why listing risks early in the project is crucial: it allows to prioritize the work to be done.

Build Measure Learn

Lean startup is often represented by the following loop, also called a Learning Loop:

Build-Measure-Learn are the three steps of each experiment:

• Build: build something in order to collect the metrics identified earlier
• Measure: run the experiment, let people interact with it, collect the data
• Learn: analyze the data to determine if the hypothesis is true or false

If the assumption appears to be false, you have to make another assumption, and test it by using the same process. If the assumption is verified during the experiment, proceed to the next assumption in the list and test it the same way.

Of course, when you build an experiment, you do not have to test only one assumption. You can test a set of hypotheses. The most important is to have a clear view of what you test, so that you maximize the learning.

There is also an alternate result to this series of experiments: if your core hypothesis are unvalidated, it affects the whole project. At this point, it is useless to test less important hypothesis. You have to pivot. We'll talk about pivot in a future post.

Designing an Experiment

In the books, testing the problem and testing the solution are consecutive. In real life, there is an activity in between: designing the solution. No Lean Startup book explains that, and it's an entire other subject. This is where experience in web development helps. We won't explain here how this works, but be aware that it's not as easy as you may think.

For admin-as-a-service, our most risky assumption is customer acquisition. Are we capable of gathering leads for a product that has no name? Now, which experiment can help us answer that question? We decide to build a demo site showcasing the possible User Interface of Admin-as-a-Service, describing a lot of product features that don't exist yet. It will also give us the opportunity to test the wording, because we found out in our interviews that people didn't share the same words to describe their pain, and the cure for that pain.

Next Steps

The story of how we built this demo is the subject of the next post in this series. Be sure to read it!

Credits:

Thumbnail picture: Steel wool loops, by David Russo

Build Measure Learn loop : Build, Measure, Learn, Rinse, Repeat by Tim Klapdor