How to Run Successful Lean Startup Experiments

Here is a high level outline and guide we follow to create successful lean startup experiments at Never Settle.|Here is a high level outline and guide we follow to create successful lean startup experiments at Never Settle.
Kenn Kelly
Kenn Kelly
September 24, 2014
February 15, 2024
how to create lean startup experiments

Being able to create and manage Lean Startup experiments is just as important as understanding the Lean Startup methodology. Without having the right parameters to navigate your new methodology - the knowledge in itself is pointless. One of Eric Reis' biggest points in Lean is to create systems that allow you to have measurable accounting. It's critical we conduct tests that have measurable accounting.Here is a high level outline and guide we follow to create successful lean startup experiments at Never Settle.  We use a simple tool to manage all of our experiments and because of interest from others we work with we published this free Lean Startup Experiment Tool for anyone to use.

how to create lean startup experiments

1. Conduct Multiple Tests and Review Them Against Each Other

The tool we use allows you to document and create multiple tests on one sheet which allows you to track your product roadmap and journey as you validate or invalidate your assumptions. Having a timeline and dashboard type of view will help you stay focused on the long term objectives without siloing into one specific experiment.

2. Understand Your Macro and Micro Leap of Faith Assumptions

It's critical to understand the difference between a macro and micro leap of faith assumption as well as define them for yourself and team so you are all accountable to the same things. The MACRO leap of faith assumption is the one assumption you are betting the farm on that your product will solve. Whereas a MICRO leap of faith assumption is the one assumption you are trying to validate through an individual test. Ideally your micro assumptions will be trying to validate your macro assumption.

3. Engine of Growth Assumption

Your engine of growth can certainly change over time, but it's critical you define what your assumed engine of growth is - again this is to provide accountability and something to track your decisions and tests against. Most would just define their engine of growth but I would argue this in itself for a new product is also an assumption itself until validated. Examples of growth engines would be:

  1. Paid Customer Acquisition: advertising, affiliate marketing, sponsorship
  2. Sticky Growth aka Organic Growth: The product itself creates growth based on customers use of it without the customer being an intentional evangelist. Example would be listening to Spotify at work or wearing your new Beats headphones outdoors.
  3. Viral Growth: This can come from intentional or unintentional user evangelism. Intentional would be a user of dropbox inviting their friends to drop box to get more space. Unintentional would be sending out a free MailChimp newsletter with MailChimps powered by icon in the footer.

4. Define Your Customer, Customer Problem, and Solution

5. Define Your Actual Testing Experiment, Success Criteria and Experiment Time Frame

Here is where you can create a true MVP and roadmap for this particular experiment as you document what the actual test is you are going to implement, how you define success or failure in that test, and what the timeframe for conducting the test will be.

6. Measure Your Results, Conclude Decisions and Actions, and Document Your Validated Learning

Yes this all takes time but it's time that is exponentially more valuable than just building. This is where the rubber meets the road and great products pull ahead driving specific value to their customers actual problem while average products get further and further behind trying to solve a problem the customer doesn't have or creating the wrong solution to the customers correct problem.

To Conclude: Have a Tool that Works and Doesn't Get in the Way

As mentioned at the beginning we use this free Lean Startup Experiment Tool, however regardless of what you do - find a or create a simple tool that works and doesn't get in the way.  Using the wrong tool is no different that solving the wrong customer problem.

Lean Startup Experimenting Tool
yellow hexagon