How to Run Successful Lean Startup Experiments

Being able to create and manage Lean Startup experiments is just as important as understanding the Lean Startup methodology. Without having the right parameters to navigate your new methodology - the knowledge in itself is pointless. One of Eric Reis' biggest points in Lean is to create systems that allow you to have measurable accounting. It's critical we conduct tests that have measurable accounting.Here is a high level outline and guide we follow to create successful lean startup experiments at Never Settle.  We use a simple tool to manage all of our experiments and because of interest from others we work with we published this free Lean Startup Experiment Tool for anyone to use.

‍

‍

1. Conduct Multiple Tests and Review Them Against Each Other

The tool we use allows you to document and create multiple tests on one sheet which allows you to track your product roadmap and journey as you validate or invalidate your assumptions. Having a timeline and dashboard type of view will help you stay focused on the long term objectives without siloing into one specific experiment.

2. Understand Your Macro and Micro Leap of Faith Assumptions

It's critical to understand the difference between a macro and micro leap of faith assumption as well as define them for yourself and team so you are all accountable to the same things. The MACRO leap of faith assumption is the one assumption you are betting the farm on that your product will solve. Whereas a MICRO leap of faith assumption is the one assumption you are trying to validate through an individual test. Ideally your micro assumptions will be trying to validate your macro assumption.

3. Engine of Growth Assumption

Your engine of growth can certainly change over time, but it's critical you define what your assumed engine of growth is - again this is to provide accountability and something to track your decisions and tests against. Most would just define their engine of growth but I would argue this in itself for a new product is also an assumption itself until validated. Examples of growth engines would be:

1. Paid Customer Acquisition: advertising, affiliate marketing, sponsorship
2. Sticky Growth aka Organic Growth: The product itself creates growth based on customers use of it without the customer being an intentional evangelist. Example would be listening to Spotify at work or wearing your new Beats headphones outdoors.
3. Viral Growth: This can come from intentional or unintentional user evangelism. Intentional would be a user of dropbox inviting their friends to drop box to get more space. Unintentional would be sending out a free MailChimp newsletter with MailChimps powered by icon in the footer.

4. Define Your Customer, Customer Problem, and Solution

• Before you can test its critical to define who the exact customer is you are trying to target. Too often businesses don't take the time to define their customer and in turn fail while trying to serve every potential user.
• There are a million problems you customer has, if you can define the one you are trying to solve you'll never have enough focus and impact to solve something worth the customer paying for.
• The solution you are going to test in this experiment. Again this whole exercise is about documenting your process so you can go back and creating measurable accounting of what was done to make better decisions in the future.

5. Define Your Actual Testing Experiment, Success Criteria and Experiment Time Frame

Here is where you can create a true MVP and roadmap for this particular experiment as you document what the actual test is you are going to implement, how you define success or failure in that test, and what the timeframe for conducting the test will be.

• It's critical you use specific metrics for defining your success criteria. This can be number of users registered, specific revenue received, downloads created - just ensure they tie to something that actual has real value not perceived value. An example of this going wrong is making page views your metric for a membership site. If you have 1M page views but 2 signups that metric is the wrong one to define success from.
• As simple as it sounds defining the timeline your test will be conducted is as important as having a battery in your watch - you'll have a watch but you'll never be able to tell the time. Without it you can easily spend too much time on creating an MVP or falsely skew your success criteria. i.e. if the success criteria is 25 signup but you don't have a timeframe of which to accomplish that it's a pointless test to conduct in the first place.

6. Measure Your Results, Conclude Decisions and Actions, and Document Your Validated Learning

Yes this all takes time but it's time that is exponentially more valuable than just building. This is where the rubber meets the road and great products pull ahead driving specific value to their customers actual problem while average products get further and further behind trying to solve a problem the customer doesn't have or creating the wrong solution to the customers correct problem.

To Conclude: Have a Tool that Works and Doesn't Get in the Way

As mentioned at the beginning we use this free Lean Startup Experiment Tool, however regardless of what you do - find a or create a simple tool that works and doesn't get in the way.  Using the wrong tool is no different that solving the wrong customer problem.