Every day, you make decisions about your product, your customers, your team, your competition and more. You base your decisions on assumptions that you probably have spent little time validating.
These unvalidated assumptions can become costly. You can lose time, money or even your job.
There is a better way to validate your assumptions: think and act like a scientist.
Let’s break it down.
Experiments Help You Make Better Decisions
To make better decisions, you have to challenge assumptions. But it’s not enough to think about assumptions or looking at data.
To challenge assumptions, you must think like a scientist. You must run experiments.
Google, Facebook, Booking.com, Kohl’s, JCPenney, Apple and others run thousands of experiments daily. And have it down to a science.
Here’s how to run an experiment to validate your assumptions:
- Observe and identify your assumptions.
- Establish your hypothesis.
- Run the experiment.
- Collect and analyze the data.
- Draw conclusions and make the decision.
Let’s break down each of these steps.
Observe and identify assumptions
Before making any decision, a little research is required. The first step is to observe the environment and identify any hidden assumptions that could skew your decisions.
Look for anomalies and any missing information. Figure out exactly what you need to learn.
This information will be needed for you to establish your hypothesis.
Establish your hypothesis.
A hypothesis is a concept or idea that you want to prove by conducting an experiment, collecting data, and analyzing it.
A hypothesis has three parts:
- Theory - something you believe to be true.
- Metrics - how will you know if the theory is true?
- Objective - the reason you need to validate the theory.
Here’s a handy template for your hypotheses:
- Based on [your observation, assumption, research], we believe that [what happens, will happen, won’t happen].
- We will know this when we will see these [effects] happen to these [metrics].
- This will affect [customers, partners, employees] in [what way].
Example:
- Based on our observation that users have difficulty finding the “Add More Items” button, we believe that if we change its colour to grey, they will find it faster.
- We will know this when we see more customers (10%) go from the cart to the product page.
- This will affect how many people click on the Add More Items button, which will eventually affect our sales numbers.
Establishing the right hypothesis is paramount to running a great experiment and making the right decision.
Run the experiment.
Running the experiment could be as simple as changing the colour of the button on your site or as complex as building a whole new landing page that will be shown to a subset of your users.
Things to remember when running the experiment:
- Define the experiment parameters - how long the experiment will last and who will participate.
- Identify what you will test - what variables will state consistent and what variable (One) will change.
- Define and collect baseline metrics.
- Run the experiment - do the work needed to prove the hypothesis.
Running experiments is both an art and a science. The steps above are your starting point.
Collect and analyze the data.
Collect the data. Record what has changed and what hasn’t changed.
The important part of collecting and analyzing the data is staying objective. Just because you want something to be true doesn’t mean it is true.
A quote from Richard Feynman makes a great point:
“It doesn’t make any difference how beautiful your guess is. It doesn’t make any difference how smart you are, who made the guess, or what his name is. If it disagrees with the experiment, it’s wrong. That’s all there is to it.”
Draw conclusions and make a decision.
Once you have your data, there is only one question to answer:
Does your data prove or disprove your hypothesis?
That’s it. There is no maybe or if we changed this to that. The data is the objective judge of your hypothesis.
It’s either a yes or no.
Once you have your answer, you can move on to making the decision.
Today’s Action Steps
Here’s how you can act on this advice today:
Before you make your next important decision, run an experiment hypothetically:
- Observe the situation, identify assumptions
- Define your hypothesis
- Run a quick experiment
- Collect and Analyze Data
- Make a decision
If this works out, do it for real.
Outro
Experimenting is one of the fundamental secrets to the success of many companies and people. You can run big experiments with well-defined research phases and hypotheses, or you can run small experiments based on observations you’ve made on a walk or in a business meeting.
The key to making effective decisions and improving your chances of success is running experiments.
—
And that’s all for this week.
See you next Monday.
P.S. I’d love to hear what challenges you are facing in your business. What can I write about to help you personally?
Reply back to this email to let me know.