Once you have filled in your canvas, you are stuck with a pile of stickies filled with assumptions. Do they all need to be tested? Is every single one of them risky? Where do I start?!
There are a couple of tools that can help you prioritize and create order in the chaos. One that is really helpful is the Hilo matrix. In a guest post on Tristan Kromers blog, Dan Toma is writing about how this has been helping teams prioritize on risk in a very easy way.
Once you have written down all of you assumptions on stickies you can use this visual method to decide which things needs derisking the most. First you order you assumptions based on the knowledge you have on that subject. Do you know the facts the mindset, the problems etc… or not at all. Order your stickies along a horizontal axis from left to right. From ‘I know a lot about it’ to ‘I know nothing about it’. Once you have ordered them, add a vertical axis. Going throught the middel of the horizontal one, dividing everything into 4 squares. This axis is going from ‘Has low impact on the Businessmodel in the bottom to ‘ Has high impact on Businessmodel’ on the top. If you order your stickies as honeslty as possible the right hand top quadrant is now containing all of the assumptions that are most risky and should somehow be tested.
Using this tool to prioritize experiments works really well in combination with our progress Canvas. The NEXT canvas is using 8 building blocks ( partially taken from Ash Maurya’s Lean canvas) and is ordered in such a way that the blocks form steps that correspond with the 4 basic phases in a startup journey:
From left to right it determines the order ( or proposed order) in which it makes sense to test assumptions first. Moving along the canvas will show you the progress you are making towards product market fit. Starting your experiments with assumptions that are marked in the top right quadrant of the HiLo matrix AND are in the first step of the canvas is a perfect place to start. Testing risky assumptions on your solution make less sense if you havent proven the problem yet.
The summer is getting near the end, most of you have been on holidays or wel deserved breaks. We have been on a different kind of break. Whilst working with our NEXT Canvas with the startups we help, we were seeing a way to help both startups and companies to actively keep track of what is happening within a startup or innovation team. Right now we use a process of using the canvas to ‘see’ actual progress and interaction with, as well as keeping track of, experiments and validation. It is on the wall with the paper version of both the card and the canvas. While we store experiments and progress over time in a Trello board. We figured it could be made so much clearer if we could get the mental model out of our heads and into a digital tool to guide you. We decided to act upon that and dedicated two weeks in August not going to Bali or Ibiza. Working solely on this online tool during our ‘internal summer hackathon’ ( with just the two of us 😉 ).
As a result a first minimal version is online, based on what we have learned from the paper version of our canvas and experiment board so far. The tool needs to be tested by a few startups right now, so we can learn as much as we can from the start. Things like; How do we convey our mental model to the users and how can you benefit from this. Right now it has a few basic functionalities. You can add a project that contains one or more canvasses. Each segment of the canvas can hold assumptions. To each ‘sticky’ with an assumption you can add one or more experiments. There is an experiment board for an overview of running experiments as well as an experiments inbox.
You are welcome to give it a try, provided you ask us when something is unclear and tell us when something is wrong. We want to learn, learn, learn, so we can continue to integrate and optimize. To be able to build something that will help you build a viable business model.
One of the most important tools, or experiments you can do throughout the search for your business model is the customer interview. Whether you are finding out if there is a problem, or want to know about what it really is that people are trying to do, the customer interview is an important tool in understanding more about the behaviour and needs of your customer. Most startup founders we know tend to be scared by the notion of actually getting out of the building and start talking to strangers. They compile surveys and questionnaires they can conduct online and send out from behind their computers. Easier, far more comfortable, but the outcome will be averages rather than patterns. Talking to real people, listening to the things they say, their expression when saying it, will help you investigate and drill down to the why behind the what. You think they will not tell you? People love to complain about what is not working for them, or to tell you all about their passion, all you have to provide is a listening ear.
Even though customer interviews remain awkward, there is a couple of things to help you feel more at ease. It is not that hard; You do not have to talk about your solution, you do not have to pitch. This is not about you or your product, this is about your customer. All you have to do is listen! Sounds easy enough right? Keep reminding yourself of this, because your first urge is to pitch your obviously brilliant solution. Don’t! Ask what they are doing to solve their problems instead, and why they are doing it. Once you have found a good way to start conversations with potential customers the process will get easier and more and more valuable. We keep conducting customer interviews regularly, even though we are past the customer discovery phase. Sometimes metrics show you a lot of things, but if you want to know the why talking to people is always the most valuable option. Want to grow? Target more customers? Understand what your current customers are doing and why, this will help you learn valuable insights.
How do you turn customer interviews into an experiment?
Box it! In time and in numbers. Be sure to interview as many people as possible in the time of a week. Or the dedicated days within your experiment loop. If you conduct interviews over a longer time, every single interview will slowly influence your overall view slightly. But by considering the whole of interviews as one experiment it will give you a chance to discover patterns and actually move forward from there.
Setting up for the interviews.
First make sure that you know what it is that you want to learn. Define your hypotheses. Personally I’d like to prepare questions that refer to a specific topic that I want to learn more about, related to the hypotheses. No follow up questions, just topic starters. In this way I am able to make sure I tick of all the topics I want to ask questions about, and still be able to drill down if something interesting comes up.
Asking the actual questions.
The other week while giving a workshop, somebody asked me; “I have talked to customers, they have all said they loved what I was doing, but now nobody is actually that enthusiastic about the real product. How do I get actual value out of those interviews because to me it felt they where just lying.”
Since people usually say one thing but do another, this is indeed a very valid question. It is important to try and ask questions that will provide you with answers of high value. Believe it or not, everybody lies to you. Out of disinterest, just to be polite, because they want something else from you, or to not to destroy your dreams. The biggest liar is your biggest fan: Your mom. She will never want to destroy your dreams with telling you your idea sucks. Rob Fitzpatrick came up with a way to get valuable answers knowing that people lie to you. Every single question you ask should be asked in a way that even your mom cannot lie to you. He called this ‘The mom test‘.
The Basics:
Stop fishing for compliments, this is not about you or your product, this is about the world of the customer and how they perceive their problem.
Compliments have no value in validating your business model
Avoid generics or abstracts, this will only lead to guesstimates. Ask about real events and what they did dealing with them.
Actual events rather than abstract averages
Never ask questions that contain would or should,nobody can predict their future. Ask what they are doing to solve their problem right now instead.
Behaviour rather than whishes
What you do want is to know what they perceive as a problem, what they are doing to solve it, and ‘why’ they are doing this.
The Analysis
Now when you have interviewed a good number of customers it is time to analyse data and make decisions from there. Start of by writing every single answer on a separate sticky note. (if it helps jot down a number or initials to refer back to the interviewee) lay the stickies out in a grid. Where the interviewee is played out across columns and the topics across rows. Make sure to leave some room in the row section since there are going to be different and more than one answers available here. Stick answers in the grid. Now start looking for similar answers or problems and group them together.
Can your hypotheses be validated?
Always make sure to take pictures.
See if there are any other patterns emerging.
It feels like a lot of work, but doing this as a team will lead to valuable insights. Your brain is (still) the best patterns recognition system there is in the world, you will ‘see’ patterns sticking out to you. Leaving through the answers that are written down will not give you the insights it will when visualizing it.
The power of visualization
For instance; We interviewed high school students on the use of a calendar. The hypothesis was that high school students have problems with existing calendar apps while no longer using paper calendars because they forget them or find them archaic. By determining a more specific segment we wanted to see if we could see if the problem was different or maybe more prevalent in order to offer more value in our intended solution. I prepared some topics that I wanted to learn more about: Current apps on their phone. Classroom and school digital approach. Paper calendar ( did they use it before , if so how). Context of given homeweork and tests. Their planning and learning rituals ( if any). We validated our hypotheses in a specific group of students we were able to identify. But the real value was the insight that their biggest problem (they all volunteered this information when we saw signed of frustration) was something we didn’t know about. They felt tests and homework seemed to sneak up on them. Tests that only showed up on the actual day, or a day before. They all blamed it on something else, the teacher, yet another crappy calendar app, the system, never themselves. But it was a problem to them because it affected their scores. This ‘job-to-be-done’ became an important part of the new value proposition and the solution. Asking ‘why’ helped us rethink our value proposition. Now, instead of focussingen on a paper calendar replacement, the app gives notifications in advance in three steps before every test. Giving some sense of control back, and help them plan.
In the last post I talked about the importance of validated learning. Trying to validate your way to a working businessmodel. You don’t validate your businessmodel by running one single experiment. Continuously testing risky assumptions is important in every phase of your startup. Helping you to derisk the path to a successful businessmodel.
It sounds easy enough but it is much harder than you think. Your first and foremost urge is to build something really awesome. Show the world your shiny new vision as a brand new product, world domination and everything. Running experiments is something that feels counterintuitive, taking away hours of your precious time and your product. Even when you know it is valuable, even when you realize that learning before building can save much needed time and money and can lead you to more awesome products, it still feels this way. In a way it feels like homework.
You are not building a product (yet) you are building a viable businessmodel.
Experiment design
With each new experiment it is important to remember that you are not testing your product, but rather your businessmodel. Every single experiment is build to learn, not to scale. Ask why. Ask yourself what it is you want to validate. Identify you riskiest assumption and think of how to test this, in the most simple and cost effective way. Do not worry about scale, you are not building your real product yet, you are testing if what you assumed would be the case, actually is the case.
The basic structure for designing experiments looks something like this;
Set a structure and timebox every experiment (1 or 2 week loops for instance)
Decide on what it is you want to learn (this loop) and make this falsifiable. (true/untrue)
Agree on a number that determines succes for this experiment and write it down beforehand.
Design an experiment that will only test what you want to learn.
Build and Run the experiment
Analyse the data.
Update your Businessmodel options with the learnings.
Repeat.
It is important to have a structure of continuous experimenting to show learning progress towards a successful businessmodel. By the time you are ready to scale, you already will have a proven customer base, and a way to grow it into something really big.
The MVP as single proof of concept?
How does the MVP fit into this? Eric Ries talks about the MVP (or Minimum Viable Product) in his book. His definition:
“A Minimum Viable Product is that version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort.”
The term has since been used, and also heavily misused, for a lot of different things. Most of the time people wrongly refer to their first release of the product as their MVP. An MVP is not your single proof of concept that will validate your business. Rather, an MVP is an experiment that will help you validate if the solution offer you have in mind, is something that has enough value for your customer segment. I’d rather avoid the term MVP. In a way your MVP is nothing more than a solution experiment. Most probable in a series of solution experiments, that test your value proposition. Is it the first product you build to capture the value you offer to your customer? Or the experiment without a product yet to test your solution? Let’s avoid this discussion and focus on continuous experimenting. Make sure that what you are building delivers value to the customer and iterate from there.
The experiments that you run when you are still validating the problem, and finding customer segments to test with will be different from the ones that you design later on. In the early phases explorative experiments like the customer interview are important. In fact, talking to your customers will remain important throughout the development of your businessmodel. Other useful experiments later on are smoke tests and concierge models. I will explain about these experiments in following posts. Just remember that with every experiment you design, you want to learn something. Think of the easiest way to proof your hypothesis. Don’t worry about scaling. Think simple, if you have to buy customers to test, buy customers. If you have to mail people by had to mimic some algorithm, do it! You are building to learn, not to scale.
In the previous post I introduced the lean startup methodology by Eric Ries. He defines two phases in a startup. There is a Customer Discovery and Customer Validation phase. The first fase is all about proving problem/solution fit. It starts of with exploring the customer segments, validating there is a real problem and deciding on a solution. The second fase starts with validating this solution offer and optimize this into something that a lot of people will actually use, and more important, pay for. If you can also grow the number of those users rapidly, you are ready to scale.
The first phase is all about validating if there is a customer that has a problem at all. Whereas the second phase is concerned with optimizing your solution offer into one that a lot of people will actually pay for to use. Although the experiments are likely to be different depending on the fase that your startup is in, the goal is the same.
The goal is to learn.
As with any new idea, the boxes in your businessmodel or lean canvas are filled with a lot of assumptions and therefor a lot of options. It is all about finding the few that really matter. Test if assumptions are true before building upon it. Not every assumption needs to be tested though. You start of by identifying the riskiest assumptions on the canvas. The one that, if not true, will have your whole idea come tumble down like a house of cards. From there you learn, update that information into your canvas and decide what is most critical to learn next.
Traditional product or service development cycles, look something like this; decide on a product, do research, build the product, put it out there and see how people react to it. The time between the idea and putting something out there is a rather long one. In this period, typically nothing is learned at all. Only when the physical product or service finally hits the market and people interact with it, something happens. By than it is usually to late to fix problems is the product is not working, all the while spending time and money. The lean startup methodology aims to increase learning and decrease risks. The goal is to learn and iterate all the way to something that works.
Business Science But how do we learn? For every assumption that needs to be tested, you design an experiment to test this. Applying a scientific method. Business science that is, a ‘pseudo’ scientific approach optimized for time and money.
The scientific approach lies in the fact that every ‘experiment’ that you run, whether this is a customer interview, an MVP or an A/B test, should validate an hypotheses. The criteria for this hypothesis to fail or succeed should be decided upon by the team and written down before the experiment. The criteria should be based on business value. (Although there are no hard numbers here, it helps to count back from the bigger picture. i.e. What is the amount of people that need to say yes if you look at it from a market segment perspective and the percentage you need to have a viable business.) Build an experiment, Learn, update the information from the experiment on your canvas and make the next step or decision.
Startups exist not to make stuff, make money, or serve customers. They exist to learn how to build a sustainable business. This learning can be validated scientifically, by running experiments that allow us to test each element of our vision -Eric Ries
The startup experience is really a series of experiments. Every single experiment is build to learn. Your Business Model or Lean Canvas is not filled in at once or only once, but is something that you iterate upon. It makes sense to look at your canvas as the product of your startup. Your goal is to learn and iterate this canvas into a working and scalable business model. Experiment by experiment.
The Lean Startup, Running lean, The four steps to epiphany, we have all read the books. We use the Business Model Canvas and build MVP’s. The Lean Startup methodology has become very popular not only within the startup industry worldwide, but also amongst companies that want to enable innovation and change. Allocating time and money as efficient as possible, shorten development cycles and make something people actually want to use.
But what does it really mean to run a lean startup? Why are we doing it in the first place? In this series I will give a short introduction to the lean startup methodology and how to implement this in your startup.
Let’s start at the beginning.
Why do startups need a different approach from all other companies, starting or not. Why did the Lean Startup methodology came to existence in the first place. The use of the word startup is immensely overused and hyped as a marketing term for every single starting company that has something new or is tech related. The term startup however is used specifically to highlight the difference between starting companies and startups. According to Steve Blank:
A Startup is a temporary organization searching for a scalable and repeatable Business model
What does that mean? It simply means that Startups have no Business Model. Yet. Me starting the most beautiful and successful hair salon in the world does not make me a startup. It makes me an entrepreneur, and my entrepreneurial skills and passion can certainly make a successful business out of it, but the Business Model behind hair salons is an existing one. I can roughly calculate the amount of money I need to put into the business, to know what I will be able to get out of it. I can differentiate, target niches, be more exclusive, but still, I am not inventing a new model to change what has been done before in a repeatable and more important scalable way.
The global reset
With the rapid rise of new technologies, easier acces to people all over the world, access to tools not available before, it also became possible to disrupt the way business was done for years and years with only a small team and an innovative idea. The digital revolution demands a global reset. New ways to accommodate the new economy and change in social structures and relations. Startups are searching for those new ways, combining new technologies with business insights. Searching for new ways to make things work using new technologies and new insights.
Lean Startups
Now no two startups are the same, everything they do is new. Paths to succes can not be copied.
What Spotify did for music might work in the world of books, but the path to succes is going to be a very different one. For startups it is important to have a way of minimizing the time and resources in this search for succes. Minimize the risk in every step of the way in order to optimize the chance of succes. You do not want to find out 100K of your savings and 2 years of hard work and no sleep later that you have build something nobody really wants.
This is where the Lean startup comes in. Lean in itself is not new, the concept was coined in the 1980’s when Toyota introduced its minimal waste manufacturing principles. This management system based on making obvious what adds value by reducing everything else has since been introduced in many industries.
Some of the main principles of Lean where taken to accommodate the typical startup search fase for a businessmodel. Be as efficient as possible, reducing both time and money spent as well as minimizing risk along the way:
Respect people (customers and employees)
Solve real problems for real customers
Improve using an iterative, scientific method
Be willing to discover the answer by testing things instead of “knowing” the answer
Do work in small batches and focus on flow and quality
Ask “why” instead of “who” when something goes wrong
With this the Lean Startup methodology was introduced, its main concept being the Build, Measure, Learn loop or validated learning.
Timan's test blog 