How To Measure & Analyze MVP Success
The question of how to measure MVP success is an important one. An MVP is a minimum viable product. One of the main reasons for developing an MVP in the first place is gaining feedback and input and understanding its success.
This means having measurement metrics and criteria in mind from the outset.
The discussion below focuses on minimum viable product success with apps and software in particular, but the same information is applicable and transferrable to other types of MVP too.
When to Plan Measurement and Analysis of MVP Success
Measuring and analyzing your MVP should be factored in from the very outset of planning an MVP. Think at the start about how you will measure and analyze the product. If you do this when planning how to build an MVP you will be more likely to succeed.
MVP Success Criteria
Minimum Viable Product success criteria are likely to take two forms. In brief, they are:
Feedback. This is mostly in the form of customer input about how useful the product is. It includes how the product might be improved to make it more user-friendly.
Metrics. Metrics are numbers you can gather. These help with understanding what your customers think.
These will now be considered in greater detail.
Customer Input
The key to your measurement ought to be what the customer thinks about your product. Measuring this may not seem straightforward. Yet there are a number of ways that this can be done.
While many of the metrics will be quantitative (numerical) in nature, customer feedback is qualitative (rich text).
You will be looking for information on their experience of your product and their perceptions of its usefulness or otherwise.
Remember, you must not overlook what customers think. This is possibly the most important information you can gather in your analysis. The numbers alone can only tell you so much. There are various ways you can gain customer input which include:
Forums – by posting on forums where your customers are found you can seek their feedback on your product and ask them how it might be improved. You might also post short surveys or polls here for customers to input.
Interviews – holding short interviews with a handful of customers will help you glean invaluable information for further product development (or not). Ask about the product's usefulness, whether they would recommend it to others, willingness to pay and how they think it could be improved.
Tip: Don’t make interviews too structured. Allow the customer the space to say what they really think, and you will get more from the meeting! Ask open questions rather than those with one-word answers.
Emailing users – once your MVP has a sizable number of users you might consider emailing them to gather feedback.
Input from the product – you can also gather feedback from within the product itself. Build in a free text field into the product that allows the customer to say what they think. This data can be collated and reviewed to improve the product.
MVP Success Metrics
Knowing how to measure Minimum Viable Product success by numbers is also extremely helpful.
| Metric Type | What the Metric Tells You |
| Number of downloads | This tells you about the likely level of interest in your product, and specifically in the need for a product to solve the problem you aim to address. |
| Percentage of active users | The number of downloads does not tell you if people actually use your product after downloading it. Understanding the percentage of active users does. The higher the better for this metric. |
| Percentage of paying users | A high number of paying users may suggest that your MVP is a success. Again, the higher the better here. It is not a success if the customers pay and then stop using it – if so you need to work out why! |
| ARPU (average revenue per user) | This can be tracked monthly and is calculated as the purchases made in the app divided by the total number of active users. |
| In-store positioning | Store placement will give you an idea of how popular your product is. It can also inform you on whether your product is easy for customers to find. |
| Churn | The percentage of people who stop using your product. You could consider this daily, weekly, monthly, or annually – or all of these. It is calculated as the number of customers lost per month divided by the number of users at the start of that month. |
| User ratings | User ratings can provide very helpful input on what customers think about your product. |
Be aware that at the outset many downloads may be one-time users only. As your MVP is refined to better meet customer needs, the number of one-time users only should reduce. If not, there is something wrong with your MVP.
What to Look for in the Numbers for MVP Success
There are some useful benchmarks you can consider, to know if your app is successful or not. These are:
- Achieving a rate of 1.5% of total users who make a purchase from the app.
- Ensuring that around a third of your registered users (30%) open the app once a month.
- Ensuring that one in ten (10%) of your registered users open the app once a day.
- That the cost of acquiring customers is not more than one-third of the total benefit you will get from that customer over their whole time using your app.
- For user ratings, you can expect fewer than 1% of customers to rate your product if it is an app.
Time and Resources Spent Versus Value of Data
Different levels of effort reap varied rewards in terms of the quality of information you will gather.
Gathering qualitative data from customer interviews will take significant time and resources. Yet it may prove invaluable in understanding whether your product will succeed or fail and why. This is tremendously useful in taking your MVP forward.
Looking only at the metrics is not especially time consuming, and it can also provide valuable information. The question it might not answer is the “Why?”. You might see that many customers stop using the app quickly through the figures but understanding why means asking them what they think.
The optimal approach in how to measure MVP success is likely to include gathering a combination of both customer input and metrics.
Of course, there is no point in measuring and analyzing if you are not going to act based on what you learned. Don’t forget this important step!
Summary
Summarizing, measuring Minimum Viable Product success needs to be planned from the time you start planning your MVP.
The data collected should include metrics that can be gathered about how your product is used. It is also very helpful to gain input from customers about what they actually think.
