“Do I really need to start measuring everything that happens on the product?”

That was the question that S., the CPO of a startup that has a great product in the market asked me with a sigh. You could hear the frustration in his voice. Going all-in into data and measurements is not trivial, and even more so when you are a startup that has good traction and hence a lot of other product investments you want to make.

While my answer to S. was that yes, he needs to start measuring because at some point this would become an amazing decision-making mechanism for the entire company, I can fully understand why it seems too big to even start. My advice to S. was to start the other way around: assume you can easily measure anything you want, and you already have amazing dashboards. Now, start shifting your thinking towards a data-driven approach.

Using this paradigm shift, you can start thinking with data without making a huge investment, and even without actually having any data at all. And while the full benefits of it would require you to implement the actual measurement at some point, there is a lot of value in simply thinking this way, even if you measure nothing right now.

Here are 3 specific benefits you can get by applying a data-driven mindset even before you have the data:

Understanding Your Product

The first thing you need to do before you can start measuring anything is to define what is it that you want to measure. To put it differently, it means defining what is important for you to know. 

At the product level, I highly recommend starting with the startup metrics for pirates framework (AARRR). I absolutely love this framework because of its simplicity on one hand and its insightfulness on the other.

In short, this framework describes 5 categories of metrics: Acquisition, Activation, Retention, Revenue, Referral. These categories outline the end to end, high level, customer journey of your product. Since it is so high level, the categories are easy to understand, however, the implementation or definition of what each category actually means for your product is very specific.

It is in this gap between the high-level category to the specific definition of it in your case that the magic happens: by thinking and debating what each category means for your product, you get into the depth of the value proposition and business drivers of it. It fosters discussions that wouldn’t typically happen otherwise, and even if you only understand what you would like to measure but never actually measure anything you will already have a much deeper understanding of your product and how it behaves over time for your customers.

These discussions also help in alignment, since you would be asking hard questions and the debate is an important part of the value you would be getting from it. Questions like “what is our users’ first happy experience” can unveil many different points of view that exist in the company, specifically in the leadership team. Debating this openly and getting to an agreement on what actually makes your customers happy at the beginning of their journey will help everyone do their job better – marketing, sales, customer success, and of course product and R&D.

The question I mentioned above is just an example that would help you with the activation category, but each category has a number of important questions to be asked.

Understanding Your Status

Another reason I love the AARRR framework so much is that it is very easy to understand your current status in each category even if you haven’t fully defined what it means yet. When you look at the categories, you can relatively quickly rate yourself in each category – for example on a red-yellow-green scale. 

Doing so helps you understand where you are in terms of your journey to product-market fit or to growth. If you are good at acquisition but activation is poor, it means people can relate to the problem you are solving, but they don’t see the value in your specific approach for it, or the product itself makes it harder for them to start. If activation is good but retention isn’t, it means that either you are solving a one-time problem and your product isn’t needed on an ongoing basis, or that the product fails to deliver the value that you intended to deliver. If you have great retention but you are struggling with revenue, it might be that the need isn’t strong enough for people to pay for a solution, or that you need to revisit your business model.

As you can see, all of these issues are important ones to address in your journey to build a successful product and company, and the metrics definition – even without any number, are a great way to identify them and take action.

Understanding the Impact

When you start applying this data-driven thinking even without real data, you can take it from the product level to the feature level. Before you even start working on a feature, ask yourself what this feature is all about, and how you will know that you have succeeded.

This helps you crisp up the goals and reasoning behind each feature, and in accordance, it helps with prioritization and even the definition of the feature itself – since now that you know what you are doing it for, it is much easier to define it so that it will actually meet the goal.

Note that in this case too, my recommendation is to not stop at the category level. That is, don’t just say that this feature is meant to increase retention. Explain exactly what kind of retention you want to increase and how. For example, some features are meant to make people return to the product more frequently, while others are meant to help them achieve more every time they enter.

Moreover, you want to ask yourself what would be the impact on the metric – both desired and realistic – to understand how important it is and whether or not it moves the needle. This can be done even without numbers, for example using t-shirt sizing. Since we are not measuring anything here, it is important to be able to explain why you believe this is truly the level of impact that this feature would bring. Actually, this is important even if you measure it eventually.

A Lightweight Approach To Becoming Data-Driven

To sum things up, even if your organization is far from being truly data-driven, and you don’t have a single dashboard or even the willingness to build one, the steps I mentioned above are a great way to start. They require no company resources other than your own thinking.

Once you implement these successfully and regularly, filling in the missing numbers would become a logical next step.