Despite what you might think (and for sure despite what I thought) it turns out that my business has some seasonality. Theoretically you would think that a consulting business isn’t impacted by the time of the year, but I learned that that’s not true. Since I deal with strategic consulting, many companies reach out to me when they start to think about next year, which happens sometimes in H2. The end of the year is also a great opportunity for all sorts of team events, and many companies reach out to book lectures for these offsites during Q4. One seasonality that definitely happens but I don’t have a clear explanation why, is with accelerator programs. For some reason, most of them want my product strategy and product-market fit lectures during May and June, every year. 

This year I couldn’t take many such lectures, since among other things I was in London participating in Marty Cagan and SVPG’s Coach the Coaches workshop. But in regular years, I could be giving the same lecture 3-4 times over a couple of weeks during these months. That’s exactly what happened a few years ago. I gave my talk on “The Journey to Product-Market Fit” three times within a two week timeframe. It was in three different accelerators and training programs, and the audience was entrepreneurs and experienced product leaders.

The feedback on all three talks was great. I don’t always get quantitative feedback, but for one of these talks I had it and it was 9.55 out of 10. I was proud and happy.

But it wasn’t always so. Prior to giving these lectures in the accelerators, I gave the exact same talk multiple times in a different training program, and the feedback was very different. My scores were constantly between 7 and 8, and one session even scored as low as 4.8. Ouch.

The qualitative feedback on these sessions also expressed a similar sense: people said it wasn’t practical enough for them and they didn’t learn anything they can use in their day job.

Something definitely didn’t work. This feedback was telling me quite explicitly that my product (my talk) is not good.

But this conclusion is not necessarily true. And in my case, it really wasn’t, as evidenced by the great feedback I got later on without changing the talk. So how can you tell if bad product feedback is a product problem or not? Here is a quick guide.

Your Product Doesn’t Live in a Void

After receiving this bad feedback, I obviously felt really bad. Who wouldn’t? But having looked at it closely I realized that the problem wasn’t with the product, it was that the product was delivered to the wrong audience.

Unlike the last 3 sessions, the audience in the other training program mostly consisted of freelancers and people who aspire to become product managers. All smart and motivated people, but still a very different audience with a very different perspective on product management and strategy.

Unlike the last 3 sessions, the audience in the other training program didn’t really need the product I had to offer. They needed something else. One of the reasons they didn’t need my product (my talk) is that their problems are very different from the ones experienced by entrepreneurs and seasoned product leaders. Their goals and motivations are different, and their challenges reside elsewhere.

The pain I was addressing with my talk, wasn’t a real pain for them, and as a result, they felt it was good for general knowledge, but not much more. A nice-to-have product at best, but not something they really need. 

Luckily, I knew exactly which audience I created the talk for, and knew that this audience wasn’t it. I decided to keep the talk as is, deliver it to the right audience and see if it works better. I still wasn’t sure it was a good enough product but decided to run this pseudo A/B test before changing anything. Note that it wasn’t a classic A/B test most of us are used to running since the variant here wasn’t in the product — it was in the audience itself. There was also nothing statistically significant in the test, but it doesn’t mean that you can’t learn from it – which is exactly what I did.

That’s another reason I was so happy with the quantitative feedback I got later on — it validated for me that not only the talk itself is good; it also showed me (again) that entrepreneurs and seasoned product leaders often need help in the area of product-market fit, which is super important to validate since it is one of the primary services I offer in my consulting business.

Bad Product Feedback Is Not Always a Product Problem

When your product gets bad feedback and isn’t getting the traction you expected or generally doesn’t deliver the results you were hoping for, it is very easy to conclude that the product isn’t good enough.

But sometimes, as in my case here, it wasn’t the product’s fault. The problem lied elsewhere.

Many CEOs I talk to, think of the product only as the actual technology delivered to the company’s customers. That thing that is being released to production and customers use. But the definition of the product is much more strategic, and should concern you nonetheless. Among other things, the product includes the selection of the right target audience for it, the reasons for your customers to be willing to pay you for it, and the processes through which it happens (sales, not just billing). It is not uncommon to have a great product that fails with a specific target audience but is a great success for others. 

That’s why the product strategy is so important – it helps you ensure you solve a good problem for the right audience. The problem and the audience always come together — a problem doesn’t exist in a void, and a problem that is relevant for “everyone” is never specific enough.

Side note and warning for B2B product leaders: if you think this is not relevant for you, because you are selling to a company and not to people, think again. The problem you are solving is always a problem of specific people with specific characteristics within the organization. You are not off the hook. If at all, your job is more complicated since the people within the organization who experience the problem are not always the ones who can pay for it, so you need to solve multiple problems for multiple stakeholders to be able to succeed. Remember: companies don’t have problems, it’s always someone within the company that owns the problem (or it’s not important enough to anyone and not worth your time).

So What Can You Learn From Product Feedback?

Always have a deep understanding of the problem you are solving and for whom.

The problem and the characteristics of the audience should be defined at a much more explicit level than what feels intuitively right. You can find more information and tips on this topic in my executive’s guide to strategic product management — Speed-Up the Journey to Product-Market Fit.

Make sure it is a real problem, painful and important enough for them. It’s always better to do this before you build the product, but even if your company didn’t do it up until now and you already have a product — better late than never. You don’t need to be worried, many companies evolve this way — they start with a generic need and only after they already have a few customers — and something isn’t working very well — they get into defining their product strategy more explicitly. They are much more knowledgeable to do it right at this stage — after their product rubber hit the market road — than on paper when they started.

Do whatever you can to reach the actual audience you intended for. It’s not always easy, and in some cases, you will find that other audiences are showing up instead. If that’s the case, ask yourself why this is happening: it could be because of marketing to the wrong audience, but could also be because the original audience and problem you thought of are not there. Or, they might be there, but the other audience you see has a much bigger need that your product can fulfill — you might have an opportunity there.

When things are not working well, understand the root cause before changing anything. This is generally a good practice in life, but let’s stick with product feedback :-).

When your product isn’t performing well, always get back to the problem you are solving and for whom.

Sometimes, it’s the wrong audience. In others, the problem is not painful enough. Changing the product blindly in these cases won’t help you. You should change your product strategy first (or create one if you didn’t have it up until now).

Sometimes, the problem and audience are just right, but the product doesn’t actually solve it (usually because it does not consider everything the audience needs to solve this problem, and either only takes them part of the way towards a real solution or conflicts with other constraints and needs they have). These are the cases where changing the product without changing the problem and the audience can actually make a difference.

We live in an agile world, so listening to feedback and changing accordingly should be part of our DNA. But changing without understanding what needs to change and why can do more harm than good. Always keep an eye on the bigger picture.