Many years ago, I was a system architect at the Israeli Air Force (IAF). I was part of a team of extremely talented people, who were tasked with defining the next generation of the central intelligence, command, and control system for the IAF.

This system was meant to replace over a dozen existing systems, which meant extreme complexity.

Determined to do it right, two of our team members did some research and embraced then-modern development processes like Extreme Programming. I started to hear terms like agile, pair programming, refactoring, and others.

We felt we were leading a revolution.

But our managers weren’t all that happy. I still remember that in a monthly meeting with our boss’s boss, as we were describing what we were doing, he stopped the discussion angrily and said he was unwilling to hear any more about refactoring. That it was merely a pseudonym for rewriting, and had we done our job right, no rewriting effort would have been needed.

I remember that I only thought that he didn’t get it, but I didn’t know exactly how to explain it to him or even to myself. It took me a few more years to understand the full power of refactoring and why you must embrace it. 

The idea that if only you planned well enough your code wouldn’t need to change seems so naive today, but back then it was the reality. It was all he knew, and words matter.

By the way, as I explained last week, today people tend to err in the other direction and avoid planning altogether since it will surely change. Wrong again. 

To the same extent that my boss’s boss didn’t like the word refactoring and it made him angry when it shouldn’t have, I see people these days using words that they love and make them confident and happy, but that’s also an illusion.

One such word is “validation”. 

That term that you use when you embark on a new initiative, a new product, or a new company, and is aimed at helping you ensure you do the right thing.

I’ll start from the bottom line: there is no such thing as validation. It’s not in the sense that the process is wrong but in the sense of the expected outcome.

Here is why, and what expectations you should set instead.

By the way, since that’s the common word in the industry, I’ll keep using the word validation to describe the process itself, but not for the essence of it.

‘Validation’ Has a Scent of Certainty

When you think about the word validation it sounds as if once you are done you know things are valid. Unfortunately, that couldn’t be farther from the truth.

I’m not talking about the fact that if you do your job right, in most cases you’ll find out that you were wrong and your assumptions were not valid. It’s true, but that’s not the real problem.

I’m talking about the fact that in tech, especially when creating a new product, certainty doesn’t exist. 

The world we live in is so complex that you can’t figure it out fully, no matter how hard you try. 

So I’m not saying you shouldn’t try, what the industry calls validation is an extremely important process that too many companies skip and pay a high price later on. But you shouldn’t expect to be certain about anything at the end of this process.

Instead, you should change your mindset to a risk-management one.

The whole point of going through the validation process is to eliminate the risks you are taking but to a certain point. You cannot eliminate all risks (and even if you could, it would create an alternative cost that is not worth it – staying in the research mode forever). Therefore, you need to start managing your risks.

When you perform your validation, you need to understand what risk you are trying to assess. Having a solid product strategy (or the skeleton of one) helps in the sense that you then know which hypotheses you are trying to test, even if you don’t have a product yet.

Once you understand that it’s not about validation but rather about risk, a new set of answers is made available to you, and your decisions can be more informed.

You can feel fairly confident about certain assumptions and less confident about others and still decide to move on to development since the latter have less impact on your chances of success and you might have more time to validate them later on.

You can understand that within your complete theory about why the world needs your product, there is a certain point that if it’s untrue everything collapses. You can then assess this specific point explicitly, and perhaps leave the others less “validated”. 

It’s Not Binary

Theoretically, validation is about answering a simple yes/no question: is my assumption valid?

The problem is that the answer is not that simple. Not in tech products in general, and specifically not in phases that are more strategic and high-level, where qualitative answers are needed rather than data crunching that leads to statistical significance.

In our world, the process called validation doesn’t give you a definitive yes or no answer, after which you can move to the next step. 

Instead, it gives you insight. It gives you feedback. It gives you perspective. It gives you food for thought.

And that’s what you can and should expect from it. Instead of a yes or no answer to your question, what you should look to get is additional information that would help you refine your original thoughts.

Unfortunately, validation cannot decide for you, this part is still on you.

It’s Never-Ending

Another problem with the term validation is that it gives you a false sense of “getting it over with”. You validate, you understand that things are valid, and you are done.

But a good “validation” process doesn’t look like that. Since the answers are not binary, and since you often don’t know what you don’t know, it’s an iterative process like many others in product.

You take your assumptions to the market, get feedback, and refine them. Then you take them again, get additional feedback, and refine them again. You keep doing that until you see certain outcomes that satisfy you. But then, you are back in the market with a new hypothesis to test, or with a full-blown product that requires, well, validation. 

In that sense, discovery is a much better term.

Note that product discovery is just one part of the process. Problem discovery, solution discovery and customer discovery are also important types of discovery, and all would typically fall under the general definition of the validation process.

Now, let’s assume you have done your discovery, and got to a point where you feel you are done.

But what you learned today, might not remain the same way in the future. Our world keeps changing, the markets you operate in keep evolving, new technologies disrupt how people work, and macroeconomics impacts how people make decisions.

So even if something was valid at a certain point in time, you cannot rely on it to remain that way for the longer term.

So keep going through validation processes, since that’s what the industry calls it. If you do, you are already in a much better position to succeed than many others who skip it altogether. 

But you should also know what to expect. Validity isn’t it.