How to build better products by betting small and keeping an eye on the race

The Grand National is a National Hunt horse race held annually at Aintree Racecourse in Liverpool, England. First run in 1839, it is a handicap steeplechase over 4 miles 514 yards with horses jumping 30 fences over two laps. It is the most valuable jump race in Europe, with a prize fund of £1 million in 2017 [1].

In the UK and Ireland, the Grand National is the most watched race of the year. It is an amazing spectacle to watch for the excitement, the fierce competition, the danger that comes with the massive fences and obviously to see if the horse we backed wins.

Every year, I put a small bet on one of the horses, but I don’t remember ever winning in my last 20 attempts. Predicting the winner on a field of 40 horses is not a simple matter, it is indeed very complex.

So many things can happen during the race that can throw even the most experienced and knowledgeable horserace expert. Among the things that happen during the race, changing weather and track conditions, the form in which the horse woke up in the morning, the training or lack thereof he had the week preceding, the horse in front of yours falling at the fence and bringing you down with him, the amount of drinks that the jockey had the night before, how distracted he might be from other thoughts in his head, after all his wife is divorcing him, the effect that the noise from the crowd has on your horse, and just about another million random events that could happen in those 5 minutes of racing.

Predicting a winner among the 40 participants is hard

But what if we could bet while the race has already started? Would that help us with our prediction?Well, yes. We could discard the horses that have fallen at the first fence, the unseated ones and the ones that have fallen too much behind.

That’s interesting, does that mean that now our chance of backing the right horse has increased?

Yes indeed and this does not only work for horse racing, it works for product development and innovation too.

Let’s imagine an organisation with €1M budget. Based on their understanding of the market they decide to invest the full budget into delivering “my shiny project” that again according to their understanding of the market will give them a return on investment when finished of €10M.

In gambling terms they are making a $1M bet on the “my shiny project” horse at 9/1 odds.

So what do they do? They use the €1M for assembling a team getting them the necessary resources to get the work done and do the work to deliver the project. While they are delivering, the race is on, but they are not watching it, they are busy building the project. Once they are done, only then they will know if their bet was a winning one.

If things go well, they might even get better odds as return, they might make $100M instead of 10, but history has thought us that this happens only in statistically negligible cases, while most of the time the result is that we have lost most of our money if not all of it. There are more fallers than winners at the Grand National year after year; and indeed there are more losers than winners, in fact on a field of 40, 39 lose, and one wins.

What can we do to help our organisation win the bet? Is there a possible approach that gives us the edge over other companies?

Oh yes, there are ways to help you win the race, let’s look at one example.

We take our €1M budget and allocate 9% of it to bet on the product we believe will win based on our understanding of the market and we pay an extra 1% to watch the race.

How do we watch the race? We invest 1% of our budget in tracking how our customers use our product with the intent of understanding what our customer really needs.

This would equate to spending €90k on betting on one horse and €10k on a live stream of the race we are betting on.

bets
Don’t bet all your money when you have the least amount of information

 

While we are watching we find out that our original horse has fallen at the first fence, and we also observe that another horse, we never thought could be the winner, is jumping very well and is at the centre of the leading group, a very good strategic position.

Then obviously we decide to invest another €90k betting on that horse and another €10k to keep on watching the race.

risk
Gain an unfair advantage against your competitors by watching how your customers behave and limit the risk of your bets

You can clearly see where I am going with this analogy. By limiting the risk of the initial bet and investing in observability I am gaining a clear advantage on the people that made their big bet before the start of the race.

 

In product development we can do this by making sure we create customer observability in our product to have a live stream view of the customer behaviours that informs our future bets.

Sources:

[1] https://en.wikipedia.org/wiki/Grand_National

What can Product Development learn from our body temperature regulating system?

Systems are all around us, and inside us. Understanding the systems we live and work in is fundamental if we want to navigate the problems that they carry.

Let’s think about our body as a system. It is a pretty complex and advanced one as it went through millions of years of evolution to get to the current state. Its working is truly fascinating and even though we have been studying it for millennia, there are parts that we still don’t fully understand.

Our body, as all systems, contains feedback loops. One we are very familiar with, is our body temperature regulating system. In order for our body to work well, its temperature needs to be between 36.5 and 37.5 degrees Celsius. In extreme circumstances, if it goes under 25 degrees we can die and a rising body temperature can damage our organs very quickly.

How do we make sure that this doesn’t happen without a thermometer that measures our temperature constantly?

Our body has a very sophisticated yet simple feedback loop system that allows us to regulate the temperature to the optimal range. In fact if we go out naked in the snow, we immediately feel a cold sensation, decide to get back into the house to put on some heavy clothes and then go out again. If we are running or exercising for a prolonged period of time and our temperature raises, our system will automatically start sweating to reduce and regulate it to its optimal range.

body temperature
This in Systems Thinking terms is called a balancing feedback loop.

What happens every day of our life is that our body temperature will oscillate within a range of acceptable temperatures because, through evolution, it has designed a defence system based on fast feedback loops. By continuously knowing when it’s too hot or too cold we are guaranteed we won’t die frozen or for too much heat. A slower feedback might have caused our body to freeze, imagine for example if we felt the cold only after 24 hours in the snow, but a feedback loop fast enough prevents all that.

What a great system our body is. But even such a wonderfully evolved system has its Achilles heel, in fact it can be damaged by slow feedback loops.

Let’s think for example about the effects of smoking tobacco on our body. How come we only recently found out that smoking is so dangerous for our body? Why did it take so long?

When we smoke our first cigarette, chemicals such as tar and formaldehyde penetrate the cells in the lung tissue and change them. At first our body may be able to repair the damage. With repeated exposure, normal cells that line our lungs are increasingly damaged. Over time, the damage causes cells to act abnormally and eventually cancer may develop.

The problem is in the fact that the time that will take for the cancer to develop is so long that only recently we could only start hypothesising tobacco smoke was a potential cause. You can easily imagine why. In ~30 years our body absorbs an incredible amount of chemicals coming from all sources, the food we eat, the air we breathe and the tobacco smoke we inhale. How could we have guessed? It was very difficult and only through data driven medical studies we managed to identify a clear link between tobacco smoke and cancer.

tobacco smoke
Slow feedback loop

Tobacco smoking is a very recent habit in human history if we consider humans evolution and our body has not yet developed a defence from it. What happens unfortunately is that our body will alert us of the cancer (feedback loop) only when it has damaged one of our organs. At that point, unfortunately, it is often too late to repair the damage.

A slow feedback loop can often look like the this picture. No impact for a long period of time, then a massive impact all of a sudden.

Slow feedback loops are everywhere around us, look at how some policies that seemed harmless at first sight, have caused migrations, desertification and destruction, years after, look at how the excess co2 in the atmosphere is damaging our planet slowly but nevertheless unequivocally.

Not being able to predict the effects of policies and changes to a system is troubling, but how does this fit with modern organisations and product development?

The link is in the feedback loops. We have seen earlier how the fast feedback built in our body temperature regulating system has helped our body defend itself from changes in temperature for millions of years, can we learn something from it?

A system that oscillates within a healthy range, like our body temperature, is a balanced and healthy system. What produces the healthy oscillation is the speed at which the feedback is delivered. Our body senses the changes in temperature very quickly and allows for regulating it in a way that appears to us to be continuous.

Now let’s think about organisations that rather than regulate their own temperature are trying to achieve an outcome.

Let’s look at product development for example.

Traditional organisations spend quite a lot of time studying the market to identify new solutions or products that can help them achieve an outcome. This work is often done by internal experts. Such experts have been involved in delivering previous products and are naively expected to have the wisdom that is required to design the product that will achieve the goal.

A lot of work happens inside the organisation and a lot of time passes before the new product is given to the customers. The more time the idea stays within the organisation the more we build assumptions upon assumptions instead of satisfying real customer needs. After one year or two we finally have a new product on the market, but very often we realise that the outcome we had originally thought we would get is not materialising. We often are very far from the expected outcome. We are objectively surprised, sometimes astonished at the results.

Our plan didn’t survive contact with reality.

Reality indeed, this is the first time we get feedback from our customers and it’s not what we thought it would be.

waterfall
Effect of slow feedback loops in Product Development

 

Organisations rarely understand the cause of the problem and often an unsuccessful product causes blame games. Everybody is to blame, excluding the people that designed the product. “The market has changed!”, “The latest regulatory policy makes it more difficult to use!”, “The downturn of the economy killed us!” Et cetera. I even heard leaders blame the customers, saying things like “they don’t understand!”

Slow feedback loops for product development cause something like this picture. The delta can be very big in the presence of slow feedback loops.

What can we do to avoid the truly sobering experience of showing our customers the fruit of years of our work and seeing it rejected? Well, we can’t avoid it completely but we can make it much less painful if we introduce a balancing feedback loop like the one

fastfeedback
Fast feedback loops in Product Development

that regulates our body temperature. 

The first thing to do is to identify an outcome we want to reach over time. This outcome is for us now the ideal body temperature we want to maintain.

The second thing we need to do is to plan regular and frequent feedback loops with our customer to verify directly in the real world how we are doing against the outcome.

The third thing is to measure how we are doing against the outcome and design the next bit of functionality taking in consideration the learning from the last feedback loop.

A fast feedback loop on product development causes a behaviour like the one in the picture.

By doing this we make sure not to diverge too much from our desired outcome. We know that we will make bad decisions that might move us further away from reaching it, but we also know that through frequent feedback loops we will be able to discover our mistakes early and take actions to repair them in a cost effective way.

It looks like we can learn a lot from our body temperature regulating system and treat our customers like our body regulates our temperature, with care and continuously.