One-Trial Learning: Or, Counting on Luck

One-Trial Learning: Or, Counting on Luck

By Mark Chussil

I love data. I spent 15 years working with the PIMS (Profit Impact of Market Strategy) database alongside academic luminaries. (I was in the room but I was not in their league.) Many years later, I think it’s fun to run and analyze a billion simulations before breakfast. I love data.

But data isn’t learning, and learning isn’t just about the amount of data.

How much data do we need to learn? One data point gives us an existence proof; that is, proof that that data point is possible, because it exists. It’s also an anecdote, and the learning that people get from that one data point properly called a story.

To see why, let’s look at rats.[1]

Imagine a T-shaped maze. A rat enters at one end of the T. It scurries along and soon faces a choice: go one way or go the other.

If the rat goes one way, it gets a piece of delicious cheese. If it goes the other way, it encounters an irrevocably fatal electric shock. (Note to the nervous: This is merely a thought experiment. No real rat was harmed in the writing of this essay.)

That mythical experiment parodies a serious idea, one-trial (one data point) learning, proposed by Edwin Ray Guthrie (1886-1959), a psychologist at the University of Washington.[2] The joke, of course, is that a rat that survives hasn’t really learned. It just chose the right path by chance.

That’s all very amusing and/or academic, except not for the rat. As it happens, though, one-trial learning can teach us something about decision-making.

What has a surviving rat learned? The answer is simple: nothing. The rat doesn’t know what would have happened otherwise. (For a different reason, neither does a rat that turned the other way.) The rat has only an anecdote.

If one trial isn’t enough for learning, though, how many does it take for us to know something? Too many. We don’t usually have enough time, let alone money or appetite for risk, to run randomized controlled trials on our businesses. (Possible exception: test markets.)

Fortunately, there are options between one-trial learning and too-many-trials knowing. That option is competing as a skill, as opposed to competing by counting on luck or competing by looking for certainty. We can greatly improve the quality of our strategy decisions at costs in time and risk that we can bear. Or, as a data-lover might put it:

Greatly improved decisions ÷ Costs we can bear
= Impressive returns on investment

When the stakes are high and the possibilities are plentiful, there is simulation, computer-based (e.g., strategy tournaments) and/or role-playing (e.g., business war games). Those techniques can simulate strategies too risky to test in real life, uncover surprises in a safe environment, explore many possible futures, and offer rigor and impartiality. They’re not perfect — nothing is — but they greatly improve the odds of making good strategy decisions. Think of it this way: Gamblers lose by only a few percentage points when they spin the roulette wheel (the house edge is between 2.7% and 5.3%), but that’s how casinos make millions.

By contrast, we can accumulate anecdotes. We can see which companies succeed and which fail, we can make up stories about them, and we can believe the stories of their inevitable successes and failures in finding the cheese.

Sometimes time is too short for simulation or war-gaming, and the problem is merely whether we’re ready to take advice from anecdotes. For those situations, try a variant on the headline test: Turn the advice around and see if the opposite is possible too. For example, consider “be first to market if you want to win.” Can you win if you’re not first? (In some markets, clearly yes; in others, arguably no.) If you can win without being first, then “be first” is a strategy option, not an obligation. And when you spot new options, you spot new opportunities for competitive advantage.

You can be a furry hero telling squeaky stories, or you can learn about mazes. Don’t worry about where the cheese was yesterday. Your job is to find the cheese today.

Further Reading

My friend Ben Gilad posted a terrific essay on the shortcomings of “data-driven” decision-making. Ben and I wrote The NEW Employee Manual: A No-Holds-Barred Look at Corporate Life (Entrepreneur Press, 2019).

Professors Bart De Langhe (ESADE Business & Law School) and Stefano Puntoni (Rotterdam School of Management, Erasmus University) spoke about “Understanding Decision-Driven Analytics” in a 2021 MIT Sloan School webinar.

[1] You think people don’t find rats exciting? Put a rat in a room full of people and see what happens.

[2] For more, see, “psychology students’ best friend.”

Share This Comment

No Comments