Wal-Mart Radio Tags to Track Clothing, reported the Wall Street Journal on July 23, 2010. The giant retailer plans to use radio-frequency ID (RFID) tags in some clothing so it can keep shelves and inventory properly stocked with sizes. There are efficiencies to be had, costs to be saved, prices to be cut, customers to be thrilled.

According to the article, Avery Dennison, which makes RFID equipment, says a “pilot program at American Apparel Inc. in 2007 found that stores with the technology saw sales rise 14.3% compared to stores without the technology.” Looks like an advantage, and not inconsiderable.

Surely the 14.3% boost isn’t because customers prefer clothes with RFID tags. “Look, Ophelia, I got those jeans with an RFID tag on ‘em!” We surmise instead that American Apparel’s sales grew because customers were able to find the sizes they wanted, due to RFID-enabled systems goosing sales clerks to replenish sizes as they are sold. We’ve all had the experience of not finding our size.

Now we get to the strategically relevant point. Who gets to enjoy the 14.3% increase in sales, and for how long? Is there a first-tagger advantage, and is it worth having? Let’s think it through.

Customers aren’t going to buy more clothes because they find their size in the store. They may buy more from you, but not more overall(s). If they find their size in your store, they don’t have to look in someone else’s store. So if you get the first-tagger advantage, you may score that 14.3% increase.

As your sales rise your competitors’ sales fall. They don’t like that so they take action, possibly also adopting the RFID tagging system. Their sales recover. And your sales fall. Now the important part happens: you decide what your sales decline means.

You could conclude the first-tagger advantage is ephemeral or was oversold. You could conclude that something else has gone wrong in your stores to cause your sales to fall. You could conclude your competitors have spotted hot new trends. You could conclude that they’re simply catching up to you. The last would be reasonable and would prevent knee-jerk overreactions, and I’ve seen my clients do it by using strategy simulations to set performance expectations. But companies generally insist that results should get better, better, better, and a sales decline happens at your peril, and the short-attention-span performance reports companies use neither remember nor remark on “effects of competitors simply catching up.” So you’re going to feel some pressure and heat. (See also Do Not Overtighten and Do Something.)

Time passes. Sooner or later there is a new floor, a new baseline, a level playing field, in which pretty much everyone who could adopt the tags has adopted them. They become a part of the background in the same way that bar codes, credit cards, and electronic cash registers have. Are they worthwhile only during the time you’re the only one with the technology — the first-tagger advantage — or are they beneficial also when everyone uses them? Do they hurt when everyone has adopted them? (See also Putting the Lesson Before the Test.) That question is answerable, although we won’t answer it here. Note that the Avery study reported an increase in sales in one year; it said nothing about profit or subsequent years. (That may be due to the Journal’s reporting, not to the study itself.) The tags may be profitable if they cut costs by via fewer stock-outs, less over-ordering, and less theft. They may be unprofitable if they mostly add costs, such as equipment and systems to track the tagged items, labor to restock shelves, and higher costs from suppliers. Or they may be neutral, as prices and costs rebalance.

Ironically, the company-centric numbers we use to gauge our performance can make it appear that the benefits are better for the later adopters. The first-tagger appears to get a temporary boost; the later-taggers appear to get a lasting improvement. Translate that into performance reviews: the first-tagger decision-makers seem to have over-promised, the later-tagger decision-makers look like turn-around heroes.

So, there are two dangers in being the first one.

1. You may attract competition and thereby change the path to profits you expected to walk. A risk factor for such disappointment is whether your decision-making is done with company-centric, accounting-based spreadsheets. Recommendation: Consider how your world will look if everyone else follows you. Would you still be glad you made the move?
2. It’s easy to misinterpret the change in your results when your competitors catch up. A risk factor for such misinterpretation is whether your performance assessment looks more at effects than at causes. Recommendation: Consider what external events would have a causal impact on your results. Track those events as well as your numbers.

The best way to avoid those dangers is to think it through before you make your move. The point is not to be first, second, twelfth, or last. The point is to make good decisions.

“It’s an enviable position when you’re the only one.” — Lord Tolloller to Lord Mountararat, lamenting that he’s not the only one in love with the heroine, in Gilbert & Sullivan’s Iolanthe.

Postscript. Hmmm. 14.3% exactly equals 1/7. Could that precise percentage have come from rough estimates? I read in a terrific book, probably one of John Allen Paulos’ Innumeracy series, about the average weight of a species of hummingbird. (I’ve got the story right but perhaps not the specific numbers, so don’t quote me.) The bird’s weight was reported as 113.5 grams, which seemed awfully precise. Except that 113.5 grams exactly equals 4 ounces, and we can easily imagine someone saying “this bird weighs about ¼ pound.”

Kudos to Sharon Begley, Newsweek’s excellent science columnist, for her article “Don’t Just ‘Do Something’” in the June 21, 2010, issue of the magazine.

Tongue in cheek, Ms. Begley begins “Scientists are such spoilsports, always insisting on gathering data on the likely effects of a strategy before implementing it.” She then focuses on the oil spill and the pressure politicians feel (I add: pressure that perhaps they bring on themselves?) to do something even if there’s no evidence that the something-to-be-done will help. The pressure they feel to do something can prevail even if there is evidence that the something will make things worse. Apparently we the people believe that bad action is better than no action at all. If we didn’t believe that, we wouldn’t demand that politicians act. Not unlike, perhaps, demanding that doctors prescribe antibiotics to treat a virus.

Businesspeople feel the same pressure, of course. We feel the pressure to beat targets. We feel the pressure to turn things around. We feel the pressure to make our mark and show we’re in charge. We fall into a do-something trap.

I’ve seen it often in business war games. When given an opportunity to change their strategies, managers usually do so (and almost always do so if they’re taking over from another group of managers). They usually don’t stick with a previous strategy decision, whether or not it seems to be working. And those changed strategies often subtract value. How do I know? By simulating their businesses’ performance with and without the strategy changes.

I’ve seen it also in the tens of millions of strategy simulations I’ve run (using ACS’ patent-pending strategy decision tests) in a pricing tournament on which I’ve previously reported. Nearly 300 strategists have contributed pricing strategies to the tournament. Each person’s overall pricing strategy could include 0, 1, or 2 changes in strategy over time. About 28% of the strategists made 0 changes, 25% made 1 change, and 47% made 2 changes. The 47% making 2 changes performed slightly worse than the other groups. I am not saying that changing strategies necessarily hurts performance. I am saying that merely changing strategies — doing something when we have the chance — doesn’t necessarily help performance.

The question, of course, then becomes how can we know if changing course will help or hurt. That, Ms. Begley says, is what science is for. Where we don’t have established science, I’d add that it’s worth testing a strategy change in a safe environment before you act. And the first do something you should do is to not assume that doing something will make things better.

Further reading: A Dark and Stormy Night, Gross Galactic Product, It’s Working!, The Model Whisperer,  What You Pay For.

Preamble.

This is the third and last in a series of essays about models and competitive strategy.

From All About Models, the first in the series:

• Models describe how we believe the world works.
• Models may be in our heads (mental models) or computer-based.
• Computer-based models are always based on mental models, though they can calculate more and better.
• A model is always involved, though perhaps unconsciously, when we say or compute if we do this we will get that.
• No model is perfect. The test for whether a model is useful is whether we can make a better decision with it than without it.

From What The Model Says, the second in the series:

• What the model “says” is at least as much about its design as about the numbers that come out. For example, a model that knows about costs will give you cost-based advice, nothing more.
• The most important decision we make about our model is to choose how our model should think.
• If your model omits key variables, it will leave you vulnerable to competitors who make use of those variables.
• A key test for whether a model is valid is whether it makes conceptual sense for the problem you want to solve. Many common modeling techniques do not.

Excessively cautious notice. This essay may occasionally sound commercial because in it I describe techniques ACS uses, including one technique proprietary to ACS. The two business war-gaming techniques I’ll describe are practiced by ACS and other companies; each company has its own way of doing things. The third, strategy decision tests, is proprietary to ACS. Related techniques (e.g., Monte Carlo simulation) are not.

End of preamble.

People whisper to horses, dogs, and (according to TV) ghosts. Why not models? The model whisperer — perhaps you, savvy strategist — gently, wisely guides models into shape and helps them achieve fulfillment as oracles of your business’ future.

In our last episode I promised that we’d explore when to use mental and computer-based models in competitive strategy. I further promised that we’d talk about them in the context of three strategy-development techniques: qualitative business war games, quantitative business war games, and strategy decision tests.

In qualitative business war games we experiment with ideas and paradigms embedded in our mental models. In quantitative business war games we use computer-based models to stress-test specific strategy options. In strategy decision tests we run massive what-if simulations to gauge the risks and rewards of going down different paths.

The hypothetical examples I’ll present are actually not hypothetical. Each is an amalgam of multiple war games I’ve conducted in multiple industries, with identifying marks removed to protect confidentiality.

Qualitative business war games

Imagine. You run a business preparing a major change. You’re contemplating something unprecedented, even controversial. Your move will be highly visible and very expensive to retract. If it works, the rewards are great. But there are many interested parties out there, including your competitors, and it’s far from clear which side they’ll take. Their opposition could hurt you badly. Should you make the move?

You’ve thought it through and, although you favor making the move (your mental model at work), you’ve quickly bumped up against the chess-like complexity of your situation. You feel how easy it is to slip into rosy predictions. Your management team is split, some offering passionate support, some passionate opposition (their mental models at work). You and they have debated for some time without reaching a conclusion. Your window of opportunity is closing.

What would you do? Why, you’d run a qualitative business war game, of course.

Business war games involve role-playing. Teams of strategists role-play their business, competitors’ businesses, government regulators, consumers, investors, and so on, as needed for the problem the company wants to solve. Through a series of structured exercises and interactions, the teams construct, debate, select, execute, and react to strategy moves.

In the safe environment of a qualitative war game your strategists get to experience what could happen as they roll out a strategy… or, as a role-played competitor, as they fight against it. A qualitative war game is not unlike a mock trial or a field exercise: it doesn’t ascertain what will happen, but it shows what could happen, with far greater richness than a conference-room discussion. It reveals gaps in your thinking. It surfaces unconscious and unfounded assumptions that could haunt you later.

A qualitative business war game plays out your mental model of what you believe or hope will happen. In addition, it will probably uncover new ideas, favorable and unfavorable. You may choose to explore those new ideas with the quantitative methods I’ll describe later, or by running through another qualitative session. It’s because of those new ideas that I strongly advise you to run at least two “rounds” in a business war game; the second round turns the clock back to the kick-off point of the war game, but with the first round behind them the teams know more than they did before.

By definition, qualitative business war games don’t require computer-based models. They are flexible, time- and cost-efficient, and they can simulate just about anything. They are a great way to generate ideas because the adrenaline rush (I’m not kidding) of simulated competition stimulates creativity. Oh, and they’re fun.

Their drawbacks mirror those of mental models: humans make lousy calculators, even more so under uncertainty, and so it is difficult to get reasonable performance projections in a qualitative war game. Frankly, I don’t even try. I wouldn’t believe the projections.

Quantitative business war games

Imagine. You run a business facing a new competitive threat that’s sure to materialize. You and your colleagues have developed several mutually exclusive countermoves. The countermoves share no common ground and there’s no split-the-difference compromise to be found. Your colleagues advance persuasive arguments in favor of each possible move, and promise future glory. But how can that be, when the moves are so different? How can you know what to expect? What if competitors counter your countermove?

You ponder and reject the usual tools. Financial spreadsheet: doesn’t take competitors into account. (Notice the mental model in those spreadsheets, and refer to our discussion of Model V and Model M in What The Model Says.) Forecast: history won’t be a good guide because the new competitive threat changes too much. (Notice the mental model there too.) SWOT analysis: just a way to express your mental models using four categories.

So you turn to a quantitative business war game. You use a qualitative game when you’re looking for ideas and surprises; you use a quantitative game when you’re looking for analysis and choices. A quantitative war game uses teams and role-playing like the qualitative variety, but it adds (as you might expect) a computer-based model. The model, calibrated for your competitive environment, takes as input the strategy decisions made by the teams and provides as output estimates of sales, market share, profits, and so on.

Sidebar. Tempting and fascinating as it is, I’m not going to discuss the process of building such models in this essay. Contact me at info@whatifyourstrategy.com if you’d like to talk about it. I’ll just say 1) it’s possible to build such models, 2) it’s not as hard as it might sound, and 3) such models easily satisfy the usefulness criterion, to wit, whether you can make a better decision with the model than without it. See, for example, the Shell case at the start of Putting the Lesson before the Test from Wharton on Dynamic Competitive Strategy (Day and Reibstein, editors, 1997). That model projected a nine-figure loss if Shell were to execute a certain strategy. End of sidebar.

In qualitative war games participants are surprised by the actions teams take. In quantitative war games participants are surprised by the outcomes of those actions. Surprises are good, of course, if they’re in the safe environment of a war game.

Here’s a tip-of-the-iceberg illustration that shows the dynamic of a quantitative war game. Teams develop strategies. They all expect to perform well. Unless it begins with a near-monopoly, no team expects to lose market share. Yet each and every market in the world always, always contains exactly 100% market share, which means that for one business to gain share someone must lose share. Without a model, the teams argue endlessly about who gains and who loses. With a model, teams see who gains and who loses, and why. It makes the discussion much more fruitful.

By definition, quantitative business war games require computer-based models. They generate ideas, as do qualitative games, but their key benefit is that they let you evaluate and contrast your strategy options rigorously and objectively. They can produce consensus and action quickly because participants see the results of option A versus option B. (I’ve seen Fortune 500 companies change course overnight.) They help set reasonable targets and expectations. Oh, and they’re fun.

Their drawbacks mirror those of computer-based models: more time and cost to set up, and the model won’t be able to simulate actions not anticipated in the design phase. More subtly, some models are simply better-designed than others, so you must choose carefully.

Strategy decision tests

Imagine. You direct pricing for a business in a highly competitive market. You and your competitors watch each other’s prices like genetically modified hawks. You and they can change prices frequently. How should you set your prices?

This situation is rife with mental models and computer-based models. Some pricing strategists — your competitors? — look at prices periodically and then make a decision what to do. Those decisions, of course, come from some invisible and visceral combination of their mental models and any analysis they see. Other pricing strategists — your competitors? — may use computer-based models that figure out “optimal” pricing based on, for example, demand curves or the status of supply and demand at this microsecond.

So how should you set your prices? This much is clear: if you go too low you may trigger a price war or leave money on the table, and if you go too high you may lose share and long-time customers. If you try to duplicate others’ apparently successful moves you may make matters worse, sort of like grabbing someone else in midair instead of opening your own parachute. And of course prices are relative, and of course your prices affect their prices, and vice versa. So how should you set your prices?

You set up a strategy decision test.

Sidebar. Or perhaps you use other tools, of which there are at least several. As my focus in this essay is models and decision-making, not a pricing-technique critique, I won’t delve into methods other than strategy decision tests. I’ll just reiterate that all computer-based models are based on mental models, and it behooves the model consumer to ensure that a model is conceptually sensible and appropriate for the intended use. End of sidebar.

A strategy decision test is a massive what-if simulation. It’s useful when there are many options for you and/or your competitors and you want to understand the risks and rewards of your options. Each of your options may face many thousands of possible outcomes, and evaluating all your options may mean looking at many millions of futures. (See also The How-Likely Case.) That’s too many to do in your head. It’s also too many to do in a spreadsheet. But it’s not too many for a strategy decision test.

Strategy decision tests look at all the possibilities and summarize the results. They don’t involve teams or role-playing. They do, however, thrive on ideas for strategy options, specifically including ideas generated in war games or by brainstorming.

Obviously, strategy decision tests use computer-based models, and all my usual imprecations about model quality and sensibility apply.

One of the benefits of strategy decision tests is its comprehensiveness, which lets it show a clear view of risk; that is, of the range of what could happen. That comprehensiveness would have been very difficult not many years ago, and simply impossible not many years before that.

Another, quite intriguing, is what Malcolm Gladwell called “serendipity” in his article about drug companies screening millions of compounds to see if they might have useful medicinal effects. (“The Treatment,” The New Yorker, May 17, 2010.) “[Screening] provided a chance of stumbling across something by accident — something so novel and unexpected that no scientist would have dreamed it up. It provided for serendipity, and the history of drug discovery is full of stories of serendipity.” He cites penicillin and Viagra. “What he [a cancer researcher] found was exactly what he’d hoped for when he started his hunt: something he could never have imagined on his own.”

That serendipity happens in strategy decision tests. It’s not only that I’ve seen strategy decision tests find strategies that beat the best I could develop. I’ve also seen strategies pop out that I could never have imagined on my own.

The drawbacks of strategy decision tests are two. First, developing a strategy decision test takes somewhat longer than developing other kinds of computer-based models. Second, the analysis may appear like a black box, not because anything is hidden but because the scope is too big to be eyeballed. Oh, and maybe a third. The results may be fascinating, insightful, and even jaw-dropping — I’ve felt mine fall a few times — but it’s not as entertaining as a business war game.

The n rules for model whispering

My goodness, we made it! Thank you, friendly reader, for staying with me to the end (almost) of this series of long essays.

We end with a few rules to remember as you whisper to your models and make your competitive-strategy decisions.

There is always a model. Computer-based or mental; Model V or Model M or Model XYZ; big or small. You never decide whether to use a model. You always decide which model to use. Don’t expect a financial spreadsheet to give you advice about marketing strategy. Corollary: choose a model on the basis of sensibility for the decision you need to make.

Computer-based models are people too, sort of. Computers only know what people have fed them and they only think what people have taught them. It’s not human versus computer. It’s, again, which model you want to use, and is that model better applied in your head or in a computer. Corollary: don’t try to do too much arithmetic in your head.

Models can stimulate creativity. People get results from models — their mental models, others’ mental models, computer-based models — and they create ideas for how to do better. (War games are a prime example.) They even get ideas by designing models, as they become attuned to the wide range of variables under their control. Corollary: you can build your creativity by asking yourself “how would I model that?” when you come across an interesting strategy. That activity thrills me and makes me endlessly fascinating at parties. I know that because people can hardly wait to wander off and think about it on their own.

Surprise is good. If you can always predict what a model will say, the model doesn’t add any value. The model adds value when it tells you something you didn’t already know. Corollary: if you (i.e., your mental model) disagree with another model, take the opportunity to learn whether to update your mental model or to jettison the other model.

There is no such thing as precision or accuracy about the future. There are many, many, many possible futures. No practical model can capture every relevant variable and tell you which one will happen. (Put another way: a perfect model of the next five years would take five years to run.) Depending on the technology we employ, you can get a good / better / best view of those futures, but no one can guarantee a correct view. Incidentally, there is also no such thing as data about the future, but that’s only one reason why there’s no such thing as precision or accuracy about the future. Corollary: the objective is to improve the odds of making good strategy decisions.

Finally: ask what-if, and imagine. “Whenever you see a successful business, someone once made a courageous decision.” — Peter Drucker

Further Reading

About business war games in general
Business War Games
Learning Faster Than The Competition
The Seven Deadly Sins of Business War Games

About quantitative business war games and methods
Honey, We Shrunk The Industry
Honey, We Shrunk The Industry Again
Precision In, Garbage Out

About strategy decision tests
Predicting Competitors
Strategy decision tests
When I Was Wrong

Preamble. This essay is the sequel to All About Models. In that essay we covered several points:

• Models describe how we believe the world works.
• Models may be in our heads (mental models) or computer-based.
• Computer-based models are always based on mental models, though they can calculate more and better.
• A model is always involved, though perhaps unconsciously, when we say or compute if we do this we will get that.
• No model is perfect. The test for whether a model is useful is whether we can make a better decision with it than without it.

You don’t have to read that essay before this one, but it wouldn’t hurt. End of preamble.

So you’ve got a model, whether mental or computer-based. Or several models, put forth by several people or several computers. Congratulations! Your brainstorming, calculating, visioning, calibrating, and/or programming has paid off. Now you’re ready to get value and insight.

So, what do you do next? How do you get that value and insight?

Not so fast. It’s not so easy. In fact, you may be shocked a few paragraphs from now.

Before we listen to what a model says, we must figure out whether it speaks well-founded wisdom or well-intentioned folly.

Model V versus Model M

In All About Models I posited two models for a large manufacturing business. Both models acknowledged that our business has high fixed costs and that we must cover our costs to be profitable. They diverged from that point.

• One model said costs per unit go down as we produce and sell more units, and that selling more units also brings in additional revenue. We’ll call that Model V, for volume.
• The other model said it’s easier to cover our costs if our prices are high, and that excellent quality and a strong brand give us pricing power. We’ll call that one Model M, for margin.

Model V and Model M could be implemented as computer-based models or remain as mental models. It doesn’t matter right now.

For Model V, “what the model says” is to increase volume. For Model M, “what the model says” is to increase margin. Notice:

By listening to what the model “says,” we listen to the model’s design
at least as much as we listen to the numbers sifting through
our mental or silicon calculators.

We made the most important decision of all when we built the model: we chose which way, volume or margin, is the correct way to think about (i.e., to model) our large manufacturing business. Any subsequent analysis and action issues from that chosen paradigm. If we’ve chosen Model V, for instance, and if our profits underperform, we will look for other ways to boost volume. Why volume? Because the model we adopted, which codified the way we thought, shows us that volume drives profits.

The power to speak

What we hear from the model depends on which model we’ve given the power to speak. Choosing the wrong model messes up a lot, perhaps even everything. (See also Predictable Competitors and Predicting Competitors.)

In this example, neither Model V nor Model M is right. They’re not right because each misses something important, namely, what’s in the other model!

And that’s not all. For example, neither takes competitors’ actions and reactions into account. Sure, your unit sales may go up if you cut your price. But what if competitors cut their prices too? Okay, perhaps total demand will grow. Will it grow enough to compensate for the lower prices? Maybe yes, maybe no. The point is that we won’t even get to that question if our model ignores competitive dynamics.

The situation isn’t necessarily quite as dire as it sounds. We can take dubious, indulgent comfort in the notion that our competitors may make the same mistakes we do. If a whole industry adopts a way of thinking (and many industries do), no single business suffers a competitive disadvantage due to choosing the wrong model. On the other hand, a business could potentially enjoy the benefits of a different model if it can find one. That’s what those upstarts coming out of left field do. That’s what we call “game changing” action. Changing the game means changing our models and starts by changing our minds.

This discussion of Models V and M is highly and vitally practical. But rather than lengthen a series of long essays by reciting my evidence, I’ll just provide a link to real-life war stories. (Look about halfway down that post.) See also the story of Shell that begins Putting the Lesson Before the Test, a chapter from Wharton on Dynamic Competitive Strategy (Day and Reibstein, editors, 1997).

Does it make sense?

There is always a model when we estimate, assert, or calculate that this action will lead to that result.

The challenge in developing an effective strategy begins long before we decide what actions to take and how to execute them. The challenge begins when we decide what model to use. By selecting a model we frame the way we see and evaluate our strategy options.

Prudent strategists want to make sure a model is valid before they select it and thereby entrust their businesses and their careers to its advice. Based on what we’ve discussed about Models V and M, I suggest that “valid” refers first and foremost to whether a model makes sense.

For example:

• A model that looks only at price does not make sense if customers perceive differences in product or service quality.
• A model that calculates changes in market share does not make sense if it ignores customer loyalty, inertia, and switching costs.
• A model that accounts only for our business’ actions does not make sense if we have competitors.
• A model that extrapolates the past into the future does not make sense if we expect (or want) the future to look different from the past.

And so on. Those statements reflect what ACS calls principles of competition.

Three things to notice about that brief “a model that” list. First, none of those statements is particularly controversial. Second, none of those statements is particularly complicated or arcane. Third, according to those statements many of the models commonly used in strategy development violate the does-it-make-sense criterion.

Sensible models are difficult to handle in our heads, partly because of the arithmetic and partly because of the number of we do this, then that happens, then that, then that, then… connections. That’s why I favor computer-based models for many applications. There are times, though, when computer-based models are overkill or even a distraction.

In my next and final essay in this series, The Model Whisperer, we’ll explore when to use mental and computer-based models. We’ll apply those models in qualitative business war games, quantitative business war games, and strategy decision tests (sometimes called decision tournaments).

This essay is full of good stuff about models. Of course that depends on how you define good stuff and models. If you mean something involving delectable (your definition here too) folk lurching down a runway with sour expressions stitched onto their faces, well, sorry, I’m not that kind of manager. But if “good stuff about models” to you means cool and insightful methods to improve decision-making, as it does to me during working hours, then you’ve clicked to the right place.

The fascinating world of models and decision-making

All of us live with models. More accurately, models live within all of us. They help us every day.

As if that weren’t enough, there are models outside us too, and they help us too.

In general, we can classify models chemically: those based on carbon and those based on silicon. The former, mental models, operate inside our heads. The latter, computer-based models, operate inside computers. They behave differently.

Mental models

Mental models are the things in our heads that say if you do this then the result will be that. They reflect knowledge and/or beliefs about how the world works.

Our ability to, say, throw physical objects to a specific target without consciously solving equations comes from mental models. If asked, we could enumerate the key variables involved — the force we apply, the distance it must travel, wind resistance — but we don’t think about them when we throw an object.

Those mental models work well, and without controversy, because thrown objects precisely obey the laws of physics for everyone, every time, so far. But not everything covered by mental models is so consistent. Humans’ mental models can differ dramatically and contentiously where our experience is more varied, complex, or ambiguous. As with competitive strategy.

Imagine you and I run a large manufacturing business. A mental model for what makes it tick may go like this:

• We have heavy fixed costs.
• To be profitable, we must cover those fixed costs and all other costs.
• Fixed costs go down per-unit if we sell more units.
• Selling more units also brings in more revenue.

I’m not saying that mental model is right or wrong. I’m saying only that it is an example of a mental model.

That mental model reflects a certain list of variables (akin to those for throwing physical objects) and it leads its owner to think in a particular way. It focuses on internal operations (our costs), it recognizes the effects of volume (keeping the factory full reduces costs per unit), and it would favor moves such as price cuts or mass marketing because, given the list of variables, those actions would promise to build volume.

I’ve encountered exactly that mental model in the business war games I’ve conducted, and I’m sure you’ve heard it too. It’s common and it at least appears to make sense.

Think again about our large manufacturing business. What would be a different mental model? How about this:

• We have heavy fixed costs.
• To be profitable, we must cover those fixed costs and all other costs.
• It’s easier to cover our costs if our prices are high.
• Excellent quality and a strong brand give us pricing power.

I’m not judging that mental model either. Notice, though, that it reflects a different list of variables and it leads its owner to think in a different particular way. It focuses on customer perceptions (quality and brand), it recognizes the causes of margin (profit comes from the difference between prices and costs), and it would favor moves such as price increases or product differentiation because, given the list of variables,  those actions would promise to build margin.

I’ve encountered that mental model as well, and I’m sure you have too. It’s common and it at least appears to make sense.

Our large manufacturing business is hurting. What should we do? Your mental model says cut price, my mental model says don’t. We battle, we spreadsheet, we PowerPoint. We argue passionately about why a price cut is a good idea or a bad idea, we selectively cite anecdotes and trends and studies. Oddly, we spend less time asking how we can end at different conclusions when we start with the same data; that is, asking how our mental models differ. (See also Predicting Competitors.)

The point is not that any given mental model is right or wrong. (That said, we all know that no one is always right.) The point isn’t even that we must eventually choose one mental model or another. The point is that we all have mental models, we use them all the time, they’re unconscious to us and invisible to others, they appear self-evidently true to their owners, and they differ from person to person.

Mental models have benefits.

• They’re as flexible and creative as the human brain.
• They accommodate new information in real time. We can revise or update them quickly; we just change our minds.
• We always have them with us, they’re always ready to go, and we don’t need help from IT to operate them.
• Mental-model conflicts, annoying as they may be, help us learn. The wisdom of crowds. The marketplace of ideas. Very beneficial in qualitative business war games.

Mental models have drawbacks. For example, they require that we handle numbers in our heads. Sure, we can handle the math behind accurately throwing objects, but try this much simpler arithmetic: what’s 1,248 x 3,579 – 9,876 + 5,555? You’re right, it’s 4,362,271. I was just checking to be sure you knew too. How about this: if our sales are $1,000 today and we expect 3.15% compound annual growth, what will our sales be after 63 months? And we haven’t even gotten into any interesting problems yet. How about our profits after 5 years, assuming that brand popularity rises 9% and fixed costs are… well, you get the point.

It’s not just about handling numbers; it’s also about understanding ripple effects. We cut our price and expect volume to grow, but our competitors want volume too and they feel threatened and so they cut their price in response, so now we’re still at parity but at lower prices, so we cut our price again or cut our costs, and then, and then, and then. Unintended consequences come from unanticipated effects. Scenario-planning and role-playing programs, like business war games, help reveal those effects and consequences. (See also Do Not Overtighten.)

For an amusing and related diversion — and we probably all need one right about now — you might enjoy the lyrics to Tom Lehrer’s classic satire New Math.

Computer-Based Models

Remember that our mental models tell us if you do this then the result will be that. Say you tell about computer about “this” and ask it what “that” will be. Sorry; it will just sit there in stolid silicon silence. Before the computer can tell you about “that,” you have to tell it how. Your computer thinks as you tell it to. Your computer thinks like you.

All computer-based models start in life as mental models.

If you believe you should forecast your sales by extrapolating trend lines into the future, you or your Excelophilic proxy will insert trend-line-extrapolation equations. If you believe you  should forecast sales with a statistical model, you will commission a statistical model. If you ask someone else to figure it out, your computer will think like the person to whom you delegated thinking. And if you believe your mental model is good enough, you won’t even turn on your computer. Let it sit there.

Computer-based models think neither better nor worse than humans. That’s because they think like humans. Their benefits are in clarity, speed, precision, and scope.

• They’re explicit and visible. People can see, discuss, enhance, and ultimately share the way of thinking embodied in the model.
• They compute far, far faster than humans can. That’s more than a convenience. It means also that it’s practical to conduct serious what-if tests with many scenarios. A simulator I wrote calculates about 20,000 scenarios per second, which is handy when I ask it to process millions. (See Millions of Pricing Simulations.)
• They calculate much more accurately than humans can. (See also Precision In, Garbage Out.) Oh, by the way: the arithmetic we covered earlier? The real answer is 4,462,271. The 4,362,271 I mentioned was a sly test. You didn’t know it was wrong? That’s the point.
• They can keep track of many more variables than humans can. A strategy simulator I created for a company could handle roughly a thousand decisions for them and their competitors in multiple countries. Not a job for a mental model. Very beneficial in quantitative business war games and strategy analysis.

Computer-based models also provide a different kind of benefit. Yes, they serve as mental models mated with deft calculators. They also let strategists compare mental models. Back to our large manufacturing business, the one where you and I have been arguing whether to cut price. Use a computer-based model to run it both ways. And maybe some other ways too: cut capacity, broaden the product line, lead prices up, and more.

In other words, computer-based models allow us to run what-if tests. They can do so in quantitative business war games or even in the massive explorations ACS calls decision tournaments. (See also When I Was Wrong.)

Computer-based models sometimes are like rocket science, but their drawbacks are simple. They take time, money, and skill to create. And, of course, they need mental models for their raw material, which makes them vulnerable to lousy mental model in, garbage out. Note that the garbage-out would be the fault of the lousy mental model in, not the computer.

Is the model valid?

I think we’ve established that strategists always use models, whether mental or computer-based. Sooner or later someone asks how you know your model is valid. I think that mostly they want to know about numerical validity and accuracy; does the model work, has it been tested. That’s a good question we’ve all been trained to ask. We’ll talk about that next. However, I think it’s even more important to ask about conceptual validity; in effect, should the model work, does it make sense. We’ll talk about that in the part about rephrasing the question, just a few terrific paragraphs from now.

Actually, people ask the validity question only with computer-based models. Mental models are invisible, don’t feel like models, and are self-evidently correct to their owners. People may contest other people’s conclusions, but, as we said a while back, people rarely ask why they reach different conclusions from the same data.

When people ask if a model is valid they usually mean to ask whether it fits known data. The validity question is fair, but fitting known data is often not a useful test.

It’s always possible to put together a model that fits known data. If you expect the future to look just like the past, then such a model might even be useful. (Assuming it didn’t violate good model-building hygiene, such as using up too many degrees of freedom. See also Predictable Competitors.) But it’s not especially interesting to model a future that will look just like the past. The time you really need a good model is when the future will not look like the past. Unfortunately, there are no data about the future, so the does-it-fit-the-data test isn’t available. (See also More Internet Users than People.)

I recommend rephrasing  the question to shift from numerical validity to conceptual validity. Rather than ask if a model is accurate, ask if it is sensible. Ask how it works, ask what it takes into account, ask on what paradigm it rests. Remember that making strategy decisions is not about accounting, trend lines, or forecasting. It is about strategy, and the models you select should work with strategy concepts. (See also Pundits and Stress and With All This Intelligence, Why Don’t We Have Better Strategies?)

The bottom line

The ultimate point about validating models — and about selecting models and especially about using models — is this. You are going to make decisions no matter what. The relevant question is whether you can make a better decision with the model than without it.

No model is perfect and there are no guarantees of fabulous future performance. Improving the odds of success is all you can hope to get by using models. Fortunately, it is a lot to get. It’s the difference between the gambler and the casino.

This essay talks about models themselves. The next in the series, What The Model Says, discusses which models are worth listening to. The final essay, The Model Whisperer, connects mental and computer-based models to techniques for developing competitive strategy.

“Everything is 50/50. Either it will happen or it won’t.” — Unknown

Strategists commonly analyze best-case, worst-case, and most-likely scenarios before making a significant strategy decision. That covers about 0.000007618% of the possibilities. I’m not making up that number.

Here we’ll talk about why there are so many scenarios, why the “most-likely” scenario is hardly likely at all, and why we won’t get better predictions by getting better precision. We’ll end with some ways to make better decisions anyway.

So many scenarios

I ran a business war game with a company in which we took into account the potential actions of several organizations: their business, key competitors, government agencies, and others with vested interests in the industry’s evolution. Teams of executives role-played the organizations in accelerated real time. I designed the war game to… well, we won’t go into that.

Before the war game began I asked each team to spend 15 minutes listing the potential actions they could take. The shortest list had 8. The longest had 17. All told, there were 39,382,200 possible combinations of potential actions. If we were to look at three scenarios — best, worst, most-likely — we would be looking at 0.000007618% of the possibilities. (I told you I didn’t make up that number.) That is, assuming we could actually identify which were best, worst, and most-likely.

To put it another way, hoping that those three scenarios would tell us something meaningful about the competitive landscape would be like clipping a tiny part of a single letter in this essay and using it to infer what the essay was about.

Sidebar. The purpose of that war game wasn’t to evaluate their best, worst, most-likely (BWML) scenarios. I brought up the war game because I have enough information to calculate the number of scenarios and because the company is hardly unique in the number of possible futures it faced, which tells us that a conventional BWML analysis generally looks at a minuscule sample of what could happen. Even if a BWML analysis could assess a scenario with perfect accuracy, the odds that it would perform its perfection on the right three scenarios are vanishingly small. End of sidebar.

Of course the 39,382,200 possible scenarios were not equally likely. If a government agency issues a Thou Shalt regulation, it would presumably be much more likely that businesses would obey the rule than defy it. The combination of actions in some scenarios might be silly or even impossible. But still, however we cut it we’ve got 1) a large number of scenarios and 2) little confidence that the BWML scenarios will give us a representative picture.

Is there any hope? Let’s see how far we get with heroic assumptions. Let’s say we could:

• Efficiently and correctly eliminate 99.9% of those scenarios as being impossible, silly, redundant, or trivial. That’s probably too optimistic but we’re just thought-experimenting here. 
• And then somehow view the remaining 39,382 scenarios. It’d take roughly 600 pages if we choose to print them.
• And then efficiently and correctly spot the most-likely one.
• And then achieve consensus that we’ve spotted the right one.

So, we’ve chosen the most-likely scenario. How likely would it be? Remember, we’ve already eliminated impossibilities, redundancies, and trivialities. Would it really be likely enough for us to call it “most” likely in any meaningful sense?

Precision not to the rescue

Precision doesn’t help much. That’s because scenarios can differ not only in degree but also in kind.

A business might change its price by +2.0%, +2.5%, or +3.0%. Those are differences in degree. A business might change its price, exit the market, launch a new product, merges with a competitor, vertically integrate, and more. Those are differences in kind.

It is reasonable to assume that the outcome of its 2.5% price increase would be somewhere between the outcome of the 2.0% increase and the 3.0% increase. It might not always work out that way but it’s a defensible shortcut.

It is not reasonable to assume that the outcome of merging with a competitor is somewhere between the outcome of launching a new product and vertically integrating.

The differences-in-kind issue means we don’t get to reduce the conceptual or computational load by looking at a few representative scenarios. (Representative of what?) It means too that we won’t know which scenarios are “best” or “worst” without evaluating them all. (Which isn’t impossible. That’s what ACS strategy decision tests do. Evaluating 39,382,200 could take as little as 30 minutes.) And that’s why we won’t solve the strategy-decision problem with a BWML analysis even if we deploy a microscope through which to peer more precisely at those three scenarios.

We don’t need a microscope to dissect a few scenarios. We need a wide-angle lens to explore a lot of scenarios.

Preparing versus predicting

Predicting the future is tremendously difficult partly because of the number-of-scenarios and differences-in-kind problems. It’s difficult also because it is hard to know what other organizations will do. They themselves may not know yet. They may be trying to make decisions at the same time you are. They may even be waiting and seeing what you do.

Still, you must make decisions and those decisions will affect your future. Fortunately, preparing can help you make decisions with much better odds of success, and fortunately too, preparing is easier than predicting.

In a future essay I’ll discuss how cool analysis can help you prepare. For now we’ll end with some shortcuts that don’t require any technology more advanced than thinking caps.

Scope out the terrain. You and I might not be able to assess 39,382 scenarios in our heads — actually, there’s no “might” about it — but merely knowing something about the possibilities is useful. In the war game it was strategically stimulating to know how many options each organization had. That knowledge prevented the executives from falling into overconfidence and tunnel-vision traps.

Apply competitive intelligence. CI can hint at or even reveal which actions the other organizations might adopt. If you can eliminate a few of the potential options, you can focus your attention on those that are left.

Keep asking what if. You are considering actions A, B, and C. How would each be affected if other organizations do X, Y, and Z? Your new product might not be materially affected if another company vertically integrates, but it might be affected a great deal if that company merges with a competitor. Are some of your options less vulnerable than others to what other organizations might do? I’ve seen exactly that, many times: an initially preferred strategy was discarded when a simulation showed it was highly vulnerable to competitive response.

Remember that your actions affect theirs. As you contemplate your moves, ask yourself not only what wonderful results you hope to achieve but also how your moves might influence others’ actions. You might not get those wonderful results if you provoke them to respond. Nobody intends to start a price war but price wars get started. On the happier hand, well-chosen action may lead them to behave as you want.

Talk it through. Say you want to do X. What happens after you do it? (“What happens” is about them as well as about you.) What happens after that? After that? After that? After that? Business war games can do a great job here. Rigorous, contrarian-rich conversation can too.

Notice that those approaches start by broadening the range of what you think about; in other words, they consider multiple scenarios. Contrast that with techniques that quickly and/or invisibly encourage you to narrow your field of vision. Best, worst, most-likely analysis does the latter: it defines the future in terms of three outcomes in which many factors — such as your competitors’ actions — are necessarily assumed to be known or irrelevant.

Look through your wide-angle lens first, the microscope later.

“The purpose of writing is to inflate weak ideas, obscure pure reasoning, and inhibit clarity. With a little practice, writing can be an intimidating and impenetrable fog!” — Bill Watterson (1958-), author of the comic strip “Calvin & Hobbes”

Based on a recent discussion on the Business Strategy & Competitive Strategy Forum of LinkedIn.com, many strategists must fight their way through intimidating and impenetrable fogs of strategy.

One brave warrior, weary of the fog, asked this clear, straightforward, and provocative question:

Where is the evidence that a clear strategy makes a company more likely to succeed?

Many opinions were offered. Rather than summarize and paraphrase others’, I will share mine. I’ve gently edited what I originally wrote to include stuff I should have included before and to make me look smarter now than I did then. Here we go.

That brave question is stunningly difficult to answer.

We can quickly dispense with any suggestion that clarity is enough by itself. A clear-but-lousy strategy is still a lousy strategy. Clarity might help such a strategy perform better than lack of clarity, and it might not, but even if it helps, that’s small comfort.

I worked for 15 years at The PIMS Program (Profit Impact of Market Strategy), created by the brilliant Sidney Schoeffler. SPI had a multi-year database of thousands of real-life businesses contributed by hundreds of businesses around the world. The PIMS Program was perhaps the largest, most comprehensive effort to learn what’s different between businesses that perform well and businesses that perform badly. The research succeeded. However, whether those differentiating factors were the result of clarity, strategy decisions by the business, strategy decisions by competitors, or just plain serendipity, could not be known even with that monumental effort (note 1).

I did research on the PIMS database with Prof. Robert D. Buzzell of Harvard Business School on how much of their performance-potential businesses actually achieved. The short answer: not much (note 2).  Even strategies that seem to succeed often performed worse than they could have. Our research on how much success is possible reminds us that the strategy-clarity issue begs the question of how to gauge success. Performing better than competitors? Better than targets? Better than last year? A high percentage of potential performance? A rising stock price?

Let’s not stop with clarifying success. Let’s also clarify clarity. What do we have when we have a “clear” strategy? Agreement on each person’s role in operating a strategy? Step-by-step instructions? Does “clarity” mean execution, in the sense that it’s difficult to execute an unclear strategy well?

This is getting complicated. But wait, there’s more! And don’t worry. I see, of all things, a useful bumper sticker in our future.

Back to the idea of a strategy that succeeds. My recent work on strategy analysis, using decision tournaments or decision tests, makes clear something that’s intuitively obvious though often ignored: our performance is only partly under our control. Competitors’ strategies are relevant and can be more impactful than our own. What Netscape did made a difference to Netscape, but what Microsoft did made a bigger difference to Netscape. By most measures Netscape did not succeed, but what exactly was its failure? (note 3)

I’ve conducted many business war games with companies around the world over the last 18 years. Over and over I’ve seen seasoned strategists surprise themselves with the ideas they come up with, and when they have executed those strategies they have done well. This is hopeful for the clarity question because in war games strategists test and live through their strategy decisions, leading to an unusual level of clarity and consensus. Still, cases and “experience” are anecdotal; they’re not proof that clarity matters. Plus we have a survivor-bias problem: companies with clear strategies that failed aren’t around to tell us their stories.

Which brings us full circle to why the question is so difficult to answer. It’s difficult to satisfy a request for evidence because it is extraordinarily hard to set up an evidence-based test. We have good hints (PIMS), we have simulations (decision tournaments), we have anecdotes (business war games and other stories). We also have questions about the nature of success and clarity. But as far as I know, we don’t have evidence that links clarity to success.

The bottom line: A bumper sticker

I’ve found that when one question is too hard to answer, it’s helpful to ask a different question. Remember the bumper sticker that said “If you think education is expensive, try ignorance”? In this case, the bumper sticker would be this: “If you’re not sure it’s important to have a clear strategy, try an unclear strategy.”

So, here’s the best answer I can offer. Clarity is good, but as an effect, not a cause. To me, the well-intentioned desire to cause clarity by issuing marching orders can lead unintentionally to rigidity. Clarity as an effect of rigorous strategic thinking, stress-testing, war-gaming, and the like, is more successful. I don’t have data to prove it, but at least that interpretation is consistent with my experience, analysis, and research.

If you’re not sure it’s important to have a clear strategy, try an unclear strategy. Assuming, of course, that it’s a non-lousy strategy.

Notes

1) That’s not a flaw in the design of the PIMS database or the research conducted on it. If you think about what would be required to tease apart the necessary variables — and we thought about it, a lot — you’ll probably come to the same conclusion we did, which is that it’s extremely hard to do in any practical way. Moreover, knowing why strategists did something was not the program’s objective. The objective was to know the consequences of what strategists did regardless of why they did it.

2) The methodology is too complex to describe here but it’s available in a Sloan Management Review article that Prof. Buzzell and I wrote, “Managing for Tomorrow.” See Buzzell, R.D. and Chussil, M.J. (1985) “Managing for Tomorrow,” Sloan Management Review, Vol. 26, No. 4, pp.3–14. That article also appears as a chapter in The PIMS Principles by Buzzell and Bradley T. Gale.

3) One could argue that Netscape should not have tried in the first place. On the other hand, some people did make money. AOL bought Netscape in 1998 for stock valued at $4.2 billion. (AOL received $750 million from Microsoft in 2003 as a settlement in an antitrust lawsuit.) Then again, other people lost money. Netscape was disbanded in 2003.

This brief essay highlights key themes from my Numbers Gone Wild workshop at the 2010 SCIP International Annual Conference on March 11, 2010. It does not replay the workshop, as the workshop was energetically interactive, verging at times on raucous; well, as raucous as competitive-intelligence professionals get. Not to mention that the group enjoyed numerous punch lines and shockers, and I wouldn’t want to spoil the delights and surprises if someday you experience the workshop yourself. Rather, this essay focuses on concepts and conclusions, which will shock and delight better than would a play-by-play recap.

Introduction: Crazy

Having numbers drives us crazy.

Lacking numbers drives us crazy.

Seeking precision drives us crazy.

Living with imprecision drives us crazy.

In short, we are going crazy.

Numbers surround us. Even people who prefer qualitative methods (“quals”) use numbers every day.

Numbers bug us. Even people who like quantitative methods (“quants”) criticize numbers every day.

In this workshop prudent people explored the unintended ways in which numbers going wild cause us to suffer. We suffer as a result of the bad decisions, and the collateral career damage, that can come from untamed numbers.

How to Like Numbers

I like most numbers. When I say “most” I don’t mean that I get along with all the digits except 7 and 8. I mean that numbers are literally the only way that we can gain certain insights, knowledge, and even inspiration, and that makes them likable.

Of course, the presence of numbers does not guarantee insight, knowledge, or inspiration. On the other hand, the absence of numbers limits us to anecdotes, impressions, and hypotheses. So, we want numbers.

I like most numbers. I don’t like most factoids. Here’s an example: “GM’s market share went down at the same time the reliability of its cars went up.” That statement is true and over many years it was measured precisely in cars sold and defects per hundred vehicles, but it is not likable. Why not? Here’s one of several reasons: because it is context-free. Context would tell us what else was happening while GM improved reliability and lost market share. Context would reveal that many more-reliable Japanese cars had entered the market and the other American producers were improving reliability too. GM almost certainly would have lost more market share if it hadn’t improved reliability.

Our unlikable factoid illustrates how we can decide which numbers to like and which not. The relevant criterion for likability is not whether a number is “actionable.” After all, the factoid is actionable: advise GM management to regain market share by making their cars less reliable. The relevant criterion is whether a number is sensible.

“Sensible” needs a big tent. It includes context, robustness, and quality of analysis. It does not imply or require approval. You might not like a number, but numbers are not popularity contests. (Which is amusingly ironic, if one is easily amused, because popularity contests are judged with numbers.)

The big tent of sensibility also does not imply or require precision. It doesn’t matter a whole lot whether GM’s market share declined 11.2% or 11.3%. No one would breathe a sigh of relief to find the loss was the former and not the latter. Moreover, precision is not even an option for many numbers, such as all numbers about the future.

Notice that the sensibility criterion separates the wildness of a number from what we do with that number (e.g., approve of it or take action on it). That is a good thing. When we humans get wrapped up in strategy debates and decision-making, we confound the goodness of a number (its sensibility) with our approval of the number (its value in getting our idea adopted). We slip into judging the goodness of a number by the support it lends to our idea, rather than judging the goodness of our idea by the support it gains from a number. It is to avoid confounding sensibility and approval that I design the structure of my strategy-simulation models before calibrating the models with data, and that I calibrate the models before simulating strategies with them.

Models in Our Heads!

Think about your business. What would happen to its profits if its prices were to rise 5%?

To answer that question you will execute a series of unconscious cerebral calculations to forecast the net results of many interconnected moving parts. How do I know you will do that? Because you will have an answer to my what-would-happen question, meaning you will promptly predict how prices will prod profits.

Let’s think about that. To promptly predict how prices will prod profits you will assess:

• Competitors’ reactions to your price increase. Will they follow you up or will they tout their new price advantage? How quickly will they move, how far will they go?
• Customers’ reactions to the price increase. What will happen to your business’ sales in the short term? In the long term? Will customers expect a better product for the higher price? Can you market the increase in a soothing way? Will you try to hide the increase via smaller packages or hidden fees? Will customer loyalty dampen or conceal the effects?
• Given the changes (if any) in your business’ volume, what happens to costs? Are fixed costs fully covered? What are the implications for contracts with labor, distributors, suppliers, and shipping? What about sales support and customer service?
• And more.

It would be extraordinarily difficult for you to take all of those factors into account in your head, not to mention doing the arithmetic. And yet you will proffer a prompt prediction to the what-would-happen question. As would most business professionals. You and I hear it all the time; just listen to debates about what works and what to do.

That prediction comes from what’s called a mental model. It’s just like a computer model, except it happens in one’s head and it operates with a lower degree of computational accuracy.

A lower degree of computational accuracy? Yes. Try this. You sell a product for $129 per unit. You sold 51,500 last quarter. The market is growing at a compound annual rate of 3.8%. What will be your revenue next quarter? Write down your answer. That, by the way, is a much simpler question than the one I posed before, the one about raising prices. Even so, you probably had a confident opinion about the first question while feeling helpless to answer the second without electronic assistance.

Mental models are always switched on, even within the quals wandering among us. They matter because we use them to predict the results of actions we might take and to judge the validity of the computer models we might use. We can and should take these steps to use both mental models and computer models without producing garbage-out:

• Mental models are generally unconscious. Make them explicit. Draw them, discuss them, think through the assumptions and principles that drive them. Question mental models and their owners, respectfully, just as we question computer models. Separate their conceptual sensibility from the numbers they produce, and focus on the former first.
• Because humans tell computer models how to think, computer models think like humans. If you believe a financial analysis is appropriate, you use a financial model in which the computer has been taught that profit equals sales minus costs. There’s a flip side: when you use a financial model, you implicitly aver that the problem at hand is properly addressed by financial analysis. Worries about “garbage in” apply to the choice of analysis at least as much as to the numbers fed in.

Important point: both mental models and computer models are about thinking. Both reflect how we believe the world works.

By the way, the answer to the arithmetic-accuracy question is $6,705,733.32.

Oh, Them

Let’s compare models. We’ll go back to the $129 product arithmetic and contrast it with the raise-prices prediction. There’s a critical difference between the two.

• Yes, the $129 product arithmetic is more precise. We have actual numbers. But although it’s a real difference, it’s not a critical difference.
• The $129 product arithmetic can be calculated, monitored, and verified. Another real difference, though not yet critical.
• The $129 product arithmetic implicitly and invisibly implied that the solution was a matter of financial analysis. I reinforced that perception by 1) supplying an answer 2) that was calculated to the penny. The raise-prices prediction implied financial analysis much less, and perhaps not at all. That’s critical.

Review the list of assessments you’d make in the raise-prices prediction. You might assess well and you might assess badly, but the way you would think about that problem would probably differ dramatically from the way you would think about the $129 product arithmetic.

The superficial precision of the $129 product arithmetic nudged you toward a method of analysis. (That’s known as “framing” the problem.) Thus, precision in could cause garbage out. Why garbage? Because the $129 product arithmetic should raise exactly the same issues about competitors and customers as the raise-prices prediction, and using a financial-analysis approach would completely, presumably unintentionally, and potentially disastrously ignore competitors and customers. Oh, them.

Conclusion: Not Crazy

The cure to our initial craziness is not to dispense with numbers wholesale, nor is it to embrace them unconditionally. The way to prevent garbage-out is not to scorn precision, nor is it to demand it. The answer is not either-or or some mythical “sweet spot.”

The answer is to deploy the best of both worlds, thinking strategically and calculating rigorously. Not perfectly or exhaustively, but consciously and vigorously.

That means being careful as we frame problems and choose models. It means remembering that there’s always a model, whether it’s in your head or in your computer. It means taking relevant factors into account, even imprecisely, because omitting relevant factors hurts more than using them imprecisely.

I know it’s possible because I’ve done it in the strategy models I’ve built in my career, and I’m not the only one who’s done it.

Let’s close with lessons from the remarkable Deep Blue.

Garry Kasparov, the world’s reigning chess champion at the time, was defeated in 1997 by IBM’s Deep Blue supercomputer. There are several points we can make about that match.

First, Deep Blue was not programmed with trend lines, gap analysis, or financial fundamentals. Of course not; it was a chess match. So why use trend lines, gap analysis, or financial fundamentals to analyze competitive-strategy problems?

Second, Deep Blue was a machine but it thought like a human. That’s because humans programmed it, with their chess-playing knowledge and expertise. Computers do not contest us; they express us. They are amplifiers for our thinking the way power tools are amplifiers for our muscles.

Third, it’s virtually certain that none of the individuals who programmed Deep Blue could have beaten Kasparov. Their combined talent, though, did. It’s also virtually certain that Deep Blue would have beaten any single person whose knowledge and expertise it contained.

Finally, we know that Deep Blue didn’t win every game in the match, nor (obviously) did Kasparov. But there’s a question deeper than which of them was best. The deeper question is this: who or what could beat a Kasparov/Deep Blue team? The value of numbers and models is not numbers and models. The value is better business decisions that help you succeed.

How to avoid garbage-out on your way to better decisions? Think about it.

We’re going to talk about investors. First, though, we’ll set the stage with a few relevant words about the United States Congress. No, not those words.

The United States government was designed with a system of checks and balances. They work very well. So exquisitely is Congress balanced that it has become immobile. Which, incidentally, an economist might call an “optimal” position. That’s because motion in any direction would apparently produce an outcome so awful that Congress will not allow it. If no other position is preferable, then the current position must therefore, by definition, be optimal. Which must mean something is wrong with economists, but at least they have a sense of humor.

Anyway. If Congress suffers from a surplus of balances, then it seems to me that investors suffer from a surplus of checks. I don’t mean checks in the sense of here, let me write you a check. I mean checks in the sense of due diligence, verification, and oversight. Not that there’s anything wrong with that, except for when they are all along the lines of making sure you don’t overpay for your berth on the Titanic.

Some time ago I attended a meeting of investors, of whom I was not one. They filled a large room and listened to pitches from passionate entrepreneurs, of whom I was also not one. The entrepreneurs all sounded highly competent, knowledgeable, and professional. They knew their markets, their products, and their prospects.

I listened to the thirty or so questions asked by the investors following the entrepreneurs’ presentations. The questions dealt with checks as in due diligence, verification, and oversight. That’s fine: if I’d chosen to invest, I’d have wanted to be sure everything was legitimate too. But that’s what every question was about. Every question was a request for a reason to believe that giving you a great deal of my money would be a wise move. Not a single question dealt with what could go wrong. What if, for example, a company already in a market wanted to squash the entrepreneurial intruder? What if an applied-for patent is denied?

We humans are basically optimistic and energetic. We imagine things, we want things, we make things happen. I’m that way too; one doesn’t start one’s own business otherwise. But being an optimist doesn’t mean looking only for reasons to believe. An optimist is not the person who leaps off a 50-story building and says, after plummeting 49 stories, “so far, so good.”

Strategy development often combines reasons to believe with ambition; that is, the laudable desire to do well and get ahead. In combination — reasons to believe and ambition — we get advocacy. I’m right, this is why I’m right, and this is why we should do things my way. That’s how we get ahead. That’s how we inspire action. That’s how we make progress.

We don’t want to become check-happy, like the investors, nor do we want to become balance-happy, like Congress. At least I hope not. So, let’s hearken back to the olden days of yore when we took tests and had to answer questions like these:

Checks are to balances as advocacy is to:
A. Optimism
B. Optimal
C. Optical
D. Optional
E. Stress tests

Yes, stress tests: business war games, strategy simulations, what-if analysis, sparring-partner equivalents, corporate contrarians, etc. Techniques advocated by conditional optimists who know something can go wrong. After all, if an unexpected threat (or opportunity) can come out of left field, it is good to look in that direction before committing to a new strategy or investment, or even to the status quo.

One last point. If stress-testing a strategy (or a government policy) demonstrates that it will produce really nasty results, that’s not a good enough reason to reject it. First, we must compare the results of the stress test to the outcomes we’d get under the status quo. If we get better results with the new strategy (or policy) than we’d get without it, then, really nasty as it is, we should switch to it. Even better, look for an even-better strategy (or policy).

I wrote and you may have read an essay called Predictable Competitors. In that essay we explored the assumption of predictability and the easy-to-fall-into traps of using competitors’ previous behavior to predict their future behavior. We also discussed how to avoid those traps and, in so doing, how to open up promising opportunities to make better strategy decisions. We did it in the context of pricing.

Here we’ll talk less about illusions of predictability and more about delusions of predicting. We’ll do so in the context of a pricing tournament in which over 250 able strategists participated and for which I ran millions of simulations. The pricing tournament was a kind of massive business war game using humans’ strategies and a computer’s calculations.

Unless you already know what I’m going to say — if you think you can predict what I’m going to say, write it down so you can check later — you will have two surprises by the time this essay is over.

The case

Here’s the situation I presented to those able strategists.

You are a pricing strategist for a company with businesses in three industries. You will develop pricing strategies for each of those three businesses, covering 12 quarters (three years). In each industry your business has two competitors, and in each industry your business and your competitors’ businesses start from identical positions. You define success: you decide how much you care about profitability and market share.

I proceeded to describe the three industries in some detail. Those details included these and more:

• The Ailing industry was shrinking, had somewhat price-sensitive customers, and was capital intensive.
• The Mature industry was growing slowly, had relatively price-insensitive customers, and was labor intensive.
• The Fast Growth industry was growing rapidly, had price-sensitive customers, and was a bit on the capital-intensive side.

Because the businesses in each industry began from identical positions, everyone had an equal opportunity to win. The only thing that would determine who won was the quality of their pricing-strategy decisions.

Participants selected a pricing move for Q1 (the first of the 12 quarters), a pricing strategy for Q2-4, another pricing strategy for Q5-8 (year 2), and a third pricing strategy for Q9-12 (year 3). The three multi-quarter pricing strategies could be the same or different, in any combination. The Q1 pricing move would be cut, hold, or raise (see picture); the three subsequent pricing strategies would be selected from a list of strategy options. Just as in real life, participants had to select their strategies without knowing what their competitors would do.

(I know what you’re thinking. Rest assured, there are analytic nuances, technological marvels, and good answers for your good questions, none of which will we go into here. What we’re about to cover doesn’t depend on those nuances, marvels, and answered questions.)

What would you do?

Let’s focus on the first move, the one affecting only Q1, where in each industry (Ailing, Mature, Fast Growth) you could cut, hold, or raise your price.

What would you do in each industry?

Write down your answer, or, if you’re telling yourself you’ll remember your answers, record them legibly in mental ink.

Of course I haven’t given you as much information as I gave the tournament participants. Still, though, you probably have some idea of what you’d do, something along the lines of “in a declining market it is best to _____ prices” or “I’d calculate the effect on profitability of _____ my prices and then decide.”

Second question. I didn’t directly ask this one in the tournament but it’s relevant for our discussion here.

What do you think your competitors (i.e., the other strategists participating in the tournament) will do for the two competing businesses you’ll face in each industry?

Record those answers too.

Notice how that question explicitly focuses your attention on competitors. An approach such as “in a declining market it is best to _____ prices” addresses competitors only obliquely: you’d probably consider the role of competition in a declining market, but perhaps not think about their specific actions.

Now this third question:

Did you think your competitors would do something different from what you chose to do?

In my experience strategists tend to assume competitors will behave as they wish them to or as they have behaved in the past. Who knows, they might even assume competitors are not as clever, quick, or attentive as they are. Hence that third question. But why would you think your competitors will do something different from you in scenarios when we have explicitly said they start from positions identical to your own?

Having now encountered that question, would you change your answer to the first question, the one about what you would do in each industry?

Our first surprise

Unless you correctly predicted me (did you?) and so there was no surprise, we’ve completed our first surprise: the realization that we make assumptions unconsciously that we wouldn’t make deliberately. (Remember, you heard it first here.) Further evidence appears as the subject of our second surprise.

Our second surprise

If the right pricing strategy were obvious, we would expect our able strategists to be pretty close to unanimous in their strategy selections for the tournament. They were not. In other words: surprise, we’re all over the map on how to kick off an effective pricing strategy.

The chart above shows the percentage of strategists who chose to cut, hold, or raise price in their Q1 pricing decision for each industry. Even the most-popular choice — hold price in Q1 in the Mature industry — was preferred by only 57% of the strategists. No move even got a majority in the Ailing and Fast Growth industries. In the Ailing industry, roughly equal percentages of strategists thought it would be best to cut or to raise their prices!

I mentioned earlier that the strategists indicated their performance objectives: market share, profits, or any combination. Perhaps if we control for their objectives we’ll see something closer to consensus. Here’s as close as we get to consensus:

Sixty-seven percent of the strategists who wanted a mix of share and profit in the Mature industry chose to hold their prices in Q1. No other pricing decision, in any of the industries, got anywhere near that level. For instance, here’s the Ailing industry:

The highest we see is 49% in favor of holding prices in Q1 to achieve a mix of share and profit, and 49% who’d cut price to gain share.

We do see an effect we might expect. The strategists who preferred market share as their performance objective were more likely to cut price than to raise it, and those who sought profit were more likely to raise price than to cut. Even so, those effects seem muted: raising or cutting prices didn’t win a clear majority of strategists. (That changed only when we looked at the most-extreme of the share-seekers, and we’d have to get really extreme — the 5 or 10 most share-happy strategists, out of over 250 — to approach a consensus.)

(Sidebar. Since objectives have a demonstrable effect on pricing decisions, we might argue that predicting competitors’ pricing moves translates, at least in part, to understanding their objectives. We would come to the same conclusions and the same surprises, though, because the strategists were far from unanimous about objectives too. There were many in the Ailing industry, for example, who wanted growth, and many in Fast Growth who wanted profits. We’ll talk more about objectives presently. End of sidebar.)

Let’s flip the numbers around a bit. What are the odds that you would be wrong if you predicted your competitors would make the most-popular move? At best, you’d have a 33% chance of being wrong, if you were in the Mature industry and knew your competitors wanted some share and some profit. At worst, you’d have a 63% chance of being wrong for profit-oriented competitors in Fast Growth. (Look at the chart below. The odds of being wrong equal 100% minus the most-popular choice, which is 37% for “hold.”) Not good odds.

So far we’ve focused on one quarter’s pricing decisions. In the tournament our able strategists could choose from a longer list of strategies that ranged from aggressive to reactive, cooperative to confrontational, tend-to-raise to tend-to-cut, and so on, for their moves after Q1. There was nothing approaching consensus or even popularity in those decisions.

Why might real life feel different?

Here we’ve seen that hundreds of strategists are far from agreement when they predict what pricing moves would work and, therefore, why it would be difficult to predict what any of them would do. So why might it seem that competitors are predictable in real life?

Here are a few ideas. Although I don’t have data to prove or disprove them, they are consistent with my experience working with thousands of strategists around the world, and consistent with the way that competitive-strategy tools think. (See Further Reading, below.)

• Clean slate. In the pricing tournament, the strategists started with clean slates: no history, no politics. No one could rely on, or had to defend, previous decisions. In real life, that’s not the case.
• Safety. In the tournament, the worst that could happen was that a person wouldn’t do terribly well in a simulation. Big deal. In real life, a person could lose his or her job for a change that seems to backfire. There’s perceived safety in consistency: don’t blame me, this strategy has worked well for [fill in suitable time span and/or credible other people]. In the tournament, no strategy has an inside track.
• Clarity. In the tournament, all the facts were laid out and the decisions were relatively simple. In real life, there’s more complexity and ambiguity, which might make strategists wary of upsetting a precariously balanced system.
• No tradition. In the tournament, strategists were free to define success according to their own preferred combinations of market share and profit, unburdened by how we do things around here. In real life, different companies may assign similar missions to their businesses (e.g., “fly full” in airlines). We’ve seen that definitions of success — objectives — influence pricing decisions. That effect is doubtless magnified by real-life industries’ oral traditions of what it takes to achieve objectives.

In real life, competitors often emulate others’ moves. That happened in the tournament too, in various ways and for various reasons. However, that affected pricing after Q1, and here we focus mostly on the Q1 decisions. The phenomena after Q1 are fascinating but this essay may already be long enough to try your patience, and certainly mine.

What our surprises mean

Strategists face the problem of predicting competitors. We have just seen why doing so may be harder than we might have thought. Our first surprise suggests that we don’t ask questions that might help us predict competitors (more on that quite soon). Our second surprise suggests that our competitors may not be so easy to predict, unless the slate isn’t clean, jobs aren’t safe, issues aren’t clear, and tradition is binding.

The predicting problem has two elements, carbon and silicon, a.k.a. humans and software.

On the human side, we’ve seen that we are prone to make optimistic, or at least unexamined, assumptions about what competitors will do. We know we assume, and we work in good faith not to do so. Take, for instance, SWOT (strengths, weaknesses, opportunities, and threats) analysis. There we endeavor to give equal time to our competitors so we don’t get too convinced of our infallibility and invulnerability.

But let us look at the SW of SWOT. Its strength is that it is easy, fast, portable, and potentially insightful. (I say “potentially” because whether we get insight depends as much on what we receive as on what it transmits.) Its weakness is… well, let’s illustrate by comparing SWOT to business war games. In a business war game you walk in your competitors’ shoes. In SWOT analysis you merely look at them.

Ways to do better:

• Use competitive intelligence to learn about your competitors’ objectives. A change in objectives may well upset any pricing (or other) “pattern” you may have observed.
• Also use CI to learn about new management. Our able strategists have demonstrated that smart people differ in what they think will work. A change in management may foretell a change in strategy. That’s true especially because “now under new management” rarely means “still under previous thinking.”
• Competitive dynamics resemble chess more than accounting or trend lines. Practice your game before the big real-life match. Business war games let you do that. In my experience, surprises like those we’ve explored here are the rule, not the exception, in business war games. The good news is, it’s a lot cheaper to get surprised during practice.
• During strategy debates, ask if you were our competitors, how would you take advantage of our move. Ask what could go wrong. Ask what are we assuming and do we believe what we are assuming.

Then there’s the software side. Our software thinks like us. (Who else would it think like?) We tell it how to think. We tell it that profit equals revenue minus costs. We tell it demand will change X% for a Y% change in price. We tell it how to combine those thoughts, and others, to figure out the bottom-line effects of price changes.

Since our thinking includes assumptions, so does the thinking of the software that calculates on our behalf. Some of the able strategists, participating in the tournament at a pricing conference I addressed, actually wrote down their spreadsheet-style calculations on their strategy-decision forms: at this price, with these fixed and variable costs, here’s how much I’ll make. If such people were in their offices, it’s likely they would run those calculations in Excel to guide their decisions. Those people chose a paradigm for how to decide before deciding on pricing strategies. That paradigm, like all others, has its paradigm-specific assumptions. In that case, the paradigm assumed competitors were simply irrelevant.

Ways to do better:

• Ask your tools what assumptions they make. They’re not talking? Okay, ask the tool-designers or -wielders what assumptions their tools make. Ask especially how competitors’ moves would be taken into account.
• Ask about competitive dynamics. What gets held constant over time, what doesn’t.
• Ask about what-if. If there’s anything we should take away from the Q1 pricing decisions we explored, let alone the similar variation in the other pricing decisions that we didn’t explore, it’s that we need to test the very real possibility that they will move in a surprising direction. (Subtle point: they may even want to move in a different direction, and may be watching us before they commit.)
• Worry more about the what-ifs than about precision. Who cares if the 63% chance of being wrong for the profit-oriented competitors in the Fast Growth industry should really be 62% or 64%? It’s far more important to explore your chess opponent’s options than to measure precisely where each piece is in its square.

Bonus surprise

Did you predict there’d be only two surprises since that’s what I said at the beginning? Oh my poor student.

There’s one more surprise concerning the able strategists. We noted the variation in their pricing decisions. Every one of them believed that he or she was selecting the strategy that’d win; if she or he believed otherwise, he or she would have selected a different strategy.

Surprise: not all of them won. (I didn’t either. See Further Reading, below.)

That means that we strategists don’t know what will work. Actually, that’s a slight overstatement. Some did know what would work, and their strategies performed well in the tournament, in which I ran over 25,000,000 (no joke) what-if simulations. The problem is, we don’t know in advance whose strategies will work, nor do we know if they will be successful in the future. For what it’s worth, and to add a minor surprise: the person who did best, so far, isn’t a pricing expert. The person is a market-research practitioner.

It’s humbling and perhaps infuriating that we don’t know what will work. We may assume solace in thinking that we do know what’ll work in our industries, and those Ailing, Fast Growth, and Mature industries are pretty weird. Maybe that’s true, but I don’t think so. And I’ll end by asking one more annoying question: if we’re so good at pricing, where do price wars come from?

Update: this essay was published as “Predicting Competitors: Game Theory in Pricing” in the Journal of Professional Pricing, First Quarter 2010 (www.pricingsociety.com).

Further reading

Predictable Competitors, on using history and trends to predict competitors (or not)
Motor Swilling Forbidden, on how people use the same words and mean different things
When I Was Wrong, on the consequences of and opportunities from mistakes
House, MBA, on the envy two CEOs have for the other’s pricing strategy
The Rules, about surprises and assumptions
Decision Tournaments, on the technology behind the pricing tournament

Appendix

Participating in the pricing tournament 

If you would like to run a pricing tournament for your group, let us know. Strategies and scores will be held in confidence. For more information, please write to info@whatifyourstrategy.com.

A representative sample of strategists

In the essay I mentioned that 250+ real-life strategists have participated in the massive pricing simulation. One might ask, especially if one has been exposed to statistical analysis, whether those 250+ people constitute a representative sample. It’s a good question, and surprisingly hard to answer, and fortunately quite inconsequential.

First, what would be a representative sample? Pricing specialists? Pricing consultants? People with responsibility for pricing decisions? With how much experience? In what countries and industries? Big companies or small? Highly competitive markets, long-established markets, markets with many competitors, or not? It’s hard to know what’s representative.

It’s a little easier to know what’s relevant, which is what leads me to say the “representativeness” of our sample is inconsequential. What’s relevant is that you could be up against pricers of any kind.

The strategists in the pricing tournament:

• Came from several countries, mostly from the USA
• Came from many industries
• Included mostly corporate strategists, augmented by some consultants, academics, and MBA students
• Knew their strategies would be held in confidence.

My analysis so far shows little reason to believe that demographic characteristics (location, occupation, etc.) have a material effect on the strategy decisions the strategists made. In other words, the quality of thinking and strategizing doesn’t seem to depend much, if at all, on demographics. More research is needed.