Death by Shark

Death by Shark

by Mark Chussil

No strategic message here. This is just another in my heroic series of occasional essays on rigorous thinking.

Your odds of dying as the result of an unprovoked shark attack are roughly 1 in 300,000,000 in the USA (1). The odds are more than 20 times higher in Australia. Therefore, if you don’t want to be killed by a shark, you should flee Australia immediately for the USA. It’s safer here, except maybe for the gun thing.

The numbers and arithmetic are correct. The analysis and conclusion are silly. Yet that’s the kind of reasoning CBC News used and Premier Traveler’s April 22, 2013 newsletter reproduced.

In “Scary Flights with Happy Endings” the Canadian Broadcasting Corporation said:

“Statistics from the United States Department of Transportation suggest that the odds for the general population of dying in:

• A car crash: 1 in 7,700.
• A motorcycle accident: 1 in 91,500.
• A railroad accident, 1 in 306,000.
• A bicycle accident: 1 in 410,000.
• An airplane accident: 1 in 2,067,000.”

Succumbing to a mixed metaphor (what are the odds!), I immediately saw those numbers didn’t pass the sniff test. Your odds of dying in a motorcycle accident are less than one-tenth what they are for dying in a car crash? Really? That doesn’t sound right. And it’s not. It’s because the CBC didn’t compare risks. It compared the annual rates of motorcycle fatalities and automobile fatalities per 100,000 people. Just like the shark-attack rates with which we began.

Your odds of dying while doing something are zero if you don’t do that thing. As you relax at home in Melbourne, Florida, your odds of a fatal shark encounter are zero. The same zero as when you relax at home in Melbourne, New South Wales. You’ll enjoy the same serene sense of shark-free safety in southern North Dakota as you would in the southern Northern Territory.

It makes sense to talk about the rate of shark-attack deaths as among the causes of human expiration. Very, very few people in the world die by shark. But it doesn’t make sense to use the same rate to talk about the odds of death by shark when very, very few people go into water with sharks.

The CBC apparently took the rate of motorcycle deaths in the USA per 100,000 people and re-stated it (2). That’s fine: 1.09 per 100,000 is the same as 1 in 91,500. But the number doesn’t express the risk of taking a motorcycle out for a ride.

The rate of motorcycle deaths per 100,000 people combines 1) the rate of motorcycle deaths among the people in the 100,000 who get onto motorcycles and 2) the rate of motorcycle deaths — zero — among the people in the 100,000 who don’t get onto motorcycles. Combining the two rates dilutes the risk of motorcycling. The non-motorcyclists arithmetically become safe motorcyclists, leading us to underestimate the risk.

We can better measure your risk per hour or per mile you travel on a motorcycle. Ideally we’d also adjust for weather conditions, speed, and whether you’re wearing a helmet.

Practically everyone travels by car, which avoids the dilution problem above. Ideally, comparing the risks of motorcycles and cars (3), as the CBC did, would take a little more data. For one thing, people drive cars under almost all weather conditions. If cars were driven only under conditions conducive to motorcycling, we’d probably find cars safer than the existing statistics.

The CBC’s article focused on airline safety relative to other methods of transportation. As before: if you don’t get on an airplane you won’t die in an airplane accident. In addition, airplane accidents happen mostly at the start or the end of a flight. (You could argue they all happen at the end of a flight.) Cruising at 35,000 feet isn’t the same as landing at a busy airport. One four-hour flight carries much less risk than four one-hour flights.

What you really want to know are the odds that you will arrive safely on a specific trip. I used statistics from The New York Times and the National Highway Traffic Safety Administration to compare taking a car or an airplane. The Times quotes an MIT statistician saying that, based on the safety record of the last five years, “the death risk for passengers in the United States has been one in 45 million flights.” The NHTSA says the car fatality rate is 1.11 per 100 million vehicle miles traveled. Based on those numbers, you should drive for any roundtrip shorter than 4½ miles. (Actually, shorter than 4½ miles plus the distance you’d have to drive to and from airports. Adjust for taxi drivers in certain cities.) For distances longer than that, take a plane (4) .

We heroically slayed all those stats so we can arrive (safely) at these noble truths.

Imprecision or garbage. We people talk about garbage in, garbage out. We almost always mean imprecision when we say garbage. But the 1 in 91,500 statistic would be garbage no matter how accurately we measured it. It’s garbage not because the number is wrong but because it is the wrong number to gauge risks. You don’t have to be an expert in transportation safety to know that; all you have to do is think about whether it makes sense in its context.

A flaw in the foundation. Using a wrong (not imprecise) number is not a detail, a nicety, or an inconvenience. It is a flaw in the foundation on which we build decisions that affect money, careers, and lives. In this case, motorcycles falsely appearing safer than cars — forget about the airplane part — could wrongly encourage people to switch from cars to motorcycles for safety reasons. Wrong numbers can even affect crime and punishment. See the prosecutor’s fallacy.

Thinking about systems. When we spoke about factors such as weather conditions, wearing helmets, numbers of takeoffs, and so on, we were actually constructing models of how a system works. And we were finding it useful, not frightening or technical! Even that brief exercise in systems thinking gave us a richer understanding of the problem and helps us defend ourselves against being deceived, intentionally or not, by others who cite wrong numbers at us. Keep at it. It’s a skill you can practice and develop, while entertaining your friends the way I have for years.

By the way, there is an impressive statistic that the CBC could have used to show the marvelous safety of air travel. The New York Times reported that it’s been over four years since the last fatal airline crash in the USA.

(1) There were 17 confirmed fatalities from unprovoked shark attacks in Australia between 2003 and 2013 and 11 in the USA. http://www.sharkattackfile.net/incidentlog.htm. In July 2011 the population of Australia was 22,620,600 and it was 311,591,900 in the USA.

(2) The Centers for Disease Control and Prevention said that motorcyclist death rates in 2001 were 1.12 per 100,000 persons, which is very close to the CBC’s 1.093 per 100,000 (no year stated). The CDC says motorcyclist death rates rose to 1.74 per 100,000 persons in 2008.

(3) The CBC’s number, adjusted for population, is close to National Highway Traffic Safety Administration reports for USA motor vehicle deaths in 2001.

(4) Motorcycles aren’t an option if you care only about safety. Quoting the NHTSA, the Network of Employers for Traffic Safety says they were “involved in fatal crashes at a rate of 35.0 per 100 million miles of travel” in 2006, compared to cars’ rate of 1.7. Take the plane for distances longer than a few hundred feet.

For other reporting, see “Is Air Travel Safer Than Car Travel?”, USA Today. That article quotes “The odds of dying from…”, by the National Safety Council. The NSC notes that their odds depend on the activities in which people engage. In other words, lifetime risks depend on which risks you take. I, for one, will not die on a roller coaster.

Update, April 25, 2013. “Our Feel-Good War On Breast Cancer,” by Peggy Orenstein in The New York Times, describes more-serious consequences of numerical problems like those above. From the article:

“’The five-year survival rate for breast cancer when caught early is 98 percent. When it’s not? It decreases to 23 percent.’ [Geisel School of Medicine at Dartmouth Professor Steve] Woloshin called that [statement from a Komen Foundation print ad] willfully deceptive. The numbers are accurate, but five-year survival rates are a misleading measure of success, skewed by screening itself. Mammography finds many cancers that never need treating and that are, by definition, survivable. Meanwhile, some women with lethal disease may seem to live longer because their cancer was found earlier, but in truth, it’s only their awareness of themselves as ill that has been extended. ‘Imagine a group of 100 women who received diagnoses of breast cancer because they felt a breast lump at age 67, all of whom die at age 70,’ Woloshin said. ‘Five-year survival for this group is 0 percent. Now imagine the same women were screened, given their diagnosis three years earlier, at age 64, but treatment doesn’t work and they still die at age 70. Five-year survival is now 100 percent, even though no one lived a second longer.’”