Abraham Wald And The Missing Bullet Holes

Trading successfully is a result of thinking about the right things in the right way. This is a great illustration of how math combined with an outlook that challenges what appears to be true saved many lives in WW2. I thoroughly enjoyed the story as well as the (trading) principles that drop out of it, I hope you do to. (Excerpt taken from “How not to be wrong” – J.Ellenberg)

This story, like many World War II stories, starts with the Nazis hounding a Jew
out of Europe and ends with the Nazis regretting it. Abraham Wald was born in
1902 in what was then the city of Klausenburg in what was then the Austro-
Hungarian Empire. By the time Wald was a teenager, one World War was in the
books and his hometown had become Cluj, Romania. He was the grandson of a
rabbi and the son of a kosher baker, but the younger Wald was a mathematician
almost from the start. His talent for the subject was quickly recognized, and he
was admitted to study mathematics at the University of Vienna, where he was
drawn to subjects abstract and recondite even by the standards of pure
mathematics: set theory and metric spaces.

But when Wald’s studies were completed, it was the mid-1930s, Austria was
deep in economic distress, and there was no possibility that a foreigner could be
hired as a professor in Vienna. Wald was rescued by a job offer from Oskar
Morgenstern. Morgenstern would later immigrate to the United States and help
invent game theory, but in 1933 he was the director of the Austrian Institute for
Economic Research, and he hired Wald at a small salary to do mathematical odd
jobs. That turned out to be a good move for Wald: his experience in economics
got him a fellowship offer at the Cowles Commission, an economic institute then
located in Colorado Springs. Despite the ever-worsening political situation, Wald
was reluctant to take a step that would lead him away from pure mathematics for
good. But then the Nazis conquered Austria, making Wald’s decision
substantially easier. After just a few months in Colorado, he was offered a
professorship of statistics at Columbia; he packed up once again and moved to
New York.

And that was where he fought the war.

The Statistical Research Group (SRG), where Wald spent much of World
War II, was a classified program that yoked the assembled might of American
statisticians to the war effort—something like the Manhattan Project, except the
weapons being developed were equations, not explosives. And the SRG was
actually in Manhattan, at 401 West 118th Street in Morningside Heights, just a
block away from Columbia University. The building now houses Columbia
faculty apartments and some doctor’s offices, but in 1943 it was the buzzing,
sparking nerve center of wartime math. At the Applied Mathematics Group
−Columbia, dozens of young women bent over Marchant desktop calculators
were calculating formulas for the optimal curve a fighter should trace out
through the air in order to keep an enemy plane in its gunsights. In another
apartment, a team of researchers from Princeton was developing protocols for
strategic bombing. And Columbia’s wing of the atom bomb project was right
next door.

But the SRG was the most high-powered, and ultimately the most influential,
of any of these groups. The atmosphere combined the intellectual openness and
intensity of an academic department with the shared sense of purpose that comes
only with high stakes. “When we made recommendations,” W. Allen Wallis, the
director, wrote, “frequently things happened.

Fighter planes entered combat withtheir machine guns loaded according to Jack Wolfowitz’s* recommendations
about mixing types of ammunition, and maybe the pilots came back or maybe
they didn’t. Navy planes launched rockets whose propellants had been accepted
by Abe Girshick’s sampling-inspection plans, and maybe the rockets exploded
and destroyed our own planes and pilots or maybe they destroyed the target.”
The mathematical talent at hand was equal to the gravity of the task. In
Wallis’s words, the SRG was “the most extraordinary group of statisticians ever
organized, taking into account both number and quality.” Frederick Mosteller,
who would later found Harvard’s statistics department, was there. So was
Leonard Jimmie Savage, the pioneer of decision theory and great advocate of the
field that came to be called Bayesian statistics.* Norbert Wiener, the MIT
mathematician and the creator of cybernetics, dropped by from time to time.
This was a group where Milton Friedman, the future Nobelist in economics, was
often the fourth-smartest person in the room.

The smartest person in the room was usually Abraham Wald.

So here’s the question. You don’t want your planes to get shot down by enemy
fighters, so you armor them. But armor makes the plane heavier, and heavier
planes are less maneuverable and use more fuel. Armoring the planes too much
is a problem; armoring the planes too little is a problem. Somewhere in between
there’s an optimum.

The reason you have a team of mathematicians socked away
in an apartment in New York City is to figure out where that optimum is.
The military came to the SRG with some data they thought might be useful.
When American planes came back from engagements over Europe, they were
covered in bullet holes. But the damage wasn’t uniformly distributed across the
aircraft. There were more bullet holes in the fuselage, not so many in the

Section of plane Bullet holes per square foot
Engine 1.11
Fuselage 1.73
Fuel system 1.55
Rest of the plane 1.8

The officers saw an opportunity for efficiency; you can get the same
protection with less armor if you concentrate the armor on the places with the
greatest need, where the planes are getting hit the most. But exactly how much
more armor belonged on those parts of the plane? That was the answer they
came to Wald for. It wasn’t the answer they got.

The armor, said Wald, doesn’t go where the bullet holes are. It goes where
the bullet holes aren’t: on the engines.

Wald’s insight was simply to ask: where are the missing holes?

The ones that would have been all over the engine casing, if the damage had been spread
equally all over the plane? Wald was pretty sure he knew. The missing bullet
holes were on the missing planes. The reason planes were coming back with
fewer hits to the engine is that planes that got hit in the engine weren’t coming

Whereas the large number of planes returning to base with a thoroughly
Swiss-cheesed fuselage is pretty strong evidence that hits to the fuselage can
(and therefore should) be tolerated. If you go the recovery room at the hospital,
you’ll see a lot more people with bullet holes in their legs than people with bullet
holes in their chests. But that’s not because people don’t get shot in the chest; it’s
because the people who get shot in the chest don’t recover.

Here’s an old mathematician’s trick that makes the picture perfectly clear: set
some variables to zero. In this case, the variable to tweak is the probability that a
plane that takes a hit to the engine manages to stay in the air. Setting that
probability to zero means a single shot to the engine is guaranteed to bring the
plane down. What would the data look like then? You’d have planes coming
back with bullet holes all over the wings, the fuselage, the nose—but none at all
on the engine.

The military analyst has two options for explaining this: either the
German bullets just happen to hit every part of the plane but one, or the engine is
a point of total vulnerability. Both stories explain the data, but the latter makes a
lot more sense.

The armor goes where the bullet holes aren’t.

Wald’s recommendations were quickly put into effect, and were still being
used by the navy and the air force through the wars in Korea and Vietnam.

I can’t tell you exactly how many American planes they saved, though the dataslinging
descendants of the SRG inside today’s military no doubt have a pretty
good idea.

One thing the American defense establishment has traditionally
understood very well is that countries don’t win wars just by being braver than
the other side, or freer, or slightly preferred by God. The winners are usually the
guys who get 5% fewer of their planes shot down, or use 5% less fuel, or get 5%
more nutrition into their infantry at 95% of the cost.
That’s not the stuff war
movies are made of, but it’s the stuff wars are made of.

And there’s math every step of the way.

Leave a Reply

Your email address will not be published. Required fields are marked *