If you ever decide to do something as stupid as build an
automatic terrorism detector, here’s a math lesson you need to
learn first. It’s called “the paradox of the false positive,” and it’s a
doozy.
Say you have a new disease, called SuperAIDS.
Only one in a
million people gets SuperAIDS.
You develop a test for SuperAIDS
that’s 99 percent accurate. I mean, 99 percent of the time, it
gives the correct result true
if the subject is infected, and false if
the subject is healthy. You give the test to a million people.
One in a million people have SuperAIDS.
One in a hundred
people that you test will generate a “false positive” the
test will
say he has SuperAIDS
even though he doesn’t. That’s what “99
percent accurate” means: one percent wrong.
What’s one percent of one million?
1,000,000/100 = 10,000
One in a million people has SuperAIDS.
If you test a million
random people, you’ll probably only find one case of real SuperAIDS.
But your test won’t identify one person as having SuperAIDS.
It will identify 10,000 people as having it.
Your 99 percent accurate test will perform with 99.99 percent
inaccuracy.
That’s the paradox of the false positive. When you try to find
something really rare, your test’s accuracy has to match the rarity
of the thing you’re looking for. If you’re trying to point at a single
pixel on your screen, a sharp pencil is a good pointer: the penciltip
is a lot smaller (more accurate) than the pixels. But a penciltip
is no good at pointing at a single atom in your screen. For that,
you need a pointer a
test that’s
one atom wide or less at the
tip.
This is the paradox of the false positive, and here’s how it
applies to terrorism:
Terrorists are really rare. In a city of twenty million like New
York, there might be one or two terrorists. Maybe ten of them at
the outside. 10/20,000,000 = 0.00005 percent. One twentythousandth
of a percent.
That’s pretty rare all right. Now, say you’ve got some software
that can sift through all the bankrecords,
or tollpass
records, or
public transit records, or phonecall
records in the city and catch
terrorists 99 percent of the time.
In a pool of twenty million people, a 99 percent accurate test
will identify two hundred thousand people as being terrorists. But
only ten of them are terrorists. To catch ten bad guys, you have to
haul in and investigate two hundred thousand innocent people.
Guess what? Terrorism tests aren’t anywhere close to 99 percent
accurate. More like 60 percent accurate. Even 40 percent
accurate, sometimes.
2 Comments on “The important thing about security systems isn’t how they work, it’s how they fail.”
Very genius.
Never looked at it this way
Makes sense. No surprise since everything’s flawed nowadays.
Leave a Reply