« "Not every wrong, or even every violation of the law, is a crime." | Main | Anthrax hoax suspect indicted »

No, It's Not A Statistical Tie

Kevin Drum provides a great service by explaining what the "margin of error" in a poll really means. It does not mean, despite what all of us will hear over and over, that a poll with a result within the margin of error is a "statistical tie."

No. No. No. No. No. As Drum explains:

The idea of a "statistical tie" is based on the theory that (a) statistical results are credible only if they are at least 95% certain to be accurate, and (b) any lead less than the MOE is less than 95% certain.

There are two problems with this: first, 95% is not some kind of magic cutoff point, and second, the idea that the MOE represents 95% certainty is wrong anyway. A poll's MOE does represent a 95% confidence interval for each individual's percentage, but it doesn't represent a 95% confidence for the difference between the two, and that's what we're really interested in.

Drum provides a handy chart that allows a reader to look at the poll's margin of error and the percentage difference between the candidates and see what the chances are the person leading in the poll is actually ahead.

So, instead of Barack Obama and John McCain being "statistically tied" in the latest Pew poll, we can see that there is a 93 percent chance that Obama is actually leading McCain.

That's a slightly different story.

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)