Wednesday, October 21, 2009

Press Releases: Numbers game

Published in the Portland Phoenix

If you take a close look at the latest polls, you will find that supporters and opponents of November's same-sex marriage referendum question are locked in a neck-and-neck battle. The state's major media outlets, however, did not report the news this way. In fact, they got it backward. Here are some samples:

PORTLAND PRESS HERALD "A new poll shows an edge for supporters of same-sex marriage in Maine's Nov. 3 referendum, with 51.8 percent of those surveyed saying they plan to vote to uphold the law legalizing it and 42.9 percent planning to vote for repeal."

BANGOR DAILY NEWS "The results of a new poll released Wednesday show growing support among voters for Maine's gay marriage law."

LEWISTON SUN JOURNAL "Mainers planning to vote on Election Day favor keeping Maine's law allowing same-sex marriage."

WGME "A slight majority of Mainers support same sex marriage."

MPBN "A majority of Mainers in a new poll say they're ready to uphold the state's new gay-marriage law by voting 'no' on the people's-veto referendum question."

At first glance, they'd all appear to be right. The poll itself, by Pan Atlantic SMS Group in Portland, says 40.9 percent of people surveyed said they would vote to repeal the new same-sex marriage law; 2 percent said they were leaning toward repeal the law; 50.6 percent said they would uphold the law; 1.2 percent said they were leaning toward upholding it; 5.2 percent said they were undecided.

The total of all those wanting to repeal the law is 42.9 percent, and those who would uphold it is 51.8 percent.

The key fact, though, is the survey's margin of error, plus-or-minus 4.9 percent. All of the news outlets reported it, but failed to accurately describe what it means: it's a statistical dead heat.

If you take the numbers for people saying they plan to vote a particular way, those in favor of the law are between 45.7 and 55.5 percent of the likely voters; those desiring repeal are between 36 and 45.8 percent.

That 0.1 percent overlap is bad enough, but when adding the "leaning" voters in to each category, the media outlets failed to recognize that the dead heat actually gets closer: voters plus leaners favoring the law are between 38 and 47.8 percent of the population; voters plus leaners for repeal are between 46.9 and 56.7 percent of likely Maine voters -- an overlap of 0.9 percent.

Now you see: It is quite possible that the poll has found more people wanting to repeal the law than supporting it.

And it actually gets worse. Patrick Murphy, Pan Atlantic's president, says it is a nationally accepted fact among pollsters that surveys unavoidably under-report the number of people who oppose same-sex marriage. The reason is that people who oppose it fear being thought of poorly by the person interviewing them, and so they answer that they will support it. But when it comes to actually voting, they vote the way they feel, not the way they said they would. (Pollsters call this the "Bradley effect," after an African-American man who led in the polls but lost to a white man in the 1982 California gubernatorial election.)

And while many pro-marriage young people may be landline-less and therefore left out of the survey, Murphy says the 18-to-34 age group was properly represented in the survey, and studies have shown that young people with landlines are not statistically different from young people who have only cell phones.

All of this gets us to a position that is radically different from what the mainstream media told you. The real story should have been: A new poll shows voters who oppose same-sex marriage outnumber supporters in Maine. While the results themselves show a statistical dead heat, survey experts know that opponents are often under-counted because of unavoidable imperfections in the polling process.