Reposted from Polity.
The reporting of this morning’s final New Zealand Herald poll is nothing short of a disgrace.
First, here’s the report:
Moment of Truth gifts Team Key a late bounce in polls
Last voter survey before election day shows jump in National’s rating
The Kim Dotcom-inspired event in Auckland’s Town Hall that was supposed to end John Key’s career gave the National Party an immediate bounce in support this week, according to polling for the last Herald DigiPoll survey.
With 60 per cent of the poll done by Monday night, when the event happened, National was polling at 47.8 per cent, down on last week, said DigiPoll general manager Nandan Modak. From Tuesday it jumped to 49.1 per cent.
Party Vote results: National 48.2 (down 0.4), Labour 25.9 (up 1.3), Green 11.1 (down 0.4), NZ First 8.4 (up 0.3), Conservatives 3.3 (down 0.5), internet Mana 1 (down 1.3), Maori Party 1.1 (up 0.4), Act 0.5 (up 0.2), United Future 0.2 (up 0.2), Legalise Cannabis 0 (no change).
Two exasperated points here:
First, the “last voter survey before election day” does not show a “jump in National’s rating” at all. In fact, “National’s rating” has fallen in this “last voter survey” compared to the previous voter survey, which is normally how “jumps in support” are assessed.
That headline reporting is horribly misleading and irresponsible, especially on the last day before an election.
Second, I want to address the fig leaf the Herald is relying on to make these misleading claims. The article claims authority from a 1.3% increase in reported National support within the survey window, between the 60% of the survey completed prior to Moment of Truth, and the 40% completed afterwards.
There are simple statistical calculations you can do to see if that evidence is any good, and worth relying on. Any pollster worth their salt can point to these intuitively. I’ll lay it out:
Is there a statistically meaningful difference between those two groups of results? Absolutely not. A simple two sample difference in proportions test says the evidence cannot support any conclusion of changing support during the survey, at conventional levels of significance or anything close to them.1
In fact, if you absolutely had to say whether National’s support rose or fell post Moment of Truth, and you were simply not allowed a “no change” option, then there’s a 37% chance National’s support actuallydropped, given these data.
Put another less technical way, if there had been no change at all in National support during the survey, we would have expected to see 140 of the 293 late responders tick National. In the event, we observed 144 of the 293 people tick National. Is that four-person difference worth getting at all excited about? Since, like intuition, says “No.”
Put yet another way, the Herald’s screaming day-before-the-election pro-National headline was based on an observed difference of 1.3%, with a margin of error of plus or minus 7.4%.
That, New Zealand Herald, is a fucking disgrace.
lprent: I agree. If a politics editor at a major National newspaper doesn’t know basic statistics, then you’d have to ask how she retains her job?
I have enough of an issue viewing polls as having much validity these days due to sampling population errors. Having a jonolist apparently deliberately distort the meaning of a poll that is taken, and whose results was not linked to in the article, is a complete travesty. Get rid of the useless things. They aren’t helping inform the public. They just provide cheap and easy matter for jonolists to use when they are avoiding doing their jobs.