Imagine if, five times in 18 months, I stuck my head out the window on a random day, made a guess of the temperature that was 95% likely to be 3 degrees or less too high or too low and, then, extrapolated climate change data from my results. You would laugh at me: too much chance and margin of error, not enough data points. Still, that doesn’t stop the Herald.
And you can see why they’ve got to make a big deal of their polls: these things cost tens of thousands of dollars each. You’re talking months of a journalist’s salary for a single poll. So, no matter what it says, it’s going to be a bloody story – a series of stories if possible.
Here, for comparison, is the Roy Morgan results post-election beside the Herald’s (the big dots).
You can see two things – first, the Herald consistently puts National much higher than the Roy Morgan. Second, there just aren’t enough data points to hang a trend off for the Herald. If I put in the error bars as well, you would see that we can’t have a clue who is going up or down or staying still just relying on the Herald. The Roy Morgan, on the other hand, gives us 34 polls since the election and even that is only just enough to really tell us there’s a trend, not just statistical noise. (it’s tempting, almost instinctive, to say the Herald’s trend is divergent from the Roy Morgan trend but you have to pull yourself back and say ‘no, there can be no trend in the Herald’s numbers’)
The irony, of course, is that you see journalists poo-pooing the Roy Morgan for always ‘jumping around’. In fact, jumping around is what polls do – the same population with the same views could return polls taken at the same time with party results that are 4,5,6% different and it wouldn’t mean a thing – that’s just what happens when you ask 750 people a question and then try to extrapolate from that what over 2 million voters think. The irregular polls like the Herald jump around just as much as the Roy Morgan but, because you see them less often, you tend to put more meaning behind those movements.
Powered by WPtouch Mobile Suite for WordPress