While politicos were concentrating on Hills defeating The Donald in the first US debate*, Roy Morgan released their latest poll, showing Labour massively up and National down a chunk.
It brings Roy Morgan into line with the latest Newshub poll, and Labour’s UMR poll. Labour low 30s, National low 40s, Labour-Greens at or just above National.
The likes of Matthew Hooton are obviously rubbishing it:
But will we see the equivalent articles about how there must be panic (this time in National), like the last lot about Labour by a gender-balancing obssessed media? (a few bloke MPs lose their jobs… and?)
Roy Morgan is generally not covered well in the media as it’s not connected to a news organisation and because it has the reputation of “it moves around a lot.” And it does. But part of that is because it actually has enough data points to see that: all polls move around quite a bit. When you only get polls every 2 months (OneNews), or 3 times a year (Newshub) or indeed haven’t been seen since 2015 (Herald), suddenly every point on your graph seems highly significant.
Odds are that One News’ last one that set off such a media flurry was an outlier (95% confidence of +/- 3.6%? Or 1 in 20 polls is just plain wrong?). But we have so few data points we can’t tell.
So really it’s a great shame that Roy Morgan have moved from fortnightly to monthly. And that apparently our media can’t afford the number of polls to actually tell us much. And when they do spend the money they feel so compelled to make ‘good’ use of it, they massively over-hype what it’s actually telling us.
If you’re looking at fivethirtyeight (I know you are…) and the poll updates, you see massive discrepancies between polls. Between different polls that Nate Silver is quite happy with their methodology, you’ll see Clinton nationally up by 6 and down by 2 on the same day. If you look at the likes of Utah, Clinton could be up by 4 or down by 39 (okay the up by 4 is GCS, but there’s still >30 points between other surveys). And the nationwide tracking polls have big swings across a few days despite having consistent methodology and not always an event to attribute any swing too.
You end up looking at trends and averages, and you need enough data points to do that. We don’t have them. The Radio NZ poll of polls is better than any individual poll, but they’re still ultimately stuck with too little data.
So: I propose our media scrap polls until they have enough money to do them properly. Otherwise they paint a narrative that can become self-fulfilling on too little data – or they end up looking like dolts when they have to suddenly declare black is white when their next result comes out a few months later…
–
* we’ll have to wait a few days for the polls to validate my statement… 🙂
[And Gary Morgan: just stick to describing the results, when you try to give a reason for the latest vast swing you look silly. When National are up 6 points because the news has all been about homelessness and the housing crisis… it’s just best not to try and make up why]
Powered by WPtouch Mobile Suite for WordPress