This is a Guest Post from Matthew Whitehead. Matthew is a writer, programmer, Green party member, and electoral reform activist.
A new Colmar Brunton poll was recently released, having finished at the 27th of July, and as usual, the media is dramatizing its results without appropriate caution. It does potentially represent a shift in party voting trends, as the Greens had their recent and explosive announcement about welfare policy, so there’s a chance these changes are in fact real, but it’s still no reason to panic, you need to be looking at three polls in a row saying roughly the same thing before you can really absolutely conclude it’s probably correct, and political events often move faster than that, so a lot of the time we honestly don’t know.
I talk a lot about my feelings and speculation in online political debate, but I think it’s time to have a discussion about polling and the cold, hard facts around it, so that nobody panics at this poll, which I overall think is actually good news.
Every poll needs to be taken with four different grains of salt beyond just looking at the party vote percentages. They are:
Let’s take them one by one.
Trend Lines
Looking at just one poll in isolation is a bad idea, for several reasons. Firstly, as some have noted, polls jump up and down. Plotting polling on a graph looks a little like measuring an earthquake. This is at least partly due to margins of error, but it’s why more serious poll-watchers take a “poll of polls” approach to measuring political changes. Combine this with notations of significant political events and it makes it quite reliable to view both general trends and what campaigning tactics have been effective.
It is incorrect to say that these jumps up and down indicate “unreliability” in and of themselves. What really indicates unreliability is jumps that exceed the margin of error with no corresponding trend or explanation, and if you get such a jump more than one time in twenty1.
Overall, the trendline for polling since John Key has resigned has Labour and the Greens heading upwards when taken together, and National heading downwards.
Margins of Error and Undecided Voters
Lets look for a moment at the results of the Colmar Brunton poll. They were:
National: 47% (N/C) (about 58 seats)
Labour: 24% (-3%) (about 29 seats)
Greens: 15% (+4%) (about 18 seats)
New Zealand First: 11% (N/C) (about 14 seats)
TOP: 2% (N/C) (below threshold)
Māori Party: 1% (-1%) (single electorate seat)
ACT/UF/Mana: Let’s assume they win their electorates.
Majority: 62 seats.
Bottom line if this poll were the general election result: NZ First decides the government, with Mana and Māori party also needed for a left-wing coalition, a difficult proposition.
Now, you’ll frequently hear that the maximum margin of error for a poll like this is 3.1%. This number will differ based on the number of people polled, but almost always the goal for a polling company is to have 1,000 respondents, which lets them have a 95% chance of a 3.1% maximum margin of error.
That 3.1% doesn’t mean that any 3.1% of the results could be misallocated. What it means is, when you’re trying to draw a conclusion as to how big two mutually exclusive groups are, if each group is sized 50%, your numbers could be out by up to 3.1%. It will be less at smaller levels of support, such as Labour Voters vs Non-Labour voters, or ACT voters vs Non-ACT voters.
So, responsibly reported, the results start to look more like this:
National: 44%-50% (-3%/+3%)
Labour: 21.5%-26.5% (-5.5%/-0.5%, 2.5% MoE)
Greens: 13%-17% (+2%/+6%, 2% MoE)
New Zealand First: 9%-13% (-2%/+2%)
TOP: 1.4%-2.6% (-0.6%/+0.6%)
Māori Party: 0.7%-1.3% (-1.3%/-0.7%, 0.3% MoE)
(These are rounded, to the nearest half percent for over-threshold parties, and to the nearest per mille for small parties. It’s very difficult to generate a seat estimate for these because we can’t actually be sure it’ll add up to 100%)
Believe it or not, none of these changes are large enough to be 100% sure that the change from the previous Colmar Brunton poll to this poll is down to anything other than fluctuation within the margin of error. (you need a huge leap where neither margin “touches” the other to do that) They are big enough that Labour’s, the Greens’, and the Māori Party’s results likely are down to more than that, but we don’t know for sure, as the margins of error for each poll need to have no overlap at all for that to happen. It’s only if later polls back this one up that we can really draw a conclusion.
When looking at the margin of error, you should also remember that both the minimum and maximum bounds of that margin are very unlikely. Think as if there’s a normal curve inside that margin and you’ll get the idea. So National are very likely to be within 2% in either direction, and there are at least even odds they’re within 1% of the polled figure, but it’s still possible that the poll is up to 3% off.
Colmar hasn’t posted their full results yet, but they do tweet their undecided numbers. Adjusting to exclude people who refused to respond sufficiently, those numbers are 16.7%, or to put in the margins of error, there are only between 81% and 85.6% of voters polled who have actually decided who they’ll give their Party Vote to. That means there is a huge potential for late voters to decide this election, and if past experience is correct, those late deciders are likely to break for New Zealand First, Labour, and National.
Coalition arrangements and electorate seats
A lot of poll-watching reporting talks about likely coalition arrangements. The only definite things we should be assuming this stage is that the Greens and Labour are going to co-operate, and that United Future, ACT, and National are going to be co-operating, electorate votes permitting. We can reasonably assume that if Mana are returned to Parliament, Hone will refuse to co-operate with National, but won’t necessarily support Labour. And the Māori Party and New Zealand First have both refused to say, or give any reliable criteria for who they will decide to give preference to in any coalition talks, and any speculation on the topic is just that.
Even this poll, which trumpets itself as a “historic low” for Labour, fundamentally doesn’t change the coalition maths if we assume it will accurately depict the general election: (which it won’t, no poll has ever done better than getting within its margin of error, and some don’t even manage that) Winston is highly likely to control who can form a government.
Beyond that, nobody polls electorate seats, and they frequently change hands in unexpected ways for minor parties. There is no guarantee, even running against Greg O’Connor, (which theoretically may cancel out the advantage of the Greens withdrawing Tane Woodley from the race) that Peter Dunne will retain Ōhāriu. There is no guarantee Te Ururoa Flavell will retain Waiariki, and there is no guarantee that Kelvin Davis won’t lose Te Tai Tokerau back to Hone Harawira now that a certain German albatross is out from around his neck and the Māori Party has agreed not to contest that electorate. There’s no guarantee that even if the Māori Party lose Waiariki, that Marama Fox won’t manage to take Ikaroa-Rāwhiti, now that she’s not also fighting the Mana Party there. There’s even no guarantee that Raf Manji won’t manage to steal Ilam from Gerry Brownlee and remove the 120th List seat from Parliament2. The only information we have on these contests is guesses, and nobody is making an effort to figure this out before the election. (I don’t know why, a scoop on What’s Likely To Happen In Ōhāriu is far more valuable media fodder than the latest party vote poll, and people will be just as interested in Te Tai Tokerau)
In short, assuming previous electorate winners will retain their seats is just as wrong an assumption as assuming that all minor party challengers will win theirs.
Lean
And this is where we get to why this is actually a good result.
Different polling companies have different methods, reach different audiences, and therefore have different factors that might lean the results one way or the other compared to an actual general election. Colmar Brunton, for instance, doesn’t poll cellphones to my knowledge. They ask questions in different ways, decide if someone is an undecided voter in different ways, and talk to different numbers of respondents. All of their methods tend to be adequate practice in the industry, but that’s not to say many of them couldn’t improve.
While generally all well-intentioned, these actually influence whether a given party performs better in their poll than in elections (“over-polls”) or the reverse. (“under-polls”) There are general trends for two of the parties when compared to the poll of polls, but compared to other New Zealand polls relative to the average of all polls, and looking at the trend of their polling compared to the 2014 election results, Colmar Brunton tends to do the following:
This isn’t to say that their performance in the 2017 election will continue to show that trend, but it means you should be sceptical of Colmar Brunton offering high results for National or low results for opposition parties, and likewise, certain other companies, like say, Roy Morgan, might have the opposite trends in how their responses tend to favour certain parties. This means you should really look at Colmar Brunton to confirm if drops for National’s support in Roy Morgan are likely to be real, and at Roy Morgan for whether drops in Labour’s support in the Colmar Brunton polls are.
To illustrate, let’s look at the most recent Roy Morgan:
National: 40%-46% (about 53 seats)
Labour: 27.7%-33.3% (about 37 seats)
Green Party: 11.4%-15.6% (about 17 seats)
New Zealand First: 6.3%-9.7% (about 10 seats)
Other: (incl. TOP) 1.7%-3.3% (below threshold)
Māori Party: 1%-2% (about 2 seats)
ACT: 0.7%-1.3% (single electorate seat)
Now, I expect that National’s real level of support is actually somewhere between the 40% minimum (or 43% estimate) of Roy Morgan and the 50% maximum (or 47% estimate) of Colmar Brunton. But it doesn’t serve us to panic. CB claimed no change, so National is more likely somewhere around 45% right now. That’s about 55 seats. That likely means they will need New Zealand First to govern, even if for some reason the Māori Party’s constituents tell them to work with National before Labour. Labour’s real job leading up to the campaign is to pivot in a way similar to what the Greens seem to have just done, whatever direction that pivot actually goes, and reclaim 4% or so of the vote. (Remember, Labour may be down 3 points in this CB poll, but the Greens were up 4- that means the whole coalition is likely up 1 point.
This poll, confusingly given both the coverage and Little’s reaction this morning, is good news, as it gets us closer to a position where New Zealand First will look stupid if it works with National, or even where Labour and the Greens don’t need them, and can talk to the Māori and/or Mana Parties about ensuring they can run a minority government, assuming they get their second MP and Mana gets Te Tai Tokerau)
Labour and the Greens together in the early July Roy Morgan, before Metiria’s welfare policy announcement, were polling 44%, (+4.5%) . In the Colmar Brunton, they’re polling 39%, (+1%) so even though I’m waiting for another couple polls to celebrate, I’m remembering the important thing: Labour and the Greens, taken together, are trending upwards. The Greens, the more progressive party of the two, is also doing well within that pre-election coalition. We might need to rely on New Zealand First to change the government, and I really don’t like that, but I’ll take it if I have to, especially if Labour makes sure they don’t get in any of their more racist policies.
1 Even a completely unflawed polling method should still throw up an anomaly or “rogue poll” 5% of the time. Roy Morgan, the company frequently cited as unreliable, has had one identifiable rogue in all the time I’ve tracked it, with Colmar Brunton having had two, for instance, while delivering less polls and at irregular intervals outside election campaigns. I generally trust that Roy Morgan is the most accurate company polling in New Zealand, although I completely ignore their commentary on their numbers because it’s complete fiction. This is because it both has some superior methodology, (such as polling cellphones) and because it did very well at predicting the trend for the 2014 election.
2 And that’s a whole other mess to explain. Independents don’t get overhang seats, they remove a seat from the list calculation altogether. So effectively whoever will have won the 120th list seat in Parliament has it “stolen” from them by the independent candidate. It’s most likely to be either National or Labour, but there’s no guarantee. It could be any list candidate who loses out if Raf Manji wins, and he would side with National at least as much as Peter Dunne does, so if Labour or the Greens lose a list seat to him, it’s really bad news. And there’s no way to know until every single vote is in. Why we decided to do things this way just for independents boggles the mind, frankly, but I suppose it was considered more likely that a large number of independents would win electorates than that you’d end up with a large electorate-based party.
Powered by WPtouch Mobile Suite for WordPress