Bugger the polls

Anthony R0bins
12 years ago
Categories: election 2011, polls, spin
Tags: poll of polls, polls, rob salmond

Do polls reflect public opinion or do they shape it? Bit of both of course, but my guess is there’s more of the shaping effect than is really good for us.

I think NZ is way over-polled, and consequently polls are way over-discussed. For the last 3 years we’ve been bombarded with messages about National’s popularity. National are popular, it’s true, but in terms of specific numbers, the polls were wrong.

Individual polls are full of noise of course, and should never be trusted. But over time, averaging polls should (if their methodology is sound) give us an accurate picture. It has been claimed that “poll of polls” average measures have been accurate in the past. Not this time. Here’s Pundit’s poll of polls on the day before the election, and the corresponding analysis by Rob Salmond:

Final Poll of Polls Update

Five more polls this week. Four today. Little change. National can govern alone. Winston will not make it (but it could be close).

Five new polls have come out since we last updated our poll of polls. Four of them came in the last 24 hours. Together, they show National continuing to be able to govern alone, according to our latest estimates (available on the left hand side of the pundit front page). We estimate National will win 52% of the vote tomorrow, giving them 67 seats. There have been 57 polls released this year by the five firms we follow, and all 57 of them estimated National would win the seats to govern alone. Fifty of those also estimated that National had an absolute majority of the votes, too.

Got that citizens? 57 out of 57 polls (four of them just a day or so before the election) predict National with the seats to govern alone. Did that happen on the night? No. Let’s compare actual results with poll of polls:

		Actual	Polls
National Party	47.99	52.4
Labour Party	27.13	27.6
Green Party	10.62	11.3
NZF		 6.81	 3.3
Māori Party	 1.35	 1.5
ACT   		 1.07	 1.3
Mana		 1.00
United Future	 0.61	 0.2
Conservative 	 2.76

The error is not from Rob Salmond and the Pundit team, I’m sure their averaging methodology is fine. It’s the error in the underlying polls that’s the problem (57 out of 57 wrong). So let’s get something on record for 2014. The polls systematically over estimate National’s support to the tune of about 4%. They need to rethink their methods. Bugger the polls!