The poll gods must be crazy

Written By: - Date published: 10:07 am, February 25th, 2013 - 43 comments
Categories: polls - Tags:

nation in the polls
(the Herald poll is italicised because it’s much older than the others)

So, what can we say about this data?

First, it’s nearly impossible for all those pre-election polls to have arisen from proper random samples of the population that gave National 47.3% of the vote – either the polls are (or at least were) systemically over-counting National, or the voting intentions of the population changed rather dramatically between when those polls were conducted and a few days later when the election occurred. The second option is certainly a big part of it – remember the teapot tapes, the police raids on media, and the rise of Winston – but, this is the last time that we had a real-world sanity check on the polls, and it wasn’t good news for the polls – most of them were out by more than the margin of error.

Second, with the exception of TV3’s weird nonsense result last night, all the polls show National down. In their wildly inconsistent way, the polls seem to be trying to say that fewer people are supporting National. The Right has a one-seat majority in this Parliament, reliant on its zombie parties in Ohariu and Epsom and less than 5,000 votes (0.2%) for its 59th seat.

Third, the spread of results and the spread of declines that the different polls are showing is too variable – we shouldn’t be seeing such a wide range if they’re all properly random samples. Coupled with how disastrously wrong they were at the last election, and the wild, inexplicable jumps that some of the polls show (5% and 4.4% jumps for National, from TV1 and TV3 respectively, really? In what world did 1 in 20 people go pro-National in the last 2 months?) it looks like some of the polls are broken.

Fourth, having just pointed out how thin National’s hold on power is, the polls are still concerning for the Left. NZF could well support National in 2014, Shearer could easily lose 5% in the debates. Labour-Greens need not only to be out polling National, as they are in 2 of the 4 recent polls, but to be comfortably outpolling them to the point where there’s a debates buffer and National+NZF+zombies clearly won’t make a majority. That means we want to see National hitting 40%-43% in polls consistently and Labour+Greens at 45%-48% again and again. We’re not there yet.

Fifth, turning out the vote will be vital. Can John Key convince voters to show up on the promise that the brighter future is still just around the corner? Can David Shearer inspire people to make him the next Prime Minister? Whose potential voters will stay home instead because they just don’t believe in them?

43 comments on “The poll gods must be crazy ”

  • Comments are now closed

  1. Raa 1
    25 February 2013 at 10:25 am

    I hate to be boring, but there is only one poll which counts.

    I think there is a lot of merit in compulsory voting legislation, as in Australia,
    for a better reflection of Vox Populi, rather than just of the denizens
    of Auckland North Shore, Epsom, Wellywood, and few other quiet gated vicinities.

    • bad12 1.1
      25 February 2013 at 10:46 am

      Yeah i tend to agree with you on the compulsory voting issue, i also cannot help but view the latest TV3/Reid Research poll as anything but propaganda,

      For the past 2 years at least, that particular poll has had the National Party polling over 50%, i would never suggest that the old dear that owns Reid Research could be a National Party member making a constant donation to that particular Party…

      • aerobubble 1.1.1
        25 February 2013 at 1:49 pm

        Compulsory voting is wrong, unethical, and leads to what has happen in Oz, party factions. If the PR debate told us anything is that slight less proportionality is about right. Australia’s system is
        PR gone mad. Compulsory voting removes the choice to vote and so make the choice of one of the two major parties more pressing and so give factions in parties more incentive to hoard power.
        I agree Dunne should not have a life job in parliament, I agree one seat should not give a party a proportional booster engine. And Winston may very well be right that lowering the 5% limit may just produce even more farcical parties where one MP makes racist comment and another drops off the list into the zombie bench seat.

        What’s really wrong with NZ system is the lack of a upper chamber to contain the right wing (never left wing) fads of political incumbents. Our politics is heavy right wing, our major newspaper is heavily right wing, hell our tax system is rigged to serve the richest.

        Any capitalism worthy of the name knows that a economy won’t be subtle, stable, sustainable
        and resilient when the leading commentators all revel in softening the free markets hand on executives, while making its blows fall harder on the less powerful. We cannot breed a zoological garden of capitalist winners, many zoo animals are now totally unsuited to return to the
        jungle (the equivalent free market for humans). We don’t nee more investment, we don’t need softly softly approaches to the executive classes, we don’t need protection of the status whiners winners who fester our economy, because look at their works, leaky homes, worker work place deaths, fraud, exodus of skilled labour, lack of a living wage, worker rights dismissed diminished (not to say that workers had to many rights in the past, only that the pedulum has no swung too far the other way and the reality of the world economy requires engaged employees who are well rewarded for their ability to buidl prosperity).

        Sorry, compulsory voting is a distraction, no wonder Labour are spent.

        • Matthew Whitehead 1.1.1.1
          25 February 2013 at 6:54 pm

          Australia’s system is not a proportional system, it is an electorate system that has slightly less disproportionate results than FPP.

          And rather than compulsory voting: I’d rather you get a tax credit or something similar for voting, and then give people the option to make a deliberate “no-vote” for the electorate and/or party votes. While it amounts to roughly the same thing as compulsory voting, there’s a big difference between giving you something and taking something away. (ie. fining you for not voting)

  3. wobble 3
    25 February 2013 at 10:49 am

    I think an empty coke bottle is the reason Labour are stuck in the early 30s. Should we through it off the cliff at Slope Point?

    That might solve the problem.

  4. Brian 4
    25 February 2013 at 10:50 am

    Curious how the Herald always reports polls favourable for National in headline articles and others are nowhere to be seen….

    • scotty 4.1
      25 February 2013 at 11:07 am

      Yeah ,the Right friendly media, are happy again.
      Happy their personal buy in to JKs BS is reaffirmed by a sample of a 1000 Tv watching, conservatives ,with a land line.

  5. Matthew 5
    25 February 2013 at 11:01 am

    I think the best way to interpret these latest tv3 poll results is ‘if it bleeds it leads’. News organizations will deliberately skew a poll so they can advertise with the soundbite & get people to watch to see what the hell went wrong.
    Wit a shortage of actual natural disasters to report on, they come up with a man-made one to keep the punters glued to the box.
    I cant see any other way its possible.

    • Lanthanide 5.1
      25 February 2013 at 11:20 am

      I really don’t believe a media company would deliberately skew a poll for ratings. As soon as anyone gets a whiff of that, your credibility is mud and no one will trust your polls again.

      • Matthew 5.1.1
        25 February 2013 at 11:27 am

        you mean like tv1 colmar-brunton polls? Honestly, does anyone here, the fairly educated & political, put any weight behind polls conducted by news organizations? & i dont mean skew as in fake, i mean skew as in use poor method to produce the ‘right’ result (pun intended)

        • Lanthanide 5.1.1.1
          25 February 2013 at 11:34 am

          I don’t think they deliberately use a poor method either.

          They might use a poor method, but I don’t think the reason for the poor method would be to generate a particular type of result. I can imagine a poor method might be chosen because it was cheaper, or they were just ignorant, however.

          • bad12 5.1.1.1.1
            25 February 2013 at 12:18 pm

            The old dear that owns Reid Research has been in that game for 27 years, yet her polling organization for at least the past 2 years has polled National with at least 50% of the vote,

            i hardly think someone with 27 years experience gets it so completely wrong over such an extended period by ‘accident’,

            As far as turning the reputation to mud goes, that depends upon how deeply you think that the average TV watcher thinks about those polls and in particular the people that ‘own’ such polling companies,

            i would suggest that even here on the Standard there is enough evidence to suggest that people see ‘the numbers’ but give little consideration, if any, to exactly ‘who’ provides such numbers and whether or not the provision of such numbers has simply become a game of ‘politics’…

            • Lanthanide 5.1.1.1.1.1
              25 February 2013 at 1:01 pm

              “i hardly think someone with 27 years experience gets it so completely wrong over such an extended period by ‘accident’,”

              Go read some of Nate Silver’s analysis of the 2012 Presidental election polling.

              One of his points is that although some of the polling places look to be consistently out of line with others, it doesn’t mean there’s any deliberate intent in there. Rather, it’s another example of statistical variance: if you get enough polling organisations together, that each follow their own process in good faith, you will naturally see a few of them appear to be outliers, while most of them group around a common mid-point.

  6. Lanthanide 6
    25 February 2013 at 11:25 am

    I think in general, people don’t understand what “margin of error” means, especially the talking heads that repeat the news who are always so careful to say what it was, before they move on to the next story.

    What it means in a nutshell, is:
    1. assuming the sampling accurately reflects the population, and
    2. that all proper and recognized statistical methods have been followed accurately, that
    3. 95% of the time, the published figures will be within the margin of error calculated compared to the true population at large

    James rightly questions #1 and to some extent #2 in this article. But the other point here is that the 95% in #3 actually isn’t particularly precise: it means 1 out of 20 polls will be off by more than the margin of error, but given 20 polls taken sequentially over a period of time we still wouldn’t actually be able to pick which one was wrong.

  7. bad12 7
    25 February 2013 at 11:46 am

    Here’s another scenario, should both John(the convicted) Banks lose Epsom to National’s Goldsmith by having NOT been given another highly public chimps tea party, AND, the National candidate in the ‘Hairdo from Ohariu’s electorate win over incumbent Peter Dunne while National’s Party vote % over the whole country remains around 45-47%,

    Then National will lose the 2014 election simply by having 2 more electorate MP’s and thus having to drop 2 List members off of their Party List,

    This of course would rely on neither NZFirst or the Maori Party being in the next Parliament, (not an impossibility considering Prosser’s recent help in leveraging a % of the left wing of NZfirst away from that party),

  8. AsleepWhileWalking 8
    25 February 2013 at 11:56 am

    The Nerald headline screams out to the masses, “National’s 51pc leaves the rest far behind…”

    http://www.nzherald.co.nz/nz/news/article.cfm?c_id=1&objectid=10867501

  9. mikesh 9
    25 February 2013 at 12:36 pm

    I have recently watched Adam Curtis’s documentary series The Century of the Self. In discussing John Major’s election victory it says that many people, mainly Labour supporters, were telling the pollsters that they would vote Labour while secretly planning to vote Conservative. This conclusion was based on research with focus groups and it explains why Major won despite polls predicting a clear Labour victory. Perhaps the same thing is happening here, with people telling pollsters that they would vote National, believing it to be the popular choice,, while intending to vote Labour, Greens or NZF.

    • The Gormless Fool formerly known as Oleolebiscuitbarrell 9.1
      25 February 2013 at 12:41 pm

      …on the basis that Labour is so uncool that no-one wants to admit to voting for them? Then their problems are significantly worse than I thought.

    • felixviper 9.2
      25 February 2013 at 7:11 pm

      Possibly mikesh.

      I reckon that’s why the Greens tend to poll a bit higher than they ever get.

      • Matthew Whitehead 9.2.1
        25 February 2013 at 8:09 pm

        Honestly that probably has to do with the small sampling problem to some degree. You’re likely to either underpoll or overpoll significantly as a party below about 20%, just due to the margins of error involved in how many people are polled in New Zealand.

        It is interesting though that the Greens pretty conistently overpoll now though, while most other parties above 1% underpoll.

        • Andrew 9.2.1.1
          27 February 2013 at 7:28 am

          The margin of error reduces the further a result gets from 50%. For example, in a random sample of 1000, the margin of error on a result of 3% is +/- 1.06 percentage points at the 95% confidence level, not 3.1 percentage points.

          Here, use this: http://www.rmpd.ca/en/calculators.php

  10. Tom Gould 10
    25 February 2013 at 12:52 pm

    The ‘analysis’ spun by Tory Paddy takes it as gospel that Banks and Dunne, and the three Maori, will ‘win’ their seats, which is far from certain and is nothing more than a wishful hunch, based on no data. Yet Winston is pronounced ‘gone-burger’ on the actual numbers? How do these big Tory chooks get away with this garbage?

    • Lanthanide 10.1
      25 February 2013 at 1:10 pm

      Because there is almost no direct polling of electorates in NZ (it only happens around election time, and only for a couple of interesting electorates), it is impossible to gauge whether electorate seats would be won by the parties that currently hold them or not. So the convention in the media is to assume all electorate seats will remain as they are, regardless of how ridiculous an assumption this is, and then report the results based on that.

      That’s why Mana are projected to be in parliament (which is a good bet), but NZ First isn’t, because on the results they have, NZ First won’t meet the threshold.

  11. George D 11
    25 February 2013 at 1:18 pm

    You realise that when they conduct a poll, they ask New Zealanders what they think?

    By implying that the results are “crazy”, you’re suggesting that NZers are crazy. That’s not very polite.

    • felixviper 11.1
      25 February 2013 at 1:28 pm

      The post is mostly about methodology, but you’d probably have to read it to know that.

    • Lanthanide 11.2
      25 February 2013 at 1:54 pm

      It’s crazy to suggest the results of this poll are truly representative of the population of NZ voters, given how out of step it is with other recent polls and the prevailing media reports on National’s handling of governing the country.

  12. Tim 12
    25 February 2013 at 1:41 pm

    An emergent industrial complex: Polling opinion.
    Just like other seizures of opportunity to make ‘money’ – especially that which has no actual value behind it – here we’re wasting time, emotion, stress, ill-will et al to a complete load of kaka

    JohnKy’s riding a 51% MAGIK carpet atm. Quick – call an election! I mean – I know Jonky’s Basically fick and an animal cunning operator, but even a rat up a drainpipe has enough intellect to know the value of the spin doctors and pollsters

  13. Roy 13
    25 February 2013 at 3:33 pm

    What is the point of constantly polling and reporting poll results? Are we supposed to think ‘Oh, the guys whi represent my preferences are losing, so I had better not vote for them’? Are voters really so shallow that they just want to side with the winners?

  14. Richard Down South 14
    25 February 2013 at 5:34 pm

    My question is are they asking the same people each time? im pretty sure theyre not, so if you ask 1000 people, and get 600 this time who would never vote left, itd look like a massive jump for Key

    • Colonial Viper 14.1
      26 February 2013 at 1:30 pm

      More like they ask 50 people and a 2 person change is a 4% jump for Key. Or for Shearer.

  15. georgecom 15
    25 February 2013 at 9:05 pm

    Some really bad economic and social news coming out this month, yet the results seemingly not yet being sheeted home to the Nats. I’d think a bit of poll error at play here myself.

    Maybe also something to do with the weather as well perhaps. This assume the average kiwi voter is maybe a little shallow or whimsical. Maybe this isn’t accurate at all.

    Unless you are a farmer or someone who relies on water for their livelihood or necessities of life (or a climate change denier struggling to substantiate your claims that all is fine climate wise), you are probably enjoying the extended brilliant weather.

    Liam Dann is saying that despite all the economic gloom he is enjoying days at the beach and blue skies. http://www.nzherald.co.nz/business/news/article.cfm?c_id=3&objectid=10867482
    Could there be a case of many kiwis who still have jobs enjoying the feel good factor of a hot summer? Will grotty weather of May and June, mixed with the economic gloom, start to see some pigeons roosting for National?

  16. Maggie May 16
    26 February 2013 at 1:23 pm

    Lets remember only landlines are used in polls so how many people does that block out of participating showing their choice or how well they think this government is doing.

The server will be getting hardware changes this evening starting at 10pm NZDT.
The site will be off line for some hours.