web analytics

Granny Herald: from flawed data, false conclusions

Written By: - Date published: 11:21 am, March 23rd, 2013 - 17 comments
Categories: john key, national, polls - Tags:

Three of the Herald’s political staff has a crack at trying to explain why National polled 48.5% in the latest Herald-Digipoll. Consensus seems to be that John Key’s an angel sent to earth to do very little but keep enough of the middle class happy enough who’ll never lose. May I humbly suggest an alternative theory – that the Herald-Digipoll is broken.

Danyl at the Dimpost has the most mathematically poll of polls that, crucially, is anchored in reality – ie the last election result. Whereas political commentators and pollsters seem to shamelessly ignore how badly their poll got it wrong at the last election when breathlessly reporting the latest numbers, the Dimpost poll of polls corrects for their error.

What the Dimpost poll shows is that:

a) the Herald Digi-poll consistently over measures National – for example, the final Herald poll before the last election had National on 50.9%, that’s outside the margin of error above the 47.3% they got. The final Herald poll in 2008 had National on 47.9%, they got 44.9%. Statistically, the Herald isn’t just missing the bullseye, it’s missing the dartboard altogether. Only the 3news poll is worse. (in the image below, the light blue line is the Herald trend, the dark one is the poll of polls trend).

Dimpost poll of polls highlighting Herald Digipoll

b) even the Herald poll shows that National has lost a significant amount of support. From a pre-election high of 54.2%, a final pre-election number of 50.9%, to 48.5% today. That’s a big drop. Adjust to the Herald’s 3% over-polling of National at the last two elections, and it signals National is at around 45% – in the range that the Dimpost poll of polls has it. Remember, the Right has a one-seat majority in the House. Losing 2-3% since the election, especially when Banks will never get back in, is no laughing matter for National. If it were to translate into an election result, it would mean National would need New Zealand First (or a very large wasted vote) to govern.

c) more regular polls, like the Roy Morgan show a clear downward trend for National. You can’t possibly draw a trend from the five data points in 17 months that the Herald provides because the statistical noise that results in variation between every poll can’t be sensibly separated from the trend when you have so little to work with. The Roy Morgan has polled 29 times in that period and its trend is clear. (incidentally, the Roy Morgan was also the most accurate for National in 2011).

roy morgan labour greens v nats 3

To look at the Herald poll and conclude that National is now more popular than at the election, you have to ignore the Herald’s record of over-polling National, the fact that National’s support in the Herald poll has fallen significantly since the election, and the fact that more reliable polling trends tell us the opposite.

Only if you make all these logical errors (and there’s a strong incentive to do so because our paper spent a lot of money on the poll, which means its going to be treated as gospel no matter what, goddammit), can you then conclude National’s rule continues unblemished thanks to Bonny Prince Grinny.

But when’s a few fallacies ever stopped Granny?

[Note: none of this should be taken as a ‘if these trends continue’ argument. One current accurate polling, the Left needs Winnie too making him Kingmaker and he would have much more incentive to go with National, and there’s the risk of a Shearer campaign meltdown too. Put National at 40% and I’ll be happier].

17 comments on “Granny Herald: from flawed data, false conclusions”

  1. Tiresias 1

    “Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: “There are three kinds of lies: lies, damned lies and statistics.”
    – Mark Twain’s Own Autobiography

    Presently Labour will only be picking up those sufficiently dismayed by National’s duplicities and incompetence not to want them in Government again. Labour will only start picking up genuine support when it starts presenting an alternative programme worth genuinely supporting, and six-weeks out from an election is far too late to start thinking of doing that.

  2. Bob Tambling 2

    The worry for me then is if the polling trends continue and it looks like Shearer can just get over the line the possibilty of changing the leader will further diminish and Shearer will fight the 2014 election.
    Unfortunetly during the ensuing campaign, Key and the Nats will crucify him and we will be left with a National Government or a very weak LAb /NZF /Green one term Goverment
    I wonder from a Green party prospective (and a leftish pespective) , would it not be better if the Labour Party lost the next election with Shearer as Leader?
    If NZF play a part as may be likely, the Green can’t afford to go into a NZF /Lab/Green Gov.
    Shearer simply won’t cut the mustard and Peters is toxic. It would be a crap government and the well honed National party machine would again tear it to shreds

    The leaves the Greens in an interesting position, the lure of ministerial positions versus the pragmatic long term strategy of the Greens. A one term Lab/Nzf/Green Gov seems to serve no ones interest.
    Labour needs to have a strong leadership team and comitted activists , The reality is they have a diminishing group of activists and a crap leadership.
    The Greens on the other hand have a very strong leadership team and an ever growing group of activists ( let us not forget they are the second biggest party in Wellington).
    Labour? I can’t think of anyone in the present caucus who could take most of the country with them in an election. They need a good clean out and new blood. They don’t appear to be listening to that line of argument.
    In conclusion:
    I am committed to working very hard to getting as many Green MP’s elected as I can in 2014. I am not so sure though if I want those MP’s in a Government with the current infighting mob of the Lab party who it seems “could not pour piss out of a boot if the directions were on the heel”.

  3. tc 4

    The more granny and her shills like Johnno, o’shillivan, rortney etc keep banging the NACT drum the more irrelevant and bias they show themselves to be.

    Labour needs to grow a pair and get out there now however with the mallarfia and their front man it seems it’ll come down to the hollowmen blowing it more so than Labour winning it.

    They could easily blag another term if they weren’t such psychos given the current state of the opposition, thankfully they are what they are.

  4. ghostrider888 5

    article in todays Herald acknowledges the popularity of Key is slipping

  5. pete 6

    “Only the 3news poll is worse.”

    That’s not quite true: all five polls overestimate National’s election results, and the Digipoll isn’t too much worse than Fairfax or Roy Morgan. One News Colmar Brunton, as well as 3 News Reid Research both overestimate by more than the Herald.

    • lprent 6.1

      Points to a systematic flaw in their sampling doesn’t it…

      • pete 6.1.1

        That’s one possible explanation, although some have theorised that National lose a lot of votes close to the election once people actually start looking at their policies.

        There’s also a theory that NZF underpolls because their voters are embarrassed to admit that to callers.

        • lprent 6.1.1.1

          Possible. But less likely than the simpler explanation.

          Canvassing companies are using listed landlines*. When I was analyzing some somewhat larger samples of both door-knock canvassing and phone canvassing across several electorates, there was a statistically significant correlation between people who canvassed right and left and if they had a listed landline.

          For that matter there was an even stronger correlation between age group and having a listed land line – and it looks like if people don’t have a listed phone line in their mid 20’s then they never get one.

          * There is a common myth that goes around about auto-dialing random numbers. Asked a few people who work at or have worked at polling companies. It doesn’t happen and a few minutes thought about number ranges will give an explanation about why. It is cheaper to buy/get a phone list.

  6. Chris 7

    So,will Granny Herald be ramping up lots of little good news, happy smiley,relaxed all singing, dancing reports/photos of our poor excuse for a pm then. Got to keep him relevant.

  7. Centaur 8

    Give Key *some* credit. He may decide to call it quits. It is not as if he actually needs this gig ..

  8. Murray Olsen 9

    I’m not even sure that the margins of error as reported are very meaningful at all. From what I know, they seem to rely on unbiased sampling and the validity of the Central Limit Theorem, which then leads to a percentage presumably taken from 100 times √N/N. The concern about landlines is just one thing that could skew this.

    • lprent 9.1

      They do sample distribution normalising for many of the other factors like age, gender and geological distribution. They don’t adjust for factors like people working during the periods that they phone.

      But they don’t publish their techniques and criteria so it is pretty hard to see why there if it was unbiased sampling there would be any differences between the polls within the confidence limits. But there clearly is.

      • Centaur 9.1.1

        lprent: geological distribution ?
        Such as, where there are proven mineral resources ?

        • lprent 9.1.1.1

          I blame the iPad, it’s overly smart autocorrect, and that I use Geological at least 10x more often than Geographical.

    • Lanthanide 9.2

      No, they’re not. Even assuming they are following absolutely perfect statistical methods and the statistical methods work as advertised by the literature, the margin of error they’re reporting is presumably the 95% confidence interval. Which means 1 out of 20 polls are going to be wrong, but we won’t ever be able to tell which one without some other corroborating information (eg other polls taken at the same time).

      That’s assuming they’re using perfect methods, which of course they aren’t.

Recent Comments

Recent Posts

  • EY: TPP stamp duties on foreigners may have to apply to Kiwis
    The Government’s claim that a TPP-enabled tax on foreign buyers would amount to a ban has been exposed as folly by tax experts, who say that in most cases a tax would apply to Kiwi buyers too, says Labour’s Trade ...
    2 hours ago
  • Project 300 short on facts
    A Minister’s pet scheme to employ 300 disabled people in Christchurch seems to be short on facts, says Labour’s Disability Issues spokesperson Poto Williams.  “Nicky Wagner cannot provide solid evidence to show that her much vaunted Project 300 has actually ...
    2 hours ago
  • Who are they going to call?
    A cry for help from New Zealand’s longest-running crisis line highlights chronic underfunding of the sector by the Government, Labour’s Health spokesperson Annette King says. “Lifeline is THE go-to helpline for people in crisis, taking up to 180,000 calls each ...
    8 hours ago
  • Five months too long for homeless to wait
    New figures revealing homeless people registered with Work and Income are waiting an average of 155 days to be housed shows the Government is totally overwhelmed by the housing crisis, Labour’s Social Development spokesperson Carmel Sepuloni says. “What’s worse is ...
    23 hours ago
  • Minister in cloud cuckoo land
    Hekia Parata needs a very big reality check if she truly believes every parent has the choice of sending their child to a private school, Labour’s Education spokesperson Chris Hipkins says. Questioned in the House today about plans to pump ...
    1 day ago
  • Convention centre failure means years of uncertainty for CBD
    The failure of Gerry Brownlee’s planned convention centre deal leaves Christchurch facing uncertainty about when activity will be restored to the CBD, says Labour’s Canterbury spokesperson Megan Woods. “As one of the CBD’s major anchor projects, the convention centre complex ...
    1 day ago
  • PCE proves water quality still deteriorating
    The PCE State of the Environment Report shows that river water quality is continuing to get worse across large parts of New Zealand, says Labour’s Environment and Water spokesperson David Parker. “Water quality has deteriorated in Canterbury, Central Otago, Auckland, ...
    1 day ago
  • Families with new babies victims of today’s veto
    Families with new babies are the victims of an historical “first” for the New Zealand Parliament today. “For the first time ever, a Bill will have a third reading debate and no vote will be taken at the end because ...
    1 day ago
  • Crime on the rise…again!
    The Police Minister’s contention that Police have enough resources to meet the expectations of New Zealand communities is not reflected in the Police’s own statistics, says Labour’s Police spokesperson Stuart Nash.  “Yet again, reported burglaries have increased in every region ...
    1 day ago
  • Private schools beneficiaries of extra cash
    Plans to give more taxpayer money to private schools at a time when state schools are struggling to make ends meet says everything about the National Government’s twisted priorities, Labour’s Education spokesperson Chris Hipkins says. “Not only did this year’s ...
    2 days ago
  • Inequality getting worse under National
    Inequality is getting worse under National with almost 60 per cent of the wealth in this country concentrated in the hands of the top 10 per cent according to Statistics NZ figures released today, says Labour’s Finance spokesperson Grant Robertson. ...
    2 days ago
  • Government freezes elderly out of insulation subsidy
    Government cuts to the Warm Up New Zealand insulation subsidy means it will now only be available for rental properties and could leave many elderly homeowners cold this winter, Opposition Leader Andrew Little says. “In this year’s Budget the Government ...
    3 days ago
  • Shewan report delivers rebuke to National
    John Shewan’s report into foreign trusts is a rebuke to John Key and the National Party who have protected an industry that has damaged New Zealand’s reputation, says Labour’s Finance spokesperson Grant Robertson. “Three years ago the Inland Revenue Department ...
    3 days ago
  • Auckland Airport rail analysis must be made public
    The Government should publicly release its detailed analysis of rail to Auckland Airport before it closes off options, so the public can have an informed debate, says Labour’s Auckland Issues spokesperson Phil Twyford. The Transport Agency today said it is ...
    3 days ago
  • Minister approved OIO consent despite death and investigations
    Louise Upston must say if she knew Intueri was being prosecuted for the death of a student and under a funding investigation when she approved its overseas investment consent to buy another education provider, says Labour’s Land Information and Associate ...
    4 days ago
  • Brexit vote costs NZ effective EU voice
    Despite being extremely close the result of the referendum in Britain reflects the majority voice, Labour’s Foreign Affairs spokesperson David Shearer says. “While we respect the decision to leave the EU, it goes without saying the move will usher in ...
    6 days ago
  • Pasifika Education Centre doomed
    The Pasifika Education Centre appears doomed to close down this December, says Labour’s Pacific Island Affairs spokesperson Su’a William Sio  “In a written question I asked the Minister whether he would put a bid in for more money. His answer ...
    6 days ago
  • Onetai Station review a shameful whitewash
    A report released today on the Overseas Investment Office’s (OIO) good character test is a whitewash that does nothing to improve New Zealand’s overseas investment regime, says Labour’s Land Information spokesperson David Cunliffe. “The review of the good character test ...
    6 days ago
  • We need a national strategy to end homelessness now
    Long before I entered Parliament, housing and homelessness were issues dear to my heart. I know from personal experience just how hard it is to find an affordable home in Auckland. In my maiden speech, I talked about how when ...
    GreensBy Marama Davidson
    6 days ago
  • Capital feels a chill economic wind
      Wellington is on the cusp of recession with a sharp fall in economic confidence in the latest Westpac McDermott Miller confidence survey, says Labour’s Regional Development spokesperson David Clark.  “Economic confidence amongst Wellingtonians has dropped 12% in the past ...
    6 days ago
  • Dive school rort took six years to dredge up
    News that yet another private training establishment (PTE) has rorted the Government’s tertiary funding system since 2009 shows that Steven Joyce has no control of the sector, says Labour’s Associate Education (Tertiary) spokesperson David Cunliffe. “Like Agribusiness Training and Taratahi, ...
    1 week ago
  • National’s housing crisis hitting renters hard
    National’s ongoing housing crisis is causing massive rental increases, with Auckland renters being hit the hardest, says Labour’s Housing spokesperson Phil Twyford. ...
    1 week ago
  • A Day with the PSA
    This week, along with Labour MP Kris Faafoi, I accepted an invitation to spend a day working alongside the good folk at the Public Service Association in Wellington. As the Workplace Relations and Safety spokesperson for the Greens, I was ...
    GreensBy Denise Roche
    1 week ago
  • A Day with the PSA
    This week, along with Labour MP Kris Faafoi, I accepted an invitation to spend a day working alongside the good folk at the Public Service Association in Wellington. As the Workplace Relations and Safety spokesperson for the Greens, I was ...
    GreensBy Denise Roche
    1 week ago
  • Government holds Northland back
    New information shows Northland remains the most economically depressed region in New Zealand, says Labour’s Regional Development spokesperson David Clark. “The latest Westpac McDermott Miller regional survey found that more Northlanders believe their local economy will deteriorate this year than ...
    1 week ago
  • Rebstock report into MFAT leaks a disgrace
    An Ombudsman’s report on the Paul Rebstock investigation into MFAT leaks shows the two diplomats at the centre of the case were treated disgracefully, says Labour’s State Services spokesperson Kris Faafoi.  “The Ombudsman says one of the diplomats Derek Leask ...
    1 week ago
  • More families forced to turn to food banks for meals
    Increasing numbers of families are having to go to food banks just to put a meal on the table, according to a new report that should shame the Government into action, says Labour’s Social Development spokesperson Carmel Sepuloni. ...
    1 week ago
  • We have a housing emergency in New Zealand
    Auckland, New Zealand, where house prices have risen 20 percent in the last year alone We have a housing emergency in New Zealand.  Like many people we are ashamed and angry that in a wealthy country like ours, we have ...
    GreensBy Eugenie Sage
    1 week ago
  • We have a housing emergency in New Zealand
    Auckland, New Zealand, where house prices have risen 20 percent in the last year alone We have a housing emergency in New Zealand.  Like many people we are ashamed and angry that in a wealthy country like ours, we have ...
    GreensBy Eugenie Sage
    1 week ago
  • Aussie reforms signal trouble ahead for school funding plan
    Plans by the Government to return to bulk funding are likely to see increased class sizes and schools most in need missing out on much-needed resources, Labour’s Acting Education spokesperson Grant Robertson says. “The signaled return to bulk funding is ...
    1 week ago
  • Toxic Sites – the down low on the go slow
    In  2011, I negotiated an agreement with the National Government to advance work on cleaning up contaminated sites across the country. This included establishing a National Register of the ten worst sites where the creators of the problem could not ...
    GreensBy Catherine Delahunty
    1 week ago
  • Aucklanders face new motorway tax of up to $2500 a year
    The Government wants to tax Aucklanders thousands of dollars a year just to use the motorway network, says Labour’s Auckland Issues spokesperson Phil Twyford. “Officials estimate the average city commute is 11.8km. This means for the average Aucklander commuting five ...
    1 week ago
  • 15 corrupt bank managers identified in student fraud
    New information show 15 bank managers in India have been identified by Immigration New Zealand as presenting fraudulent documents on behalf of foreign students studying here, Labour’s Immigration spokesperson Iain Lees-Galloway says. “Documents obtained by Labour under the Official Information ...
    1 week ago
  • National leaves Kiwi savers the most vulnerable in OECD
    News last week that Israel’s Finance Minister will insure savers’ bank deposits means New Zealand will be left as the only country in the OECD that has no deposit insurance to protect savers’ funds should a bank fail. Most Kiwis ...
    GreensBy James Shaw
    1 week ago
  • Comprehensive plan for future of work needed
    A Massey University study showing many New Zealanders are unaware of the increasing role of automation in their workplace, highlights the need for a comprehensive plan for the future of work, says Grant Robertson, Chair of Labour’s Future of Work ...
    1 week ago
  • Another National Government failure: 90 day work trials
    On Friday last week, the Treasury released a report by MOTU economic consultants into the effectiveness of the controversial 90-day work trial legislation. The report found that there was “no evidence that the policy affected the number of hires by ...
    GreensBy Denise Roche
    1 week ago
  • Iraq mission extension case not made
    The Prime Minister has not made the case for extending the Iraq deployment another 18 months nor the expansion of their mission, says Opposition Leader Andrew Little.  “Labour originally opposed the deployment because the Iraqi Army’s track record was poor, ...
    1 week ago
  • Denial is a long river
    William Rolleston from Federated Farmers made the absurd claim on RNZ on Saturday that “we actually have very clean rivers”. This statement doesn’t represent the many farmers who know water quality is in big trouble and are working to clean ...
    GreensBy Catherine Delahunty
    1 week ago
  • Denial is a long river
    William Rolleston from Federated Farmers made the absurd claim on RNZ on Saturday that “we actually have very clean rivers”. This statement doesn’t represent the many farmers who know water quality is in big trouble and are working to clean ...
    GreensBy Catherine Delahunty
    1 week ago
  • Melanoma deaths could be avoided by an early access scheme
      The tragic death of Dunedin’s Graeme Dore from advanced Melanoma underlines the cruelty of this Government in promising a treatment but delaying for months, says Labour’s Health Spokesperson Annette King.  “Graeme was diagnosed with Melanoma last year. He used ...
    1 week ago
  • Assessing the Defence White Paper
    The Government’s recently released Defence White Paper has raised questions again about New Zealand’s defence priorities, and in particular the level and nature of public funding on defensive capabilities. The Green Party has a longstanding belief that priority must be ...
    GreensBy Kennedy Graham
    1 week ago
  • Kiwis’ confidence drops again: Economy needs a boost
    Westpac’s consumer confidence survey has fallen for the seventh time in nine quarters, with middle income households ‘increasingly worried about where the economy is heading over the next few years’, says Labour’s Finance spokesperson Grant Robertson. “This survey is a ...
    1 week ago
  • Relocation grant simply kicks can down the road
    The response by state house tenants and social agencies to the Government’s rushed plan to shift families out of Auckland tells us what we already knew – this is no answer to the chronic housing shortage, Opposition Leader Andrew Little ...
    1 week ago
  • Peace hīkoi to Parihaka
    On Friday a Green crew walked with the peace hīkoi from Ōkato to Parihaka. Some of us were from Parliament and some were party members from Taranaki and further afield. It was a cloudy but gentle day and at one ...
    GreensBy Catherine Delahunty
    1 week ago
  • Children’s Commissioner right to worry about CYF transition
    The Government must listen to the Children’s Commissioner’s concerns that young people under CYF care could be ‘negatively impacted’ as the new agency’s reforms become reality, says Labour’s Children’s spokesperson Jacinda Ardern. “Dr Russell Wills has used the second annual ...
    1 week ago
  • Bill English exaggerates PPL costs to justify veto
    The Finance Minister has used trumped-up costings to justify a financial veto against parents having 26 weeks paid parental leave, says Labour MP Sue Moroney. “Bill English’s assertion on RNZ yesterday that the measure would cost an extra $280 million ...
    2 weeks ago
  • Government must refund overcharged motorists
    Labour is calling on the Government to refund motor registration fees to three-quarters of a million Kiwi motorists whose vehicles were wrongly classified under National’s shambolic ACC motor vehicle risk rating system, Labour’s ACC spokesperson Sue Moroney says.“Minister Kaye’s ridiculous ...
    2 weeks ago
  • 90-day work trials an unfair failure which must change
    A new Treasury report shows the Government’s 90-day trials haven’t helped businesses and are inherently unfair, Labour’s Workplace Relations spokesperson Iain Lees-Galloway says. “The Motu report found that 90-day trial periods had no impact on overall employment and did not ...
    2 weeks ago
  • Massey East houses a start but Nick Smith should think bigger
    The Massey East 196-home development is a start but the Government must think bigger if it is to end the housing crisis, Labour’s Housing spokesperson Phil Twyford says. “It is great the Government is finally realising it needs to build ...
    2 weeks ago
  • More changes needed to ensure fewer cases like Teina Pora’s
    Teina Pora spent 21 years behind bars for a crime he didn’t commit, shafted by a Police investigation that prioritised an investigator’s hunch over the pursuit of credible evidence. Yesterday’s announcement that the government is to pay him $2.5m in ...
    GreensBy David Clendon
    2 weeks ago

Public service advertisements by The Standard

Current CO2 level in the atmosphere