web analytics

Granny Herald: from flawed data, false conclusions

Written By: - Date published: 11:21 am, March 23rd, 2013 - 17 comments
Categories: john key, national, polls - Tags:

Three of the Herald’s political staff has a crack at trying to explain why National polled 48.5% in the latest Herald-Digipoll. Consensus seems to be that John Key’s an angel sent to earth to do very little but keep enough of the middle class happy enough who’ll never lose. May I humbly suggest an alternative theory – that the Herald-Digipoll is broken.

Danyl at the Dimpost has the most mathematically poll of polls that, crucially, is anchored in reality – ie the last election result. Whereas political commentators and pollsters seem to shamelessly ignore how badly their poll got it wrong at the last election when breathlessly reporting the latest numbers, the Dimpost poll of polls corrects for their error.

What the Dimpost poll shows is that:

a) the Herald Digi-poll consistently over measures National – for example, the final Herald poll before the last election had National on 50.9%, that’s outside the margin of error above the 47.3% they got. The final Herald poll in 2008 had National on 47.9%, they got 44.9%. Statistically, the Herald isn’t just missing the bullseye, it’s missing the dartboard altogether. Only the 3news poll is worse. (in the image below, the light blue line is the Herald trend, the dark one is the poll of polls trend).

Dimpost poll of polls highlighting Herald Digipoll

b) even the Herald poll shows that National has lost a significant amount of support. From a pre-election high of 54.2%, a final pre-election number of 50.9%, to 48.5% today. That’s a big drop. Adjust to the Herald’s 3% over-polling of National at the last two elections, and it signals National is at around 45% – in the range that the Dimpost poll of polls has it. Remember, the Right has a one-seat majority in the House. Losing 2-3% since the election, especially when Banks will never get back in, is no laughing matter for National. If it were to translate into an election result, it would mean National would need New Zealand First (or a very large wasted vote) to govern.

c) more regular polls, like the Roy Morgan show a clear downward trend for National. You can’t possibly draw a trend from the five data points in 17 months that the Herald provides because the statistical noise that results in variation between every poll can’t be sensibly separated from the trend when you have so little to work with. The Roy Morgan has polled 29 times in that period and its trend is clear. (incidentally, the Roy Morgan was also the most accurate for National in 2011).

roy morgan labour greens v nats 3

To look at the Herald poll and conclude that National is now more popular than at the election, you have to ignore the Herald’s record of over-polling National, the fact that National’s support in the Herald poll has fallen significantly since the election, and the fact that more reliable polling trends tell us the opposite.

Only if you make all these logical errors (and there’s a strong incentive to do so because our paper spent a lot of money on the poll, which means its going to be treated as gospel no matter what, goddammit), can you then conclude National’s rule continues unblemished thanks to Bonny Prince Grinny.

But when’s a few fallacies ever stopped Granny?

[Note: none of this should be taken as a ‘if these trends continue’ argument. One current accurate polling, the Left needs Winnie too making him Kingmaker and he would have much more incentive to go with National, and there’s the risk of a Shearer campaign meltdown too. Put National at 40% and I’ll be happier].


17 comments on “Granny Herald: from flawed data, false conclusions”

  1. Tiresias 1

    “Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: “There are three kinds of lies: lies, damned lies and statistics.”
    – Mark Twain’s Own Autobiography

    Presently Labour will only be picking up those sufficiently dismayed by National’s duplicities and incompetence not to want them in Government again. Labour will only start picking up genuine support when it starts presenting an alternative programme worth genuinely supporting, and six-weeks out from an election is far too late to start thinking of doing that.

  2. Bob Tambling 2

    The worry for me then is if the polling trends continue and it looks like Shearer can just get over the line the possibilty of changing the leader will further diminish and Shearer will fight the 2014 election.
    Unfortunetly during the ensuing campaign, Key and the Nats will crucify him and we will be left with a National Government or a very weak LAb /NZF /Green one term Goverment
    I wonder from a Green party prospective (and a leftish pespective) , would it not be better if the Labour Party lost the next election with Shearer as Leader?
    If NZF play a part as may be likely, the Green can’t afford to go into a NZF /Lab/Green Gov.
    Shearer simply won’t cut the mustard and Peters is toxic. It would be a crap government and the well honed National party machine would again tear it to shreds

    The leaves the Greens in an interesting position, the lure of ministerial positions versus the pragmatic long term strategy of the Greens. A one term Lab/Nzf/Green Gov seems to serve no ones interest.
    Labour needs to have a strong leadership team and comitted activists , The reality is they have a diminishing group of activists and a crap leadership.
    The Greens on the other hand have a very strong leadership team and an ever growing group of activists ( let us not forget they are the second biggest party in Wellington).
    Labour? I can’t think of anyone in the present caucus who could take most of the country with them in an election. They need a good clean out and new blood. They don’t appear to be listening to that line of argument.
    In conclusion:
    I am committed to working very hard to getting as many Green MP’s elected as I can in 2014. I am not so sure though if I want those MP’s in a Government with the current infighting mob of the Lab party who it seems “could not pour piss out of a boot if the directions were on the heel”.

  3. tc 4

    The more granny and her shills like Johnno, o’shillivan, rortney etc keep banging the NACT drum the more irrelevant and bias they show themselves to be.

    Labour needs to grow a pair and get out there now however with the mallarfia and their front man it seems it’ll come down to the hollowmen blowing it more so than Labour winning it.

    They could easily blag another term if they weren’t such psychos given the current state of the opposition, thankfully they are what they are.

  4. ghostrider888 5

    article in todays Herald acknowledges the popularity of Key is slipping

  5. pete 6

    “Only the 3news poll is worse.”

    That’s not quite true: all five polls overestimate National’s election results, and the Digipoll isn’t too much worse than Fairfax or Roy Morgan. One News Colmar Brunton, as well as 3 News Reid Research both overestimate by more than the Herald.

    • lprent 6.1

      Points to a systematic flaw in their sampling doesn’t it…

      • pete 6.1.1

        That’s one possible explanation, although some have theorised that National lose a lot of votes close to the election once people actually start looking at their policies.

        There’s also a theory that NZF underpolls because their voters are embarrassed to admit that to callers.

        • lprent

          Possible. But less likely than the simpler explanation.

          Canvassing companies are using listed landlines*. When I was analyzing some somewhat larger samples of both door-knock canvassing and phone canvassing across several electorates, there was a statistically significant correlation between people who canvassed right and left and if they had a listed landline.

          For that matter there was an even stronger correlation between age group and having a listed land line – and it looks like if people don’t have a listed phone line in their mid 20’s then they never get one.

          * There is a common myth that goes around about auto-dialing random numbers. Asked a few people who work at or have worked at polling companies. It doesn’t happen and a few minutes thought about number ranges will give an explanation about why. It is cheaper to buy/get a phone list.

  6. Chris 7

    So,will Granny Herald be ramping up lots of little good news, happy smiley,relaxed all singing, dancing reports/photos of our poor excuse for a pm then. Got to keep him relevant.

  7. Centaur 8

    Give Key *some* credit. He may decide to call it quits. It is not as if he actually needs this gig ..

  8. Murray Olsen 9

    I’m not even sure that the margins of error as reported are very meaningful at all. From what I know, they seem to rely on unbiased sampling and the validity of the Central Limit Theorem, which then leads to a percentage presumably taken from 100 times √N/N. The concern about landlines is just one thing that could skew this.

    • lprent 9.1

      They do sample distribution normalising for many of the other factors like age, gender and geological distribution. They don’t adjust for factors like people working during the periods that they phone.

      But they don’t publish their techniques and criteria so it is pretty hard to see why there if it was unbiased sampling there would be any differences between the polls within the confidence limits. But there clearly is.

      • Centaur 9.1.1

        lprent: geological distribution ?
        Such as, where there are proven mineral resources ?

        • lprent

          I blame the iPad, it’s overly smart autocorrect, and that I use Geological at least 10x more often than Geographical.

    • Lanthanide 9.2

      No, they’re not. Even assuming they are following absolutely perfect statistical methods and the statistical methods work as advertised by the literature, the margin of error they’re reporting is presumably the 95% confidence interval. Which means 1 out of 20 polls are going to be wrong, but we won’t ever be able to tell which one without some other corroborating information (eg other polls taken at the same time).

      That’s assuming they’re using perfect methods, which of course they aren’t.

Recent Comments

Recent Posts


  • Freedom on the Seas – A gift from your friendly Government
    When the Government announced it would need to uphold public safety during the upcoming International Naval Review, you’d be forgiven for thinking it was talking about wearing life jackets. But actually, the Government is applying restrictions that stop people boating ...
    GreensBy Kennedy Graham
    16 hours ago
  • Time for Paula Bennett to front up on HNZ P Fiasco
      Social Housing Minister Paula Bennett needs to rein in Housing NZ and sort out the mess that’s been created by the organisation’s misuse of methamphetamine testing procedures, says Labour’s Housing spokesperson Phil Twyford.   ...
    1 day ago
  • A charge on plastic bags – debunking some myths
    The launch of my Members’ Bill last week, which would introduce a 15 cent charge on single-use plastic bags at the check-out, has generated a lot of comment on mainstream and social media. From The Paul Henry Show at the ...
    GreensBy Denise Roche
    3 days ago
  • National’s $1trillion property sandcastle
    The National government's failure to fix the housing crisis has seen the ballooning and unsustainable property market touch the $1 trillion mark, Labour’s Finance spokesperson Grant Robertson says. "Labour wants an economy that creates high wage work that is based ...
    3 days ago
  • Government failure on housing crisis drives Reserve Bank to add tools
    If the Government was delivering a comprehensive plan to fix the housing crisis, it is unlikely that the Reserve Bank would be continuing to pursue debt to income limits for lending for housing, says Labour’s Finance spokesperson Grant Robertson. ...
    4 days ago
  • International embarrassment for NZ likely over National’s failure to protect Maui dolphin
    New Zealanders who care about Maui dolphin should prepare to feel embarrassed: the Government is about to be put to shame on the international stage for its lack of action to protect Maui’s dolphin. The International Whaling Commissions’ 66th Biennial ...
    GreensBy Eugenie Sage
    6 days ago
  • Why don’t we spend $1b to keep people out of jail, rather than spending it on keeping them in?
    Earlier this week, Corrections Minister Judith Collins announced the government’s ‘solution’ to our burgeoning prison population. It seems that most, if not all, of Bill English’s hard-won surplus is going to disappear into another round of prison-building.  That must be ...
    GreensBy David Clendon
    1 week ago
  • PKE Ship Sent Packing – Not Too Soon
    It is appropriate that the palm kernel expeller (PKE) ship off Tauranga has been sent packing. For weeks I have been saying this ship needed to be sent away, but it seems as if MPI has been trying to find ...
    GreensBy Steffan Browning
    1 week ago
  • Do you #LoveSnow?
    I was a lucky kid. When I was about five or six my mum and auntie took me up to Whakapapa on Mt Ruapehu and taught me to ski. As a young kid I thought there was no bigger ...
    GreensBy James Shaw
    1 week ago
  • Awa Kairangi/Hutt River – Swimmable?
    On Thursday night I hosted a great swimmable rivers meeting organised by the local Greens in Heretaunga (Hutt Valley). It was great to see about 70 people attend and engage in the topic. We were welcomed by Te Atiawa representative ...
    GreensBy Catherine Delahunty
    1 week ago
  • September benefit figures disappointing
    The Government is out of touch with the reality that fewer people are going off the benefit and into employment or study, says Labour’s Social Development spokesperson Carmel Sepuloni.  “The quarterly benefit numbers for September are concerning. They show that ...
    1 week ago
  • MFAT officials refuse to back Prime Minister on Saudi sheep claims
    An Ombudsman’s interim decision released about the existence or otherwise of legal advice on the multimillion dollar Saudi sheep deal shows MFAT has failed to back up the Prime Minister’s claims on the matter, says Labour MP David Parker. “The ...
    1 week ago
  • Barry Coates on his first weeks in Parliament
    Week one in Parliament has been quite an occasion. I would like to share the experience. I had given up on the prospect of getting into Parliament before the election and had been enjoying the diverse work I was doing ...
    GreensBy Barry Coates
    1 week ago
  • Nats still planning to take Housing NZ dividend
    Housing New Zealand’s Statement of Performance Expectations shows that the National Government intends to pocket $237m from Housing New Zealand this year including a $54m “surplus distribution”, despite promises that dividends would stop, says Labour’s Housing spokesperson Phil Twyford. “After ...
    1 week ago
  • Parliament must restore democracy for Ecan
    Parliament has a chance to return full democracy to Canterbury with the drawing of a member’s bill that would replace the Government’s appointed commissioners with democratically elected councillors, says Labour’s Canterbury Spokesperson Megan Woods. “In 2010, the Government stripped Cantabrians ...
    1 week ago
  • Police struggle to hold the line in Northland
    Labour’s promise of a thousand extra police will go a long way to calming the fears of people in the North, says the MP for Te Tai Tokerau Kelvin Davis.  “Police are talking about the Northland towns of Kaitaia and ...
    1 week ago
  • Vote Sooty Shearwater/Tītī for Bird of the Year
    Sooty shearwater (Puffinus griseus) are amazing and deserve your vote in Forest and Bird’s Bird of the Year competition.  They make one of the longest known bird migrations, flying an annual round trip of 64,000 kms across the entire Pacific ...
    GreensBy Eugenie Sage
    1 week ago
  • Urgent action on agriculture emissions needed
    Immediate action is required to curb agricultural emissions is the loud and clear message from Climate change & agriculture: Understanding the biological greenhouse gases report released today by the Parliamentary Commissioner for the Environment, says Labour’s Climate Change spokesperson Megan ...
    1 week ago
  • Super Fund climate change approach a good start
    Labour Finance Spokesperson Grant Robertson and Climate Change Spokesperson Dr Megan Woods have welcomed the adoption of a climate change investment strategy by the New Zealand Super Fund. “This is a good start. It is a welcome development that the Super ...
    1 week ago
  • Energy use going in the wrong direction
    New data out this week from Statistics NZ paints a concerning picture of energy use across the economy under this National Government. You won’t be surprised to hear that there is some seriously worrying information here about how dirty our ...
    GreensBy Gareth Hughes
    1 week ago
  • Raising the age the right thing to do
    The announcement today that the Government will leave the door open for young people leaving state care still means there is a lot of work to do, says Labour's Spokesperson for Children, Jacinda Ardern "The Government indicated some time ago ...
    1 week ago
  • Junior Doctors go on Strike
    Thousands of junior doctors took strike action for 24 hours this week for better working conditions and safer working hours.  The Green Party supports their cause, and particularly their claims to reduce the number of days worked from up to ...
    GreensBy Denise Roche
    1 week ago
  • Coleman plays down the plight of junior doctors
    Junior doctors are crucial to our health services and the industrial action that continues tomorrow shows how desperately the Government has underfunded health, says Labour’s Health spokesperson Annette King.  “Jonathan Coleman’s claim that he has not seen objective evidence of ...
    2 weeks ago
  • Inflation piles pressure on National and Reserve Bank
    While many households will welcome the low inflation figures announced today, they highlight serious questions for both the National government and the Reserve Bank, Labour’s  Finance Spokesperson Grant Robertson said.  "While low inflation will be welcomed by many, the ...
    2 weeks ago
  • Officials warned Nat’s $1b infrastructure fund ineffective and rushed
    Treasury papers show the Government rushed out an infrastructure announcement officials told them risked making no significant difference to housing supply, says Labour’s housing spokesperson Phil Twyford.  “Like so much of National’s housing policy, this was another poll-driven PR initiative ...
    2 weeks ago
  • More cops needed to tackle P
    New Police statistics obtained in Written Questions show John Key is losing his War on P, highlighting the need for more Police, says Opposition Leader Andrew Little.  “New Zealanders expect serious action on P but today’s hodgepodge of half-measures won’t ...
    2 weeks ago
  • Strengthening our relationship with the Rātana movement
    It was a privilege to visit Rātana Pā last week with fellow Greens’ Co-leader James Shaw, our Māori Caucus and senior staff to meet with the leaders of te iwi mōrehu, to strengthen the ties between the Green Party and ...
    GreensBy Metiria Turei
    2 weeks ago
  • MBIE docs show country needs KiwiBuild, not Key’s pretend “building boom”
    John Key’s spin that New Zealand is in a building boom does not change the massive shortfall in building construction as new MBIE papers reveal, says Labour Party housing spokesperson Phil Twyford.  “We can fix the housing crisis, by the ...
    2 weeks ago
  • 1 in 7 Akl houses now going to big property speculators
    Speculators are running riot in the Auckland housing market making life tougher for first home buyers, says Labour’s housing spokesperson Phil Twyford.  Newly released data from Core Logic shows a 40 per cent increase in the share of house sales ...
    2 weeks ago
  • Disconnected thinking dirties the water
    Iain Rabbitts’ belief that drinking water quality, charging for water use and the land use that leads to water quality degradation should be treated separately is part of the problem we have right now in this country. The connection is ...
    GreensBy Catherine Delahunty
    2 weeks ago
  • Report back from Hands Off Our Tamariki hui
    This week I attended a hui in Otaki organised by Hands Off Our Tamariki about the proposed reforms to the Child Young Persons and their Families Act. Moana Jackson and Paora Moyle spoke.  They expressed deep, profound concern about the proposed ...
    GreensBy Jan Logie
    2 weeks ago
  • Labour mourns passing of Helen Kelly
    Helen Kelly was a passionate advocate for working New Zealanders and for a safe and decent working life, Leader of the Opposition Andrew Little says.  “Helen Kelly spent her adult life fighting for the right of every working person to ...
    2 weeks ago
  • National’s visionless immigration policy
    National’s recent immigration announcement is a continuation of the visionless approach to government that it has displayed in the last three terms. Rather than using the levers of government to implement a sustainable immigration policy that benefits new and current ...
    GreensBy Denise Roche
    2 weeks ago
  • Andrew Little: Speech to the Police Association Conference 2016
    Police Association delegates, Association life members and staff, representatives from overseas jurisdictions. Thank you for inviting me here today. The Police Association has become a strong and respected voice for Police officers and for policing in New Zealand. There is ...
    2 weeks ago
  • 1,000 more police for safer communities
    Labour will fund an extra 1,000 Police in its first term to tackle the rising rate of crime, says Leader of the Opposition Andrew Little. “Labour will put more cops on the beat to keep our communities safe. ...
    2 weeks ago
  • Call for all-party round table on homelessness
    Labour is calling on the Government to take part in a roundtable meeting to hammer out a cross-party agreement on ending homelessness.  Labour’s housing spokesperson Phil Twyford said the country wanted positive solutions to homelessness, and wanted the political parties ...
    2 weeks ago
  • Seclusion rooms in schools
    Schools are undoubtedly stretched and underfunded to cope with students with high learning support needs. But this cannot justify the use of rooms (or cupboards) as spaces to forcibly isolate children. It has emerged via media that this practice continues ...
    GreensBy Catherine Delahunty
    2 weeks ago
  • Public should get a say on new Waikato power station
    I had an opinion piece published in the Waikato Times about a controversial proposal to build a new gas-fired power station. It’s not on their website yet, so here it is: If you think the public would get a say ...
    GreensBy Gareth Hughes
    2 weeks ago
  • MSD and their investment approach
    The Government talks about investment but there is no investment. It is not investment if it isn’t over the whole of life and if there is no new money  — Shamubeel Eaqub   Investment sounds like adequate resourcing but this ...
    GreensBy Jan Logie
    2 weeks ago
  • Certainty needed for community services
    A couple of months ago I was at a seminar where three community organisations were presenting. Two of the three presenters were waiting to find out if their organisation would get a contract renewed with MSD. Not knowing if their ...
    GreensBy Jan Logie
    3 weeks ago
  • Domestic Violence – some advice for the media
    For the purpose of this piece, I’m going to use Domestic Violence (DV) as a proxy for intimate partner violence. DV is not isolated to physical abuse in a relationship between people with the same power. DV is a pattern of ...
    GreensBy Jan Logie
    3 weeks ago
  • Leroy’s New Paw Prints
    Leroy, an Auckland great dane recently received a new 3D printed bionic leg after cancer was discovered. I think this is a fantastic story and highlights the real potential of additive manufacturing, or 3D printing Leroy’s prosthetic was printed in titanium and was ...
    GreensBy Gareth Hughes
    3 weeks ago