web analytics

Granny Herald: from flawed data, false conclusions

Written By: - Date published: 11:21 am, March 23rd, 2013 - 17 comments
Categories: john key, national, polls - Tags:

Three of the Herald’s political staff has a crack at trying to explain why National polled 48.5% in the latest Herald-Digipoll. Consensus seems to be that John Key’s an angel sent to earth to do very little but keep enough of the middle class happy enough who’ll never lose. May I humbly suggest an alternative theory – that the Herald-Digipoll is broken.

Danyl at the Dimpost has the most mathematically poll of polls that, crucially, is anchored in reality – ie the last election result. Whereas political commentators and pollsters seem to shamelessly ignore how badly their poll got it wrong at the last election when breathlessly reporting the latest numbers, the Dimpost poll of polls corrects for their error.

What the Dimpost poll shows is that:

a) the Herald Digi-poll consistently over measures National – for example, the final Herald poll before the last election had National on 50.9%, that’s outside the margin of error above the 47.3% they got. The final Herald poll in 2008 had National on 47.9%, they got 44.9%. Statistically, the Herald isn’t just missing the bullseye, it’s missing the dartboard altogether. Only the 3news poll is worse. (in the image below, the light blue line is the Herald trend, the dark one is the poll of polls trend).

Dimpost poll of polls highlighting Herald Digipoll

b) even the Herald poll shows that National has lost a significant amount of support. From a pre-election high of 54.2%, a final pre-election number of 50.9%, to 48.5% today. That’s a big drop. Adjust to the Herald’s 3% over-polling of National at the last two elections, and it signals National is at around 45% – in the range that the Dimpost poll of polls has it. Remember, the Right has a one-seat majority in the House. Losing 2-3% since the election, especially when Banks will never get back in, is no laughing matter for National. If it were to translate into an election result, it would mean National would need New Zealand First (or a very large wasted vote) to govern.

c) more regular polls, like the Roy Morgan show a clear downward trend for National. You can’t possibly draw a trend from the five data points in 17 months that the Herald provides because the statistical noise that results in variation between every poll can’t be sensibly separated from the trend when you have so little to work with. The Roy Morgan has polled 29 times in that period and its trend is clear. (incidentally, the Roy Morgan was also the most accurate for National in 2011).

roy morgan labour greens v nats 3

To look at the Herald poll and conclude that National is now more popular than at the election, you have to ignore the Herald’s record of over-polling National, the fact that National’s support in the Herald poll has fallen significantly since the election, and the fact that more reliable polling trends tell us the opposite.

Only if you make all these logical errors (and there’s a strong incentive to do so because our paper spent a lot of money on the poll, which means its going to be treated as gospel no matter what, goddammit), can you then conclude National’s rule continues unblemished thanks to Bonny Prince Grinny.

But when’s a few fallacies ever stopped Granny?

[Note: none of this should be taken as a ‘if these trends continue’ argument. One current accurate polling, the Left needs Winnie too making him Kingmaker and he would have much more incentive to go with National, and there’s the risk of a Shearer campaign meltdown too. Put National at 40% and I’ll be happier].

17 comments on “Granny Herald: from flawed data, false conclusions”

  1. Tiresias 1

    “Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: “There are three kinds of lies: lies, damned lies and statistics.”
    – Mark Twain’s Own Autobiography

    Presently Labour will only be picking up those sufficiently dismayed by National’s duplicities and incompetence not to want them in Government again. Labour will only start picking up genuine support when it starts presenting an alternative programme worth genuinely supporting, and six-weeks out from an election is far too late to start thinking of doing that.

  2. Bob Tambling 2

    The worry for me then is if the polling trends continue and it looks like Shearer can just get over the line the possibilty of changing the leader will further diminish and Shearer will fight the 2014 election.
    Unfortunetly during the ensuing campaign, Key and the Nats will crucify him and we will be left with a National Government or a very weak LAb /NZF /Green one term Goverment
    I wonder from a Green party prospective (and a leftish pespective) , would it not be better if the Labour Party lost the next election with Shearer as Leader?
    If NZF play a part as may be likely, the Green can’t afford to go into a NZF /Lab/Green Gov.
    Shearer simply won’t cut the mustard and Peters is toxic. It would be a crap government and the well honed National party machine would again tear it to shreds

    The leaves the Greens in an interesting position, the lure of ministerial positions versus the pragmatic long term strategy of the Greens. A one term Lab/Nzf/Green Gov seems to serve no ones interest.
    Labour needs to have a strong leadership team and comitted activists , The reality is they have a diminishing group of activists and a crap leadership.
    The Greens on the other hand have a very strong leadership team and an ever growing group of activists ( let us not forget they are the second biggest party in Wellington).
    Labour? I can’t think of anyone in the present caucus who could take most of the country with them in an election. They need a good clean out and new blood. They don’t appear to be listening to that line of argument.
    In conclusion:
    I am committed to working very hard to getting as many Green MP’s elected as I can in 2014. I am not so sure though if I want those MP’s in a Government with the current infighting mob of the Lab party who it seems “could not pour piss out of a boot if the directions were on the heel”.

  3. tc 4

    The more granny and her shills like Johnno, o’shillivan, rortney etc keep banging the NACT drum the more irrelevant and bias they show themselves to be.

    Labour needs to grow a pair and get out there now however with the mallarfia and their front man it seems it’ll come down to the hollowmen blowing it more so than Labour winning it.

    They could easily blag another term if they weren’t such psychos given the current state of the opposition, thankfully they are what they are.

  4. ghostrider888 5

    article in todays Herald acknowledges the popularity of Key is slipping

  5. pete 6

    “Only the 3news poll is worse.”

    That’s not quite true: all five polls overestimate National’s election results, and the Digipoll isn’t too much worse than Fairfax or Roy Morgan. One News Colmar Brunton, as well as 3 News Reid Research both overestimate by more than the Herald.

    • lprent 6.1

      Points to a systematic flaw in their sampling doesn’t it…

      • pete 6.1.1

        That’s one possible explanation, although some have theorised that National lose a lot of votes close to the election once people actually start looking at their policies.

        There’s also a theory that NZF underpolls because their voters are embarrassed to admit that to callers.

        • lprent 6.1.1.1

          Possible. But less likely than the simpler explanation.

          Canvassing companies are using listed landlines*. When I was analyzing some somewhat larger samples of both door-knock canvassing and phone canvassing across several electorates, there was a statistically significant correlation between people who canvassed right and left and if they had a listed landline.

          For that matter there was an even stronger correlation between age group and having a listed land line – and it looks like if people don’t have a listed phone line in their mid 20’s then they never get one.

          * There is a common myth that goes around about auto-dialing random numbers. Asked a few people who work at or have worked at polling companies. It doesn’t happen and a few minutes thought about number ranges will give an explanation about why. It is cheaper to buy/get a phone list.

  6. Chris 7

    So,will Granny Herald be ramping up lots of little good news, happy smiley,relaxed all singing, dancing reports/photos of our poor excuse for a pm then. Got to keep him relevant.

  7. Centaur 8

    Give Key *some* credit. He may decide to call it quits. It is not as if he actually needs this gig ..

  8. Murray Olsen 9

    I’m not even sure that the margins of error as reported are very meaningful at all. From what I know, they seem to rely on unbiased sampling and the validity of the Central Limit Theorem, which then leads to a percentage presumably taken from 100 times √N/N. The concern about landlines is just one thing that could skew this.

    • lprent 9.1

      They do sample distribution normalising for many of the other factors like age, gender and geological distribution. They don’t adjust for factors like people working during the periods that they phone.

      But they don’t publish their techniques and criteria so it is pretty hard to see why there if it was unbiased sampling there would be any differences between the polls within the confidence limits. But there clearly is.

      • Centaur 9.1.1

        lprent: geological distribution ?
        Such as, where there are proven mineral resources ?

        • lprent 9.1.1.1

          I blame the iPad, it’s overly smart autocorrect, and that I use Geological at least 10x more often than Geographical.

    • Lanthanide 9.2

      No, they’re not. Even assuming they are following absolutely perfect statistical methods and the statistical methods work as advertised by the literature, the margin of error they’re reporting is presumably the 95% confidence interval. Which means 1 out of 20 polls are going to be wrong, but we won’t ever be able to tell which one without some other corroborating information (eg other polls taken at the same time).

      That’s assuming they’re using perfect methods, which of course they aren’t.

Recent Comments

Recent Posts

  • Sticking Plaster Budget fails the test
    Bill English’s penultimate Budget fails to tackle the structural challenges facing the economy – a housing crisis, rising unemployment, underfunded health and creaking infrastructure, says Labour’s Finance spokesperson Grant Robertson. “This Budget applies a sticking plaster to a compound fracture.… ...
    59 mins ago
  • John Key fails middle New Zealand with no fix for housing crisis, more underfunding of health
    Middle New Zealand has again missed out in this year’s Budget with not a single fix for the housing crisis, and health and education woefully underfunded again, Leader of the Opposition Andrew Little says. “This Budget is just a patchwork… ...
    1 hour ago
  • Labour Bill would back Kiwi jobs
    The Government’s $40 billion of buying power would go towards backing Kiwi businesses and jobs under a Labour Member’s Bill which will be debated by Parliament, Leader of the Opposition Andrew Little says. “My Bill – which was pulled from… ...
    3 hours ago
  • Julie Anne Genter: My Budget 2016 wish is fairness
    When my parents first visited me in Auckland ten years ago, they remarked on how there were no homeless people on the streets. Coming from Los Angeles, they were used to seeing the impacts of horrendous inequality and a lack… ...
    GreensBy Julie Anne Genter
    5 hours ago
  • Steffan Browning: Pesticide reduction and Organic Growth Strategy in Budget 2016
    Pesticide reduction The Budget is an opportunity for the Government to launch a pesticide reduction strategy that multiplies the Environmental Protection Authority’s (EPA) and the Ministry for Primary Industries’ capacity to reassess pesticides and other toxins.  The Agricultural Compounds and… ...
    GreensBy Steffan Browning
    6 hours ago
  • Steffan Browning: Pesticide reduction and Organic Growth Strategy in Budget 2016
    Pesticide reduction The Budget is an opportunity for the Government to launch a pesticide reduction strategy that multiplies the Environmental Protection Authority’s (EPA) and the Ministry for Primary Industries’ capacity to reassess pesticides and other toxins.  The Agricultural Compounds and… ...
    GreensBy Steffan Browning
    6 hours ago
  • Minister won’t fess up on wrong figures
    The Minister of Health was caught out telling porkies in Parliament today when he was asked about the number of people getting access to mental health and addiction services, says Labour’s Health spokesperson Annette King. ...
    24 hours ago
  • Budget 2016 and our LGBTQI communities
    LGBTI people make up about a tenth of our population, and our communities face a unique set of needs and challenges. These challenges are caused or exacerbated by discrimination, invisibility and barriers to appropriate support. We have a long way… ...
    GreensBy Jan Logie
    1 day ago
  • Budget 2016 and our LGBTQI communities
    LGBTI people make up about a tenth of our population, and our communities face a unique set of needs and challenges. These challenges are caused or exacerbated by discrimination, invisibility and barriers to appropriate support. We have a long way… ...
    GreensBy Jan Logie
    1 day ago
  • Scrambled announcement policy on the hoof
    Paula Bennett’s scrambled desperate announcement that she will pay homeless people to move to the regions is just the latest evidence of the disarray this Government’s housing policy is in, Leader of the Opposition Andrew Little says. “This is policy… ...
    1 day ago
  • Police Minister admits resolution rates fall short of expectation
    Police Minister Judith Collins has admitted in Parliament current burglary resolution rates are not meeting the expectations of our communities, says Labour’s Police spokesman Stuart Nash “Out of 284 police stations in New Zealand in 2015, 24 stations recorded zero… ...
    1 day ago
  • Mojo Mathers: A better deal for animals in Budget 2016
    Currently we are failing animals in NZ. On the face of it farmed and domestic animals in this country have strong legal protection from abuse, cruelty and neglect. In reality it seems that only the very worst, most extreme cases… ...
    GreensBy Mojo Mathers
    1 day ago
  • Metiria Turei: What we need from Budget 2016
    Every family deserves a warm decent home.  Everyone believes that. This housing crisis is just the latest consequence of a Government who puts the interests of the few wealthy people above the needs of NZ families.  Families are doing it… ...
    GreensBy Metiria Turei
    1 day ago
  • Dairy exports fall of 11%: Budget action on diversification needed
    Dairy exports have fallen 11 per cent compared to this time last year, a fall of almost $1.5b, showing the Government must take clear action on diversifying the economy in tomorrow’s Budget, says Labour’s Trade and Export Growth spokesperson David… ...
    1 day ago
  • Investors driving families out of homes in South and West Auckland
    Investors cashing in on skyrocketing Auckland house prices are driving families out of homes in South and West Auckland and causing homeownership rates in some of our poorest suburbs to plummet, Labour’s Housing spokesperson Phil Twyford says. “New analysis shows… ...
    1 day ago
  • Budget must deliver on paid parental leave
    Budget 2016 must deliver 26 weeks paid parental leave by April 2018 – anything less will be short-changing families, says Labour MP Sue Moroney. “My Bill which is before Parliament this afternoon has majority support and does just that. I… ...
    1 day ago
  • Key’s “brain fart” on tax cuts news to English
    John Key didn’t tell his own Finance Minister he was about to go on radio and announce he wanted $3b of tax cuts, just days after Bill English ruled them out, says Labour’s Finance spokesperson Grant Robertson. “In Parliament today… ...
    2 days ago
  • What I’m looking for in Budget 2016 – A better start for our tamariki
    Ensuring the best start for our tamariki is a priority for me in everything I do. And so in Budget 2016, my first budget as an MP, I looking for the Government to make a real investment in the wellbeing… ...
    GreensBy Marama Davidson
    2 days ago
  • What I’m looking for in Budget 2016 – A better start for our tamariki
    Ensuring the best start for our tamariki is a priority for me in everything I do. And so in Budget 2016, my first budget as an MP, I looking for the Government to make a real investment in the wellbeing… ...
    GreensBy Marama Davidson
    2 days ago
  • Denise Roche: What I’m looking for in Budget 2016 Pt II
    Aotearoa’s new New Zealanders,  come to our country in vulnerable position: – often away from the culture, communities and families they know, sometimes in neighbourhoods without familiar faces and often encountering barriers to employment. With net migration at 50,000+ a… ...
    GreensBy Denise Roche
    2 days ago
  • Equal Pay and Budget 2016
    The last few years we’ve seen equal pay for women flagged as an undefined risk in the budget. This year we should expect to see this, as well as budgeted money to deliver equal pay to caregivers and funding for,… ...
    GreensBy Jan Logie
    2 days ago
  • Equal Pay and Budget 2016
    The last few years we’ve seen equal pay for women flagged as an undefined risk in the budget. This year we should expect to see this, as well as budgeted money to deliver equal pay to caregivers and funding for,… ...
    GreensBy Jan Logie
    2 days ago
  • A great Budget would
    A great Budget would embrace the challenge of our polluted rivers and move the money away from justifying the status quo water rules into cleaning up waterways. A great Budget would take the Ministry for the Environment freshwater budget and… ...
    GreensBy Catherine Delahunty
    2 days ago
  • Budget building materials policy backfires
    On the eve of this year’s Budget official figures show Nick Smith’s Budget 2014 centrepiece to reduce the cost of building materials has backfired, says Labour’s Housing spokesperson Phil Twyford. “Ministry of Business, Innovation and Employment officials have spent the… ...
    2 days ago
  • Smarter, Better, Cleaner, Stronger
    This Thursday Bill English will deliver his eighth Budget. Will it continue the trend of previous National budgets, making tertiary education less affordable, putting only token funds into innovation, and subsidising polluters? Budgets aren’t what they used to be. Once… ...
    GreensBy Gareth Hughes
    2 days ago
  • Govt must come clean on tax cuts in Budget
    National is making a mockery of the Budget process by dangling the promise of tax cuts but failing to include them in the Budget, says Labour’s Finance spokesperson Grant Robertson. “National’s tax cut promises have turned into a farce. One… ...
    3 days ago
  • Grant Robertson Pre-Budget Speech
    Today I want to talk about success. As we know success can come in many different forms, from the fact you all made it here at such an early hour on a Monday, for which I am very grateful, to… ...
    3 days ago
  • Budget must deliver for middle New Zealand
    The Government must ensure next week’s Budget stops the squeeze on middle New Zealand and delivers shared prosperity for all New Zealanders, Leader of the Opposition Andrew Little says. The call follows new research commissioned by Labour that shows working… ...
    4 days ago
  • Our housing emergency – why we have to act
    Marama and Metiria at Homes Not Cars launch On Thursday, Metiria Turei announced the Green Party’s plan to start addressing the emergency housing crisis facing our country. Too many people are without homes right now – homeless. It is the… ...
    GreensBy Marama Davidson
    5 days ago
  • Will funding boost for sexual violence services go to the right places?
    This week the Government announced $46million for sexual violence services. This announcement was a result of decades of work by advocates and everyone who submitted to the Select Committee inquiry into funding for sexual violence services that I initiated with… ...
    GreensBy Jan Logie
    6 days ago
  • Will funding boost for sexual violence services go to the right places?
    This week the Government announced $46million for sexual violence services. This announcement was a result of decades of work by advocates and everyone who submitted to the Select Committee inquiry into funding for sexual violence services that I initiated with… ...
    GreensBy Jan Logie
    6 days ago
  • Denise Roche – What I’m looking for in this year’s Budget
    Two of the things I’ll be looking for in the Budget next week are more funding for refugees and for our arts and culture sector. More funding for refugees I’m a strong supporter of the #DoubleTheQuota campaign and its goals… ...
    GreensBy Denise Roche
    6 days ago
  • Denise Roche – What I’m looking for in this year’s Budget
    Two of the things I’ll be looking for in the Budget next week are more funding for refugees and for our arts and culture sector. More funding for refugees I’m a strong supporter of the #DoubleTheQuota campaign and its goals… ...
    GreensBy Denise Roche
    6 days ago
  • Car rego victims must get a refund
    Motorists who have been overcharged for their car registration should get a refund, says Labour’s Transport spokesperson Sue Moroney.  “Minister Nikki Kaye’s ‘faulty risk’ rating scheme has blown up in her face with over 170 different models of car having… ...
    6 days ago
  • Council statement shows they just don’t get it
    The Auckland Council’s statement today shows they don’t understand the problems created by the urban growth boundary, says Labour’s housing spokesperson Phil Twyford.  “I have been the first to defend the Auckland City Council when Bill English has been blaming… ...
    6 days ago
  • Inspecting electronic devices a potential privacy threat
    Labour is expressing concern for New Zealanders’ privacy rights as the Government signals Customs will have the power to inspect electronic devices coming across the border, says Labour’s Customs Spokesperson Rino Tirikatene. “We agree that customs officers should have the… ...
    6 days ago
  • The Price of Water
    This week I hosted a public meeting at EIT in Hawkes Bay to discuss how we might put a price on the commercial use of water, so that water may be valued and treated more sustainably. I invited a… ...
    GreensBy Catherine Delahunty
    6 days ago
  • Caption It NZ!
    Today I received a petition from the NZ Captioning Working Group urging the government to legislate for accessibility via closed captioning for deaf and hard of hearing New Zealanders. It was timely because today is the fifth Global Accessibility Awareness… ...
    GreensBy Mojo Mathers
    1 week ago
  • Older Kiwis to miss out on electives
    The Government is not doing enough elective surgery to keep up with New Zealand’s ageing population, says Labour’s Health spokesperson Annette King.  “It’s damning that the targeted national intervention rate for cataract and knee and hip surgery is the same… ...
    1 week ago
  • Most principals say their college is underfunded
    The Government must substantially increase funding for secondary schools in next week’s Budget after a new survey found 86 per cent of principals consider their college under-resourced, Labour’s Education spokesperson Chris Hipkins says. “Just 14 per cent of secondary principals… ...
    1 week ago
  • Bill English and Nick Smith on different pages
    The Government’s support for Labour’s policy to remove the Auckland urban growth boundary is good news, but National needs to clarify its position, Labour’s Housing and Auckland Issues spokesperson Phil Twyford says. “The Acting Prime Minister has acknowledged our position… ...
    1 week ago
  • Bill English and Nick Smith on different pages
    The Government’s support for Labour’s policy to remove the Auckland urban growth boundary is good news, but National needs to clarify its position, Labour’s Housing and Auckland Issues spokesperson Phil Twyford says. “The Acting Prime Minister has acknowledged our position… ...
    1 week ago
  • Labour calls for independent inquiry into illegal fish dumping
    The Labour Party is reiterating its call for an independent inquiry into New Zealand’s fishing industry after two reports revealed the Ministry for Primary Industries turned a blind eye to widespread fish dumping in New Zealand waters, says Labour’s Fisheries… ...
    1 week ago
  • Mt Karangahake and Newcrest Mining
    On Wednesday and Sunday of last week the local residents of the Karangahake mountain in the Karangahake gorge of Hauraki/Coromandel peacefully protested against a gold mining drill rig on private land adjacent to the DOC land. The drilling rig was… ...
    GreensBy Catherine Delahunty
    1 week ago
  • Robbing Aucklanders to pay Rio Tinto
    New Zealand’s national electricity grid stretches the length of the country and contains some 11,803 kilometres of high-voltage lines and 178 substations. It wouldn’t make sense for competing power companies to duplicate and build their own expensive electricity transmission system… ...
    GreensBy Gareth Hughes
    1 week ago
  • Government should abolish Auckland urban growth boundary
    The Government should rule out any possibility of an urban growth boundary in Auckland Council’s Unitary Plan if it is serious about fixing the housing crisis, says Labour’s Housing spokesperson Phil Twyford. “Over 25 years the urban growth boundary hasn’t… ...
    1 week ago
  • Kiwis don’t want iPads for Land deals
     It is outrageous that schools are relying on money and iPads from foreign land investors to meet the learning needs of their students, says Labour’s Education spokesperson Chris Hipkins.  “Several OIO land applications by offshore investors have claimed that without… ...
    1 week ago
  • Homelessness – National has failed all of us
    A young South Auckland Māori woman recently tried to get hold of me around midnight. I missed her call. The woman wanted me to know the sharp reality facing too many families looking for a stable place to live. Things… ...
    GreensBy Marama Davidson
    1 week ago
  • Moko case should never have been manslaughter deal
    Confirmation again yesterday that the manslaughter charge in the Moko Rangitoheriri case was a deal done by the Crown Prosecution Service is justifiably the cause of outrage, says Labour’s Children’s spokesperson Jacinda Ardern.“This should never have been a case where… ...
    1 week ago
  • Overseas investor funds school’s digital devices
    The Government must address the inequality laptops and tablets in classrooms are causing after a Queenstown school was forced to use a donation from an overseas investor to get their students digital devices, Labour’s Education spokesperson Chris Hipkins. “Documents obtained… ...
    1 week ago

Public service advertisements by The Standard

Current CO2 level in the atmosphere