web analytics
The Standard
Advertising

Granny Herald: from flawed data, false conclusions

Written By: - Date published: 11:21 am, March 23rd, 2013 - 17 comments
Categories: john key, national, polls - Tags:

Three of the Herald’s political staff has a crack at trying to explain why National polled 48.5% in the latest Herald-Digipoll. Consensus seems to be that John Key’s an angel sent to earth to do very little but keep enough of the middle class happy enough who’ll never lose. May I humbly suggest an alternative theory – that the Herald-Digipoll is broken.

Danyl at the Dimpost has the most mathematically poll of polls that, crucially, is anchored in reality – ie the last election result. Whereas political commentators and pollsters seem to shamelessly ignore how badly their poll got it wrong at the last election when breathlessly reporting the latest numbers, the Dimpost poll of polls corrects for their error.

What the Dimpost poll shows is that:

a) the Herald Digi-poll consistently over measures National – for example, the final Herald poll before the last election had National on 50.9%, that’s outside the margin of error above the 47.3% they got. The final Herald poll in 2008 had National on 47.9%, they got 44.9%. Statistically, the Herald isn’t just missing the bullseye, it’s missing the dartboard altogether. Only the 3news poll is worse. (in the image below, the light blue line is the Herald trend, the dark one is the poll of polls trend).

Dimpost poll of polls highlighting Herald Digipoll

b) even the Herald poll shows that National has lost a significant amount of support. From a pre-election high of 54.2%, a final pre-election number of 50.9%, to 48.5% today. That’s a big drop. Adjust to the Herald’s 3% over-polling of National at the last two elections, and it signals National is at around 45% – in the range that the Dimpost poll of polls has it. Remember, the Right has a one-seat majority in the House. Losing 2-3% since the election, especially when Banks will never get back in, is no laughing matter for National. If it were to translate into an election result, it would mean National would need New Zealand First (or a very large wasted vote) to govern.

c) more regular polls, like the Roy Morgan show a clear downward trend for National. You can’t possibly draw a trend from the five data points in 17 months that the Herald provides because the statistical noise that results in variation between every poll can’t be sensibly separated from the trend when you have so little to work with. The Roy Morgan has polled 29 times in that period and its trend is clear. (incidentally, the Roy Morgan was also the most accurate for National in 2011).

roy morgan labour greens v nats 3

To look at the Herald poll and conclude that National is now more popular than at the election, you have to ignore the Herald’s record of over-polling National, the fact that National’s support in the Herald poll has fallen significantly since the election, and the fact that more reliable polling trends tell us the opposite.

Only if you make all these logical errors (and there’s a strong incentive to do so because our paper spent a lot of money on the poll, which means its going to be treated as gospel no matter what, goddammit), can you then conclude National’s rule continues unblemished thanks to Bonny Prince Grinny.

But when’s a few fallacies ever stopped Granny?

[Note: none of this should be taken as a ‘if these trends continue’ argument. One current accurate polling, the Left needs Winnie too making him Kingmaker and he would have much more incentive to go with National, and there’s the risk of a Shearer campaign meltdown too. Put National at 40% and I’ll be happier].

17 comments on “Granny Herald: from flawed data, false conclusions”

  1. Tiresias 1

    “Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: “There are three kinds of lies: lies, damned lies and statistics.”
    – Mark Twain’s Own Autobiography

    Presently Labour will only be picking up those sufficiently dismayed by National’s duplicities and incompetence not to want them in Government again. Labour will only start picking up genuine support when it starts presenting an alternative programme worth genuinely supporting, and six-weeks out from an election is far too late to start thinking of doing that.

  2. Bob Tambling 2

    The worry for me then is if the polling trends continue and it looks like Shearer can just get over the line the possibilty of changing the leader will further diminish and Shearer will fight the 2014 election.
    Unfortunetly during the ensuing campaign, Key and the Nats will crucify him and we will be left with a National Government or a very weak LAb /NZF /Green one term Goverment
    I wonder from a Green party prospective (and a leftish pespective) , would it not be better if the Labour Party lost the next election with Shearer as Leader?
    If NZF play a part as may be likely, the Green can’t afford to go into a NZF /Lab/Green Gov.
    Shearer simply won’t cut the mustard and Peters is toxic. It would be a crap government and the well honed National party machine would again tear it to shreds

    The leaves the Greens in an interesting position, the lure of ministerial positions versus the pragmatic long term strategy of the Greens. A one term Lab/Nzf/Green Gov seems to serve no ones interest.
    Labour needs to have a strong leadership team and comitted activists , The reality is they have a diminishing group of activists and a crap leadership.
    The Greens on the other hand have a very strong leadership team and an ever growing group of activists ( let us not forget they are the second biggest party in Wellington).
    Labour? I can’t think of anyone in the present caucus who could take most of the country with them in an election. They need a good clean out and new blood. They don’t appear to be listening to that line of argument.
    In conclusion:
    I am committed to working very hard to getting as many Green MP’s elected as I can in 2014. I am not so sure though if I want those MP’s in a Government with the current infighting mob of the Lab party who it seems “could not pour piss out of a boot if the directions were on the heel”.

  3. tc 4

    The more granny and her shills like Johnno, o’shillivan, rortney etc keep banging the NACT drum the more irrelevant and bias they show themselves to be.

    Labour needs to grow a pair and get out there now however with the mallarfia and their front man it seems it’ll come down to the hollowmen blowing it more so than Labour winning it.

    They could easily blag another term if they weren’t such psychos given the current state of the opposition, thankfully they are what they are.

  4. ghostrider888 5

    article in todays Herald acknowledges the popularity of Key is slipping

  5. pete 6

    “Only the 3news poll is worse.”

    That’s not quite true: all five polls overestimate National’s election results, and the Digipoll isn’t too much worse than Fairfax or Roy Morgan. One News Colmar Brunton, as well as 3 News Reid Research both overestimate by more than the Herald.

    • lprent 6.1

      Points to a systematic flaw in their sampling doesn’t it…

      • pete 6.1.1

        That’s one possible explanation, although some have theorised that National lose a lot of votes close to the election once people actually start looking at their policies.

        There’s also a theory that NZF underpolls because their voters are embarrassed to admit that to callers.

        • lprent 6.1.1.1

          Possible. But less likely than the simpler explanation.

          Canvassing companies are using listed landlines*. When I was analyzing some somewhat larger samples of both door-knock canvassing and phone canvassing across several electorates, there was a statistically significant correlation between people who canvassed right and left and if they had a listed landline.

          For that matter there was an even stronger correlation between age group and having a listed land line – and it looks like if people don’t have a listed phone line in their mid 20’s then they never get one.

          * There is a common myth that goes around about auto-dialing random numbers. Asked a few people who work at or have worked at polling companies. It doesn’t happen and a few minutes thought about number ranges will give an explanation about why. It is cheaper to buy/get a phone list.

  6. Chris 7

    So,will Granny Herald be ramping up lots of little good news, happy smiley,relaxed all singing, dancing reports/photos of our poor excuse for a pm then. Got to keep him relevant.

  7. Centaur 8

    Give Key *some* credit. He may decide to call it quits. It is not as if he actually needs this gig ..

  8. Murray Olsen 9

    I’m not even sure that the margins of error as reported are very meaningful at all. From what I know, they seem to rely on unbiased sampling and the validity of the Central Limit Theorem, which then leads to a percentage presumably taken from 100 times √N/N. The concern about landlines is just one thing that could skew this.

    • lprent 9.1

      They do sample distribution normalising for many of the other factors like age, gender and geological distribution. They don’t adjust for factors like people working during the periods that they phone.

      But they don’t publish their techniques and criteria so it is pretty hard to see why there if it was unbiased sampling there would be any differences between the polls within the confidence limits. But there clearly is.

      • Centaur 9.1.1

        lprent: geological distribution ?
        Such as, where there are proven mineral resources ?

        • lprent 9.1.1.1

          I blame the iPad, it’s overly smart autocorrect, and that I use Geological at least 10x more often than Geographical.

    • Lanthanide 9.2

      No, they’re not. Even assuming they are following absolutely perfect statistical methods and the statistical methods work as advertised by the literature, the margin of error they’re reporting is presumably the 95% confidence interval. Which means 1 out of 20 polls are going to be wrong, but we won’t ever be able to tell which one without some other corroborating information (eg other polls taken at the same time).

      That’s assuming they’re using perfect methods, which of course they aren’t.

Important links

Recent Comments

Recent Posts

  • Dunedin Hospital needs more than drip feed
    An ongoing and embarrassing pattern of major building leaks and equipment failures at Dunedin Public Hospital has been revealed in papers released under the Official Information Act, Dunedin North MP David Clark says. “Documents released under the Official Information Act… ...
    14 hours ago
  • Dunedin Hospital needs more than drip feed
    An ongoing and embarrassing pattern of major building leaks and equipment failures at Dunedin Public Hospital has been revealed in papers released under the Official Information Act, Dunedin North MP David Clark says. “Documents released under the Official Information Act… ...
    14 hours ago
  • 17 too young for teens to be shown the door
    Laws which see young people under the care of CYFS abandoned once they turn 17 will mean at least a dozen young Kiwis will be left to fend for themselves over the December festive season, Labour’s Children’s spokesperson Jacinda  Ardern… ...
    14 hours ago
  • 17 too young for teens to be shown the door
    Laws which see young people under the care of CYFS abandoned once they turn 17 will mean at least a dozen young Kiwis will be left to fend for themselves over the December festive season, Labour’s Children’s spokesperson Jacinda  Ardern… ...
    14 hours ago
  • National’s albatross, taxpayers’ curse
    Government consideration of further corporate welfare hand-outs to SkyCity for its convention centre shows just how weak the original contract was, Labour’s Economic Development spokesperson David Clark says. “Taxpayers will be appalled to hear that on top of the humiliating… ...
    15 hours ago
  • Recognizing Palestine: The European Parliament Votes
    Last week I wrote a blog drawing attention to Sweden’s formal recognition of the state of Palestine (the second Western state to do so after Iceland).  That move has created ripples throughout the international community. In recent months the parliaments… ...
    GreensBy Kennedy Graham MP
    3 days ago
  • Recognizing Palestine: The European Parliament Votes
    Last week I wrote a blog drawing attention to Sweden’s formal recognition of the state of Palestine (the second Western state to do so after Iceland).  That move has created ripples throughout the international community. In recent months the parliaments… ...
    GreensBy Kennedy Graham MP
    3 days ago
  • Recognizing Palestine: The European Parliament Votes
    Last week I wrote a blog drawing attention to Sweden’s formal recognition of the state of Palestine (the second Western state to do so after Iceland).  That move has created ripples throughout the international community. In recent months the parliaments… ...
    GreensBy Kennedy Graham MP
    3 days ago
  • Minister has work to do over Xmas
    Red flags raised in a multi-agency review into how Phillip Smith was able to flee the country highlight the inadequacies of those very same agencies not having red flags in place that would have notified them of his plans, says… ...
    4 days ago
  • Minister has work to do over Xmas
    Red flags raised in a multi-agency review into how Phillip Smith was able to flee the country highlight the inadequacies of those very same agencies not having red flags in place that would have notified them of his plans, says… ...
    4 days ago
  • Minister has work to do over Xmas
    Red flags raised in a multi-agency review into how Phillip Smith was able to flee the country highlight the inadequacies of those very same agencies not having red flags in place that would have notified them of his plans, says… ...
    4 days ago
  • Gerry Brownlee’s revolving airport door story
    A new report shows Gerry Brownlee is the latest Cabinet Minister to have contracted the infectious tell-porkies-until-you-are-caught disease, Labour’s Chief Whip Chris Hipkins says. “A Civil Aviation Report out today shows that despite being an extremely recognisable figure, Gerry Brownlee… ...
    4 days ago
  • Gerry Brownlee’s revolving airport door story
    A new report shows Gerry Brownlee is the latest Cabinet Minister to have contracted the infectious tell-porkies-until-you-are-caught disease, Labour’s Chief Whip Chris Hipkins says. “A Civil Aviation Report out today shows that despite being an extremely recognisable figure, Gerry Brownlee… ...
    4 days ago
  • Govt spend on transport out of step with reality
    The National Government is planning to allocate ever increasing amounts of taxpayer funding to build expensive new motorways despite record numbers of New Zealanders flocking to buses and trains, said the Green Party. The Government released its Government Policy Statement… ...
    GreensBy Julie Anne Genter MP
    4 days ago
  • Govt spend on transport out of step with reality
    The National Government is planning to allocate ever increasing amounts of taxpayer funding to build expensive new motorways despite record numbers of New Zealanders flocking to buses and trains, said the Green Party. The Government released its Government Policy Statement… ...
    GreensBy Julie Anne Genter MP
    4 days ago
  • Solar homes stymied by Govt inaction
    Government inaction is allowing the big power companies to discourage the nascent solar power sector, the Green Party said today. Green Party MP Gareth Hughes launched a petition today calling on the Government to empower the Electricity Authority to act… ...
    GreensBy Gareth Hughes MP
    4 days ago
  • Foreign buyers for iconic island must add value
    Labour will look very closely at any Overseas Investment Office application to purchase Pakatoa Island if it is not bought by a Kiwi, says Labour’s Land information Spokesperson Stuart Nash. “Pakatoa is an iconic island in the middle of Hauraki… ...
    4 days ago
  • Way opening for April Sun in Cuba
    The United States of America’s President’s historic announcement yesterday to restore diplomatic ties with Cuba should be applauded by the New Zealand Government. The announcement marks a turning point in more than five decades of hostility between the two countries… ...
    GreensBy Kennedy Graham MP
    4 days ago
  • Minister ducking for cover over ‘Diplomat Case’
    Apparently the Ministerial Inquiry into what now seems to be being referred to as ‘The Diplomat Case’ ( I have a few other names for it) has been completed and is in front of Foreign Affairs Minister McCully. Initial Reports seem to… ...
    GreensBy Jan Logie MP
    5 days ago
  • Energy users need answers on Vector share plans
    Energy Minister Simon Bridges needs to stop ducking for cover about whether or not the Government will support plans to nationalise and then privatise $2.1 billion of shares in the Auckland Electricity Consumer Trust, Labour's Energy spokesperson Stuart Nash says. “It… ...
    5 days ago
  • Biosecurity it’s everyone’s responsibility
    Biosecurity costs New Zealand millions of dollars in attempting pest eradication and much more in ongoing management of pests in farming, horticulture, beekeeping and conservation, as well as in our own backyards and recreation areas. More work must happen at… ...
    GreensBy Steffan Browning MP
    6 days ago
  • Is the Health Minister accountable to the public? He doesn’t seem to thin...
    Lately I’ve been involved in a sort of farcical standoff with the Health Minister, who seems to be under the illusion that I have no right to ask questions about conflicts involving Health Promotion Agency Board member Katherine Rich, and… ...
    GreensBy Kevin Hague MP
    7 days ago
  • Irresponsible tax cuts lead to seventh successive deficit
    National's borrowing to pay for cutting the top tax rate was irresponsible and will likely lead to a seventh successive deficit, the Green Party said today. Treasury have forecast a $572 million deficit this year in its Half Year Economic… ...
    GreensBy Russel Norman MP
    7 days ago
  • Heartfelt sympathy for Sydneysiders
    The Labour Party has offered its heartfelt sympathy to the people of Sydney after the hostage situation in the city, says Labour’s Acting leader Grant Robertson.  “Our thoughts are with all those who went through this horrific and traumatic experience. ...
    7 days ago
  • Farewell at Phillipstown
    Last Wednesday, I attended the farewell for Tony Simpson, Principal of Phillipstown School. It was a very emotional event where many of us in the large crowd shed tears. Bagpipes and tiny tamariki performing kapahaka brought the house down and… ...
    GreensBy Catherine Delahunty MP
    1 week ago
  • The CIA Torture Report
    Earlier this week, the United States Select Committee on Intelligence released the Committee Study of the Central Intelligence Agency’s Detention and Interrogation Program.  The report, which was five years in the making, looked into the CIA’s interrogation techniques from 2001… ...
    GreensBy Kennedy Graham MP
    1 week ago
  • Haere Rā 2014
    We’ve almost reached the end of the Parliamentary year so I wanted to take a moment to reflect on some of my highlights of the term in this blog post. It’s been an absolutely hectic year juggling an election campaign… ...
    GreensBy Gareth Hughes MP
    1 week ago
  • Labour applauds High Court decision on Ruataniwha
    Today’s decision by the High Court on the Ruataniwha scheme is a victory for NewZealand’s environmental groups, says Labour’s Conservation spokesperson RuthDyson. ...
    1 week ago
  • A welfare system for the 21st Century
    Today Child Poverty Action Group released a background paper on ‘The complexities of ‘relationship’ in the welfare system and the consequences for children.‘ The report includes 16 recommendations to modernise our welfare system which is no longer fit for the… ...
    GreensBy Jan Logie MP
    1 week ago
  • Welfare system out of date and out of touch
    A new Child Poverty Action Group report released today highlights another example of how our outmoded social welfare system is harming kids, says Labour’s Social Development Spokesperson Carmel Sepuloni.  “The complexities of how a ‘relationship’ is defined in the welfare… ...
    1 week ago

Public service advertisements by The Standard

Current CO2 level in the atmosphere