web analytics
The Standard

Granny Herald: from flawed data, false conclusions

Written By: - Date published: 11:21 am, March 23rd, 2013 - 17 comments
Categories: john key, national, polls - Tags:

Three of the Herald’s political staff has a crack at trying to explain why National polled 48.5% in the latest Herald-Digipoll. Consensus seems to be that John Key’s an angel sent to earth to do very little but keep enough of the middle class happy enough who’ll never lose. May I humbly suggest an alternative theory – that the Herald-Digipoll is broken.

Danyl at the Dimpost has the most mathematically poll of polls that, crucially, is anchored in reality – ie the last election result. Whereas political commentators and pollsters seem to shamelessly ignore how badly their poll got it wrong at the last election when breathlessly reporting the latest numbers, the Dimpost poll of polls corrects for their error.

What the Dimpost poll shows is that:

a) the Herald Digi-poll consistently over measures National – for example, the final Herald poll before the last election had National on 50.9%, that’s outside the margin of error above the 47.3% they got. The final Herald poll in 2008 had National on 47.9%, they got 44.9%. Statistically, the Herald isn’t just missing the bullseye, it’s missing the dartboard altogether. Only the 3news poll is worse. (in the image below, the light blue line is the Herald trend, the dark one is the poll of polls trend).

Dimpost poll of polls highlighting Herald Digipoll

b) even the Herald poll shows that National has lost a significant amount of support. From a pre-election high of 54.2%, a final pre-election number of 50.9%, to 48.5% today. That’s a big drop. Adjust to the Herald’s 3% over-polling of National at the last two elections, and it signals National is at around 45% – in the range that the Dimpost poll of polls has it. Remember, the Right has a one-seat majority in the House. Losing 2-3% since the election, especially when Banks will never get back in, is no laughing matter for National. If it were to translate into an election result, it would mean National would need New Zealand First (or a very large wasted vote) to govern.

c) more regular polls, like the Roy Morgan show a clear downward trend for National. You can’t possibly draw a trend from the five data points in 17 months that the Herald provides because the statistical noise that results in variation between every poll can’t be sensibly separated from the trend when you have so little to work with. The Roy Morgan has polled 29 times in that period and its trend is clear. (incidentally, the Roy Morgan was also the most accurate for National in 2011).

roy morgan labour greens v nats 3

To look at the Herald poll and conclude that National is now more popular than at the election, you have to ignore the Herald’s record of over-polling National, the fact that National’s support in the Herald poll has fallen significantly since the election, and the fact that more reliable polling trends tell us the opposite.

Only if you make all these logical errors (and there’s a strong incentive to do so because our paper spent a lot of money on the poll, which means its going to be treated as gospel no matter what, goddammit), can you then conclude National’s rule continues unblemished thanks to Bonny Prince Grinny.

But when’s a few fallacies ever stopped Granny?

[Note: none of this should be taken as a ‘if these trends continue’ argument. One current accurate polling, the Left needs Winnie too making him Kingmaker and he would have much more incentive to go with National, and there’s the risk of a Shearer campaign meltdown too. Put National at 40% and I’ll be happier].

17 comments on “Granny Herald: from flawed data, false conclusions”

  1. Tiresias 1

    “Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: “There are three kinds of lies: lies, damned lies and statistics.”
    – Mark Twain’s Own Autobiography

    Presently Labour will only be picking up those sufficiently dismayed by National’s duplicities and incompetence not to want them in Government again. Labour will only start picking up genuine support when it starts presenting an alternative programme worth genuinely supporting, and six-weeks out from an election is far too late to start thinking of doing that.

  2. Bob Tambling 2

    The worry for me then is if the polling trends continue and it looks like Shearer can just get over the line the possibilty of changing the leader will further diminish and Shearer will fight the 2014 election.
    Unfortunetly during the ensuing campaign, Key and the Nats will crucify him and we will be left with a National Government or a very weak LAb /NZF /Green one term Goverment
    I wonder from a Green party prospective (and a leftish pespective) , would it not be better if the Labour Party lost the next election with Shearer as Leader?
    If NZF play a part as may be likely, the Green can’t afford to go into a NZF /Lab/Green Gov.
    Shearer simply won’t cut the mustard and Peters is toxic. It would be a crap government and the well honed National party machine would again tear it to shreds

    The leaves the Greens in an interesting position, the lure of ministerial positions versus the pragmatic long term strategy of the Greens. A one term Lab/Nzf/Green Gov seems to serve no ones interest.
    Labour needs to have a strong leadership team and comitted activists , The reality is they have a diminishing group of activists and a crap leadership.
    The Greens on the other hand have a very strong leadership team and an ever growing group of activists ( let us not forget they are the second biggest party in Wellington).
    Labour? I can’t think of anyone in the present caucus who could take most of the country with them in an election. They need a good clean out and new blood. They don’t appear to be listening to that line of argument.
    In conclusion:
    I am committed to working very hard to getting as many Green MP’s elected as I can in 2014. I am not so sure though if I want those MP’s in a Government with the current infighting mob of the Lab party who it seems “could not pour piss out of a boot if the directions were on the heel”.

  3. tc 4

    The more granny and her shills like Johnno, o’shillivan, rortney etc keep banging the NACT drum the more irrelevant and bias they show themselves to be.

    Labour needs to grow a pair and get out there now however with the mallarfia and their front man it seems it’ll come down to the hollowmen blowing it more so than Labour winning it.

    They could easily blag another term if they weren’t such psychos given the current state of the opposition, thankfully they are what they are.

  4. ghostrider888 5

    article in todays Herald acknowledges the popularity of Key is slipping

  5. pete 6

    “Only the 3news poll is worse.”

    That’s not quite true: all five polls overestimate National’s election results, and the Digipoll isn’t too much worse than Fairfax or Roy Morgan. One News Colmar Brunton, as well as 3 News Reid Research both overestimate by more than the Herald.

    • lprent 6.1

      Points to a systematic flaw in their sampling doesn’t it…

      • pete 6.1.1

        That’s one possible explanation, although some have theorised that National lose a lot of votes close to the election once people actually start looking at their policies.

        There’s also a theory that NZF underpolls because their voters are embarrassed to admit that to callers.

        • lprent 6.1.1.1

          Possible. But less likely than the simpler explanation.

          Canvassing companies are using listed landlines*. When I was analyzing some somewhat larger samples of both door-knock canvassing and phone canvassing across several electorates, there was a statistically significant correlation between people who canvassed right and left and if they had a listed landline.

          For that matter there was an even stronger correlation between age group and having a listed land line – and it looks like if people don’t have a listed phone line in their mid 20’s then they never get one.

          * There is a common myth that goes around about auto-dialing random numbers. Asked a few people who work at or have worked at polling companies. It doesn’t happen and a few minutes thought about number ranges will give an explanation about why. It is cheaper to buy/get a phone list.

  6. Chris 7

    So,will Granny Herald be ramping up lots of little good news, happy smiley,relaxed all singing, dancing reports/photos of our poor excuse for a pm then. Got to keep him relevant.

  7. Centaur 8

    Give Key *some* credit. He may decide to call it quits. It is not as if he actually needs this gig ..

  8. Murray Olsen 9

    I’m not even sure that the margins of error as reported are very meaningful at all. From what I know, they seem to rely on unbiased sampling and the validity of the Central Limit Theorem, which then leads to a percentage presumably taken from 100 times √N/N. The concern about landlines is just one thing that could skew this.

    • lprent 9.1

      They do sample distribution normalising for many of the other factors like age, gender and geological distribution. They don’t adjust for factors like people working during the periods that they phone.

      But they don’t publish their techniques and criteria so it is pretty hard to see why there if it was unbiased sampling there would be any differences between the polls within the confidence limits. But there clearly is.

      • Centaur 9.1.1

        lprent: geological distribution ?
        Such as, where there are proven mineral resources ?

        • lprent 9.1.1.1

          I blame the iPad, it’s overly smart autocorrect, and that I use Geological at least 10x more often than Geographical.

    • Lanthanide 9.2

      No, they’re not. Even assuming they are following absolutely perfect statistical methods and the statistical methods work as advertised by the literature, the margin of error they’re reporting is presumably the 95% confidence interval. Which means 1 out of 20 polls are going to be wrong, but we won’t ever be able to tell which one without some other corroborating information (eg other polls taken at the same time).

      That’s assuming they’re using perfect methods, which of course they aren’t.

Recent Comments

Recent Posts

1 2 3 8

  • Tragedy must be impetus for better training
    The Police Minister needs to explain why unsworn and inadequately trained custody officers were put in a situation of caring for a medically unwell prisoner on a busy Saturday night, Labour’s Police spokesperson Kelvin Davis says. Commenting on an IPCA… ...
    4 hours ago
  • Government must be more transparent on investor state clauses
    The Government must be more transparent around the draft investor state dispute settlements in the TPPA, says David Parker, Labour’s Export Growth and Trade spokesperson. “Labour is pro trade, and is proud of the FTA we negotiated with China, which… ...
    8 hours ago
  • Protect university staff and student voices
    The Green Party believes ensuring student and staff representation on university councils is important. National recently passed a law reducing the size of university governance councils while increasing the proportion of the members nominated by, guess who… Steven Joyce. The… ...
    GreensBy Gareth Hughes MP
    1 day ago
  • C’mon Nick what’s the truth on the RMA
     “Nick Smith has got to fess up and tell us what is happening to his much vaunted RMA reform, Labour’s Environment spokesperson Megan Woods says.  “With just a day and a half to go before the polls open in Northland,… ...
    1 day ago
  • SSC salaries sink National’s spending spin
    Massive pay rises at the State Services Commission prove National’s claims of clamping down on spending in the public sector are simply fantasy, Labour’s State Services spokesman Kris Faafoi says. “Salaries in this one department are almost $70,000 more than… ...
    1 day ago
  • We can fix Christchurch and keep our assets
    The Christchurch City Council is seeking public feedback on its proposed 10 year plan for Council revenue and spending. This is probably one of the most significant 10 year plans ever to be written by a local council because of… ...
    GreensBy Eugenie Sage MP
    1 day ago
  • Epidemic of serious assaults in our prisons
    Labour wants stab proof vests and pepper spray for all corrections officers to keep them safe from the epidemic of serious prison assaults that are occurring around the country’s jails, says Labour’s Corrections Spokesperson Kelvin Davis.  “There have been five… ...
    1 day ago
  • Listen to the locals Hekia!
    Minister Hekia Parata needs to understand what consultation is, Port Hills MP Ruth Dyson says. “It means you have to listen to what people say in their submissions and then be able to demonstrate you have considered their views when… ...
    2 days ago
  • Thanking our caregivers
    Let’s celebrate and thank our caregivers. This week is caregivers’ week. It’s a chance to acknowledge the thousands of women, and occasional other person, who are caring for the elderly and disabled in our country. They hold people’s lives in… ...
    GreensBy Jan Logie MP
    2 days ago
  • Mana Post shop the best outcome for community
    Labour MP for Mana Kris Faafoi has welcomed the move to place the services from the Mana Post shop to a local small business. “This is the best outcome for the community we could ask for. All the vital services… ...
    2 days ago
  • Labour looks to put the tea back into entitlements
    Labour is moving to restore the rights of Kiwis to take tea and rest breaks, Leader Andrew Little says. “Within months of the Government’s Employment Relations Amendment Bill becoming law we are already seeing some of our largest companies, including… ...
    2 days ago
  • Desperate money grab to keep Ruataniwha afloat
    The Hawke’s Bay Regional Investment Company’s decision to borrow $4 million to keep the Ruataniwha project afloat is a case of throwing ratepayer’s good money after bad, says Labour’s Water spokesperson Meka Whaitiri and Napier MP Stuart Nash.   “This bridging… ...
    3 days ago
  • Roundup: UN finds it “probably” causes cancer
    At last the UN has spoken out against the widely-used weedkiller Roundup. The UN’s International Agency for Research on Cancer has identified glyphosate, the principle ingredient in Roundup, as a probable carcinogen. They also include as probable carcinogens the insecticides… ...
    GreensBy Steffan Browning MP
    3 days ago
  • Invermay petition delivered to Parliament
    Labour’s Economic Development spokesperson David Clark handed over a 12,450 signature Save Invermay petition to Dunedin South MP Clare Curran on the steps of Parliament today.  “The level of support that the petition has received across New Zealand is overwhelming,”… ...
    3 days ago
  • Redcliffs School closure plan wrong
    The Government’s proposal to consult on the closure of Redcliffs School not only goes against the best geotechnical advice, but more importantly goes against the best educational outcomes for Redcliffs children and the health of our community, Port Hills MP… ...
    3 days ago
  • Cotton On first to test the tea breaks law
    Australian corporate Cotton On, the first major business operating in New Zealand to exploit the new tea breaks law, could walk away from negotiations if it doesn’t get its own way, says Labour Party Labour Relations spokesperson Iain Lees-Galloway.  “Cotton… ...
    4 days ago
  • World water day: eight rivers in one day
    Our photo journey started by the Waioweka (also known as Waioeka) River which flows from Te Urewera to Opotiki, and is surrounded by beautiful forest. The water looked great! Kopeopeo Canal It contrasted greatly with the Kopeopeo Canal near Whakatane,… ...
    GreensBy Catherine Delahunty MP
    4 days ago
  • Council can stop Port’s encroachment on harbour
    As owner of the Port of Auckland, Council can stop the wharf extension and reclamation if it wants to, says Labour’s Auckland Issues spokesperson Phil Goff. ‘As owner the council is custodian of the port and harbour on behalf of… ...
    4 days ago
  • We all benefit when education meets everyone’s needs
    As Dyslexia week comes to a close,  Dyslexia NZ have reminded us that around 10% of our citizens are dyslexic and are entitled to better support. One of their strongest arguments is that failure to provide identification and support for… ...
    GreensBy Catherine Delahunty MP
    4 days ago
  • State house sell-off fiasco a gift for developers
    The Government’s property developer mates are the only people who can salvage National’s state house sell-off now the Salvation Army has torpedoed the policy, Labour’s Housing spokesperson Phil Twyford says. “Having been cynically used by the Government as the poster… ...
    4 days ago
  • National reinforces inequality in schools
    The National Government’s flagship programme Investing in Educational Success is clearly reinforcing inequality in the school system, Labour’s Education spokesperson Chris Hipkins says. “The analysis released today by the NZEI clearly shows schools in wealthier suburbs are the main beneficiaries… ...
    4 days ago
  • Big change starts small
    Today marks Race Relations Day in New Zealand. Race Relations Day coincides with the International Day for the Elimination of Racial Discrimination.  The United Nations General Assembly chose this day as it marks the day in 1960 when 69 peaceful… ...
    GreensBy Denise Roche MP
    6 days ago
  • Israel, Palestine and the question of statehood
    The knife-edge election in Israel complicates the Middle East situation, even more than usual. The Prime Minister-elect, Binyamin Netanyahu, is moving to form a government. Netanyahu has indicated that, during his term, a Palestinian state would not be established. That… ...
    GreensBy Kennedy Graham MP
    1 week ago
  • Christchurch transport goes backwards
    The Green Party has a vision of a liveable, accessible Christchurch with a sense of identity and strong connected communities. Instead, 2013 census figures released by Statistics New Zealand reveal a fractured community, and tell a story of frustrated Christchurch commuters… ...
    GreensBy Eugenie Sage MP
    1 week ago
  • Super Fund should divest $140 million in high risk coal
    The Green Party is calling on the New Zealand Super Fund to divest their $140 million investment in coal companies that are vulnerable to becoming financially stranded according to a damning new report from Oxford University. The Smith School of… ...
    GreensBy Russel Norman MP
    1 week ago
  • Learn to count with Mark Osborne: 0 + 1 = ?
    The adage about the first casualty of war being truth is one that might often be applied to the political battle for hearts and minds, and of course votes. A rather unfortunate example of this has been arriving in the… ...
    GreensBy David Clendon MP
    1 week ago
  • Is it still a safety net when the holes are this big?
    Over the last few weeks I’ve been wondering how safe our income support system is for people, especially those with cognitive or learning disabilities. I’ve been trying to support a young man who was severely injured in a workplace accident… ...
    GreensBy Jan Logie MP
    1 week ago
  • Pasifika – protecting the Pacific needed now more than ever.
    Over the weekend thousands of Aucklanders flocked to celebrate our city’s diverse Pacific communities and cultures at the annual Pasifika festival and the Greens were there to join them. The Pasifika festival has been held every year for 23… ...
    GreensBy Denise Roche MP
    1 week ago
  • Sounds Stakeholders Seek a Sustainable Future
    It was heartening to see a large number of people who care about the Marlborough Sounds come together at the Marlborough Marine Futures’ forum in Picton on March 8. Fellow Green MP Steffan Browning, who lives in Marlborough, and I… ...
    GreensBy Eugenie Sage MP
    2 weeks ago
  • Solid Energy, who will clean up the mess?
    What can you say? This state-owned coal miner is facing some very serious problems. They haven’t run a profit in years, have required two Government bailouts, laid-off more than 700 staff and look like they need a third injection of… ...
    GreensBy Gareth Hughes MP
    2 weeks ago

Removed at the request of The Daily Blog.
Public service advertisements by The Standard

Current CO2 level in the atmosphere