Granny Herald: from flawed data, false conclusions

Written By: - Date published: 11:21 am, March 23rd, 2013 - 17 comments
Categories: john key, national, polls - Tags:

Three of the Herald’s political staff has a crack at trying to explain why National polled 48.5% in the latest Herald-Digipoll. Consensus seems to be that John Key’s an angel sent to earth to do very little but keep enough of the middle class happy enough who’ll never lose. May I humbly suggest an alternative theory – that the Herald-Digipoll is broken.

Danyl at the Dimpost has the most mathematically poll of polls that, crucially, is anchored in reality – ie the last election result. Whereas political commentators and pollsters seem to shamelessly ignore how badly their poll got it wrong at the last election when breathlessly reporting the latest numbers, the Dimpost poll of polls corrects for their error.

What the Dimpost poll shows is that:

a) the Herald Digi-poll consistently over measures National – for example, the final Herald poll before the last election had National on 50.9%, that’s outside the margin of error above the 47.3% they got. The final Herald poll in 2008 had National on 47.9%, they got 44.9%. Statistically, the Herald isn’t just missing the bullseye, it’s missing the dartboard altogether. Only the 3news poll is worse. (in the image below, the light blue line is the Herald trend, the dark one is the poll of polls trend).

Dimpost poll of polls highlighting Herald Digipoll

b) even the Herald poll shows that National has lost a significant amount of support. From a pre-election high of 54.2%, a final pre-election number of 50.9%, to 48.5% today. That’s a big drop. Adjust to the Herald’s 3% over-polling of National at the last two elections, and it signals National is at around 45% – in the range that the Dimpost poll of polls has it. Remember, the Right has a one-seat majority in the House. Losing 2-3% since the election, especially when Banks will never get back in, is no laughing matter for National. If it were to translate into an election result, it would mean National would need New Zealand First (or a very large wasted vote) to govern.

c) more regular polls, like the Roy Morgan show a clear downward trend for National. You can’t possibly draw a trend from the five data points in 17 months that the Herald provides because the statistical noise that results in variation between every poll can’t be sensibly separated from the trend when you have so little to work with. The Roy Morgan has polled 29 times in that period and its trend is clear. (incidentally, the Roy Morgan was also the most accurate for National in 2011).

roy morgan labour greens v nats 3

To look at the Herald poll and conclude that National is now more popular than at the election, you have to ignore the Herald’s record of over-polling National, the fact that National’s support in the Herald poll has fallen significantly since the election, and the fact that more reliable polling trends tell us the opposite.

Only if you make all these logical errors (and there’s a strong incentive to do so because our paper spent a lot of money on the poll, which means its going to be treated as gospel no matter what, goddammit), can you then conclude National’s rule continues unblemished thanks to Bonny Prince Grinny.

But when’s a few fallacies ever stopped Granny?

[Note: none of this should be taken as a ‘if these trends continue’ argument. One current accurate polling, the Left needs Winnie too making him Kingmaker and he would have much more incentive to go with National, and there’s the risk of a Shearer campaign meltdown too. Put National at 40% and I’ll be happier].

17 comments on “Granny Herald: from flawed data, false conclusions”

  1. Tiresias 1

    “Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: “There are three kinds of lies: lies, damned lies and statistics.”
    – Mark Twain’s Own Autobiography

    Presently Labour will only be picking up those sufficiently dismayed by National’s duplicities and incompetence not to want them in Government again. Labour will only start picking up genuine support when it starts presenting an alternative programme worth genuinely supporting, and six-weeks out from an election is far too late to start thinking of doing that.

  2. Bob Tambling 2

    The worry for me then is if the polling trends continue and it looks like Shearer can just get over the line the possibilty of changing the leader will further diminish and Shearer will fight the 2014 election.
    Unfortunetly during the ensuing campaign, Key and the Nats will crucify him and we will be left with a National Government or a very weak LAb /NZF /Green one term Goverment
    I wonder from a Green party prospective (and a leftish pespective) , would it not be better if the Labour Party lost the next election with Shearer as Leader?
    If NZF play a part as may be likely, the Green can’t afford to go into a NZF /Lab/Green Gov.
    Shearer simply won’t cut the mustard and Peters is toxic. It would be a crap government and the well honed National party machine would again tear it to shreds

    The leaves the Greens in an interesting position, the lure of ministerial positions versus the pragmatic long term strategy of the Greens. A one term Lab/Nzf/Green Gov seems to serve no ones interest.
    Labour needs to have a strong leadership team and comitted activists , The reality is they have a diminishing group of activists and a crap leadership.
    The Greens on the other hand have a very strong leadership team and an ever growing group of activists ( let us not forget they are the second biggest party in Wellington).
    Labour? I can’t think of anyone in the present caucus who could take most of the country with them in an election. They need a good clean out and new blood. They don’t appear to be listening to that line of argument.
    In conclusion:
    I am committed to working very hard to getting as many Green MP’s elected as I can in 2014. I am not so sure though if I want those MP’s in a Government with the current infighting mob of the Lab party who it seems “could not pour piss out of a boot if the directions were on the heel”.

  3. tc 4

    The more granny and her shills like Johnno, o’shillivan, rortney etc keep banging the NACT drum the more irrelevant and bias they show themselves to be.

    Labour needs to grow a pair and get out there now however with the mallarfia and their front man it seems it’ll come down to the hollowmen blowing it more so than Labour winning it.

    They could easily blag another term if they weren’t such psychos given the current state of the opposition, thankfully they are what they are.

  4. ghostrider888 5

    article in todays Herald acknowledges the popularity of Key is slipping

  5. pete 6

    “Only the 3news poll is worse.”

    That’s not quite true: all five polls overestimate National’s election results, and the Digipoll isn’t too much worse than Fairfax or Roy Morgan. One News Colmar Brunton, as well as 3 News Reid Research both overestimate by more than the Herald.

    • lprent 6.1

      Points to a systematic flaw in their sampling doesn’t it…

      • pete 6.1.1

        That’s one possible explanation, although some have theorised that National lose a lot of votes close to the election once people actually start looking at their policies.

        There’s also a theory that NZF underpolls because their voters are embarrassed to admit that to callers.

        • lprent 6.1.1.1

          Possible. But less likely than the simpler explanation.

          Canvassing companies are using listed landlines*. When I was analyzing some somewhat larger samples of both door-knock canvassing and phone canvassing across several electorates, there was a statistically significant correlation between people who canvassed right and left and if they had a listed landline.

          For that matter there was an even stronger correlation between age group and having a listed land line – and it looks like if people don’t have a listed phone line in their mid 20’s then they never get one.

          * There is a common myth that goes around about auto-dialing random numbers. Asked a few people who work at or have worked at polling companies. It doesn’t happen and a few minutes thought about number ranges will give an explanation about why. It is cheaper to buy/get a phone list.

  6. Chris 7

    So,will Granny Herald be ramping up lots of little good news, happy smiley,relaxed all singing, dancing reports/photos of our poor excuse for a pm then. Got to keep him relevant.

  7. Centaur 8

    Give Key *some* credit. He may decide to call it quits. It is not as if he actually needs this gig ..

  8. Murray Olsen 9

    I’m not even sure that the margins of error as reported are very meaningful at all. From what I know, they seem to rely on unbiased sampling and the validity of the Central Limit Theorem, which then leads to a percentage presumably taken from 100 times √N/N. The concern about landlines is just one thing that could skew this.

    • lprent 9.1

      They do sample distribution normalising for many of the other factors like age, gender and geological distribution. They don’t adjust for factors like people working during the periods that they phone.

      But they don’t publish their techniques and criteria so it is pretty hard to see why there if it was unbiased sampling there would be any differences between the polls within the confidence limits. But there clearly is.

      • Centaur 9.1.1

        lprent: geological distribution ?
        Such as, where there are proven mineral resources ?

        • lprent 9.1.1.1

          I blame the iPad, it’s overly smart autocorrect, and that I use Geological at least 10x more often than Geographical.

    • Lanthanide 9.2

      No, they’re not. Even assuming they are following absolutely perfect statistical methods and the statistical methods work as advertised by the literature, the margin of error they’re reporting is presumably the 95% confidence interval. Which means 1 out of 20 polls are going to be wrong, but we won’t ever be able to tell which one without some other corroborating information (eg other polls taken at the same time).

      That’s assuming they’re using perfect methods, which of course they aren’t.

Recent Comments

Recent Posts

  • More proof National not delivering for New Zealanders
    The Ministry of Social Development’s latest Household Incomes Report is more hard evidence of the growing poverty in New Zealand, says Leader of the Opposition Andrew Little. “National claims to be delivering for New Zealanders. This report puts the lie ...
    19 hours ago
  • Strong support from medical community for Healthy Homes Bill
    More than a thousand medical professionals, medical students, and other New Zealanders have shown their support for Labour’s Healthy Homes Guarantee Bill, says Leader of the Opposition Andrew Little. ...
    21 hours ago
  • Labour will fund Buller Health Centre
    A Labour Government will fully fund the Buller Integrated Family Health Centre in Westport, confirming its commitment to the people of the West Coast and the role of central government to fund essential public services, says Labour’s Health spokesperson David ...
    22 hours ago
  • National’s affordable promise for Auckland more smoke and mirrors
    The 26,000 additional houses National plans to build in Auckland on public land over the next decade include fewer than 5,000 affordable homes, while more than 60 per cent will be available for speculators to buy, says Leader of the ...
    1 day ago
  • National out of touch over immigration
    National’s abrupt backflip on their recently-announced changes to immigration shows they never understood the problem and just came up with a confused knee jerk response, says Leader of the Opposition Andrew Little. “Regional communities and businesses were quite right to ...
    2 days ago
  • English out of touch on homelessness
    Bill English’s comments that he doesn’t know why people are complaining about the blowout in the number of homeless families the government is putting up in motels just shows how tired and out of touch National is after nine years, ...
    2 days ago
  • All Kiwis to have same standard of cancer care
    Labour is promising that all New Zealanders will have access to the same level of cancer care no matter where they live in the country, says Leader of the Opposition Andrew Little.   “As someone who has survived cancer I ...
    3 days ago
  • Infrastructure announcement too long coming
    “What took you so long?” is Labour’s response to the Government’s announcement of a new infrastructure investment vehicle. Labour’s Auckland Issues spokesperson Phil Twyford says Labour announced its policy in 2015 to debt-finance infrastructure and service that debt with targeted ...
    3 days ago
  • Time for a breather on immigration
    National has no idea how to house the record number of people entering New Zealand, let alone cope with the pressure on health, education, and transport from this record population growth, says Labour’s Immigration spokesperson Iain Lees-Galloway. ...
    5 days ago
  • Labour to invest $4 billion in education
    Labour’s Education Manifesto will bring positive change across the education sector and is backed by a massive investment, says Labour’s Education Spokesperson Chris Hipkins.  “Labour’s plan will see an extra $4 billion invested over the next four years. It’s organised ...
    5 days ago
  • National’s shame: worst homelessness in the OECD
    National’s legacy is a housing crisis that has given New Zealand the worst homeless rate in the developed world, says Labour’s Housing spokesperson Phil Twyford. ...
    5 days ago
  • Labour taking action on school donations
    Labour will end so-called voluntary school donations for the majority of parents across the country under its $4 billion plan to revitalise the education sector, says Labour Leader Andrew Little. “Labour has always been committed to a world-class free education ...
    5 days ago
  • Labour to work with Queenstown to build more houses
    Labour will work with Queenstown-Lakes District Council, iwi, and the Community Housing Trust to build the modern, affordable housing Queenstown desperately needs, says Leader of the Opposition Andrew Little. ...
    6 days ago
  • Nats blow the Budget on motels after bowling state houses
    National is spending $140,000 a day putting homeless families in motels, the legacy of nine years of selling off and knocking down state houses, says Labour Housing spokesperson Phil Twyford. ...
    6 days ago
  • New revelations in Joanne Harrison report
    The State Services Commission’s report into the treatment of whistle-blowers by Joanne Harrison has revealed new accusations against the convicted fraudster, says Labour MP Sue Moroney.  “The report found that four staff inside the Ministry of Transport who had raised ...
    6 days ago
  • Snafu at Princess Margaret
    Jonathan Coleman has to stop the stalling over a new building for mental health services in Christchurch to replace the quake damaged Princess Margaret Hospital, says Labour’s Health spokesperson David Clark. “The Government must accept that Christchurch is still recovering ...
    6 days ago
  • Labour’s fiscal plan to build a fairer New Zealand
    Labour will re-build our housing, health and education while responsibly managing New Zealand’s finances, says Leader of the Opposition Andrew Little.  “Under Labour’s Fiscal Plan we will deliver big investments in the services we all need and care about, invest ...
    7 days ago
  • Nats show they’re the tax dodgers’ best friends
    The government is taking the knife to IRD at a time when we need a highly skilled department to ensure that multinationals and speculators don’t get away with dodging tax, says Labour’s Revenue spokesperson Michael Wood. ...
    7 days ago
  • Labour secures the future for NZ Super
    A Labour Government will secure the future for New Zealand Superannuation so we can continue to provide superannuation to those retiring at age 65, says Leader of the Opposition Andrew Little. “One of the first things a Labour-led Government will ...
    1 week ago
  • Multinationals must pay fair share of tax
    A Labour Government will crack down on multinational companies that are dodging paying their fair share of tax, says Labour Leader Andrew Little. “New Zealanders are missing out by hundreds of millions according to the IRD because multinational companies can ...
    1 week ago
  • ACT’s approach to children backward and ill informed
    Act’s new deputy leader’s claim that Labour’s support for families could “extend the misery of child poverty and even child abuse” is ill informed and offensive, says Labour’s Deputy Leader Jacinda Ardern. ...
    2 weeks ago
  • Canterbury hatchet job a disgrace
    The Government’s glib acceptance of advice that the Canterbury District Health Board doesn’t need more money is a hatchet job and a disgrace, says Labour’s Health Spokesperson David Clark. “To claim that the DHB was using tactics to leverage more ...
    2 weeks ago
  • Quality for Kiwi kids at ECE
    After more than a decade of rapid growth in the number of children participating in Early Childhood Education (ECE), it’s time to take stock and map out a clear plan for the future, says Labour’s Education spokesperson Chris Hipkins. ...
    2 weeks ago
  • Labour to boost ECE quality
    Labour will ensure kids get the best start in life by boosting funding for Early Childhood Centres to employ 100 per cent qualified and registered teachers, says Leader of the Opposition Andrew Little. ...
    2 weeks ago
  • Labour will stump up a million dollars for Maniototo Hospital
    A Labour led Government will make a million dollars available to rebuild the Maniototo Base hospital in Ranfurly, says the Leader of the Opposition Andrew Little.  “This will be a much needed boost for a long overdue rebuild that has ...
    2 weeks ago
  • No vision for the West Coast
    The West Coast welcomes any Government investment in our region but the lack of any real alternative vision for the West Coast’s economy is disappointing, says Damien O’Connor Labour’s West Coast-Tasman MP.  “The establishment of a Mining Research Unit will ...
    2 weeks ago
  • National’s youth work scheme too little too late
    After nine years, National’s belated attempt to provide work opportunities for unemployed youth should be seen for what it is, a half-hearted, election gimmick from a party that’s ignored the problem till now, says Leader of the Opposition Andrew Little. ...
    2 weeks ago
  • Kiwis won’t fall for Joyce’s spin
    Steven Joyce’s embarrassingly obvious spin on Labour’s Families Package won’t fool anyone, says Labour’s Finance spokesperson Grant Robertson. ...
    2 weeks ago
  • Labour prioritises families and public services
    Labour’s Families Package delivers a bigger income boost to more than 70 per cent of families with children than Budget 2017. By not spending $1.5 billion a year on tax cuts, Labour is able to do more for lower and ...
    2 weeks ago
  • Kiwis can’t sleep in your ghost houses, Nick
    The Government’s housing infrastructure announcement is another Nick Smith special – over-promising with no detail on delivery, says Labour’s Housing spokesperson Phil Twyford. ...
    2 weeks ago
  • Labour helps older New Zealanders and low income families with winter heating bills
    Labour will further boost its commitment to warm, healthy housing with a Winter Energy Payment for superannuitants and people receiving main benefits, says Leader of the Opposition Andrew Little. “Everyone deserves a warm, healthy home to live in. But that’s ...
    2 weeks ago
  • National must rule out retrospective override for Ruataniwha
    National must categorically rule out using retrospective legislation to override the Supreme Court’s decision that the land swap of conservation land flooded by the proposed Ruataniwha Dam was illegal, says Labour’s Shadow Attorney General David Parker. “Having not got their ...
    3 weeks ago
  • Flavell’s failure a win for Māori landowners
    The Māori Development Minister’s admission that his unpopular Ture Whenua Māori Bill won’t pass into law prior to the election is a victory for Māori landowners, but only a change of government will keep the Bill gone for good, says ...
    3 weeks ago
  • Stats confirm growing housing shortfall
    National’s failure to fix the housing shortage has been starkly illustrated by new statistics, says Labour Housing spokesperson Phil Twyford. ...
    3 weeks ago
  • Systemic abuse of kids in state care
    After admitting there was systemic abuse of children in State care the Government must do the right thing and launch an independent inquiry, says Labour’s Deputy Leader Jacinda Ardern. ...
    3 weeks ago
  • Migrant worker exploitation needs sharper focus
    The astonishing number of employers found guilty of exploiting migrants shows that migrant exploitation is a serious problem in New Zealand, says Labour Workplace Relations and Safety spokesperson Iain Lees-Galloway. “A total of 53 companies have been banned from recruiting ...
    3 weeks ago
  • Minister faces questions over dam debacle
    Today’s Supreme Court ruling dismissing an appeal to allow a land swap for the controversial Ruataniwha Dam is a victory for our conservation estate and Hawke’s Bay ratepayers, but leaves the Conservation Minister with serious questions to answer, says Ikaroa-Rāwhiti ...
    3 weeks ago
  • Too little too late on Wellington housing
    The announcement today on social housing in Wellington by the National Government is a pitiful and cynical election ploy, says Labour’s Wellington Central MP Grant Robertson. “In 2012 Housing New Zealand emptied out the Gordon Wilson Flats, taking 130 places ...
    3 weeks ago