web analytics
The Standard

Granny Herald: from flawed data, false conclusions

Written By: - Date published: 11:21 am, March 23rd, 2013 - 17 comments
Categories: john key, national, polls - Tags:

Three of the Herald’s political staff has a crack at trying to explain why National polled 48.5% in the latest Herald-Digipoll. Consensus seems to be that John Key’s an angel sent to earth to do very little but keep enough of the middle class happy enough who’ll never lose. May I humbly suggest an alternative theory – that the Herald-Digipoll is broken.

Danyl at the Dimpost has the most mathematically poll of polls that, crucially, is anchored in reality – ie the last election result. Whereas political commentators and pollsters seem to shamelessly ignore how badly their poll got it wrong at the last election when breathlessly reporting the latest numbers, the Dimpost poll of polls corrects for their error.

What the Dimpost poll shows is that:

a) the Herald Digi-poll consistently over measures National – for example, the final Herald poll before the last election had National on 50.9%, that’s outside the margin of error above the 47.3% they got. The final Herald poll in 2008 had National on 47.9%, they got 44.9%. Statistically, the Herald isn’t just missing the bullseye, it’s missing the dartboard altogether. Only the 3news poll is worse. (in the image below, the light blue line is the Herald trend, the dark one is the poll of polls trend).

Dimpost poll of polls highlighting Herald Digipoll

b) even the Herald poll shows that National has lost a significant amount of support. From a pre-election high of 54.2%, a final pre-election number of 50.9%, to 48.5% today. That’s a big drop. Adjust to the Herald’s 3% over-polling of National at the last two elections, and it signals National is at around 45% – in the range that the Dimpost poll of polls has it. Remember, the Right has a one-seat majority in the House. Losing 2-3% since the election, especially when Banks will never get back in, is no laughing matter for National. If it were to translate into an election result, it would mean National would need New Zealand First (or a very large wasted vote) to govern.

c) more regular polls, like the Roy Morgan show a clear downward trend for National. You can’t possibly draw a trend from the five data points in 17 months that the Herald provides because the statistical noise that results in variation between every poll can’t be sensibly separated from the trend when you have so little to work with. The Roy Morgan has polled 29 times in that period and its trend is clear. (incidentally, the Roy Morgan was also the most accurate for National in 2011).

roy morgan labour greens v nats 3

To look at the Herald poll and conclude that National is now more popular than at the election, you have to ignore the Herald’s record of over-polling National, the fact that National’s support in the Herald poll has fallen significantly since the election, and the fact that more reliable polling trends tell us the opposite.

Only if you make all these logical errors (and there’s a strong incentive to do so because our paper spent a lot of money on the poll, which means its going to be treated as gospel no matter what, goddammit), can you then conclude National’s rule continues unblemished thanks to Bonny Prince Grinny.

But when’s a few fallacies ever stopped Granny?

[Note: none of this should be taken as a ‘if these trends continue’ argument. One current accurate polling, the Left needs Winnie too making him Kingmaker and he would have much more incentive to go with National, and there’s the risk of a Shearer campaign meltdown too. Put National at 40% and I’ll be happier].

17 comments on “Granny Herald: from flawed data, false conclusions”

  1. Tiresias 1

    “Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: “There are three kinds of lies: lies, damned lies and statistics.”
    – Mark Twain’s Own Autobiography

    Presently Labour will only be picking up those sufficiently dismayed by National’s duplicities and incompetence not to want them in Government again. Labour will only start picking up genuine support when it starts presenting an alternative programme worth genuinely supporting, and six-weeks out from an election is far too late to start thinking of doing that.

  2. Bob Tambling 2

    The worry for me then is if the polling trends continue and it looks like Shearer can just get over the line the possibilty of changing the leader will further diminish and Shearer will fight the 2014 election.
    Unfortunetly during the ensuing campaign, Key and the Nats will crucify him and we will be left with a National Government or a very weak LAb /NZF /Green one term Goverment
    I wonder from a Green party prospective (and a leftish pespective) , would it not be better if the Labour Party lost the next election with Shearer as Leader?
    If NZF play a part as may be likely, the Green can’t afford to go into a NZF /Lab/Green Gov.
    Shearer simply won’t cut the mustard and Peters is toxic. It would be a crap government and the well honed National party machine would again tear it to shreds

    The leaves the Greens in an interesting position, the lure of ministerial positions versus the pragmatic long term strategy of the Greens. A one term Lab/Nzf/Green Gov seems to serve no ones interest.
    Labour needs to have a strong leadership team and comitted activists , The reality is they have a diminishing group of activists and a crap leadership.
    The Greens on the other hand have a very strong leadership team and an ever growing group of activists ( let us not forget they are the second biggest party in Wellington).
    Labour? I can’t think of anyone in the present caucus who could take most of the country with them in an election. They need a good clean out and new blood. They don’t appear to be listening to that line of argument.
    In conclusion:
    I am committed to working very hard to getting as many Green MP’s elected as I can in 2014. I am not so sure though if I want those MP’s in a Government with the current infighting mob of the Lab party who it seems “could not pour piss out of a boot if the directions were on the heel”.

  3. tc 4

    The more granny and her shills like Johnno, o’shillivan, rortney etc keep banging the NACT drum the more irrelevant and bias they show themselves to be.

    Labour needs to grow a pair and get out there now however with the mallarfia and their front man it seems it’ll come down to the hollowmen blowing it more so than Labour winning it.

    They could easily blag another term if they weren’t such psychos given the current state of the opposition, thankfully they are what they are.

  4. ghostrider888 5

    article in todays Herald acknowledges the popularity of Key is slipping

  5. pete 6

    “Only the 3news poll is worse.”

    That’s not quite true: all five polls overestimate National’s election results, and the Digipoll isn’t too much worse than Fairfax or Roy Morgan. One News Colmar Brunton, as well as 3 News Reid Research both overestimate by more than the Herald.

    • lprent 6.1

      Points to a systematic flaw in their sampling doesn’t it…

      • pete 6.1.1

        That’s one possible explanation, although some have theorised that National lose a lot of votes close to the election once people actually start looking at their policies.

        There’s also a theory that NZF underpolls because their voters are embarrassed to admit that to callers.

        • lprent 6.1.1.1

          Possible. But less likely than the simpler explanation.

          Canvassing companies are using listed landlines*. When I was analyzing some somewhat larger samples of both door-knock canvassing and phone canvassing across several electorates, there was a statistically significant correlation between people who canvassed right and left and if they had a listed landline.

          For that matter there was an even stronger correlation between age group and having a listed land line – and it looks like if people don’t have a listed phone line in their mid 20’s then they never get one.

          * There is a common myth that goes around about auto-dialing random numbers. Asked a few people who work at or have worked at polling companies. It doesn’t happen and a few minutes thought about number ranges will give an explanation about why. It is cheaper to buy/get a phone list.

  6. Chris 7

    So,will Granny Herald be ramping up lots of little good news, happy smiley,relaxed all singing, dancing reports/photos of our poor excuse for a pm then. Got to keep him relevant.

  7. Centaur 8

    Give Key *some* credit. He may decide to call it quits. It is not as if he actually needs this gig ..

  8. Murray Olsen 9

    I’m not even sure that the margins of error as reported are very meaningful at all. From what I know, they seem to rely on unbiased sampling and the validity of the Central Limit Theorem, which then leads to a percentage presumably taken from 100 times √N/N. The concern about landlines is just one thing that could skew this.

    • lprent 9.1

      They do sample distribution normalising for many of the other factors like age, gender and geological distribution. They don’t adjust for factors like people working during the periods that they phone.

      But they don’t publish their techniques and criteria so it is pretty hard to see why there if it was unbiased sampling there would be any differences between the polls within the confidence limits. But there clearly is.

      • Centaur 9.1.1

        lprent: geological distribution ?
        Such as, where there are proven mineral resources ?

        • lprent 9.1.1.1

          I blame the iPad, it’s overly smart autocorrect, and that I use Geological at least 10x more often than Geographical.

    • Lanthanide 9.2

      No, they’re not. Even assuming they are following absolutely perfect statistical methods and the statistical methods work as advertised by the literature, the margin of error they’re reporting is presumably the 95% confidence interval. Which means 1 out of 20 polls are going to be wrong, but we won’t ever be able to tell which one without some other corroborating information (eg other polls taken at the same time).

      That’s assuming they’re using perfect methods, which of course they aren’t.

Recent Comments

Recent Posts

  • Government has no credible climate change plan
    Today’s announced climate change target falls short of the ambition required to meet even our existing targets, says Labour’s Environment spokesperson Megan Woods. “The target announced today amounts to a decrease of only 11 per cent from 1990 levels. This… ...
    11 hours ago
  • Auckland house prices now 10 times incomes
    Auckland house prices have risen so steeply the typical house in our biggest city now costs 10 times the median Auckland household income, Labour’s Housing spokesperson Phil Twyford says. “Barfoot and Thompson reports the median house sale price in June… ...
    16 hours ago
  • Time for economic spin is over
     Business confidence in the latest NZIER Quarterly Survey of Business Opinion falling to its lowest level in three years is yet another warning of a staggering economy that cannot be ignored, says Labour's leader Andrew Little.   “This comes on the back of dairy prices falling… ...
    16 hours ago
  • Bullying contributes to Auckland being stripped of ICU training
    Complaints of bullying and harassment by supervisors which have contributed to Auckland’s critical care department losing its training accreditation are further evidence of the appalling culture at executive level, Labour’s Health spokesperson Annette King says. “The department had its accreditation… ...
    4 days ago
  • Broadband failure sucks up more cash
    The Commerce Committee has blocked an inquiry into the $300 million rural broadband initiative (RBI) despite mounting evidence it’s a massive policy failure and waste of money, says Labour’s ICT spokesperson Clare Curran. “The Government is about to spend an… ...
    4 days ago
  • TISA – Another secret trade deal you may never have heard of
      This post first appeared on The Daily Blog You’ve probably heard of the Trans-Pacific Partnership Agreement (TPPA) by now and the widespread concerns around it but what about the Trade in Services Agreement (TISA) also being currently negotiated by… ...
    GreensBy Gareth Hughes MP
    5 days ago
  • Health chickens coming home to roost as Dunedin loses right to train doctor...
    News today that Dunedin Hospital has lost orthopaedic training accreditation is a major blow and proves the Government’s prevarication is having devastating consequences, Labour’s Health spokesperson Annette King says. “Losing orthopaedic advanced training is serious. There is a knock on… ...
    6 days ago
  • $74,000 quarterly rise shows crisis out of control
    New figures out today showing Auckland house prices have spiked by a massive $74,000 in the past quarter is further evidence the city’s housing crisis has spiralled out of control, Labour’s “In spite of constant announcements and photo opportunities from… ...
    6 days ago
  • Democracy for Nauru now
    Murray McCully must send the strongest possible message to the Nauruan Government that New Zealand does not condone its actions given the disturbing developments there, Labour’s Foreign Affairs spokesperson David Shearer says. “Right now we are seeing Nauru stripped of… ...
    6 days ago
  • Recovery needs more than a rebrand
    Today’s announcement of new governance arrangements for Canterbury seems to be nothing more than a fresh coat of paint on the same old approach, says Labour’s Canterbury Earthquake Recovery spokesperson Ruth Dyson. “The Canterbury Recovery has been too slow, with… ...
    6 days ago
  • Copper decision a victory for status quo, not Kiwi households
    New Zealanders hoping for cheaper copper broadband will be disappointed by the Commerce Commission’s latest decision in the long running saga to determine the price of copper, Labour’s ICT spokesperson Clare Curran says. “In an apparent attempt to appease everyone,… ...
    6 days ago
  • It’s time for hard decisions in the Bay
     The Ruataniwha dam project is turning into a huge white elephant as the economics fail to stack up, says Labour’s Water spokesperson Meka Whaitiri.  “Ruataniwha simply doesn’t make economic sense when you look at other major irrigation schemes around the… ...
    6 days ago
  • More testing won’t lift student achievement
    Hekia Parata’s latest plan to subject school students to even more testing and assessment won’t do anything to lift the educational achievement of the kids who are struggling, Labour’s Education spokesperson Chris Hipkins says. “New Zealand school students are already… ...
    6 days ago
  • Bad week for NZ economy gets worse
    The bad news for the New Zealand economy got worse this morning with the 8th successive drop in dairy prices at this morning’s global dairy auction, again exposing the absence of any Plan B from the National Government, Labour’s Finance… ...
    6 days ago
  • System failing to protect women and children from family violence
    Last week we called for mandatory child safety investigations in domestic violence cases. This came after the coronial inquiry into the deaths of Bradley and Ellen Livingstone and the verdict in the trial of the west Auckland boys charged with… ...
    GreensBy Jan Logie MP
    6 days ago
  • Backers banking on social bonds cash?
    The Government is refusing to say what the $29 million it has set aside for its controversial social bonds programme is for, raising suspicions it is an upfront payment to the project backers, Labour’s Health spokesperson Annette King says. A… ...
    6 days ago
  • Plastic Free July
    Today is the start of Plastic Free July. Since its inception in Perth, Western Australia four years ago, more and more people and organisations from around the world have joined the call to refuse single use plastic products. Nearly all… ...
    GreensBy Denise Roche MP
    6 days ago
  • State house sell off Bill gives extraordinary powers
    The Government is about to give Ministers extraordinary powers to take direct personal control of selling state houses, exempting Ministers from normal legal requirements and leaving the sale process wide open for corruption, Labour's Housing spokesperson Phil Twyford says. The… ...
    6 days ago
  • Cash for charter schools, mould for state schools
    At a time when state schools are struggling in old, cold, mouldy buildings and can barely make ends meet, the National Government is shovelling cash at charter schools which aren’t even spending the funding on kids’ education, Labour’s Education spokesperson… ...
    6 days ago
  • New Zealand needs a wise response to climate change
    Today in Parliament I got to hear from a group of New Zealanders who are concerned for the future of our country. Called Wise Response, the group is a broad coalition of academics, engineers, lawyers, artists, sportspeople and others who… ...
    GreensBy Russel Norman MP
    6 days ago
  • No alternative as waste scheme trashed
    Nick Smith must explain how he is going to prevent contamination of New Zealand’s ground and water with liquid and hazardous waste after scrapping the only monitoring scheme and offering no replacement, says Labour’s Environment Spokesperson Megan Woods. “From today,… ...
    7 days ago
  • Flawed system rates death traps as safe
    ACC Minister Nikki Kaye needs to come clean about what really lies behind the reclassification of 18 vehicles in her new motor vehicle registration system introduced today, Labour’s ACC spokesperson Sue Moroney says. "New Zealanders deserve the truth about the… ...
    7 days ago
  • Tiwai Smelter and 800 workers left in limbo
     Workers at Tiwai smelter and the people of Southland have once again been left in limbo over their future in the ongoing debacle over whether the plant stays open, says Labour’s Leader Andrew Little.  “It’s not good enough that after two years of… ...
    7 days ago
  • New twist in state house sell-off saga
    The Government has opened the door to buyers of state houses simply being landlords and not required to provide social services, Labour’s Housing spokesperson Phil Twyford says. The Prime Minister said at his post-Cabinet press conference buyers would not “have… ...
    7 days ago
  • Government fees will hit charities hard
    National’s decision to ignore the concerns of charities will see the voluntary sector face hundreds of thousands of dollars in new costs if the Policing (Cost Recovery) Amendment Bill passes, says Labour's Community and Voluntary Sector spokesperson Poto Williams. “National’s… ...
    1 week ago
  • Four out of ten for Simon’s Bridges
    The Transport Authority’s decision to fund only four of the 10 bridges promised in National’s shameless Northland by-election bribe is a huge embarrassment for Transport Minister Simon Bridges, Labour’s Transport spokesperson Phil Twyford says. “After one by-election poll showed they… ...
    1 week ago
  • Falling consents adding to Auckland housing woes
    Falling numbers of building consents being issued in Auckland will add to the city’s housing shortfall and fuel skyrocketing house prices, Labour’s Housing spokesperson Phil Twyford “The Productivity Commission found there was a shortfall of around 32,000 houses by the… ...
    1 week ago
  • So Mr English, do you have a plan?
    DIpping confidence about jobs, wages and shrinking exports are highlighting the lack of a plan from the government to diversify the economy and build sustainable growth, Grant Robertson  Labour’s Finance Spokesperson said. " Data released over the last week… ...
    1 week ago
  • Serious risks to tenants and assets in sell-off
    Overseas evidence shows there are serious risks around the Government's plan to sell off state houses to social housing providers, Opposition Leader Andrew Little says. “In the Netherlands – where community housing providers supply the majority of social housing –… ...
    1 week ago
  • Land of milk and money
    Kiwi families are paying over the top prices for their milk and someone is creaming off big profits, says Labour’s Consumer Affairs spokesperson David Shearer. “In 2011 the Government told us high New Zealand milk prices were a natural result… ...
    1 week ago
  • MoBIE largesse doesn’t stop with TVs and hair-straighteners
    The number of MoBIE staff earning more than $150,000 has risen 23 per cent in just a year, Labour’s Economic Development Spokesperson David Clark says. Documents obtained from the Ministry of Business, Innovation and Employment show there are now nearly… ...
    1 week ago
  • English wants to flog state houses to Aussies
    Bill English’s admission that he would sell hundreds of New Zealand’s state houses to the Australians is the latest lurch in the Government’s stumbling, half-baked housing policy, Labour’s housing spokesperson Phil Twyford says. “Bill English should face reality and admit his… ...
    2 weeks ago
  • Exports continue to fall as Government fails to diversify
    The Government quickly needs a plan to diversify our economy after new figures show that exports are continuing to fall due to the collapse in dairy exports, Labour's Finance Spokesperson Grant Robertson says. “Dairy exports fell 28 per cent compared… ...
    2 weeks ago
  • Government inaction leads to blurring of roles
    The Treasury wouldn’t have had to warn the Reserve Bank to stick to its core functions if the Government had taken prompt and substantial measures to rein in skyrocketing Auckland house prices, Labour’s Finance spokesperson Grant Robertson says. “The problems… ...
    2 weeks ago
  • Courthouse closures hitting regions
    The Government’s decision to shut down up to eight regional courthouses, some supposedly only temporarily for seismic reasons, looks unlikely to be reversed, Labour’s Justice spokesperson Jacinda Ardern says.“The move has hit these regions hard, but appears to be a… ...
    2 weeks ago
  • A Victory for Te Tiriti o Waitangi
    This week my partner, who has a number of professions, was doing an archaeological assessment for a District Council. He showed me the new rules around archaeologists which require them to demonstrate “sufficient skill and competency in relation to Māori… ...
    GreensBy Catherine Delahunty MP
    2 weeks ago
  • Tough bar set for Ruataniwha dam
     Today’s final decision by the Tukituki Catchment Board of Inquiry is good news for the river and the environment, says Labour’s Water spokesperson Meka Whaitiri. “Setting a strict level of dissolved nitrogen in the catchment’s waters will ensure that the… ...
    2 weeks ago
  • Minister for Women and National missing the mark – part two
    The Minister for Women was in front of the select committee yesterday answering questions about her plans for women. Some useful context is that we used to have a Pay and Employment Equity Unit within the then Department of Labour… ...
    GreensBy Jan Logie MP
    2 weeks ago
  • Lavish penthouse spend confirms culture of extravagance
    At the same time thousands of New Zealanders are being locked out of the property market, the Government is spending up on a lavish New York penthouse for its diplomats, Labour’s Foreign Affairs spokesperson David Shearer says. News that taxpayers… ...
    2 weeks ago
  • Māori Television exodus cause for concern
    The shock departure of yet another leading journalist from the Native Affairs team raises further concern the Board and Chief Executive are dissatisfied with the team’s editorial content, says Labour’s Māori Development spokesperson Nanaia Mahuta. “Annabelle Lee is an experienced… ...
    2 weeks ago
  • Million-plus car owners to pay too much ACC
    More than a million car owners will pay higher ACC motor vehicle registration than necessary from July, Labour’s ACC spokesperson Sue Moroney says. “During a select committee hearing this morning it was revealed that car owners would have been charged… ...
    2 weeks ago
  • Bill will restore democracy to local councils
    A new Labour Member’s Bill will restore democracy to local authorities and stop amalgamations being forced on councils. Napier MP Stuart Nash’s Local Government Act 2002 (Greater Local Democracy) Bill will be debated by Parliament after being pulled from the… ...
    2 weeks ago
  • Minister for Women again misses the mark – part one
    Yesterday I asked the Minister for Women about the government’s poor performance on it’s own target of appointing women to 45% of state board positions. I challenged why she’d put out a media release celebrating progress this year when the… ...
    GreensBy Jan Logie MP
    2 weeks ago
  • Banks enter Dragon’s Den in pitch for Government’s mental health experi...
    Overseas banks and their preferred providers were asked to pitch their ideas for bankrolling the Government’s social bonds scheme to a Dragon’s Den-style panel, Labour’s Health spokesperson Annette King says. Dragon’s Den was a reality television series where prospective ‘entrepreneurs’… ...
    2 weeks ago
  • Global Mode bullying won’t stop people accessing content
    It’s disappointing that strong-arm tactics from powerful media companies have meant Global Mode will not get its day in court. Today a settlement was reached terminating the Global Mode service, developed in New Zealand by ByPass Network Services and used… ...
    GreensBy Gareth Hughes MP
    2 weeks ago
  • More questions – why was the Former National Party President involved wit...
    Today in Parliament Murray  McCully said the reason Michelle Boag was involved in 2011 in the Saudi farm scandal was in her capacity as a member of the New Zealand Middle East Business Council. The problem with that answer is… ...
    2 weeks ago
  • Minister must explain Maori TV interference
    Te Ururoa Flavell must explain why he told Maori TV staff all complaints about the CEO must come to him – months before he became the Minister responsible for the broadcaster, Labour’s Broadcasting Spokesperson Clare Curran says. “Sources have told… ...
    2 weeks ago
  • KiwiSaver takes a hammering after the end of kick-start
    National seems hell bent on destroying New Zealand’s saving culture given today’s news that there has been a drop in new enrolments for KiwiSaver, says Labour’s Finance spokesperson Grant Robertson.  “New enrolments for the ANZ Investments KiwiSaver scheme have plunged… ...
    2 weeks ago
  • Straight answers needed on CYF role
    The Government needs to explain the role that Child, Youth and Family plays in cases where there is evidence that family violence was flagged as a concern, Labour’s Children’s spokesperson Jacinda Arden says. “The fact that CYF is refusing to… ...
    2 weeks ago
  • Prime Minister confuses his political interests with NZ’s interest
    The Prime Minister’s statement in Parliament yesterday that a Minister who paid a facilitation payment to unlock a free trade agreement would retain his confidence is an abhorrent development in the Saudi sheep scandal, Opposition leader Andrew Little says.  ...
    2 weeks ago

Public service advertisements by The Standard

Current CO2 level in the atmosphere