Granny Herald: from flawed data, false conclusions

Written By: - Date published: 11:21 am, March 23rd, 2013 - 17 comments
Categories: john key, national, polls - Tags:

Three of the Herald’s political staff has a crack at trying to explain why National polled 48.5% in the latest Herald-Digipoll. Consensus seems to be that John Key’s an angel sent to earth to do very little but keep enough of the middle class happy enough who’ll never lose. May I humbly suggest an alternative theory – that the Herald-Digipoll is broken.

Danyl at the Dimpost has the most mathematically poll of polls that, crucially, is anchored in reality – ie the last election result. Whereas political commentators and pollsters seem to shamelessly ignore how badly their poll got it wrong at the last election when breathlessly reporting the latest numbers, the Dimpost poll of polls corrects for their error.

What the Dimpost poll shows is that:

a) the Herald Digi-poll consistently over measures National – for example, the final Herald poll before the last election had National on 50.9%, that’s outside the margin of error above the 47.3% they got. The final Herald poll in 2008 had National on 47.9%, they got 44.9%. Statistically, the Herald isn’t just missing the bullseye, it’s missing the dartboard altogether. Only the 3news poll is worse. (in the image below, the light blue line is the Herald trend, the dark one is the poll of polls trend).

Dimpost poll of polls highlighting Herald Digipoll

b) even the Herald poll shows that National has lost a significant amount of support. From a pre-election high of 54.2%, a final pre-election number of 50.9%, to 48.5% today. That’s a big drop. Adjust to the Herald’s 3% over-polling of National at the last two elections, and it signals National is at around 45% – in the range that the Dimpost poll of polls has it. Remember, the Right has a one-seat majority in the House. Losing 2-3% since the election, especially when Banks will never get back in, is no laughing matter for National. If it were to translate into an election result, it would mean National would need New Zealand First (or a very large wasted vote) to govern.

c) more regular polls, like the Roy Morgan show a clear downward trend for National. You can’t possibly draw a trend from the five data points in 17 months that the Herald provides because the statistical noise that results in variation between every poll can’t be sensibly separated from the trend when you have so little to work with. The Roy Morgan has polled 29 times in that period and its trend is clear. (incidentally, the Roy Morgan was also the most accurate for National in 2011).

roy morgan labour greens v nats 3

To look at the Herald poll and conclude that National is now more popular than at the election, you have to ignore the Herald’s record of over-polling National, the fact that National’s support in the Herald poll has fallen significantly since the election, and the fact that more reliable polling trends tell us the opposite.

Only if you make all these logical errors (and there’s a strong incentive to do so because our paper spent a lot of money on the poll, which means its going to be treated as gospel no matter what, goddammit), can you then conclude National’s rule continues unblemished thanks to Bonny Prince Grinny.

But when’s a few fallacies ever stopped Granny?

[Note: none of this should be taken as a ‘if these trends continue’ argument. One current accurate polling, the Left needs Winnie too making him Kingmaker and he would have much more incentive to go with National, and there’s the risk of a Shearer campaign meltdown too. Put National at 40% and I’ll be happier].

17 comments on “Granny Herald: from flawed data, false conclusions”

  1. Tiresias 1

    “Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: “There are three kinds of lies: lies, damned lies and statistics.”
    – Mark Twain’s Own Autobiography

    Presently Labour will only be picking up those sufficiently dismayed by National’s duplicities and incompetence not to want them in Government again. Labour will only start picking up genuine support when it starts presenting an alternative programme worth genuinely supporting, and six-weeks out from an election is far too late to start thinking of doing that.

  2. Bob Tambling 2

    The worry for me then is if the polling trends continue and it looks like Shearer can just get over the line the possibilty of changing the leader will further diminish and Shearer will fight the 2014 election.
    Unfortunetly during the ensuing campaign, Key and the Nats will crucify him and we will be left with a National Government or a very weak LAb /NZF /Green one term Goverment
    I wonder from a Green party prospective (and a leftish pespective) , would it not be better if the Labour Party lost the next election with Shearer as Leader?
    If NZF play a part as may be likely, the Green can’t afford to go into a NZF /Lab/Green Gov.
    Shearer simply won’t cut the mustard and Peters is toxic. It would be a crap government and the well honed National party machine would again tear it to shreds

    The leaves the Greens in an interesting position, the lure of ministerial positions versus the pragmatic long term strategy of the Greens. A one term Lab/Nzf/Green Gov seems to serve no ones interest.
    Labour needs to have a strong leadership team and comitted activists , The reality is they have a diminishing group of activists and a crap leadership.
    The Greens on the other hand have a very strong leadership team and an ever growing group of activists ( let us not forget they are the second biggest party in Wellington).
    Labour? I can’t think of anyone in the present caucus who could take most of the country with them in an election. They need a good clean out and new blood. They don’t appear to be listening to that line of argument.
    In conclusion:
    I am committed to working very hard to getting as many Green MP’s elected as I can in 2014. I am not so sure though if I want those MP’s in a Government with the current infighting mob of the Lab party who it seems “could not pour piss out of a boot if the directions were on the heel”.

  3. tc 4

    The more granny and her shills like Johnno, o’shillivan, rortney etc keep banging the NACT drum the more irrelevant and bias they show themselves to be.

    Labour needs to grow a pair and get out there now however with the mallarfia and their front man it seems it’ll come down to the hollowmen blowing it more so than Labour winning it.

    They could easily blag another term if they weren’t such psychos given the current state of the opposition, thankfully they are what they are.

  4. ghostrider888 5

    article in todays Herald acknowledges the popularity of Key is slipping

  5. pete 6

    “Only the 3news poll is worse.”

    That’s not quite true: all five polls overestimate National’s election results, and the Digipoll isn’t too much worse than Fairfax or Roy Morgan. One News Colmar Brunton, as well as 3 News Reid Research both overestimate by more than the Herald.

    • lprent 6.1

      Points to a systematic flaw in their sampling doesn’t it…

      • pete 6.1.1

        That’s one possible explanation, although some have theorised that National lose a lot of votes close to the election once people actually start looking at their policies.

        There’s also a theory that NZF underpolls because their voters are embarrassed to admit that to callers.

        • lprent 6.1.1.1

          Possible. But less likely than the simpler explanation.

          Canvassing companies are using listed landlines*. When I was analyzing some somewhat larger samples of both door-knock canvassing and phone canvassing across several electorates, there was a statistically significant correlation between people who canvassed right and left and if they had a listed landline.

          For that matter there was an even stronger correlation between age group and having a listed land line – and it looks like if people don’t have a listed phone line in their mid 20’s then they never get one.

          * There is a common myth that goes around about auto-dialing random numbers. Asked a few people who work at or have worked at polling companies. It doesn’t happen and a few minutes thought about number ranges will give an explanation about why. It is cheaper to buy/get a phone list.

  6. Chris 7

    So,will Granny Herald be ramping up lots of little good news, happy smiley,relaxed all singing, dancing reports/photos of our poor excuse for a pm then. Got to keep him relevant.

  7. Centaur 8

    Give Key *some* credit. He may decide to call it quits. It is not as if he actually needs this gig ..

  8. Murray Olsen 9

    I’m not even sure that the margins of error as reported are very meaningful at all. From what I know, they seem to rely on unbiased sampling and the validity of the Central Limit Theorem, which then leads to a percentage presumably taken from 100 times √N/N. The concern about landlines is just one thing that could skew this.

    • lprent 9.1

      They do sample distribution normalising for many of the other factors like age, gender and geological distribution. They don’t adjust for factors like people working during the periods that they phone.

      But they don’t publish their techniques and criteria so it is pretty hard to see why there if it was unbiased sampling there would be any differences between the polls within the confidence limits. But there clearly is.

      • Centaur 9.1.1

        lprent: geological distribution ?
        Such as, where there are proven mineral resources ?

        • lprent 9.1.1.1

          I blame the iPad, it’s overly smart autocorrect, and that I use Geological at least 10x more often than Geographical.

    • Lanthanide 9.2

      No, they’re not. Even assuming they are following absolutely perfect statistical methods and the statistical methods work as advertised by the literature, the margin of error they’re reporting is presumably the 95% confidence interval. Which means 1 out of 20 polls are going to be wrong, but we won’t ever be able to tell which one without some other corroborating information (eg other polls taken at the same time).

      That’s assuming they’re using perfect methods, which of course they aren’t.

Leave a Comment

Show Tags

Recent Comments

Recent Posts

  • Nats’ housing policy fails to keep pace with population growth
    Auckland got less than half the new houses it needed in the past year to keep up with record population growth, says Labour’s Housing spokesperson Phil Twyford. ...
    23 hours ago
  • Urgent action needed on dirty rivers
    The Our Fresh Water Environment 2017 report re-confirms that we need urgent action to clean up our rivers. Meanwhile, National is standing by as our rivers get even more polluted, says Labour’s Environment spokesperson David Parker. “This report is yet ...
    2 days ago
  • Where there’s smoke and mirrors, there’s Steven Joyce
    Steven Joyce’s much vaunted pre-Budget speech is simply an underwhelming response to the infrastructure deficit National has created, says Labour’s Finance Spokesperson Grant Robertson. “Steven Joyce has belatedly come to the realisation that everyone else has a long time ago, ...
    2 days ago
  • Time to stamp out cold, mouldy rentals
    New figures show a small number of landlords are letting down the sector by renting cold, mouldy rentals. These houses need to be brought up to a decent standard for people to live in by Andrew Little’s Healthy Homes Bill, ...
    3 days ago
  • Time for fresh approach on immigration
    Latest figures showing another record year for immigration underlines the need for an urgent rethink on how this country can continue to absorb so many people, says Labour Leader Andrew Little. “New Zealand needs immigrants and is all the better ...
    3 days ago
  • Bring back the Mental Health Commission
    The People’s Mental Health Review is a much needed wake up call for the Government on mental health, says the Leader of the Opposition Andrew Little.   “I applaud their proposal to restore a Mental Health Commission and their call for ...
    5 days ago
  • And the band played on…
    Making Amy Adams the Housing Minister five months out from the election is just the orchestra playing on as National’s Titanic housing crisis slips below the waves – along with the hopes and dreams of countless Kiwi families, says Labour’s ...
    5 days ago
  • Hotel no place for children in care
    ...
    1 week ago
  • Maybe not, Minister? Nick Smith’s housing measure suppressed
    Sir Humphrey: Minister, remember the Housing Affordability Measure work you asked us to prepare back in 2012? Well, it’s ready now.Minister Smith: Oh goodie, what does it say?Sir Humphrey: Nothing.Minister Smith: Nothing?Sir Humphrey: Well, sir, you asked us to prepare ...
    1 week ago
  • Inflation data shows many New Zealanders are worse off under National
    The latest inflation data from Statistics New Zealand shows that too many New Zealanders are now worse off under the National Government, said Labour’s Finance Spokesperson Grant Robertson “Consumer Price Inflation (CPI) is now running at 2.2 per cent, and ...
    1 week ago
  • Another emergency housing grant blow out
      Emergency housing grants data released today show another blow out in spending on putting homeless people up in motels, says Labour’s Housing spokesperson Phil Twyford.   ...
    1 week ago
  • Families struggle as hardship grants increase
    The considerable increase in hardship grants shows that more and more Kiwi families are struggling to put food on the table and pay for basic schooling, says Labour’s Social Development spokesperson Carmel Sepuloni. ...
    1 week ago
  • More tinkering, no leadership from Nats on immigration
    National’s latest tinkering with the immigration system is another attempt to create the appearance of action without actually doing anything meaningful, says Leader of the Opposition Andrew Little. ...
    1 week ago
  • Suicide figures make for grim reading
    The 506 suspected suicides of Kiwis who have been in the care of mental health services in the last four years show that these services are under severe stress, says Labour’s Health spokesperson David Clark.   “If you do the ...
    2 weeks ago
  • Pay equity deal a victory for determination and unions
    The pay equity settlement revealed today for around 55,000 low-paid workers was hard-won by a determined Kristine Bartlett backed by her union, up against sheer Government resistance to paying Kiwis their fair share, says Labour Leader Andrew Little. “Labour welcomes ...
    2 weeks ago
  • DHB’s forced to make tough choices
    The Minister of Health today admitted that the country’s District Health Boards were having to spend more than their ring fenced expenditure on Mental Health, says Labour’s Health spokesperson David Clark.   “The situation is serious with Capital and Coast ...
    2 weeks ago
  • Nats break emergency housing pledge – deliver just five more places
    Despite National’s promises of 2,200 emergency housing beds, just 737 were provided in the March Quarter, an increase of only five from six months earlier, says Labour’s Housing spokesperson Phil Twyford. ...
    2 weeks ago
  • Research underlines need for KiwiBuild
    New research showing the social and fiscal benefits of homeownership underlines the need for a massive government-backed building programme like KiwiBuild, says Labour’s housing spokesperson Phil Twyford. ...
    2 weeks ago
  • Social data security review too little, too late
    The independent review into the Ministry of Social Development’s individual client level data IT system is too little, too late, says Labour’s Social Development spokesperson Carmel Sepuloni. “The Minister of Social Development has finally seen some sense and called for ...
    2 weeks ago
  • More questions raised on CERA conflicts
    With the admission that three more former CERA staff members are under suspicion of not appropriately managing conflicts of interest related to the Canterbury rebuild, it’s imperative that CERA’s successor organisation Ōtākaro fronts up to Parliamentary questions, says Labour’s Canterbury ...
    2 weeks ago
  • Labour to tackle Hutt housing crisis
    Labour will build a mix of 400 state houses and affordable KiwiBuild homes in the Hutt Valley in its first term in government to tackle the housing crisis there, says Leader of the Opposition Andrew Little. “Housing in the Hutt ...
    2 weeks ago
  • Farewell to John Clarke
    This wonderfully talented man has been claimed by Australia, but how I remember John Clarke is as a young Wellington actor who performed satirical pieces in a show called “Knickers” at Downstage Theatre. The show featured other future luminaries like ...
    GreensBy Catherine Delahunty
    3 weeks ago
  • Valedictory Speech
    Te papa pounamu Aotearoa NZ Karanga karanga karanga; Nga tupuna Haere haere haere; Te kahui ora te korowai o tenei whare; E tu e tu ... tutahi tonu Ki a koutou oku hoa mahi ki Te Kawanatanga; Noho mai noho ...
    3 weeks ago
  • Buck stops with Gerry Brownlee
    The fact that the State Services Commission has referred the CERA conflict of interest issue to the Serious Fraud Office is a positive move, but one that raises serious questions about the Government’s oversight of the rebuild, says Labour Canterbury ...
    3 weeks ago
  • Teachers deserve a democratic Education Council
    Teachers around New Zealand reeling from the news that their registration fees could more than double will be even angrier that the National Government has removed their ability to have any say about who sits on the Council that sets ...
    3 weeks ago
  • Free trade backers are simply out of touch
    Are the backers of free trade out of touch with public opinion? This was the question asked when the Chartered Accountants launched their Future of Trade study. I was astonished by the answer in a room of free trade enthusiasts ...
    GreensBy Barry Coates
    3 weeks ago
  • John Clarke aka Fred Dagg will be missed by all Kiwis
    The man who revolutionised comedy on both sides of the Tasman, John Clarke, will be sadly missed by Kiwis and Aussies alike, says the Leader of the Opposition Andrew Little.   “I grew up with Fred Dagg and I am ...
    3 weeks ago
  • Labour’s modern approach to monetary policy
    A commitment to full employment and a more transparent process to provide market certainty are the hallmarks of Labour’s proposals for a new approach to monetary policy, says Labour’s Finance spokesperson Grant Robertson. ...
    3 weeks ago
  • Greens back Labour’s plan for monetary policy reform
    Labour plans to change the way we do monetary policy in New Zealand and the Green Party supports them fully. We’re now of a single mind on this. Labour will move away from our reliance on a single, unelected person ...
    GreensBy robert.ashe
    3 weeks ago
  • Greens back Labour’s monetary policy reform
    Labour plans to change the way we do monetary policy in New Zealand and the Green Party supports them fully. We’re now of a single mind on this. Labour will move away from our reliance on a single, unelected person ...
    GreensBy James Shaw
    3 weeks ago
  • Govt drops ball on Masters Games housing squeeze
    Families currently living in emergency accommodation face being forced out onto the street as motel accommodation in Auckland is filled up by contestants and visitors of the World Masters Games in coming weeks, says Labours social development spokesperson Carmel Sepuloni. ...
    3 weeks ago
  • State inquiry for Nga Morehu – The Survivors of State Abuse
    The Prime Minister must show humanitarian leadership and launch an independent inquiry into historic claims of abuse of children who were in State care, says Labour’s Deputy Leader Jacinda Ardern. ...
    3 weeks ago