Polity: Shameful, innumerate reporting of Herald poll

Written By: - Date published: 8:48 am, September 19th, 2014 - 32 comments
Categories: election 2014, Politics, polls - Tags: , , ,

polity_square_for_lynnReposted from Polity.

The reporting of this morning’s final New Zealand Herald poll is nothing short of a disgrace.

First, here’s the report:

Moment of Truth gifts Team Key a late bounce in polls

Last voter survey before election day shows jump in National’s rating

The Kim Dotcom-inspired event in Auckland’s Town Hall that was supposed to end John Key’s career gave the National Party an immediate bounce in support this week, according to polling for the last Herald DigiPoll survey.

With 60 per cent of the poll done by Monday night, when the event happened, National was polling at 47.8 per cent, down on last week, said DigiPoll general manager Nandan Modak. From Tuesday it jumped to 49.1 per cent.

[…]

Party Vote results: National 48.2 (down 0.4), Labour 25.9 (up 1.3), Green 11.1 (down 0.4), NZ First 8.4 (up 0.3), Conservatives 3.3 (down 0.5), internet Mana 1 (down 1.3), Maori Party 1.1 (up 0.4), Act 0.5 (up 0.2), United Future 0.2 (up 0.2), Legalise Cannabis 0 (no change).

Two exasperated points here:

First, the “last voter survey before election day” does not show a “jump in National’s rating” at all. In fact, “National’s rating” has fallen in this “last voter survey” compared to the previous voter survey, which is normally how “jumps in support” are assessed.

That headline reporting is horribly misleading and irresponsible, especially on the last day before an election.

Second, I want to address the fig leaf the Herald is relying on to make these misleading claims. The article claims authority from a 1.3% increase in reported National support within the survey window, between the 60% of the survey completed prior to Moment of Truth, and the 40% completed afterwards.

There are simple statistical calculations you can do to see if that evidence is any good, and worth relying on. Any pollster worth their salt can point to these intuitively. I’ll lay it out:

  • The Herald’s survey had 775 people, of whom 94.4% (732 people) had a party preference.
  • 60% of them (439 people) were interviewed before Moment of Truth, of whom 47.8% supported National.
  • 40% of them (293 people) were interviewed after Moment of Truth, of whom 49.1% supported National.

Is there a statistically meaningful difference between those two groups of results? Absolutely not. A simple two sample difference in proportions test says the evidence cannot support any conclusion of changing support during the survey, at conventional levels of significance or anything close to them.1

In fact, if you absolutely had to say whether National’s support rose or fell post Moment of Truth, and you were simply not allowed a “no change” option, then there’s a 37% chance National’s support actuallydropped, given these data.

Put another less technical way, if there had been no change at all in National support during the survey, we would have expected to see 140 of the 293 late responders tick National. In the event, we observed 144 of the 293 people tick National. Is that four-person difference worth getting at all excited about? Since, like intuition, says “No.”

Put yet another way, the Herald’s screaming day-before-the-election pro-National headline was based on an observed difference of 1.3%, with a margin of error of plus or minus 7.4%.

That, New Zealand Herald, is a fucking disgrace.

 

lprent: I agree. If a politics editor at a major National newspaper doesn’t know basic statistics, then you’d have to ask how she retains her job?

I have enough of an issue viewing polls as having much validity these days due to sampling population errors. Having a jonolist apparently deliberately distort the meaning of a poll that is taken, and whose results was not linked to in the article, is a complete travesty. Get rid of the useless things. They aren’t helping inform the public. They just provide cheap and easy matter for jonolists to use when they are avoiding doing their jobs.

 

32 comments on “Polity: Shameful, innumerate reporting of Herald poll ”

  • Comments are now closed

  1. Chooky 1
    19 September 2014 at 8:53 am

    Smacks of Desperation alright!….another case of the Herald ferrets manipulating the polls…John Key is on the way out and they know it!

  2. Mark 2
    19 September 2014 at 9:06 am

    Poll in the Press this morning has the Nats down 5.1% and Labour up 3.7% which is the first time in living memory in a Stuff poll that the Nats have been under 50%.
    With the poll bias to the Nats thrown into the mix, they are in real trouble.

  3. Tom Gould 3
    19 September 2014 at 9:07 am

    Poor old Audrey is just running true to form for her employer, the National Herald. When I opened up the on-line edition this morning, I almost didn’t notice the wrap-around National Party advertising, until I took a second look, so used have I become to blazing pro-Tory headlines and photos on the front page. The paper is simply a Tory rag, a National party daily newsletter, and is irrelevant to most people, which is why their circulation is tanking and their company is on the ropes, gasping for breath. Ironic, really, given the now-all Tories who run it?

    • thatguynz 3.1
      19 September 2014 at 10:30 am

      What’s more Stuff is exactly the same. Pretty disgraceful.

    • JRT 3.2
      19 September 2014 at 9:25 pm

      I live in a North Island provincial city, and have a subscription for the local paper to be delivered, but today instead of our local morning paper we instead got delivered a Herald, with wall to wall National party propaganda inside. Just disgusted. The Herald owns the local paper so easy for them just to deliver to us what they want us to read. Really got to wonder, who is behind all of this electoral manipulation? The lengths they are going to and the money behind this years campaign seem extreme, why are they so incredibly desperate to get National into power this time? Is it about the TPPA? Wish the sheeple of NZ would wake up, before they vote us into oblivion.

  4. Puddleglum 4
    19 September 2014 at 9:11 am

    It’s probably worse than Rob states.

    In the final part of polling there’s usually an attempt to complete quotas to make the sample mirror the general electorate more closely.

    For example, women tend to be more willing and able to respond than men so the last part of the polling period would target men disproportionately.

    It’s possible, that is, that the early and late part of the overall sample are actually from different statistical populations.

  5. brian 5
    19 September 2014 at 9:12 am

    There has been far too much “news” time being taken up by polls this election. It seems to be getting worse as each election occurs. The polls are ok, in themselves, but the endless time taken up by asking politicians their comments on the polls is just absurd.

    It’s like sports games where we now have at least an hour of commenting drivel before the game, asking the participants what is going to happen, and a mutual masturbation session of journalists speculating about what is likely to happen.

    I’d love a few politicians to say at the start of a campaign that they (a) are not going to comment on polls, and (b) not going to continually update comment on coalition possibilities every day, based on polls.

    Polls are over rated, and their over use are a poor substitute for analysis of policy options, for which there has been extremely little in all MSM and in all blogs.

    Corruption involving poll results becomes a greater possibility, the greater that we allow polls to guide voting rather than underlying policies, and issues of trust.

    • Lanthanide 5.1
      19 September 2014 at 9:22 am

      Probably the only way a politician could get away with this is if they said they wouldn’t comment on the public polls, but instead would publish their own internal polling and comment solely on that, because they can trust it as reliable etc.

      But no party would ever do that, because:
      1. Sharing poll data that you paid for with your political opponents isn’t a clever strategy
      2. Inevitably their internal polling will sometimes show gains, and sometimes show losses, and when it shows a loss they’ll still have to face the same old questions

    • mike 5.2
      19 September 2014 at 7:21 pm

      Look at the poll fiasco in the US at the last presidential election.
      It was either going to be a win for Romney or incredibly close.
      Obama didn’t lose a vote from his first win.
      It could be described as a landslide.
      The humiliation was noted but quickly forgotten.
      We all got a good laugh from that one.

  6. Molly 6
    19 September 2014 at 9:20 am

    ,,, yes, I too conducted a comprehensive poll – and at the the last minute National was ensured a 100% victory.

    If you consider that the last person I asked was Audrey Young, you can see that other parties didn’t even get a mention in that particular result. Until then the poll was reflecting the feeling on the ground that MMP encourages a diverse support for different political parties, but at the very last we can see that National will indeed triumph triumphantly.

    (Disclaimer: Although there may have been atrocities committed on accuracy, statistics and democracy – no National party supporters were harmed in the conducting of this poll, and Audrey Young may or may not have been asked)

  7. wonderpup 7
    19 September 2014 at 9:34 am

    Are there any sites that cover woeful use of statistics that might be interested in showcasing this?

    • ghostwhowalksnz 7.1
      19 September 2014 at 4:03 pm

      Its called jumping the shark- some insignificant detail is highlighted which ignores the overall conclusion.
      A bit like saying ‘tobacco smokers are living longer’ when in fact they dying sooner than everyone else.

  8. Disturbed 8
    19 September 2014 at 9:46 am

    NZ polls are clearly corrupted, is this a crime?

    This is just one case of the 1.4 million sites on Goggle in India which may show what is happening here in NZ.
    quote;

    New Delhi: Public opinion gathered by leading opinion poll agencies is often tweaked to give misleading results, Operation Prime Minister, a sting operation by a private news network, News Express, has revealed Tuesday. Well-known faces from leading opinion poll agencies have been caught on hidden camera agreeing to such malpractices.

    Operation Prime Minister shows how opinion polls are conducted and manipulated at the instance of political parties, their results traded to show a particular party in a favourable position, for a price. Presenting snippets from the sting operation at a press conference, Editor-in-chief of News Express, Vinod Kapri said, “Our motivation behind conducting the sting operation was a letter written by the Election Commission of India to all regional and national parties inviting their views on the publication of opinion polls.

    We wanted to investigate the concerns of the Commission.” In its letter dated 4 Oct, 2013, the Election Commission had said, “The Commission has been suggesting to the government that there should be a similar prohibition or restriction on opinion polls also as there could be several manipulated opinion polls which could impact the voting pattern.”

    Another motivation behind the sting operation was the mushrooming of opinion polls. “There used to be one or two opinion polls every election. But now, one sees an opinion poll almost every week. Which leads us to the question- how is the data generated so quickly and processed,” said Kapri.

    “Operation Prime Minister has exposed eleven opinion poll agencies, whose surveys are published in leading newspapers and magazines besides being broadcast by leading news channels”, it was claimed in the sting opertaion.

    It demonstrates how the 810 million voters of our country are duped into believing trends or waves that are manipulated. – See more at:

    “Opinion polls seem to have become the latest weapon in the poll campaign.

    For a price, the prediction of seats tally can be changed to suit the interests of political parties.

    The agencies have no qualms accepting even black money for this purpose”,

    News Express claimed. The influence of opinion poll agencies goes beyond mere opinion polls. In some cases, the poll agencies have claimed they can even prop up dummy candidates in the constituencies where the rival candidate is on a strong wicket.

    It has also been claimed that some leading editors are hand in glove with these poll agencies.

    This is just a few of the 1.4 million sites on Goggle. – See more at:

    http://www.ummid.com/news/2014/February/26.02.2014/opinion-poll-companies-exposed.html#sthash.wqSJ2ylu.dpuf

    http://www.sciences360.com/index.php/statistics-16350/

    http://www.ummid.com/news/2014/February/26.02.2014/opinion-poll-companies-exposed.html#sthash.h8rTzw8V.dpuf

    http://ibnlive.in.com/videos/471548/exit-polls-manipulated-modi-will-never-be-pm-samajwadi-party.html

    http://www.rediff.com/news/report/general-impression-that-opinion-polls-can-be-manipulated-sibal/20131109.htm

    http://timesofindia.indiatimes.com/india/Many-agencies-manipulating-opinion-poll-projections-claims-sting-operation/articleshow/31013534.cms

    http://www.sunnewsnetwork.ca/video/2764415112001

    http://en.wikipedia.org/wiki/Misuse_of_statistics

    http://www.hurriyetdailynews.com/mhp-furious-over-opinion-poll-manipulation-claims.aspx?pageID=238&nID=62184&NewsCatID=338

  9. Jono 9
    19 September 2014 at 9:48 am

    I see in a separate article from Audrey that the male share of the Labour votes has been increasing for the last three weeks of Herald polling. That’s a very encouraging trend if it holds. It would be fascinating to nail down the cause of the change, I must say I found seeing the mongrel in DCs debate performances thrilling and I wonder what effect that had.

    • Maz 9.1
      19 September 2014 at 8:54 pm

      The pre debate discussions had John Key down to win. If Cunliffe managed a draw, that was deemed a win for him. So, much to the disappointment of the right and their media puppets, Cunliffe defied their odds. We on the other hand didn’t doubt his ability to ‘out debate’ Key. I think this would have affected voters opinions.

  10. Rodel 10
    19 September 2014 at 9:50 am

    How does political editor Audrey retain her job?
    Simple. By writing what her employers want her to write.

    • ghostwhowalksnz 10.1
      19 September 2014 at 10:16 am

      Shes the daughter of a former national MP, and sister to a current national MP.

      But of course once election day results are in, as far as Audrey is concerned what she wrote two days ago is never mentioned again

  11. Treetop 11
    19 September 2014 at 9:51 am

    Until I read a poll which states how many landline numbers and how many cell phones numbers were used to get the result, the only poll which counts is on election day. I suppose this is why a person casts their vote in a polling booth.

    • ghostwhowalksnz 11.1
      19 September 2014 at 10:21 am

      Most of them dont call mobile phones at all.

      The reason for this is they SELL the phone numbers of people who they have polled to telemarketing companies. ( or use themselves for TM)

      Remember the demographic information they ask at the end of the poll, age, employment, income, etc . This gold for TM.

      Once they have finished polling its grouped into juicy demographic bands with the land line numbers and bingo.

    • Nic the NZer 11.2
      19 September 2014 at 12:21 pm

      Forget land-lines vs cell phones, there is a much better way if you can control the sampling method.

      Just poll some of the electorates more than others, if you poll a number of National party safe electorates guess what the poll is likely to favor National but with a smattering of other parties in there. People actually doing the polling may not even realize whats going on if you feed them numbers to call from a list.

      This would explain why for example the Stuff/Ipsos poll is consistently over-rating National and is frequently over-rating them by more than the polls margin of error. Piece of piss and almost completely un-detectable without being able to look at the whole polling process in high detail.

  12. Blue 12
    19 September 2014 at 10:00 am

    The Herald have really gone downhill. They aren’t even pretending anymore. Wall to wall National Party advertising practically every single day and editorial content to match.

    They are really just in it for the money. The National Party has money to burn on advertising and when the Herald put their paywall up they will be trying to appeal to business types to buy subscriptions.

    • Tom Gould 12.1
      19 September 2014 at 10:41 am

      The paywall will kill off the Herald for good. There is no ‘news’ that cannot be found elsewhere, and generally better quality. Other than its value as a National Party newsletter for Auckland Tories, it has been a ‘dead man walking’ as a newspaper for some time. The more shrill and sensational it gets the more even the Tories will turn away.

    • David H 12.2
      19 September 2014 at 7:12 pm

      When the paywall goes up I won’t bother going there. I can get the Political News elsewhere.

  13. tinfoilhat 13
    19 September 2014 at 10:25 am

    Simple solution ban all reporting of polling from election minus 3 months.

    This allows the political parties to continue with their own polling if they wish but cuts the MSM out of the process and hopefully gets them to talk about policies although with their concentration on personalities and all things Dotcom this election cycle I’ve pretty much given up hope on the journalists in this country.

    • ghostwhowalksnz 13.1
      19 September 2014 at 11:55 am

      Just try to do this and the headlines will scream:

      End of democracy!

      btw, Im not in favour of ending media polls. One silly jonolist doesnt make it wrong

      • Draco T Bastard 13.1.1
        19 September 2014 at 1:28 pm

        How would you make it so that the reporting of the polls was accurate?

        Personally, I’m in favour of banning the reporting of polls during the election period.

  14. appleboy 14
    19 September 2014 at 12:11 pm

    Press Council official complaint?

  15. Tigger 15
    19 September 2014 at 12:16 pm

    Time to ban polling from a certain point in the election. They’re literally criminal in the way they are constructed and reported.