Roy Morgan

37 comments on “Roy Morgan ”

• Comments are now closed

1. barnsleybill 1

   14 April 2008 at 2:30 pm

   and labour should improve further once everybody has seen the latest singing effort by the fugly’s.
   A clearer example of brain development stopping at sixth form level we are never likely to see beaten.
2. Steve Pierson 2

   14 April 2008 at 2:31 pm

   Roy Morgan has a reputation for being the most accurate polling company (they certainly make Colmar Brunton look like amatuers) but their analysis is always a joke

   Its understandable that being Australia-based they might not get the subtles of political events here but surely the story of the poll trend is National falling with Key being shown as out of his depth and on the wrong side of popular opinion on a number of issues (AIA especially). Labour will have picked up votes off National there but lost others to the Greens over the FTA.
3. mike 3

   14 April 2008 at 2:49 pm

   “As the year progresses, it will not get any easier for Helen Clark and her tired Government to bridge the gap that the National Party has held consistently for over a year. The Government needs to get on the front foot before worsening economic news condemns the Labour Party to Electoral defeat later in the year.’

   Pretty fair analysis I’d say and still 13 points behind after all yr “big” announcements. With the Nats powder still bone dry it’s looking pretty sad for the pinko’s
4. illuminatedtiger 4

   14 April 2008 at 2:53 pm

   That’s 52 seats versus National’s 49 assuming the Maori Party sides with Labour and UF go with National. I know it’s just a poll but there is certainly an upwards trend here.
5. Andrew Jull 5

   14 April 2008 at 2:54 pm

   I am going to repeat a post I put up when the last poll came out (but at the end of 56 comments, it didn’t travel well).

   The chief problem with most political polls, one that is poorly understood and never articulated in newspaper reports, is the default assumption that underpins sample size calculation. A polling sample size is determined by the choice of margin of error and a sample size of 700-800 will deliver a margin of error of 3%-4%. But this margin of error is based on the assumption that the poll is a two horse race ie will the voter go with National or Labour. No New Zealand poll I have seen is powered to accurately address what is, in essence, a subgroup analysis ie what percentage of the population will vote for the minor parties. Therefore any poll with a sample size based on a the assumption of two horse race will not tell you anything useful about the other horses. A different sample size calculation is needed and will result in larger sample size to address the question of minor parties. The variation in minor party support from poll to poll will be almost totally accounted for by the inadequacies in the polling methodology. My guess is that those commissioning the polls are so innumerate that they don’t understand the product that they are purchasing – worse the pollsters probably don’t either and just use a web-based calculator to show the client they only need 700-800 people. That would be true if it was only a two horse race. But in the age of MMP, it is time to upgrade the polls to accurately count the minor parties’ support.
6. roger nome 6

   14 April 2008 at 2:58 pm

   “With the Nats powder still bone dry”

   You’re assuming that the public will like their policies. A mistake i’d say – especially the 90 DAYS OF NO WORK RIGHTS bill that they propose (its equivalent was almost universally hated in Australia). Labour has every reason to think that it can hammer National in the next election.
7. Steve Pierson 7

   14 April 2008 at 3:11 pm

   “with their powder bone dry”

   that’ll be the magical invisible powder Key has that will solve every problem with tax cuts and a grin?

   Andrew Jull. I agree. Just look at the way the Greens’ numbers jump around.

   mike. we’re not in FPP anymore, the gap between labour and national is the focus only for amatuers and the press gallery. You need to look at the balance between potential blocs.
8. Monty 8

   14 April 2008 at 4:06 pm

   So let me get this correct – National take some big hits because of a sustained and vile attack at every opportunity by Devious Cullen and the rest of the desperate Labour MPs and the best they can do is a .5% increase in the polls.

   This has also been during a period where National have been very quiet – attacking the EFA (which many people do not understand – least of all Labour MPs and President)) and Labour are in the news everyday with the success of the Free Trade agreement. Looking very sad for you lefties if a miserable .5% is all you can come with.

   Labour may be trying to put on a brave face – but essentially no one is interested in Labour anymore. They have done their time and Kiwis with an extraordinary sense of fairness want the other team to have a turn at batting. And young Johnny is going to step up first and hit a maiden century on his debut.
9. Patrick 9

   14 April 2008 at 4:25 pm

   It should also be noted that the poll was conducted between March 24th and April 6th. I really don’t think recent events would help stem National’s electoral bleeding.

   The last 8 years have been amazing for New Zealand, and I’m sure people will realise that change for the sake of change is a stupid idea, especially when the alternative is someone as vacuous as John Key and as dangerous as his National party.
10. Andrew Jull 10

    14 April 2008 at 4:30 pm

    A second of my concerns with polls is that illustrated by Monty (and he is not alone in this – all the newspaper headlines and editorialising do the same thing) – namely that the poll result is treated as a certainty rather than an estimate with a level of error around that estimate.

    In many sciences such estimates are required to be reported with the level of uncertainty. Thus the estimate is reported with the 95% confidence interval ie the likely range of results if the experiment (or poll) was conducted again. Elsewhere in The Standard I have explained that the 95% confidence interval is calculated by multiplying the standard error (what the polls report) by 1.96 (the z value for a 95% confidence interval). Thus using the margin or error of 3%, the 95% confidence interval for these polls is plus or minus 5.9% around the poll’s estimate for the two major parties. In other words, Labour’s result would be 34.5% (28.9%, 40.4%) with the range in brackets being the other likely values (although they are not all equally as likely). This explanation might sound too “pointy headed”, but the only reasonable interpretation of a poll result is one that includes the uncertainty around the estimates (for the two large parties at least) and then Monty could understand that the estimated increase in results might be 0.5%, but could equally be 6%.
11. randal 11

    14 April 2008 at 4:36 pm

    so the tory party mantra is this government is tired…what rot…just more weasel words from a party that is so bereft of policy that they have to rely on polling organisations to get any traction whatsoever which unfortunately will do them no good when the voters already know how much is at stake should the tories win.
12. Tamaki Resident 12

    14 April 2008 at 4:45 pm

    “This has also been during a period where National have been very quiet – attacking the EFA …”

    But what people (outside the blogosphere) are seeing is free speech still happening (e.g. One and half protests outside the Labour Party Congress), so they are wondering what the fuss is about when it comes to the EFA.
13. outofbed 13

    14 April 2008 at 6:44 pm

    Why don’t people understand that for the Nats to govern they need at least 46% It does not matter if they were 20% ahead of labour it only matters to get ahead of the COMBINED votes of Lab Green and the MP.
    National even at the height of their polling have never been outside the margin of error in their ability to form a Government.

    This is why Key is swallowing dead rats rats as fast as he can. he needs the centre vote
    Unfortunately for him the Nats are not a centrist party they are a right wing party so they have to pretend to be one.
    If the electorate see through the lie HC is in
    If they don’t Key is.
    Pretty simple really
14. mike 14

    14 April 2008 at 7:53 pm

    “that they have to rely on polling organisations to get any traction”
    Unbelievable – polling organisations are now to blame for Labours sagging hopes. Keep it up guys this is hilarious
15. AncientGeek 15

    14 April 2008 at 8:38 pm

    Andrew Jull comments about polls further up are correct. But he simply isn’t pessimistic enough about how inaccurate they actually are.

    They’re phone polls – specifically land line polls. They don’t cover people who don’t have land lines because they can’t afford them, don’t cover people who have unlisted numbers, don’t cover people who use cellphones, people who don’t use caller-id to screen calls etc…

    You’d have to ask who the polls actually cover. Old people, people who like telemarketers, people who answer land-lines, not the poor avoiding luxuries like a phone, and the techno-phobic.

    If people like mike think thats a random sample, then I have a nice bridge I’d like to sell. It is located under close to water.
16. AncientGeek 16

    14 April 2008 at 8:43 pm

    Bugger – strike-through doesn’t work. The under was meant to have a line through it.
17. Razorlight 17

    14 April 2008 at 9:45 pm

    Yes yes yes. This poll has National scared to death. After a month of positive headlines for the government what do we see. National well out in front.

    I was expecting the race to be close to even again now. Both parties floating around the 40% mark. What this shows though is National support is solid. Policy is being “rolled out” ( a very strange term Clark uses) by Labour but people are not listening.

    Cullen is making the hits in the house, but noone is listening.

    Natinal is scoring own goals. Noone cares.

    Really what else can Labour do. Any neutral would actually expect them to be ahead.

    Finally do you really expect Turia to go into coalition with Labour. The party was set up as a protest to Labour.
18. Ari 18

    14 April 2008 at 9:57 pm

    Andrew Jull- I’ve long suspected this was the case, but it’s nice to hear it with some actual statistical background to back it up. Maybe you should try to get in touch with the organisations commissioning the polls? 😉
19. burt 19

    14 April 2008 at 11:02 pm

    Add up the support for Labour, NZ1, Jolly Jim & Dunne. Look how many people want the current govt – 39.5%.

    An election is the only honorable thing for Labour to do, poll after poll shows that the current govt have no mandate but yet they drag it out and spend our money to try and make us think otherwise.

    It’s time for NZ to move on!
20. r0b 20

    14 April 2008 at 11:32 pm

    Righto Burt. I expect “President” Bush with his roughly 30% approval rating will be wanting to pack it in too. And every other government in the same position. Yeah right.
21. AncientGeek 21

    14 April 2008 at 11:39 pm

    Hey burt – so why did the nats bother fighting an election in 1999 or 2002? Seemed like a waste of time to me then, but they did it.

    Obviously greater believers in democracy than you are. It is all about giving the electorate choices. Even if you don’t win, the struggle will move the playing field. That is important, and I’m sure that (eventually) they will give some policy to do the same again. Of course they will be disappointed again. Looks to me like it will be too little, too late.

    You really have to wonder about the Nat’s apparent inability to articulate what they stand for.
22. burt 22

    15 April 2008 at 8:24 am

    The best defense that the Labour apologists can muster is – Others do it too. Classic – hands over ears shouting “la la la la – I’m not listening”.

    rOb at least has the mental capacity to take the “they did it too” comparison off shore so that he’s not looking like a “Labour good – National bad” muppet. AncientGeek (the brains trust) on the other hand uses the classic Labour defense of “National do it to”.

    Keep it up guys, keep supporting the party that thinks it’s OK to break their own laws to combat their slump in the polls.
23. r0b 23

    15 April 2008 at 8:30 am

    OK Burt, if you want to seriously argue that any government registering less than 50% in public opinion polls should resign, then you carry on! I think it’s easily your best and most insightful contribution to the political debate so far. You should write some letters to the editor about this exciting new theory Burt. Maybe write a whole book? You and Wishart could do one together. Seriously Burt, I think you’re really on to something here. Go for it Burt! Go!
24. Andrew Jull 24

    15 April 2008 at 9:19 am

    AncientGeek comments that I am not pessimistic enough when it comes to accuracy of polls – when I get some time I will estimate the sample size for polls that want to test the smaller parties support at a 3% margin of error.

    AncientGeek then suggests that because the polls are based on landlines they are likely to over-represent some people and under-represent others. This comment is a reasonable one and it would certainly be interesting to test the results of a polls drawn from such a sample against those drawn from a sample that included people without landlines. But to correct AncientGeek (and only in the interests of precision), such a test would evaluate the external validity of the sampling technique not its internal validity. There is absolutely no doubt in my mind that the polls do currently use a random sampling technique and are thus internally valid (if the interpretation of the data is limited to those which the sampling techniques assumptions would support ie a two horse race). The question of how representative the polls are is a separate issue (ie external validity) and one that would be best assessed by a table that compared the sample to the NZ population on key characteristics (age, sex, income, education level etc). With that information at hand and a sample size that is driven by the small party support, one could usefully interpret party support in New Zealand and understand the limitations of the data.
25. Matthew Pilott 25

    15 April 2008 at 10:10 am

    Andrew – I sense a guest post 🙂

    Burt – do you read what you are saying? You’re saying Labour should do something (what – stand down, snap electon? You haven’t even articulated what they should be doing, yet you attack the ‘apologists’ for its defence!) because they haven’t got support over 50%, and that it is bad to justify Labour not doing whatever it is because National also didn’t do this unarticulated act.

    What an odd line to pursue.

    P.S didn’t your mate Robert actually have a go at that back in the 80’s? Didn’t work out that well, but then he was pretty drunk.

    P.P.S burt – I see a twinkling of Socialism peeking through. I’m not sure you would see it, but I can’t help but think that Liberal Representative Democracy isn’t fulfilling its promise to you. So, my friend, let me introduce you to its cousin, Socialist Participatory Democracy…
26. David 26

    15 April 2008 at 10:49 am

    I’m not sure where Andrew Jull is getting all this. According to polling theory, the absolute estimates for small parties are more accurate, not less, in a random poll. The relative error might be large (doubling from 1% to 2% for a small party but we wouldnt see a large party going from 30% to 60%), but the absoulte error is smaller. For example, In a large population, if we sample 1000 people the 95% confidence interval at 50% is (50-3.1,50 3.1)= (46.9%,53.1%) whereas at 2% the 95% confidence interval is (2-.87,2 .87) = (1.13%,2.87%). A simple introduction to this stuff is available at http://www.surveysystem.com/sscalc.htm where there is also a nice little calculator. Note that the theory behind binomial samples (two parties) and multinomial samples (multiparty) is almost identical.

    So the variation we are seeing is not sampling variation – it probably shows greater awareness of a smaller party for some reason. Around election times, the polling numbers for small parties fluctuates less as we see more of them and people are actually thinking about who they might vote for rather than responding to the latest political news.
27. David 27

    15 April 2008 at 11:34 am

    Sorry second sentence above was incomplete, should be

    According to polling theory, the absolute estimates for small parties are more accurate, not less, in a random poll than for large parties.
28. James 28

    15 April 2008 at 11:59 am

    Further to David’s comment, Roy Morgan actually list the Margin of Error for a variety of percentages (down to 5%) for two different sample sizes. That actually makes them less misleading from average. You can therefore reasonable approximate the Margin of Error for the Greens, and see that it is quite a change.
29. Andrew Jull 29

    15 April 2008 at 1:34 pm

    I was wrong. David is right. I retract my comments about polls.
30. r0b 30

    15 April 2008 at 2:04 pm

    was wrong. David is right. I retract my comments about polls.

    Bravo to you both for the discussion – very useful!
31. steve black 31

    15 April 2008 at 3:05 pm

    Don’t retract your comments about the uncertainty of Polls just yet. Do get your statistics correct. Yes the variance of the estimates is greatest at 50% and gets smaller as you move to the extremes (1 or 2%). But the exact binomial confidence limits at the extreme are poorly approximated by the tables research companies give. For starters, they aren’t even symmetrical about the percentage. So it isn’t (made up numbers here!) 4% plus or minus 5%. It is 4% with the confidence interval from 3 to 8%. I’ve got an exact binomial confidence limit calculator around here somewhere. Next, the reported sampling errors (assuming perfect simple random sampling) are only a small coverage of the full set of errors involved. Last time I looked, Roy Morgan Research was better than others, but all could do better. Research companies routinely don’t add sample frame bias, response bias, and non response bias into their estimations. Sample Frame bias arises when your sampling frame differs in some relevant way from the population you would like to sample from. If you want to predict an election the population you want to correctly represent is those who will vote. People who are at home and agree to answer telephone polls is a different group. I’ve published research on the bias of using phone surveys versus face to face, but that was in 1990. Moblie phones have made it more complicated. If you don’t reach everybody you want to interview (they refuse, or are never available when you call back) then your sample starts to drift from the age/sex/race/etc groups you should be getting. This drift is not random. Younger people are always hard to get. Instead of spending lots of dollars on higher numbers of callbacks, research companies use post hoc weighting be certain demographics to “adjust” the sample. They do this without taking into account that the “adjustment” comes at a cost (it decreases your effective sample size) when they report results. But it gets worse. Most companies use a technique called “rim weighting” which can lead to improved estimates or worse estimates depending on the correlation structure underlying the demographic weightings. The catch 22 is that when you use “rim weighting” it is because you don’t have the full matrix and thus can’t know the underlying correlation structure. You don’t know when it helps and when it doesn’t, and the added uncertainty due to weighting isn’t usually calculated. More issues arise with non response. If 6.5% of people don’t know which way they will vote (of the 90% of the sample who say they will vote…) then this all decreases the effective sample size. So 1000 respondents have now become…1000 * .9 * .935 = 841. And the list of factors continues. As a rule of thumb I tend to go for effective sample size being half of what the survey sample size is…but then of course the effective sample size varies from question to question. Note that if you are making comparisons from one time to another in the same survey (and the methodology hasn’t been changed) then some of the design bias cancels out. However, there are still a number of factors which will lead to the true estimates for uncertainty being larger than reported. There, I feel better now.
32. Draco TB 32

    15 April 2008 at 3:17 pm

    They have done their time and Kiwis with an extraordinary sense of fairness want the other team to have a turn at batting.

    Anybody who actually thinks like this is stupid.

    And young Johnny is going to step up first and hit a maiden century on his debut.

    If he gets in I’d say it would be the exact opposite. He’s shown conclusively that he has no leadership capability, doesn’t know his parties policies, doesn’t seem to have even a basic grasp of economics or, more importantly IMO, psychology.

    The only things he’s shown that he can do is smile, flip flop and prevaricate. He doesn’t do those very well either.
33. steve black 33

    15 April 2008 at 3:19 pm

    um. that’s supposed to be weighting BY certain demographic groups not weighting BE certain demographic groups. And yes, I do have a PhD and have worked in the survey biz (university and private research companies) from the late 70s to the early 00s. It’s a jungle out there.
34. David 34

    15 April 2008 at 6:00 pm

    I agree, Steve, but the main point is that the tables provided by the likes of Roy Morgan give a good estimate of the sample variance assuming a simple random sample and given that we are not talking about the extreme values (2% away from the boundaries seems ok).

    The issues you raise about effective sample size are all important ones, but even if your rule of thumb is roughly accurate, we only see the error bars increasing by a factor of 1/sqrt 2.

    So here, we dont expect to see the green poll result to go from 3% to 9% because of sampling error. The smaller party poll results at this time of the election cycle seem to be very volatile and could well relate to specific news events. In some ways, polls at this time are irrelevant because we are not having an election tomorrow so people arent actually thinking about who they will vote for.
35. steve black 35

    15 April 2008 at 9:00 pm

    The Green results did not change statistically between the last two polls. Here are the confidence limits done with approximate (but much more accurate than the Roy Morgan table) estimates using the F distribution (Zar 2nd Ed 1984, p378)

    previous poll 6.5% confidence limits 4.28% — 8.88%
    this poll 9% confidence limits 6.4% — 12.1%

    the confidence limits overlap between the two polls thus no change is seen at the 95% confidence level. This is based on an effective sample size of 400 (down from their 800 but I’ve still been generous given the reductions due to all the differences between a “simple random sample” and what is actually done in the field).

    Roy Morgan just offered an explanation for a non significant result. He’s not the only one and I don’t mean to single him out. Most of the changes in polling which are commented on are not significant. Could do better.

    David, you said “we dont expect to see the green poll result to go from 3% to 9% because of sampling error”. Perhaps not, but the Roy Morgan comment was based on Greens going from 6.5% to 9% and that’s what I’ve used in my analysis. I just don’t see the point of spending the money on collecting the data and doing the analysis and not bothering to use the correct formulas. But then that’s just me. Bugger the pollsters.

    BTW, the percentage cutoff is generally 20% or 80% for using the kind of normal based approximation which accompanies the Roy Morgan report (Zar p379). That means every analysis for smaller parties should be based on a more accurate calculation of confidence limits.

Links to post

1. The election 7 months out « the Nicholas O’Kane blog: posted on 14 April 2008 at 4:25 pm
2. Roy Morgan at thestandard.org.nz: posted on 24 April 2008 at 10:11 am

• Comments are now closed

• Log in
• Donate
• Contribute post

• Articles RSS