Aug: Horizon Poll Easter 2011

21 comments on “Aug: Horizon Poll Easter 2011 ”

• Comments are now closed

1. Bill 1

   27 April 2011 at 12:03 pm

   “..despite the general uselessness of our government the left is failing to reach the people of this panel, …”

   Wouldn’t have anything to do with the Labour Party compromising with uselessness, would it?
2. peteremcc 2

   27 April 2011 at 12:44 pm

   Anyone who knows any statistics knows that Horizon is useless – it’s self selecting.
   • Blighty 2.1

     27 April 2011 at 1:17 pm

     a phone poll is self-selecting too. You don’t have to take part unless you want to. You can’t unless you have a landline.
     • peteremcc 2.1.1

       27 April 2011 at 1:22 pm

       There’s a huge difference between being randomly selected and declining (and then the company randomly selecting someone else) and signing up to a website specifically for the purpose of taking polls.

       A few internet polls are starting to get some credibility in the US by using sample sizes of about 20,000-50,000, compared to a usual US phone poll sample size of about 2,000.

       The usual NZ phone poll sample size is about 1,000, so an online poll should have 10,000-25,000.

       The Horizon poll has 2,000 (which they then claim is good because it’s higher than 1,000).

       Problem is that with a sample size of 2,000, one hundred people is 5%. How hard would it be for a party to get 100 of their supporters to sign up to the website?
       • felix 2.1.1.1

         27 April 2011 at 3:20 pm

         How hard would it be for a party to get 100 of their supporters to sign up to the website?

         Probably quite tricky for ACT at the moment.

         Speaking of which, I note that you’ve had some “hair reform” lately. When will you be changing the name to “ReformOnCampus”?
   • lprent 2.2

     27 April 2011 at 1:47 pm

     …it’s self selecting.

     Huh? So are the normal polls. It isn’t that much different from land line polling.

     You have to have a land line which removes a large proportion of the population from sampling.

     You have to be present to be sampled. When we’re phone canvassing, it is quite typical to have more than 50% of the phone calls not to be answered or to go to answer phones.

     You have to agree to be sampled – ie to answer questions. The rejection rate is high.

     You have to actually answer. This is more of a problem with the reporting of the polls. One of the characteristics of many of the polls is that they don’t show the people who didn’t answer particular questions. At least Horizon and Morgan polls do so, which means that you can figure out what the undistorted percentages were.

     Anyone who knows any statistics…

     Evidentially this set does not include you. The key to getting a good statistical match between samples and reality is that the sample you are drawing from is representative of the population. All of the current polling techniques have some serious problems with that.
     • peteremcc 2.2.1

       27 April 2011 at 1:51 pm

       No, self selecting means YOU take the initiative to opt in.

       Everything you’ve said is true, that people can choose NOT to participate – but all of the examples you’ve given still include the polling company RANDOMLY selecting you from a list.

       Does no-one know basic statistics methods anymore?
       • lprent 2.2.1.1

         27 April 2011 at 2:03 pm

         But the nett effect is the same. You’re sampling from an unrepresentative population instead of a representative one. It really doesn’t matter how you got to that because the techniques required to correct for it are the same.

         The most extreme example is when you look at the very very low number of landlines amongst households that are in high deprivation index meshblocks (about 20-30% when I’m looking around Auckland). I gather that the Horizon has a similar distortion because the same areas have the very low participation rates in the internet.

         In either case you’re going to have to look at how unrepresentative the sampled population is and adjust the results to closer reflect reality. The details of how that adjustment (or indeed if it is) are not published by any of the polling companies.
     • peteremcc 2.2.2

       27 April 2011 at 1:54 pm

       And no, things like lack of landlines don’t affect the polls.

       Polling companies choose randomly but then filter to ensure an accurate representation.

       More younger people might not have landlines, that’s true. But that just means they will keep ringing until they get more younger people and fill the quota.

       If not having a landline is the KEY factor in making you vote different, then we’d have a problem, but if there is simply a correlation between say not having a landline and being young, and it’s being young that is the KEY factor, then there’s no problem.
       • lprent 2.2.2.1

         27 April 2011 at 2:26 pm

         Remember you’re talking about listed landlines. There are quite a few choices about why people don’t have landlines and it is systematic not random. In other words the population is divergent from the base population.

         Young is one correlation with land lines. Another is incomes. Another is ethnicity. Another is the type of residential area you live in – central urban areas all have the least number of listed lines.

         But that just means they will keep ringing until they get more younger people and fill the quota.

         That really doesn’t help. If there is a systematic bias about why people don’t have landlines then all you are doing is accentuating the sampling problem.

         For instance if the under 25’s you can contact are living with their parents, then they are completely unrepresentative of those who are off living in flats without landlines. Similarly if the proportion of rural young are far more likely to have landlines, then you’re under representing urban youth.

         In practice, both of these are the case. The young group with the least access to landlines are urban and living away from their parents. Since that group (from canvassing) are significantly less conservative than the group that do have listed landlines then they are underrepresented in the polling. Continuing to collect more of the conservative group just means that you’re weighing that group more.

         The same problem applies to most other groups.

         It doesn’t matter if you are choosing randomly when the population set that you are choosing from is itself biased. You wind up sampling the population who have landlines rather than the voting population. The two are increasingly divergent.
3. ak 3

   27 April 2011 at 1:21 pm

   They’re all self-selecting. No gunpoint polls as of yet, and consider the average >70% refusal rate of the landliners (“piss off I’m eating tea”). At least this one outlines a more sophisticated methodology – and gives it around even stevens Left/Right % if you flip the MP over, which is highly likely, never mind what may happen to ACT or even GERIACT….
   • ianmac 3.1

     27 April 2011 at 2:49 pm

     GERIACT ? Great! Mind if I use it ak?
     • ak 3.1.1

       27 April 2011 at 6:02 pm

       Not a wee bit ian – and might I suggest lots more of DonKey govt, a vote 4 John is a vote 4 Don, Tail-wagging Banker, Two Tory Bankers etc.

       On the landline thingo, Lew found Pew who found cellies do (make a diff)
       http://pewresearch.org/pubs/1806/growing-gap-between-landline-and-dual-frame-election-polls

       (massive probs loading comments here Lynn…..)
4. vidiot 4

   27 April 2011 at 1:33 pm

   So who won this months $1000 & iPad ?
5. wtl 5

   27 April 2011 at 2:13 pm

   Edit: Meant to be a reply to peteremcc above

   If not having a landline is the KEY factor in making you vote different, then we’d have a problem, but if there is simply a correlation between say not having a landline and being young, and it’s being young that is the KEY factor, then there’s no problem.

   It does not need to be a key factor in itself. All that is required is that there is a bias introduced by the selection criteria (including not having a landline, not answering etc.) that is not corrected by the weighting they use. Given that the weighting can only be done by a limited number of factors, I seriously doubt they are able to remove all systematic basis from the polls.

   If not having a landline was a key factor in making one vote differently, this actual creates a very serious issue in the poll, as these people are COMPLETELY unrepresented in the poll, and there is no amount of weighting that could be done to correct this. (I don’t this probability is that unlikely – one can easily imagine that many university students living away from home might not have a landline. These people might not vote the same way as others the same age – e.g. those already working – or even other students living at home – though I doubt the polls are corrected for ‘occupation’ anyway).

   Are these factors serious issues? It is impossible to tell – because there are no polls that don’t suffer from these issues at all. The fact the results of the ‘randomly’ selected polls appear more different from each other than would be by chance suggests that there is a degree of uncorrected bias there. The difference between the Horizon poll and the other polls may be due to the Horizon poll being self-selecting, whereas the other polls are ‘correct’. Or it could be that the Horizon poll is ‘correct’ and the others are biased by the above issues. The reality is obviously somewhere in the middle, but it is not clear how far it is. I agree that the Horizon poll is probably not as good as the others, but the issue is not as cut and dry as you make it out, especially as you claim to know a lot about statistics.

   Edit: Well, there is obviously a poll that doesn’t suffer from these issues – the only one that actual matters.
   • Draco T Bastard 5.1

     27 April 2011 at 4:13 pm

     The reality is obviously somewhere in the middle,

     I really hate that assumption as there is no evidence for it at all. In fact, you even made that point in your comment.
   • Deadly_NZ 5.2

     27 April 2011 at 9:36 pm

     “If not having a landline is the KEY factor in making you vote different,”

     But you then have to look at the demographics of who has the landline.

     They would more than likely be white middle to upper class with grown up children and plenty NACT voters . Or people like me who need a landline for medical / work reasons and are low paid or on a benefit. Labour/Green/NZP and maybe even Hone voters. And then you have people between say 17 and 30 say who are flatting, all have prepay cell phones, Low paying jobs, or who are studying, and are living week to week or even day to day. Also labour/Green/NZP and maybe even Hone voters.

     This is a huge block or people, they use the library and University for Internet or parents on the time they have at home. And they are NEVER polled but they will vote. So I would imagine that the polls are skewed towards National

     And now there are a few more variables to add in now.
     1: The internal Crumbling of support for the ACT party re old Rodders.
     2: The withdrawl of the stake from Brashes cold dead heart.
     3: Dracbrash to rodders give the party or i’ll bleed you dry.

     And then you have the way the questions are asked and how they are worded.
     You would have to account for the political affiliation of the person doing the polling.

     Yes all in all I reckon a poll is accurate if I was a NACT.

     just a thought
6. Carol 6

   27 April 2011 at 2:29 pm

   Is the Horizon Poll any less representative than Hooton’s I-Predict thingy? – I don’t understand it, but it sounds like self-selecting speculative gambling to me.
   • higherstandard 6.1

     27 April 2011 at 2:38 pm

     It’s all a load of wank Carol, wait till the day after the election then we can all be expert pollsters after the fact.
     • Carol 6.1.1

       27 April 2011 at 2:54 pm

       Well, I watch all the polls with a detached kind of interest. The only problem I see with them is that they become a self-fulfilling prophesy. But my approach is to stick with supporting the policies I think are best for a fair and democratic country & to try to promote or support those. Word-of -mouth & the digital equivalent, is very important for the left, I think.
     • lprent 6.1.2

       27 April 2011 at 10:40 pm

       Yeah, all of the polls are inaccurate in terms of predicting the actual outcomes this far out. They tend to become more accurate getting closer to an election as people firm up their opinions and get more willing to answer. Even then, they are often significianly variant from the actual result.

       What they are interesting for is to look at longer term trends in each pollsters series than just a single poll. The Morgan is the most useful because of it’s two week poll cycle. The Horzion poll is looking interesting because it seems to be mostly monthly. Because they are in a series using a consistent sampling and processing technique, they yield quite a lot of information about trends.

       Almost every other poll in NZ is pretty useless because they seem to sample about every 3 months or so, which is damn near a lifetime in politics. Worse, the talking heads who pontificate about them compare between these unalike polls, which clearly have quite different biases in their sampling and processing. It makes you want to send many political analysts and news readers back to schools for a reeducation (or just education) on statistics.

• Comments are now closed

• Log in
• Donate
• Contribute post

• Articles RSS