It was a limited poll done around Wellington on Monday using a true random polling technique. It also reported all of the figures including the numbers for which there was no answer.
As you can see from the pie chart left, the actual number of people who gave a party preference was about a quarter of those sampled.
The highest proportion were those who didn’t answer the phone. Because the poll was on a single night, they were not able to retry the phone numbers. The word from people I’ve talked to says that polling companies often have to call up to seven times in the one or two week period to be able to contact someone. Even then there is high proportion from whom they do not get an answer.
But even this is not the full story. The methodology of this and every other phone poll has problems apart from the short survey period. It relies on land lines..
Selecting random numbers from the Wellington phone book and speaking to whoever answered the phone, 24 NewsWire reporters called 1147 residential numbers between 7pm and about 8.30pm on September 15.
It took that many calls to get answers from 770 people, with 200 of those who picked up the phone declining to participate.
I just checked on the availability of landlines with the Wellington Central electorate. Of the 44.5 thousand odd voters enrolled a few months ago, only 57% are contactable by land-line using the white pages. Between cell phones, unlisted phone numbers, and people who just don’t have a phone the samples look highly selective and almost certainly self-selecting. Wellington Central is close to the average of the electorates for phones. In Auckland the percentage of voters contactable by land-line can range from about 35% in South Auckland electorates to 70% in the North Shore electorates.
In my opinion, most of the published polls are about as useful as online polls. Read 08Wire’s take on those in “Memo to Whale: Online Polls Stink” . I particularly liked the story of how Hank the Angry Drunken Dwarf won an online poll for most beautiful person of the year.
The published telephone polls are self-selecting samples of who has a phone, who is willing to answer, and who has a decided opinion. At best they are indications of trends in a poll series from a single company. Different methodologies between companies make cross analysis of polls almost useless.
At worst, well just read the pontificating about polls and across poll series by the local news media. You get the distinct impression that they consider that the polls absolutely say what the electorate will do, and they treat the published error rates as gospel. It would be interesting to see what their reaction would be if they were exposed to some of the requirements of a valid statistical study, and then shown how far their phone polls deviate from it. In the end there is only one poll that counts, and it happens on November 8th.
Chris Trotter also comments on this poll in Counting the horse’s teeth. It appears that newswire is produced by Whitireia Journalism School. Perhaps a new generation of journalists who are better informed?
Update: Matthew Hooten misses the point of this post (as usual with his comments about this site).
hat-tip: bill browne
Powered by WPtouch Mobile Suite for WordPress