Making the polls better
December 3rd, 2004NOTE: LIMITED SITE UPDATES FOR THE NEXT WEEK BECAUSE OF OVERSEAS WORK COMMITMENTS
-
Establishing whether those in polling surveys actually vote
One of the obvious ways that the polls are flawed is that currently about 70% of those interviewed say that they voted last time but we know that less than 60% actually did so. Thus one in ten are either telling lies, wishful thinking or have poor memories. Whatever it must distort the polling outcome.
-
Would it be possible, we have been wondering, to devise a research project to identify this group and to examine whether they have different characteristics from the rest. Thus are the non-voting people who say they vote more likely to, say, be declared Labour supporters than those of other parties. If we knew that for sure polls could be made better.
An ideal research project would be to find a means of verifying what those who were interviewed in the surveys actually did on election day. Clearly within the confines of a secret ballot that is not possible. But there is a way of mounting a research project that could provide information that was nearly as good.
It is not widely known but after each election a copy of the “marked register” is made available by the authorities. This is a public document and shows those who have voted and those who have not. Many smart local campaigners use this use this data to plan future operations.
Is it feasible if after the election the polling organisations in conjunction, possibly, with some University politics departments could seek to verify whether all of those interviewed in selected surveys actually turnout? To cross-check this with polling data could provide a mine of information that would help create better polling methodologies for the future.
This could work with pre-election polls on the “likely to turn-out question” and on post-election ones on what those in the surveys actually did.
-
Thus it might be that 90% of Labour voters who say thay are “100%” certain to vote actually did so compared with, say, 96% for the Tories and 94% for the LDs. For future polls you could then weight the turn-out reply for each party.
In post-election surveys it would be invaluable if you had data that showed, say that only 85% of those saying they had voted Labour had actually turned out. These are all hypothetical figures to demonstrate the value of the data that could be obtained from such a survey.
General Election spread prices are unchanged and Labour has still to return to the 346-354 position of late July following the Leicester and Birmingham by-elections when it was just 1% ahead in the polls. LAB 345-353: CON 200-208: LD 71-75
MessageSpace Advertising
Mike
This sounds like a useful exercise. However, something similar should already be achievable from existing data. The British Election Study (large-scale academic survey carried out each election by Essex University and others) already contains data on both self-reported turnout and verified turnout for each respondent. (As well as the usual data on which party was voted for, etc.) This data is used for example in Rosie Campbell\’s paper on turnout and gender (link provided). One should also be able to use this data to answer precisely the questions you ask. Note however that there are large gaps in the verified or validated turnout data — they were unable to verify whether up to 25% of people had voted.
http://www.essex.ac.uk/ECPR/events/jointsessions/ paperarchive/turin/ws22/Campbell.pdf
Couldn\’t part of the difference be simply that those who are more likely to agree to respond to pollsters are those who are more politically minded anyway and therefore more likely to vote? Polls can only ever be representative of those who agree to participate.
I think an interesting idea to improve polls longterm would be to use the detailed marketing systems such as Acorns or Experian, they are uncanningly accurate of most peoples neighbourhood and I am sure some way of weighting them by likelehood to vote could be found. Then a more picture of what might happen in each constituency would be easier to build up, bar local campaigning.
There is indeed an academic literature on the topic of non-voting, and marked registers are used to try to identify why participants in longitudinal panel surveys (interviews of same respondents at t1, t2, t3 etc including before and after elections) like the British Election Study didn\’t vote.
Of course we cannot be sure that exactly the same happens with conventional polls, but my recollection is that the non-voters in the BES were as one would expect: heavily skewed by age (the younger you are the less likely to vote) and class (the more working-class you are the less likely to vote). Less important but still measureable are the marginality of the seat (higher turnout in marginals), race (less Afro-Caribbean) and, rather controversially, the amount of targeting by parties (may or may not exist as separate from the above).
In most of these cases the hypothesis is supported that non-voters, including those who say they will vote in surveys and polls, are lost to Labour significantly more than the other parties.
This is indeed probably one of the reasons for continued over-statement of Labour in the polls. I am convinced that it was a major factor in 2001, with its massive and unexpected drop in turnout.
Incidentally, an easy way for non-academic enthusiasts to follow the latest (though that\’s not quick) research is to join the EPOP (Elections, Parties and Public Opinions) sub-group of the PSA (Political Studies Association). This currently costs 25 pounds per year for which you get cpies of the EPOP newsletter about four times a year, regular e mail updates, a copy of the annual British Elections and Politics Review, and an invitation to the annual conference. Just imagine it - 150 psephologists in one place, all weekend!
Contact Justin Fisher (justin.fisher@brunel.ac.uk) or Jonathan Tonge (j.tonge@salford.ac.uk).
I\’m one of those people who if asked for my postcode(to help in a survey at a retail centre for example) always give the wrong one on purpose.I\’ve never been asked for my political opinion (yet) but again I would lie - not just say\”don\’t know or no opinion\”.Why do I do this?I\’m not sure,I thinks it\’s because I feel the pollster should pay for the privilege of extracting useful information(to him)from me.
When asked to fill in \”ethnic monitoring or lifestyles\”questionaires then I\’m afraid my imagination runs riot.
I thought the marked register was kept confidential and only used in alleged fraud cases.
No the marked register is freely available to the political parties so if you lie about whether you are going to vote we can find out!!!
I too am in the habit of giving false information to researchers. Most research is consumer research and the latent socialist in me doesn\’t like the idea of helping out the marketing departments of large corporations.
Were I to be asked for my political affiliation I would probably be honest though, largely because (like others here) I follow opinion polls and would like them to be as accurate as possible.
However, even in politics, there must be many who deliberately lie. Do (can) the pollsters allow for this?
As an aside, some years ago I knew someone in a research company. Whenever he couldn\’t find enough people of the right demographic to attend a focus group he would call me and ask me to pretend to be a Telegraph reader/home owner/whatever was required at the time.
For hard cash and travel expenses it was well worth it, and I know from post-group discussions that many others were in similar positions. Easy money and hours of fun!
Research is only as good as those people being researched.
Jon - I always give the party tellers false information -when asked for my poll number outside the station I always add 5,000 to it if asked on the way out to check I apologise and give them a totally wrong number.
It\’s good fun.
Even the marked registers are not foolproof - being one of those people who gets copies after local elections, I am able to compare it with the tellers\’ results, which include actual poll cards from people who hand them over (hence not in a position to give false information), as well as people stating their numbers (who could give false information).
A significant percentage of polling cards are not recorded as having voted on the marked register,
8 Vino - complete mistake to give wrong number as if you are on any party\’s list of supporters, you will be called on endlessly until you have voted. talk about shootin yerself in the foot ! My father in law says yes to every canvasser thinking he\’s saving himself some bother. As he\’s a facist and I\’m a LD, I see no reason to put him straight ! The best way to be ignored by all the parties is to say no to all canvassers and give your correct no. at the polling station. I do not understand why those who do not want to be disturbed think anything else will make us leave you alone
BTT - thanks for the advice - I always vote anyway(even in parish elections) - to be honest the only party to have ever contacted me was Labour.At the Euro elections in June the only parties who sent information were the Tories and BNP.
Vino - what constituency are you in ? Judging by contact, sounds English and maybe Humber or Yorks ?
play chess online Heh. How it goes? Buy it all. ASAP. Last discount in your live (AAAAA!!!!!). Take a rest.