How much can we rely on YouGov?
January 23rd, 2005-
Why does Kellner’s firm have the lowest Labour figures?
With today’s YouGov poll likely to take the edge off Labour on the spreadbetting markets we ought to look at how the internet pollster operates.
For since starting political surveys at the last General Election YouGov, headed by the well-known politcal writer Peter Kellner (above), has consistently produced figures that set it apart from the rest of the pollsters who rely on traditional interviews . Today’s 34% for Labour is 4-6% below the latest ratings from ICM, NOP, Populus and Communicate Research. It was YouGov, punters might recall, that first picked up the surge for UKIP at the Euro Election last June although, in the end, the pollster had an exaggerated figure.
-
The main criticisms of YouGov are that those polled are restricted to members of the public who have registered and who have internet access. This, it is argued, could produce a skewed sample and that the people who volunteer to go on the list might be atypical.
The biggest oddity, as the polling commentator Anthony Wells has observed, is that YouGov operates in a way that should produce much higher Labour totals. He has noted:-
YouGov does not take into account the likelihood that someone surveyed will vote - this usually helps Labour.
YouGov weights the past voting recall of its samples at LAB 56%: CON 28%: LD 13.4%. This is very different from standard pollsters such as ICM which operates on LAB 46.5: CON 30.1 LD 17.6. So nearly a quarter more previously declared Labour supporters go into each sample than with other firms.
YouGov’s age profile seems to have the over 65s at less than 9% compared with more than 20% for the adult population as a whole. Given that this group is the one most likely to vote and to support the Tories this surely has an impact.
So why, in spite of all of this, does YouGov tend to be the least favourable pollster for Labour? We can only conclude that some people complete an internet survey in a totally different way than when answering questions over the phone to an interviewer. Could it be that being pro-Labour is something you are more likely to be in public than in private?
Another feature of YouGov, some say, is that its methodology tends to magnify opinion shifts like UKIP last May. Could the latest LD surge be another example?
The pollster usually responds to criticism by pointing to its record which when tested against real votes is the best of any firm currently carrying out UK surveys. The majority of entrants to the site’s New Year competition predicted that the firm would by the top pollster at the General Election.
Latest IG Index spread prices: LAB 359-366: CON 183-190: LDs 70-74.
Latest Spreadfair prices: LAB 359-361.5: CON 188-190: LDs 68-71.9.
Mike Smithson
MessageSpace Advertising
It’s curious that Yougov consistently rate Labour worse than other pollsters, yet no longer rate the Conservatives better than other pollsters (since May).
They consistently give UKIP 5-6% of the vote, and I suspect that prior to May, most of that group were saying they would vote Conservative.
I doubt if UKIP will manage anything like 5-6% at the election, and so, by default, some of those people will vote Conservative.
As Anthony Wells says, Yougov’s methodology should favour Labour. OTOH, my impression is that internet users are much more eurosceptic and much more anti-socialist (but not necessarily Conservative) than the population at large. Perhaps one polling bias cancels the other out.
Sporting Index now have a full range of seat markets - though they are very wide on Respect and rather lofty on the SNP.
Jon - IG has an even bigger range and has just put on all the Northern Ireland parties this weekend.
Perhaps one of the other pollsters would like to include a question on internet access in their next poll and publish the result of the sub-sample? I think internet access has spread beyond self-employed techies and noisy libertarian bloggers thse days…
As internet penetration increases the number of people with landlines that have numbers that get picked up by the pollsters’ auto-diallers, and who are in at the right time, don’t have call screening, answer the phone and are ready to co-operate is declining rapidly.
The phone pollsters make 5-6 calls for every interview carried out.
Sean at 1: As the MP who operates possibly the largest internet list, it’s certainly my impression that Internet users are distinctively different from the average, even when they appear to fulfill other criteria. Internet users who sign up with YouGov seem likely to be more distinctive still - basically highly-motivated people with a keen interest in current affairs. I would expect this to produce greater volatility and sensitivity to recent events - someone who doesn’t read a paper and picks up a vague impression of news from Radio 1’s hourly bulletin is probably less likely to be affected by, say, good or bad news from Iraq. I suspect the GMW Lab-Lib floaters are present in greater numbers on the panel than in the electorate as a whole.
Mike at 5: yes, that’s true too. I followed a Democrat web site (electoralvote.com) throughout the US elections which kept eagerly predicting that the polls were wrong because of factors like this - alas, no. But it’s certainly possible that this too will introduce distortions, though the group you describe sounds less distinctive than the YouGov panel. Ironical if we gradually sink back into the pre-1948 fog and have to actually wait for elections to find out who’s going to win…
Nick
Nick - I must admit to being slightly uneasy about YouGov. It’s not just the internet which is the problem but the fact that you sign up. Those who are polled are self-selecting. YouGov make great play of the efforts they make to attract non-politicos and no doubt that works. I’ve been polled three times by YouGov in the past four months and I would suggest that I am atypical.
There was some research that showed that internet users are much more likely to embrace the new and this, I believe, leads to the magnification of trends - like with UKIP at the Euro elections. YouGov picked this up first but then over-estimated it. The same effect seems to have happened last year when Michael Howard became Tory leader. YouGov’s people rushed to embrace the new and we saw those 40% ratings.
I think that the latest move to the LDs by the pollster is as a result of a similar effect.
Having said that they are by far the most accurate pollster with the Lab-Con split.
re 3. Good to see the Northern Ireland parties quoted on IG Index. If I was a spread betting man I’d be tempted to sell DUP at 8 and Sinn Fein at 5.5.
Innocent - ICM and Populus do ask respondents about their internet access, although I have never seen them publish those people as a sub-sample. FWIW, in Populus’s last survey 66% of respondents said they had internet access.
I should say from the outset though that it would not be a particularly worthwhile exercise anyway. Internet users are not a sub-sample of people who answer phone polls, there are some people who answer internet polls who could not be contacted by phone polls (people who use the internet but only have a mobile phone, to pick an obvious example).
Mike. 2 points. The phone methodology should (and does) include up to 10 phone backs of a number, so if an answer is not immediately answered it is called back at different times of the day, and on different days. We also use screening techniques to ameliorate the effect of unbalanced samples so that the effect of weighting by socio-economic class is lessened. Our current response rate on the South West Poll is about 40%. I don’t know what YouGov’s is, because they use a panel system (self selecting as we have discussed before) and payment - so I presume that the response to the actual survey is relatively high.
The problem with panels is that - as I have shown before - they have some serious weaknesses in that they tend to attract atypical people. I don’t think that internet access is now the problem for them, but the fact that they have people who ACTIVELY want to participate in surveys (in my experience a relatively small group).
This having been said, YouGov have a number of things going in their favour, that to a certain extent does not back up Nick’s argument. YouGov tend to be the least volatile poll - this they argue is because they have very large samples which are less prone to sampling error - they therefore argue that changes they do pick up are not Margin of Error / Sampling error, but actual changes in public perception.
All of this discussion brings me back to the same old point. Polls are not about getting figures bang on. They are about plotting trends and giving guides. The trend on You Gov is that Labour are 1% down compared to Jan 2004 (i.e. no change). The Conservatives are 9% down on Jan 2004 and the Lib Dems are 6% up.
If YouGov are right in their assessment of their polls (only charting real changes in public opinion) then the only conclusion that anyone could realistically make is that the party with the biggest problem continues to be the Conservative Party, in that, no matter what they try they appear unable to convince people that they are in a position / state to become the next Government. And more worrying still for them, is that contrary to previous theories espoused (including by me), the movement over the last year doesn’t seem to be Lab to LD, but Con to LD.
If I was going to be terribly reductionist and take the yearly trend of YouGov’s poll at face value - with the Cons losing an av. 0.75% a month in 3 months they will be at 28.75% - Lab losing .083% a month = 33.75% and the LDs gaining .5% per month = 26.5%. Those figures on Baxter (no tactical) give Lab 379 Con 159 LD 77.
I should think it more likely that the decline in the Conservatives’ Yougov rating over the past year is explained by UKIP going from 0-1%, 12 months ago, to 5-6% today.
Anthony (9) - thanks.
Graham (10) - I think on those figures we would be looking at an election such as there hasn’t been in our lifetimes (i.e. since 1923)and whilst not impossible, it would be most surprising if UNS worked in such conditions.
Not to seem cynical but didn’t YouGov pick up a UKIP surge around the time they took some big cheques off UKIP to do polling?
Mike - you commented in an earlier thread that YouGov changed their methodology just the other day so that party of choice is a write-in thing rather than a tick box thing (i.e. it is now unprompted). Your view (and I tend to agree) was that this would probably slightly hit the LDs and hit UKIP very badly.
Do you know whether the latest poll was conducted with this new method?
Just seen the ITV lunchtime news, lots of coverage of the tory immigration plans, largely positive, with only a couple of seconds of tony blair and a very podgy looking charles kennedy moaning about multi-ethnic blah. Its certainly getting a lot of press coverage.
RE Graham at 10. Surely voters tend to move right to left (or vice versa) so Tory to Labour, Labour to Lib Dem fits quite well. Actually the voters have stayed where they were and the parties (especially the Labour Party) has moved.
Agree that UKIP are tending to split right wing vote - Why are they so quiet - did Kilroy have a point?
To 14. I was asked to do 2 YouGov polls last week. The first was the Sunay Times one - I think - and the second one, on Friday, had the new “write in” the party name feature.
Icarus. I don’t think that vote switching is ever a simple process. Few people demonstrate a completely consistent set of beliefs. Many people will not agree with large swathes of the party they vote for’s policies. Voting decisions are made up from a number factors, including perception, gut feeling and peer pressure. It is thus possible for a ’soft conservative’ to switch to the Lib Dems, because they are against Labour, but are not happy with the Conservative Party, but also for a ‘peacenik’ social democrat Labour voter who was disappointed over the war, but had misgivings about Respect also making that switch. To characterise the electorate as left / right would be an oversimplification.
Him Mike, George Trefgarne from the Telegraph here. I’d love to have a chat at some point. Could you e-mail me on george.trefgarne@telegraph.co.uk? Thanks
Icarus at 16 and Graham at 18: I think that Charles Kennedy would also disagree with the perception that the LibDems are to the left of Labour - my LibDem friends generally reject the simple left-right model. It’s difficult to generalise, but I’d say that the LibDems tend to be ‘left’ of Labour on what you might call Guardian issues (e.g. the Tobin tax idea has more LibDem supporters), and ‘right’ of Labour on Daily Mirror issues - for instance, the LibDems opposed the Minimum Wage, arguing (as did the Conservatives) that it would disrupt the market economy and cause unemployment, and in general LibDem MPs don’t attend debates in Westminster on industrial/urban/domestic poverty issues any more than Conservative backbenchers do.
Nick
I recently changed job and moved location. After updating my Yougov profile I started to recieve more requests to participate in polling. It could be coincidence but it could also be due to the fact that I now had a rarer profile and so was of more use to them in ensuring they had a fairly broad sweep of voter types.
The fact that yougov have more demographic data on the participants in their surveys may help balance out any presumed net access bias. It’s not as though yougov survey their entire voting club at any one time.
YouGov havent asked me to respond since 8 Dec but asked 5 times in November. They now credit me with £8.95 - has anyone ever been paid out?
plaque McGinnis.interns!aboveground acclimated .
VCDbHHKm37z f3KDA5GVkBu9Oo IvBMzyZZ00HC
home insurance quotes…
stripper bricklayers?systematic digestible brushed funds rapid galleries:specialties?insurance quotes online http://insurancequotesonline.groupxq.com/ …