Why Polls Go Wrong

Prepared by AMSR using the Archive

Poll accuracy and polling failures

A-Level Students are expected to know that polls can be wrong, and to understand why. Students should be familiar with:

sampling issues
margins of error
shy voter effects
late swings
methodological differences between polling companies

The Elections used as examples are the following :

1970 (surprise Conservative Victory)
1992 (unexpected Conservative win)
2015 (underestimated Conservative vote)
2016 EU referendum expectations
2017 and 2019 polling debates

These examples help students meet the requirement to evaluate the reliability of evidence used in political analysis.

Introduction

The following report summarises what the AMSR Archive contains on why opinion polls sometimes go wrong, organised by the major UK General Elections where notable polling failures occurred. These cases — 1970, 1992, and 2015 — are the ones most frequently cited in the Archive and together provide a clear historical picture of how methodological choices, voter behaviour, and changing technologies have shaped polling accuracy over time. Remember that the strength of the AMSR Archive is that it reports things as they were at he time.

Summary

It is worthwhile stating that in statistical theory, opinion polls based on samples of around 1,000 adults should deliver estimates with an accuracy of roughly ±3%. If the actual election result differs from this range, we either have an outlier (a rare but possible deviation within sampling theory) or something has happened during the polling processto distort the estimate. The most common problems identified across the Archive include late swings in voting intention, differential turnout, differential refusal or ‘shy’ voters, unrepresentative samples, and methodological differences between pollsters (random vs quota sampling, telephone vs online, weighting strategies).

Why polls go wrong: The five big reasons

1. Late swings in voting intention

A surprising number of people make up their minds right at the end of an election campaign — sometimes in the last few days, or even the last 24 hours. Studies show this can be 8%, and in some elections as high as 14% of voters. Polls can only measure opinion at the moment they’re taken, so if interviewing finishes days before polling day, the poll might reflect yesterday’s opinion rather than today’s. People may tell pollsters one thing, then vote differently once they’ve had time to think — or panic.

2. Differential turnout

This is when supporters of one party are less likely to actually vote than supporters of another. A classic example is 1970, when younger voters were more likely to support Labour — but also more likely to stay at home. Older voters, who leaned Conservative, turned out in higher numbers. This is why you sometimes hear the phrase “they didn’t get their vote out”. Polls can’t perfectly predict who will actually show up.

3. Differential refusal — the ‘shy voter’ effect

Some people simply don’t want to say who they’re voting for. Maybe their party is seen as unfashionable, selfish, or controversial. In 1992, many Conservative supporters were reluctant to reveal their true intention — creating the famous “shy Tory” problem. If one group refuses to be interviewed more than another, the poll ends up with a distorted picture of the electorate.

In 1992 after Thatcher’s final years, many people felt uncomfortable openly supporting the Conservatives. Policies associated with the party — especially the Poll Tax, NHS pressures, and economic difficulties — had made Tory support feel, in some circles, selfish, harsh, or unfashionable.

4. Unrepresentative samples

Pollsters must interview a group of people that reflects the whole country. But several things can go wrong:

Interviewers might talk to whoever answers the phone (or the door), rather than selecting a random person in the household.
People who are at home may have different views from people who are out.
Some groups — for example ethnic minorities — might be under-sampled.
Online panels may attract people who are more interested in politics than the general population.

If the sample doesn’t match the real electorate, the poll won’t match the real result.

5. Methodological differences

Polls can be done face-to-face, by phone, or online. In theory, the method shouldn’t matter if the sample is representative — but in practice, each method has quirks. Pollsters also use different weighting systems to adjust their data to match the population. After 1992, for example, many companies started weighting by likelihood to vote. Today, some weight their results so that the previous election outcome comes out correctly — but this only works if people accurately remember how they voted last time. In short, even small differences in method or weighting can shift the final numbers.

Historical inquiries into polling failures — notably 1970, 1992, and 2015 — show that errors rarely stem from a single cause. Instead, they arise from a combination of factors: interviewing finishing too early to catch late movement; turnout patterns favouring one party; refusals disproportionately concentrated among one group; or quota controls and weights that fail to match the true electorate. To see how these problems play out, the Archive gives us three classic examples: 1970, 1992, and 2015.

In 1970, every poll except one predicted a Labour victory. The problem wasn’t simple sampling error. The Archive shows that interviewing finished too early, missing a late swing to the Conservatives. Newspapers also misreported poll data, often ignoring “don’t knows” and sampling dates. And turnout mattered: Labour supporters were more likely to abstain, especially younger voters. The one pollster who got it right — ORC — kept interviewing until the day before the election, capturing movement that others missed. So 1970 teaches us that timing and turnout can matter more than sample size.

In 1992, the failure was even more dramatic. All five major pollsters predicted a narrow Labour lead, but the Conservatives won comfortably. The Archive shows three main causes. First, another late swing, with many voters deciding in the final 24 hours. Second, weaknesses in quota sampling, which didn’t fully reflect the social profile of the electorate. And third, differential refusal: Conservative voters were more reluctant to reveal their intention, creating what became known as the “spiral of silence”. The inquiry concluded that no single factor caused the failure — it was the interaction of all three.

In 2015, the polls again underestimated the Conservative vote. This time the problem was largely unrepresentative samples: online panels and quota controls didn’t match the true electorate closely enough. The British Polling Council inquiry also highlighted confirmation bias — pollsters tended to accept results that looked plausible rather than challenge them. And when pollsters applied the recommended fixes, some of those adjustments actually made the 2017polls worse, showing how volatile modern voting behaviour has become.

Across all three elections, the Archive shows a clear pattern. Polls usually perform well, but when they fail, the reasons are almost always behavioural and methodological, not flaws in statistical theory. Polls measure a moving target — public opinion — and sometimes that target shifts.

Introduction to the content in the AMSR Archive: Poll accuracy and polling failures

Opinion polls: how accurate are they? – Nick Moon (2023)

It is important to question whether polls are historically getting worse, and/or are now so inaccurate as to be worthless at best, and misleading at worst
There are two reasons why polls might be expected to become less accurate over time:
- Declining response rates
- The emergence of new polling methods – particularly online polling and automated telephone interviewing
Despite these plausible causes of long-term decline, there is no trend towards polls getting less accurate. Indeed, errors may actually have declined. One of the reasons for this is that pollsters, like all good researchers, are constantly striving to find what might be making their surveys less accurate than they had hoped, and striving also to find ways of compensating for this
Polls have always evolved their methodology over time, to cope with changing circumstances and new technologies. When political polling started in the UK, probability sampling was the norm, but the time taken to achieve even a part way respectable response rate led to a switch to more time-sensitive quota sampling has generally been considered to have performed well
Telephone interviewing offered both cost and time advantages. It was used in the UK for the first time by Audience Selection, in 1983, and went on to become the norm for all pollsters. The next big change came in 2001, when YouGov conducted the first online general election poll and now most polls are conducted online, although using different methodologies
It has been argued that both these technologies have the potential to improve accuracy by allowing access to more representative sets of respondents
Over the last decade there have probably been more changes in polling methodology than previously, and there are certainly far more variations in methodology between pollsters than ever This has come about because the pollsters have been engaged in an ongoing programme of making changes in methodology, testing them out, and then if necessary refining them further
It matters to the pollsters themselves if the polls get it wrong as it is clearly not good for their reputations – but does it matter beyond that? This leads on to the related question, which crops up every few years in the UK, of whether polls should be banned, but the view is that this is undesirable and impractical in a global age
Polls get it right far more than they get it wrong which lends support to the industry

1970 UK Election: Surprise Conservative victory

Why a Sample of the Polls Suggests a Vote in Favour. Nick Moon: Research Plus issue 360 supp (50 years of market research) 1996 May – Research Magazine (1992-2012) – The AMSR Online Archive (oclc.org)

By the early 1970’s, there was a change from using random samples to quota sampling and, by the mid-1970’s, quota had almost entirely replaced random sampling as a method of conducting opinion polls (reason because random sampling considered too costly and slow)

Pollsters, the Press, and Political Polling in Britain. Robert M Worcester(1980).

During the 1970 election campaign, there were 5 polls being published, the same as in 1966, but in 1970 they were read by an enormously enhanced audience as exclusive publication was abandoned. Prior to this, newspapers had jealously guarded the copyright they had on their polls
Polls dominated the 1970 election campaign reporting, with poll results often being the main front page story, and, during the following September, The Harris Poll found 62% of the country remembered seeing polls (71% recalled seeing them on television)
As a result, in 1970 when the polls got it so badly wrong, they were very much in the public eye; the Conservatives were the surprise winners of the ‘sunshine’ election of 1970
Between 1966 and 1970, public opinion, previously extraordinarily stable, began to swing wildly. Labour won a landslide victory, enjoying a 20% lead in May 1966, but just 2 years later was 20% behind
In May 1970, when the election was called, Labour was ahead again. However, the Press reports during the interim period provided misleading data by rarely giving interviewing dates, hardly ever reported ‘Don’t Knows’ or ‘Refusals’, seldom offered breakdowns of top line figures, and showed little interest in issues. Sampling error was seldom mentioned, and headlines reported shifts of electoral support based on statistically insignificant changes
At the beginning of the 1970 campaign, led by Humphrey Taylor of ORC, the pollsters banded together to publish a Code of Practice to curb this misreporting of data by the newspapers
All the polls got the winning party wrong in straight poll figures, with all predictions being too Labour, but ORC adjusted its final raw figures by projected turnout to correctly predict a Tory victory
In 1970, 3 polls were conducted with random samples, 2 by quota. The mean error of the former on the Conservative party lead was 7.6%; by quota samples it was 5.4%
Reasons why the polls got it wrong in 1970: The method of sampling did not appear to be the answer and there was only suggestive rather than conclusive evidence for a late shift. There were many hypotheses (eg differential voting by non-contacts, organized deception by Tory supporters, an anti-poll bandwagon, collusion on the part of pollsters, selective turnout etc), but these were mostly inconclusive, post-election analyses tend to support the late swing hypothesis (Abrams, 1970; Butler and Pinto-Duschinsky, 1971; Hodder-Williams, 1970; Market Research Society, 1972; Rose, 1970; Teer and Spence, 1973)

Polling in Britain: A Historic Route to the Forecasting Future. Professor Robert M Worcester, MORI, traces the origins and track record of pollsters, politics and the media (1996)

The polls’ reputations were met with disaster in the 1970 election when, of the 5 final forecasts only one, ORC for The Standard, correctly forecast the Conservative victory: the average share error was 2.2 points, the same as in 1951, but the average error on the Conservative lead over Labour was as high as 6.6 points
After 1970, considerable effort was expended by the pollsters and an MRS inquiry to correct polling methods and prevent similar mistakes recurring. Post-mortems were for the most part inconclusive but tended to support the hypothesis of a late swing and higher turnout by Conservative supporters. Random sampling, which some companies had used but which necessitated a longer fieldwork period – making the data older and hence more vulnerable to late swings – was phased out in favour of (faster turn-around) quota sampling

NOP Bulletin – Opinion Polls and the 1970 General Election and Post Election Survey. Frank Teer, July 1970. (NB This whole document is extremely useful)

There is no doubt that the opinion polls did not come out of this election with much credit. All the polls with the exception of ORC were showing a Labour victory in their final surveys and although the Harris Poll and NOP detected the move back to the Conservatives at the last minute, their final polls still had Labour ahead
Many commentators take the view that this result does discredit opinion polling, but the result should be put in context. Polls have an almost incredible record of accuracy in forecasting the results of elections in this country. Until this election NOP had successfully forecast the results of 3 General Elections with an error on the gap between the 2 main parties of about 1%. Considering that the sampling error on this measurement is about 5% this is a good record by any standards. The record of predicting the results of by-elections is similarly accurate. This time, NOP put Labour 4% ahead of the Tories in their final poll, whereas in fact the Tories were about 3% ahead of Labour in the result
Were the polls completely wrong in thinking that Labour were ever ahead? Until this March opinion poll results have shown that the Conservatives were substantially ahead of Labour. These results were confirmed by the enormous swings to Conservative in by-elections and local government elections during the same period. Labour suffered vast losses in municipal and other local elections and lost what were generally considered safe seats at by-elections. In April of this year, the situation changed. The Conservative lead which had been falling since the beginning of the year virtually disappeared. At the same time, Labour gained ground in the GLC elections and following these there was a considerable swing back to Labour in the borough elections, a swing sufficient, according to The Economist, to give them a majority of 50 seats in the House of Commons at a General Election
Since that date virtually all the polls have put Labour consistently in the lead, though some of them showed a move back to the Conservatives towards the end of the campaign. The polls therefore appear to have read the trend correctly. Why, then, did they fail to get the final result right?
There seem to be 2 possible causes of the polls’ failure. In the first place, most of the interviewing for the final published surveys was completed during the weekend before the election. This is the normal procedure in pre-election polling, especially for the polls based on random samples which make use of the weekend to get a good success rate. ORC, who got the result right, continued interviewing until the day before the election (it is interesting to note that their result based on interviews for the same time period as NOP’s produced the same Labour lead as we did). It seems probable, therefore, that there was a late swing to the Conservatives, the beginning of which had, in fact, been measured by both NOP and the Harris Poll. This movement may well have been accelerated during the early part of election week by concern about the trade figures, unemployment and the possibility of further devaluation
If the Conservatives did win the election as a result of a late swing in their favour, this may be evidence of increasing volatility in the electorate. The causes of such volatility would not be difficult to find. Voters are increasingly cynical about politics and politicians and are finding it difficult to differentiate between the 2 main parties. Party loyalties could, in these circumstances, be breaking down. In the past, as much as 80% of the electorate were loyal supporters of one party or the other. This body of committed voters may be shrinking and this could make life more difficult for the opinion polls in the future. However, we should not read too much into the circumstances of one election, and further evidence must be collected before this judgement can be confirmed
The second possibility is that Labour voters abstained. This seems somewhat less likely than the first suggestion but remains a possibility. Only 72% of those registered to vote actually voted, which is a low turnout by modern standards. Certainly the turnout was lower than in 1966 and 1964. Opinion polls have always had difficulty in estimating turnout. We ask questions which give an indication of the relative enthusiasm of Labour and Conservative voters to actually turn out and vote and these questions did measure a differential in enthusiasm which would have affected the voting if the poll were low. What the opinion poll did not predict, however, was that the vote would be so low, and this is an area to which NOP will pay particular attention in the future. We must try to develop a technique which will predict the actual proportion of the electorate that will turn out and vote
Whatever the outcome of our enquiry, we at NOP will certainly take at least 2 lessons from this election. In the first place, we must find a method of conducting efficient random samples until the last minute before the election, and secondly we must develop and validate a turnout predictor

Post-Election survey 1970 – Summary of main findings:

The additional strength required to carry the Conservatives to an Election victory was found as follows:

The Tories made considerable gains from those who had voted Labour in 1966
Labour lost more of their 1966 strength through abstention than did the Tories
Whilst young electors are much more likely than their elders to support Labour theoretically, in practice there was substantial abstention amongst this group; and most of those who did not vote were Labour supporters
The main reason for the Tories being able to win this support was the Cost of Living situation, the enduring inability of the Labour government to beat the problem, and Conservative promises to do so. Comparison with a pre-Election NOP survey shows that the problem retained constant importance throughout the campaign; but the majority who thought the Conservatives the best team to beat the problem doubled during the 3 weeks leading up to 18 June
In terms of personal popularity, the Conservatives won despite rather than because of Ted Heath; more electors would still prefer to see Harold Wilson than Mr Heath as Prime Minister. However, the Tories seem to have fought a substantially better campaign than Labour, particularly on television. The campaign in general generated much less interest than did the 1966 campaign. And what interest existed was aroused by policies, whilst in 1966 more interest was aroused by personalities

1992 UK election: Unexpected Conservative win

Research Magazine issue 349 1995 June: Verbatim (Gordon Heald), June 1995.

The 1992 Election was described in the MRS report as ‘the failure ….. was the most spectacular in the history of British election surveys’ and it triggered fundamental questioning of the methodology of opinion polls
Many newspapers lost confidence in the polls because of the 1992 debacle, some drastically cut back, switched between pollsters, or even conducted their own

In the 1950s, polls were all conducted using random samples. By the early 1970s, a change to quota sampling was well under way and indeed by the mid-1970s quota had almost entirely replaced random sampling as a method of conducting opinion polls. This was clearly a major change in technique, but in most other respects there was little such change between the 1974 General Elections and that of 1992
The reason for the continued use of quota sampling is that practical reasons, particularly over timing, make it difficult to use random sampling for opinion polls conducted in the heat of an election campaign
The failure of the 1992 General Election polling was blamed on use of quota sampling, although this argument is too simplistic as it fails to acknowledge the highly successful run of polling using quota samples throughout the 1970’s and 1980’s
There are many people who believe that quota sampling has no place of any kind in serious research and these, mainly from the academic world, were the first to say ‘I told you so’ after the 1992 general election and point the finger of blame at quota sampling as a technique
The polling disaster of 1992 led to a radical revision of all aspects of opinion poll methodology
Although this led to further innovation, moving further from the 1940’s and 1950’s, in other ways there were moves back to some of the techniques used back then ie random sampling
In truth, pollsters tend to believe that both random and quota sampling both have their place in different circumstances
One of the clear signs to have emerged from the enquiry into 1992 is that the pollsters need to take the voting question less at face value and to look more at alternative questions, such as including questions regarding party identity, party leaders and best party for running the economy

Report into the performance of Opinion Polls in the 1992 UK Election (initial report and press release) (1992):

The opinion polls published on the day of the 1992 General Election produced estimates of party support significantly different from the actual share of votes cast. The total discrepancy over-estimated the Labour share of the vote by 4%, under-estimated the Conservatives by 4.5%, and over-estimated the Liberal Democrats by 1%
The net effect was to underestimate the Conservatives’ winning margin by 8.5 points
The error was shared by each of the 5 major pollsters ie no one predicted the result correctly
The Market Research Society (MRS) inquiry into the error in the opinion polls conducted during the General Election was able to account for more than half of the difference but felt that the remainder may point to some fundamental problems in the way the polls were conducted
Key factors were:
- A late change in voting intention as identified in recall polls (conducted since the election) accounting for 3% of the difference
- Refusal to disclose voting intention appears to have been higher at this election, at around 5% compared with the usual 3%, and therefore more significant. More importantly, these refusals seem to have come disproportionately from intending Conservative voters. This factor accounted for up to 2% of the difference
- Poll tax de-registration of voters and overseas voters has been identified as accounting for more than 1% of the difference
- There is some evidence suggesting that the electorate in 1992 was particularly volatile, with more people than usual unsure of their voting intention at the beginning of the campaign and many making very late decisions on whether and how to vote
- Apart from these effects peculiar to the 1992 election polls, the Inquiry Team identified a residual error which may well have existed in polls since 1959. This led to an overestimate of the Labour vote by 1.5% to 2% and a similar underestimate of the Conservative share

Polling in Britain: A historic route to the forecasting future. Research Plus issue 358 supp (into the future) 1996 March:

In 1992, the five major pollsters averaged a 1.3% Labour lead, whereas the final result was 7.6% lead for the Conservatives due to a very late Tory swing. The polls backed the wrong winner by a wider margin than can be accounted for by sampling error and were widely criticised for not being able to perform to expectation
The Market Research Society Inquiry Report (MRS, 1994) isolated three root causes of the discrepancy: late swing, as in 1970, was one, but there were also inadequacies in the implementation of the quota sampling methodology and there was also some differential refusal – Conservative supporters were less likely to reveal their loyalties than Labour supporters, a ‘spiral of silence’ (Noelle-Neumann, 1984)
There was no single major technical fault with polling methodology which made it more difficult to address the problems for future Election polling

After the poll is over ….. Peter Hutton. The Market Research Society Newsletter Issue 314 1992 May:

To some extent, the polls were a hostage to their own past success. In the 13 General Elections since the war, up to1987, the average error per party of the final polls was only 1.4%. Over 60% of these polls estimated the Labour and Conservative share of the vote to within ±2%. Therefore, journalists, other political commentators and lay observers have come to expect that they will always serve as a good ‘predictor’ of the final outcome
The eve of poll polls showed the Tories at an average 38% and Labour at 40%, but in order to meet newspaper publication timelines, the voters were asked their voting intention some 36-48 hours before they actually voted, ie they had a day or 2 to change their mind after being polled
Furthermore, exit polls could not be conducted until polling ended at 10pm on Election Day, but finished by 4pm in order to allow time to enable data analysis. This clearly made the assumption that those voting after 4pm would show the same pattern as those voting before 4pm. But in 1992 this did not happen; in 1992 the Tories came out in their droves on the evening of April 9, possibly spurred on by the thought of a Labour victory
Therefore, just maybe, the polls were not wrong after all and did accurately reflect public opinion and voting intention at the time they were taken
The NOP poll was conducted in 100 marginals and did detect a swing to the Conservatives as the evening progressed, but polling in the marginals has not provided the best basis for predicting overall seat changes in previous elections
Key factors were a late change in voting intention which accounted for 3% of the difference, refusal to disclose voting intention (around 5% compared with the usual 3%, and therefore more significant)
Furthermore, in 1992, the electorate were far less set in their voting patterns than elections since 1979 and MORI found 8% of voters said they made up their mind in the last 24 hours, while a Gallup poll conducted just after polling day found 14% of voters decided who they would vote for ‘during the last few days’
In 1992, the turnout was relatively high at 77%
The lesson learned from 1992 was that in measuring public opinion, we are measuring something which is inherently subject to change, particularly during a close-fought campaign

The Opinion Polls and the 1992 General Election (Full Report) (1994) The Opinion Polls and the 1992 General Election (Full Report) – Opinion Poll Reports – The AMSR Online Archive (oclc.org)

The gap between the polls’ findings and the final result in 1992 was greater than their consumers are entitled to expect; indeed, it was greater than had ever been delivered since polls began
All the polls indicated that the election appeared to be so close – seemingly the closest for 20 years – meaning that the overall outcome was in doubt which added to the shock; the fact that the poll error misled most observers as to which party was likely to form the next government considerably magnified its impact
There were 3 main factors that seem to have contributed more or less equally to the error in the polls’ forecasts in April 1992. The nature of the evidence makes it impossible to ascribe specific percentages to each factor:
- There was a late swing: some voters changed their minds after the end of interviewing; furthermore, Conservative supporters proved more likely to vote on the day. The Conservatives gained from both these tendencies
- Some inadequacies were revealed in the operation of the quota system to select representative samples.This arose partly because quotas and weights did not reflect sufficiently accurately the social profile of the electorate, and partly because the variables used as the basis of quotas and in corrective weighting were not closely enough correlated with voting behaviour to ensure that the samples were fully representative of the distribution of political support amongst the electorate
- Conservative supporters were less likely to reveal their loyalties than Labour supporters (as has also been the case in earlier elections). This certainly operated through ‘item refusal’, reluctance by those interviewed to answer the voting intention question, both by outright refusal and by disingenuously answering ‘don’t know’. A similar effect was probably also caused by a differential refusal to be interviewed, although there is no solid evidence on this point. It is not possible to assign more than a tentative weight to the effect of each of these factors, especially since they interacted with each other. In particular, the quota method is in part designed to overcome the differential refusal of some groups to take part, and it was because of the weaknesses in its implementation that differential refusal could have so much impact. Late swing, the most easily isolated of the factors, probably contributed between a fifth and a third of the total discrepancy

In addition to these major causes of error, a number of other factors had a minor impact:

The selection of constituencies as sampling points by the companies had weaknesses; all but one of the companies used a selection of points that was slightly biased to Labour. This might have had a slight effect on the result of the final polls, but most of its potential risk was probably counteracted by the operation of the quotas and weighting
There was probably a slight bias to Labour created by interviewing respondents who were not in fact on the electoral register and hence could not vote. However, it seems likely that the impact of this was small, especially as the polls based their social profiles source that drew upon the electoral register rather than on the whole adult population, and some of the polling companies attempted to exclude those who said they were not registered didn’t know whether they were or not. Particular concern has been expressed about the impact of deregistration specifically to evade the ‘poll tax’, but its effect on the opinion polls appears to have been small

A number of other factors which have been suggested as partial explanations for the error of the polls did not, in our view, have any measurable effect:

We do not believe that deliberate lying to pollsters occurred to any significant extent.
The behaviour of postal voters who may not have been interviewed by pollsters probably had little effect. Less than 2% of voters voted by post and, in contrast to earlier elections, their support seems to have been fairly evenly divided between the 2 main parties
The result was not affected by the votes of Overseas Voters
Interviewing procedures played no apparent role. There were no systematic differences between those polls which interviewed in street and those which were conducted in home, those which interviewed for just one day and those for more than one day, or between polls which interviewed at weekends and those which did not
Sample size was not a contributory factor – bigger samples would not have produced significantly better or worse results

2015 UK election: Underestimated Conservative vote

Researching the public: post-war politics, policy and polling. Phyllis Vangelder and Judith Staig (eds) (2023):

After the failure of the polls in the 1992 Election, pollsters made many changes as a result, and again did fairly well at several following elections until disaster struck again in 2015
There was another enquiry, this time commissioned by the British Polling Council (BPC) in conjunction with the MRS, and conducted by an academic panel led by Professor Patrick Sturgis. This enquiry again found that there were several factors contributing to the error, and made a number of recommendations for future polling. Ironically, by implementing some of those recommendations, the polls in 2017 were less accurate than they would have been if they had carried on in the same way as in 2015.
This is not to say that the recommendations were wrong: they were right for the circumstances that prevailed in 2015 but such is the volatile nature of voting behaviour nowadays that some of the problems that beset the polls in 2015 did not recur in 2017, and some corrections made things worse rather than better

Applying artificial intelligence tools in research. Peter Mouncey, 2018.

The issue of unconscious bias also surfaced in the MRS ‘Methodology in Context’ conference, held in London on 22 November 2018, which featured a panel session on this topic based on the latest MRS Delphi report, ‘Deconstructing Bias’, covering 3 main areas where such bias can appear in research: Cultural, Managerial and Data.
The report recommended a hypothesis approach to building algorithms, taking into account relevant legislation, such as GDPR and equal opportunities
The report concludes with a case study summarising the salutary lessons learned from the inquiry into the failure of the polls to predict the outcome of the 2015 UK general election, citing unrepresentative samples and confirmation bias at the heart of the issues. In the 2015 general election, the final estimates of polling companies had the Conservatives and Labour neck-and-neck on 34%. The Conservatives won 38% of the vote, Labour 31%

IJMR Editor blog 7 2105. Researching voting intentions: the BPC (British Polling Council)/MRS enquiry. Peter Mouncey 2015.

The British Polling Council (BPC) and MRS launched an enquiry into the performance of the opinion polls in the UK preceding the May 2015 General Election, particularly focussing on methodology, with a need for transparency from pollsters and access to unweighted data
Unlike the 1992 enquiry set up by MRS, the 2015 enquiry was totally independent from the polling sector, comprising mainly academics, whereas in 1992 leading pollsters predominated. It was hoped the 2105 report would be available promptly ie by March 2016
Representatives of each of the main polling companies (ICM, Opinium, ComRes, Survation, Ipsos-Mori, YouGov, Populus) agreed that being within ‘sampling error’ was not a good enough excuse in predicting the outcome
There was little evidence to support a late swing of any significance in 2015, based on the results of post-Election polls, but the results showed the situation of understanding the Conservative vote and overstating support for Labour, seen in 1992 and 1970, was still at the heart of the problem
Martin Boon (ICM) pointed out that there had been no Tory lead in the polls since 2012
Neither the panel nor the polling companies are advanced in their investigations and, whilst all the polls misrepresented the mood of the electorate, that does not mean there was a common view on the reasons
In today’s world the 2 main options for data collection by the pollsters are telephone or online. With the mobile revolution, landline based methods no longer guarantee adequate coverage, so mobile top-up quotas are used, but the proportions varied in 2015 polling
Issues identified by pollsters were: sample bias (telephone and online methods), identifying the voting public and estimating likely turnout, weighting methods/modelling, differential turnout, tactical voting, factors possibly unique to the 2015 Election, such as a favoured Prime Minister, overstating Labour and understating Conservative support (a seemingly systematic problem)
Consideration was given to: seat projection modelling for the future, a review of the constituency level polls commissioned by Lord Ashcroft in Scotland which appeared to have produced a higher degree of accuracy, a question as to whether ‘modelling’ rather than sampling error lay at the heart of the problem, consideration of the importance and impact of the image of individual party leaders, and should the voting intentions question replicate the voting slip content for each constituency in which opinion polls take place?
Interpretation and media coverage must be considered and current evidence points to 3 key areas: sampling, question design, weighting/adjustment/modelling methods
The enquiry comprised a review of the overall research design used in opinion polling, especially the challenge of representing the voting public, rather than the general public
There was concern that solving the 2015 problem may not provide a generic, long term solution
A big challenge was predicting turnout (ie ICM predicted a turnout of 86% in 2015 but this was well above the actual level of 66%), with apparently distinct differences in the turnout behaviour between Conservative and Labour voters, which needed to be investigated
We now live in a much more complex world. In 1992, telephone polling using fixed line phones was a relatively new methodology and face-to-face polling was still used by pollsters, but now we employ the use of internet, mobile phones and social media¹
There was a need for ‘fit for purpose’ data collection methods and current methods needed to be scrutinised accordingly

2016 EU Referendum expectations

Trusting Opinion Polls After Brexit ‘Failures’. Matt Howley, 2016.

The surprising outcome of the Brexit Referendum led to opinion pollsters being widely criticised with many headlines asking how the polls could get it ‘so wrong’
There was a lot of uncertainty but, according to the bookies, most punters bet on ‘remain’, making it the favourite. This was an interesting test of the wisdom of betting crowds, who collectively have access to vast amounts of valuable information in addition to polls
Financial markets were hit by a massive shock despite traders and investors theoretically having access to teams of analysts with access to vast sources of information
Pollsters, on the other hand, have access to only one source: how their respondents say they plan to vote at any one moment
Opinion pollsters were criticised because, in the month leading up to Brexit, around half of the polls had pointed to ‘remain’, but what most critics failed to mention is that the polls also told us 3 other important things about Brexit: that support for ‘remain’ had weakened over the past 6 months, that many voters were still undecided and that this was shaping up to be a very close race. Meanwhile, no major pollster was actually labelling its poll a forecast. Even polls that successfully pointed to ‘leave’ came with paragraphs of caveats
Variation between polls about Brexit highlights that survey design plays as big a role as survey methods. The Financial Times poll tracker showed that TNS pointed to a ‘leave’ outcome a week out before the referendum while other online surveys pointed to the opposite
In 2016, attention was on the performance and accuracy of the newer methods of polling: mobile, online and telephone surveys. The Huffington Post reported that online polls consistently put the ‘leave’ vote ahead by 1.2 percentage points, giving them the edge. The mobile survey platform, Pollfish, now proudly displays the claim: ‘Traditional polls got Brexit wrong. Pollfish and other smartphone platforms got it right’.
Rising above varied individual polls and headlines, an Economist ‘Poll of Polls’ wrapped up the overall polling view as a tie, with 44% on both sides and 9% undecided.
In the real referendum outcome, we know that ‘leave’ emerged with a 4% lead so this ‘Poll of Polls’ wasn’t a successful prediction. But, an interpretation of this poll data as ‘this could go either way’ was potentially more helpful than most other forecasts and the interpretation in the media was varied. In Australia, ABC News had called Brexit ‘too hard to predict’ as did Bloomberg. This ‘either way’ view provided by polling was missing from many other forecasts and hearing it could have led to some different actions by investors, politicians and punters.
Brexit was a knife’s edge scenario with few precedents and perhaps the EU commission were working on a Plan B in the event that Britain voted to leave because they recognised this fact.
To say that opinion polling failed is to hold polls responsible for reading the minds of the undecided and suggests that that no one changes their mind as time passes. Critiques have also lumped varieties of polls together regardless of design. We can be wiser than this. To walk away from polls is to discard an entire frame of reference. We can trust that only polls can show us the closeness of the race and in turn this knowledge points us quite clearly to the probability of being right. By using the feedback loop, advances in methodology and by taking a consensus of quality data points, we can do even better

Referendums: how do voters decide? Sir John Curtice, 2023.

The economy was a central issue in the EU referendum in June 2016. Those on the Remain side of the debate emphasised what they regarded as the economic disadvantages of leaving the EU
No less than 93% of those who told the British Election Study shortly before the referendum that they thought the economy would be worse as a result of leaving the EU indicated shortly afterwards that they had voted Remain. Conversely, as many as 90% of those who took the opposite view of the economic consequences backed Leave
Meanwhile, those campaigning for Leave often argued that exiting the freedom of movement provisions of the European single market would result in lower immigration. This appears to have been a somewhat less persuasive argument, but even so, 70% of those who thought immigration would be lower if the UK left the EU backed Leave, while 83% of those who thought it would not make much difference voted Remain
The British Election Study shows that parties are not always successful in persuading their supporters to back their position. The attempt by the then Conservative Prime Minister, David Cameron, to persuade his party’s supporters to vote to Remain in the EU bore little fruit – a majority (54%) voted to The problem with asking voters to decide for themselves is that they do not always give the answer for which the politicians were hoping

Research: Matters December 2018.

Studies like the British Social Attitudes survey have shown that Euroscepticism has been the mood of the majority for over 2 decades. Until 1996, no more than 40% wanted either to see the UK leave the European Union or for its powers to be reduced: but from that point onwards (with a few exceptions) the balance tipped in favour of a Eurosceptic majority (Curtice and Evans, 2015), As we know, the consequences became apparent in June 2016, when 52% voted in support of the UK leaving the EU

Explaining Cameron’s Catastrophe: The Definitive Guide to the 2016 European Referendum (Archive Book). Tim Bale, 2017. Background to the 2016 EU Referendum (information obtained online as whole book not available on AMSR, just cover page and contents)

The definitive guide to the 2016 European Referendum: how we got here; Cameron’s mistakes; the context, content and process; what Remain did wrong; why the Leave campaign was so successful; voters’ attitudes; and the aftermath. Robert Worcester, Roger Mortimore, Paul Raines, Mark Gill

Tim Bale looks back at the 2016 Brexit vote and considers why David Cameron decided to call the referendum on the UK’s membership of the European Union – and why he went on to lose it.

Perhaps the biggest critical juncture in Britain’s recent political, economic, and diplomatic history was David Cameron’s promise, made public in January 2013, to hold an in/out referendum on the UK’s membership of the European Union if the Conservatives were returned at the next general election.
The Conservative Party’s preoccupation with Europe had done much over the years to ensure that, in spite of the successes achieved over the years by British diplomacy in Brussels, the country’s voters, particularly if they were readers of Tory-supporting newspapers, rarely, if ever, heard anything positive about the EU – something that was always going to make it difficult to sell the advantages of membership if and when they were offered the chance to register a verdict on it.
A politician as crafty as Cameron – even one criticised by his admirers as more of a game-playing tactician than a serious-minded strategist – must have known that calling a Referendum on Europe was a distinct risk. Why, then, did he choose to poke the hornet’s nest? And why, having done so, did Leave win and Remain lose?
Cameron chose to commit to a vote, not because the country’s population was clamouring for one but because a significant minority of his own MPs, many of them frustrated by the constraints of coalition, were demanding that he do so – some because they feared that UKIP would cost them their seat (or the seats of too many of their colleagues) at the next election, some because they wanted out of the European Union and were more than happy to leverage that fear to their advantage.
Naturally, Cameron was aware he was running a risk – after all, his best friend in politics, Chancellor George Osborne, told him so again and again. But he thought he would win the vote, not least because he believed – mistakenly it turned out – that he could (a) negotiate a decent deal with the EU and, just as importantly, (b) rely on the personal loyalty of influential colleagues (not least Michael Gove and Boris Johnson) to trump their Euroscepticism (and, some would say, their opportunism).
David Cameron lost the referendum for a whole host of reasons. For one thing, the deal he negotiated was too complex and too much of a contrast with what he had earlier promised to extract from Brussels to persuade most Eurosceptics to back it. For another, he underestimated how much of a role immigration would play in the debate and how willing Conservatives in the Leave campaign would be to tap into the concerns of ‘identity conservatives’ on the issue.
Cameron also underestimated the extent to which the Leave campaign would prove able to mobilise its supporters to turn out and vote, even people who didn’t normally bother. Conversely, he overestimated the willingness of the Eurosceptic Leader of the Opposition, Jeremy Corbyn, to help the Remain campaign win over traditional Labour voters. He also overestimated the extent to which voters would be swayed by the economic arguments against withdrawal, either because the economy wasn’t delivering for them anyway or because they believed they were sufficiently comfortably-off to weather the potential downsides of leaving the EU.
Hoping and believing that Remain would win, Cameron was so anxious about putting the Conservative Party back together again after the campaign that he pulled his punches when it came to criticising Tories on the Leave side, refusing again and again to fight fire with fire. He might also have avoided such a big split in the first place had he not decided to suspend collective responsibility – a decision which made it much easier for influential and often highly ambitious (as well as highly Eurosceptic) colleagues to plump for Leave.
Ultimately, though, Cameron lost the referendum because he failed spectacularly to persuade his own party, both at Westminster and at the grassroots, to listen to and then to back its own leader. ‘Loyalty’, one of post-war Conservatism’s big beasts, David Maxwell-Fyfe, is said to have claimed, was ‘the Tory Party’s secret weapon’. Whether or not that was ever really the case, that weapon has now – definitively – been put firmly beyond use.

David Cameron called the 2016 European referendum primarily to appease Eurosceptic Conservative MPs and neutralize the electoral threat of the UK Independence Party (UKIP). Instead of permanently resolving the issue, the vote to leave the EU triggered a massive political crisis, resulting in Cameron’s immediate resignation and years of parliamentary deadlock.

The Catalyst: Why the Referendum happened

The Rise of UKIP: Under Nigel Farage, UKIP surged in popularity, siphoning votes from the Conservative Party in traditional heartlands and pressuring Cameron to take a tougher stance on Europe
Tory Party Management: Cameron pledged to hold an in-out referendum in his 2013 Bloomberg speech. It was a strategic gamble intended to silence infighting within the Conservative Party, assuming a coalition would block it
Unexpected Majority: The 2015 General Election delivered an outright Conservative majority, forcing Cameron to honour his campaign pledge and stage the referendum

The flawed strategy: Cameron’s gamble

Failed Renegotiations: Ahead of the vote, Cameron toured Europe attempting to secure a new, looser settlement for the UK within the EU. Critics and Eurosceptics viewed the concessions he won as superficial

The flawed strategy: Cameron’s gamble

Failed renegotiations: Ahead of the vote, Cameron toured Europe attempting to secure a new, looser settlement for the UK within the EU. Critics and Eurosceptics viewed the concessions he won as superficial
Hubris over unity: Cameron was highly confident of a “Remain” victory. As a result, his campaign strategy largely underestimated public frustration with immigration and relied on technical economic warnings (so-called “Project Fear”) rather than a positive, emotional vision for the EU

The campaign: How “Leave” won

The slogan: The Vote Leave campaign utilized the powerful, memorable, and easily understood slogan “Take Back Control,” which appealed to a broad sense of lost sovereignty
The immigration factor: While “Remain” focused heavily on economics, the “Leave” side focused on immigration and the NHS, tapping into deep-seated anti-establishment sentiment among working-class voters and communities feeling left behind by globalization
Conservative disunity: The Conservative Party was deeply divided, with prominent figures like Boris Johnson and Michael Gove spearheading the Leave campaign against their own Prime Minister

The aftermath

The result: On June 23, 2016, the UK voted to leave the European Union by a margin of 51.9% to 48.1%. Turnout was 72.2%
Cameron’s exit: Cameron resigned the morning after the result, declaring it would be “wrong to try to be the captain that steers the country to its next destination” after losing the campaign
Long-term fallout: The referendum triggered years of political turmoil, including negotiations over the UK-EU withdrawal agreement, the invocation of Article 50, and deeply polarizing debates that transformed the British political landscape

2017 and 2019 polling debates

Researching voters: how parties use polls to develop strategy. Peter Kellner, 2023.

As research costs have fallen, large samples have become more affordable. Combined with computing power that used not to be available, parties have been able to explore the views of specific groups of voters in granular detail. This has helped them to be more sophisticated in selecting which seats to target in general elections – and how best to reach the swing voters in those seats. A significant example of this came in the 2017 election when Andrew Cooper, the founder of Populus, persuaded the Conservatives that it would be worth spending money in a dozen or so specific constituencies in Scotland
The Tories had won only one Scottish seat in 2015 – and either none or one in each of the 4 elections before that. Scotland seemed a lost cause. Yet Cooper’s research, dissected by Populus’s data analyst, James Kanagasooriam, demonstrated that gains could be achieved in seats with certain specific characteristics The result of this advice being accepted and acted upon was that the Tories gained 12 seats north of the border, while losing 25 in the rest of Britain. These Scottish gains enabled Theresa May to cling to office. Without them, the Tories would have been out of power. (Cooper, who had worked closely with David Cameron, was frozen out of the Conservative campaign in England and Wales. One wonders what the party might have done had Cooper and Kanagasooriam overseen Tory research throughout Britain.)Today, private polling occupies its place on a continuum of research into policy, campaigning and communication.
In recent years, although precise amounts are not available, it is clear that the Conservatives have employed research far more extensively than the Labour Party. This has largely flowed from the differences in financial circumstances, though an extra factor has been the recent Conservative practice of buying in outsiders to manage election campaigns. Following the 2019 election, Labour’s new leader, Sir Keir Starmer, recruited Deborah Mattinson as his full-time head of strategy. Unlike a number of the people central to Conservative election campaigns, who generally remained with their companies and worked under contract, she severed her links with her own research company in order to work for Starmer. Yet the fact that a polling expert was appointed head of strategy, rather than research, showed how integral polling information, both quantitative and qualitative, has become to the party’s strategic direction as well as campaigning.

Post-war developments in market research. Vangelder Phyllis (ed) (2021)

In December 2019 UK polls averaged only an average 0.6% error per party in the General Election. Many times polls are declared defunct, but every election there are more of them! As with democracy, polls may be the worst system, until you look at all the others. The one lie politicians all over the world routinely tell is that the only poll that matters is the one on election day – while commissioning more polling than ever.

2026: Why the Polls are all over the Place: A look at the current situation (mid 2026) Peter Kellner, mid-2026.

And finally here’s an interesting very modern look at the polls between elections.