AMSR held its third prestigious Event at the IPA on 4 February, with a record attendance of 75 people. Once again, we were able to highlight how much has been achieved since it was founded. In addition to the presentations by distinguished speakers, Sir John Curtice and our President, Professor Denise Lievesley, attendees were able to see online demonstrations of the Archive and search the ever-growing material now available.
Paddy Barwise, newly appointed Chairman of the Board of Trustees, welcomed the many supporters and friends in the audience, emphasising that without their help the Archive could not have become what has been demonstrated that evening.
Photo: Paddy Barwise
Paddy thanked Liz Nelson, the outgoing Chairman, for the amazing work she has done in helping to establish the Archive.
Paddy Barwise went on to introduce, Professor Denise Lievesley. She became AMSR’s President last year and this was the first time she had the chance to meet AMSR’s many friends and supporters. She is the Principal of Green Templeton College, University of Oxford. She has had a very distinguished career, inter alia, as Executive Dean of the Faculty of Science and Public Policy at King’s College London, Chief Executive of the English Health and Social Care Information Centre, Director of Statistics at UNESCO, where she founded the Institute for Statistics, and Director of the UK Data Archive at the University of Essex.
“We are honoured to have her with us tonight”.
Denise Lievesley acted as host to the proceedings. Before introducing the speakers, she expressed her excitement about the Archive. As Head of the UK Data Archive for Economic and Social Research based at the University of Essex, she had tried to get market research data into that Archive, with limited success. It was more successful in getting data from government departments and academic institutions, than from private organisations. It was sad and worrying that there was not access to the really rich and original sources of data relating to market research.
She referred to an Annual Lecture series at her College, which this year is on the topic of ‘Commons’. This involves sharing resources, working together and using voluntary expertise. She concluded. “AMSR is a classic example of ‘Commons’ and I would encourage you to give any support you can, whether financial, time, helping to promote the Archive or encouraging people to deposit their data”.
Denise Lievesley introduced Adam Phillips, CEO of AMSR, who presented a review of what AMSR has done this year and plans for the future.
He reminded those who came to the event last year that AMSR’s long- term aim is to become the ‘go-to place’ for people wanting to find out about what the Market and Social Research industry has learned in the UK and around the world. Where AMSR does not have the information, it will become a hub to guide enquirers to other places where they may be able to find what they want.
In particular, the Archive is aiming to build a collection of qualitative and quantitative ad hoc and panel survey results which reflect the changing habits and attitudes of the UK population over time.
The primary focus over the last four years has been on identifying and rescuing important paper material produced by the industry since the 1950s, that has not been collected elsewhere. It is regrettable that so much has been thrown away by the organisations that produced and commissioned it. Nevertheless, the Archive is still being offered interesting material and not all of it is from the UK. We have recently been talking to the Australian Market Research Society about maintaining their archive and also to ESOMAR about how we might be able to cooperate with them.
At present AMSR stores everything it has been given in paper form at the History of Advertising Trust. This is in Raveningham, near Lowestoft: not the easiest place to get to, but a lot cheaper than London!
Online archive home page
Most of what AMSR holds is available online. It now contains over 5,000 documents and over 70,000 pages in the online archive. The website has been improved, to make it more attractive and accessible. The website portal includes guidance about how to search the online Archive. In the near future, this written guidance will be augmented by an instructional video for those who prefer not to read.
AMSR has continued to produce a quarterly newsletter and, for the last year, we have been publishing regular ‘stories from the Archive’ at the rate of about two a month. These stories are being promoted through social media and AMSR now has over 150 followers on Twitter. Adam Phillips asked, “If any of you are Twitter users, we would very much like you to follow us, if you are not already doing so, and to re-tweet our tweets to your followers”. A show of hands revealed that a surprising number of the audience is on Twitter, so there is every hope that the number of followers will increase.
In the coming year AMSR will be increasing its publicity activity, since there is sufficient material in the Archive for it to be useful to a wide range of users. AMSR will now need to become more proactive and find a way to encourage more people to release material which they possess, particularly if they are unsure whether that they can give permission for it to be made public. This constraint applies to large amounts of qualitative research where the primary objective may have been to research a product which is no longer of interest, but the information collected about people’s attitudes and behaviour at that time has real historical value.
Contents Committee members will be approaching those who have not already been in contact, to see if they have anything that could be worth preserving. AMSR is prepared to take responsibility for checking on ownership of publication rights. It is offering to embargo information for 10 or more years, if the material will be of interest to historians and it is no longer clear who can give permission for publication. “If concerns about ownership and publication of interesting material that you hold have stopped any of you here tonight from offering your research, please talk to Phyllis Macfarlane, Chairman of the Contents Committee, and we shall try to find a way to preserve it”.
Since AMSR’s inception four years ago, it has focused almost entirely on collecting material. It is now in a position to start moving towards developing the Archive’s use for education and journalism.
In order better to understand the needs of users of AMSR, St Mary’s University in Twickenham generously carried out research among university librarians last spring. AMSR is now working with them on a much bigger piece of work aimed at understanding how it can provide material that will be useful for teaching students.
A list of history and sociology academics at leading universities who are working in subject areas where the Archive has relevant material has been assembled. AMSR has been talking and demonstrating the Archive to some of them. The objective is to get a few of them to use AMSR in their research. “We hope to be able start collecting evidence of the use of the Archive by their students towards the end of this year. Unfortunately, the academic world moves very slowly”.
Coding to help searching
Talking with academics has helped develop a coding framework using headings relevant to academic historians, sociologists and journalists. This coding will make it easier for people with no experience of market research to find relevant material more easily. He thanked Colin McDonald and Christine Eborall for developing this code list and for coding the first six collections in the last three months.
Many research agencies and companies have thrown away their libraries in the process of moving to paperless offices. One of the few libraries that still remains is the MORI library. This contains over 30,000 reports, with an emphasis on the public sector. We are looking for significant finance to preserve this library and will be approaching the National Heritage Lottery Fund for a grant later this year. We think this library is a National Asset.
It costs around £35,000 a year to run AMSR. The share of expenditure is roughly two thirds for storage of paper records, the online archive and website, a quarter for collecting material, publicity and fundraising and a sixth for administration. The financial cost of running AMSR bears no relationship to the real cost. If it were to pay commercial rates for the 50+ unpaid volunteers who do the curating, scanning, publicity, fundraising and online presentation of the material in the Archive, the running cost would be over £200,000. “At present the money provided by our generous supporters just matches our outgoings, but there is no surplus. We believe that to provide a sustainable future for the Archive, we need to be able to answer questions from users and the industry in addition to providing links to other archives in our website portal. We have to integrate with the community of archivists and to do this, we shall need to pay a part-time archivist to lead a team of volunteers. Allowing for the Archive to continue expanding, we expect that our running costs in two years’ time will rise to £55,000”.
AMSR has set up a fundraising group with the objective of increasing its income over the next two years to meet this level of expenditure. Currently the mix of income is 60% from individual donations, 36% from commercial organisations and 4% from grant-giving organisations.
AMSR believes that there is significant demand for the service that it will be able to provide. “I hope”, Adam concluded, “that you will continue to support us with financial donations and as volunteers”
Sir John Curtice, the distinguished political scientist and well-known media ‘guru’ of political polling, was the guest speaker. Sir John is currently Professor of Politics at the University of Strathclyde and Senior Research Fellow at the National Centre for Social Research.
Photo: Sir John Curtice
He gave a fascinating presentation on ‘Using Archives to learn from the past, using the EU as a case study’. His paper illustrated not only the value of AMSR, but of archives more broadly.
2016 was, of course, the second EU referendum: the first was during the Harold Wilson government in 1975. John Curtice has delved into our Archive and the UK Data Archive, using history to understand the present, to cast light on why the results were different.
He reminded the audience of what had happened in 2016 and the assumptions drawn; people voted according to age: younger people were more likely to vote remain; people’s behaviour varied by education: Remainers tended to have tertiary education; immigration was an important issue. The referendum showed the divisions between social liberals who were comfortable living in a diverse society and social conservatives, who represented a degree of conformity.
In 1973 the UK entered the EU without a Referendum. The Referendum was called by the Labour Government under Harold Wilson, which had been elected in 1974. The Labour Party’s stance was that it wanted to renegotiate the Tory Terms of Agreement. This was largely a financial issue: the UK was at risk of having to pay a higher contribution than its GDP might justify. Harold Wilson renegotiated the terms, came back with a Deal, called a referendum in 1975 and the result was a 2-1 victory for staying in the EU.
John Curtice summarised the questions we should ask about Referenda:
- What are the circumstances giving rise to the referendum?
- What is the question wording?
- What is the role of endorsement?
- What are the patterns of voting?
In 2016 David Cameron decided to call a Referendum. He was worried about UKIP and the divisions within his party on Europe. In the 1970s the Conservatives were the Remain party. The similarities were that in 2016 there were internal divisions in the Conservative Party; in 1975 there were internal divisions in the Labour Party. 2016 was a case of history repeating itself.
Wording of course is always a subject of dispute in a referendum. In 2016 the Electoral Commission tested the intelligibility and neutrality of the questions. In our Archives John Curtice found an NOP study conducted by the Daily Mail in January 1975 which tested seven different ways of asking the question. NOP’s definition of neutral was in fact no question, just boxes labelled ‘in’ or ‘out’.
In experimental work, you can see the impact of biased questions. A question asking people’s intentions after a statement of government endorsement. (‘How would you vote if the government says it should be ‘yes’?) indicated the potential volatility of public opinion. A Gallup Poll in August 1974, which asked how people would vote if the Government said ‘yes’ showed a dramatic shift.
In 2016, David Cameron’s opinion did not affect the result, compared with that of Harold Wilson.
Patterns of voting
Comparing the evidence of 1975 and 2016, in 1975 there were no significant differences in the age groups. Social class rather than education was of importance in 1975. Education was not so important in 1975, though there were fewer graduates in the ’70s. The social base differed: there was a much clearer link to social background, using standard social class breakdowns.
The agenda in the two referenda differed. A MORI survey in May 1975, found in the AMSR archive, asked what would be the principal disadvantages of staying in the Common Market. Prices, particularly food prices and the impact of the Common Agricultural Policy were the main concerns. This was a social economic argument: those with a working class background were more concerned about food prices.
Regarding immigration, which was the main issue of concern in 2016, such was the unimportance of this in 1975, that there was no question about immigration in the British Election Studies of that period. The issues of identity and culture in 2016 were difficult to shift. The yes/no vote broken down by perception of migrants are in the British Election Studies ’74 and ’79 which are available from the UK Data Archive, but they show nothing like the differences concerning immigration found in 2016.
John Curtice’s fascinating comparisons demonstrated that both referenda were motivated by party divisions. Both raised questions about society and the issues of concern at the time. He believes that the concern with immigration, which was the leading issue in the 2016 election, was much more difficult to change (it remained largely constant throughout the campaign), compared with concerns about the economic consequences of being a member of the EEC, which was the most important issue in 1975. He concluded that there are lessons to be learnt from history which archival material can provide help in understanding.
Denise Lievesley followed with an insightful account of the value of archives.
Sir John Curtice had illustrated the power of the wonderful narratives you can gain from the Archive and how important it is to look at changing methodologies. As a survey methodologist, Denise Lievesley stressed the use of historical information to look at changing methodology over time. What is of interest is what is not collected, as well as what is collected (as John Curtice demonstrated in relation to the issue of migration), indicating the priorities of society at the time.
Denise Lievesley discussed why a data archive is really important and the changing world of academia in this regard. As the first Director of the UK Data Archive, she was instrumental in getting the ESRC to develop its data policy: if you received money from the ESRC to collect primary data, you had to offer this data to the Data Archive. “The Data Archive did not have to take it. We did not want some of the datasets because, in our judgement, they were unlikely to be used in the future. And it takes resources to collect. Acquisitions policy is a really important issue”. The scientific principle is that research findings, together with the data, should be available for others to refute, to confirm, to clarify or to extend the results. That is part of public accountability. The Declaration of Professional Ethics of The International Statistics Institute states that the principle of all scientific work is that it should be open to scrutiny, to assessment and possible validation by fellow scientists. The principle of scientific openness extends to a responsibility to funders and to society to use data resources efficiently. Data should be used – so much is under-exploited. Denise Lievesley cited her former boss and hero, the late Sir Roger Jowell, who used to talk about “the amount of data that was untouched by human minds”. Data is underused, even by academics. She said that “we are all worried about response burden and collecting too much data”. Those were the reasons at that time that she was arguing for the importance of data archives. And this still applies. She stressed that we want to encourage responsible data collection: deliberate replication, but not duplication and ignorance of previous research. There is a growing awareness that our failure to exploit the full potential of data has meant costs to society.
The Times Higher Education has talked about the shift towards an open access movement. It has been policy to share publications for many years. Now we are addressing the tougher questions of sharing data.
Denise Lievesley was due to go to The Netherlands where she is a member of a panel convened to assess funding for a research bid for a major social science data archive that the Dutch want to set up. It will incorporate a lot of data from private organisations, as well as data from government organisations and academia. The Times Higher Education has said that what is needed is a radical change in mindset about the value of academic contributions to research in order to get an equally radical opening up of the scientific method. It argued that as we get better computational ability to crunch large datasets, significant insights can come out of having access to that data – perhaps information that was never even thought of by the people who collected it.
There are a lot of problems in sharing data in the academic sector. There are difficulties in the area of incentives. There are other issues: in addition to funding, there is huge competition. How do you encourage sharing in an environment when competition is extraordinarily fraught?
Denise Lievesley stressed the importance of data for teaching purposes especially in respect of methodology and data analyses. Too many of our young people spend a lot of time collecting really poor quality data and never get time to analyse it. Today’s ethos is they have to do their own primary data collection instead of getting them to analyse secondary data and their own data. Denise Lievesley recalled that while she was at the UK Data Archive, a number of datasets were deposited for teaching purposes. In one case a wonderful subset from the University of Surrey using the General Housing Survey was analysed to address a series of policy questions. “It was used more than the General Household Survey itself. In fact, it was the most used dataset in the Data Archives. This rich information must have influenced the statistical and social science skills of so many students”. She stressed the importance of working with academics to create good teaching datasets.
Reproducibility is an issue particularly in relation to medical research. Recently there have been a number of papers published in reputable journals, where people have gone back to attempt to reproduce research which has been done previously, and failed to do so or have only managed to do so in a small number of cases. There have been discussions as to why this should be so. Publication bias is one reason: only positive results tend to get published. This is an argument for having access to ‘grey’ material – unpublished material and raw datasets not just published material. Increasingly, in large datasets, the statistical paradigm falls to pieces. Statistical significance no longer holds up. In huge datasets everything is deemed significant. People pick and choose and often choose what falls in with their prejudices. We have a crisis in a lot of research, and it is particularly being discussed in relation to medical research. It will be a cause for discussion in the social sciences community. She concluded, “I hope the AMSR will be able to inform these areas of debate”.
Discussion from the floor
There was limited time for questions, but the discussion from the floor was an indication of the interest and involvement of the audience in the work of AMSR.
Peter Mouncey, who has recently retired as Editor of the International Journal of Market Research, commented that while some Journals discourage replication studies, the IJMR encourages this.
Philip Talmage asked if there was any possibility of disaggregated data being incorporated into the Archive. Phyllis Macfarlane, Chairman of the Contents Committee, responded that one would like to think so. Sir John Curtice remarked that he would encourage the Archive to go down this road: there is certainly a gap. Adam Phillips, CEO AMSR, pointed out that the problem is the scale of data we should have to hold. At the moment we are very selective about raw data. He suggested some datasets could be curated, or re-deposited when they have been analysed. Denise Lievesley agreed that this should be encouraged.
Professor Merlin Stone is concerned about getting students to use datasets. He noted that methodological articles tend to receive a high volume of citation. The challenge is to write articles on methodology and secondary research and get them published.
Denise Lievesley referred to the Royal Statistical Society’s excellent critique of the Teaching Excellence Framework. “It points out that it says nothing about teaching quality! However, one of the successes we have had in respect of the Research Excellence Framework, is that a well-documented dataset which is used by other people, counts in the same way as a publication”.
The drinks party after the formal addresses was an opportunity for old friends to gather and gossip. There were further demonstrations of the Archive, and people enjoyed seeing former colleagues and welcomed the opportunity to catch up with the achievements of AMSR. There were also several promises of attic and garage raids to rescue valuable reports and papers.
The Archive welcomes volunteers who wish to be involved in its myriad activities and is particularly seeking financial donations so that future plans can be secured. Please contact firstname.lastname@example.org and, of course, visit the web site at www.amsr.org.uk.