Measures to prevent data falsification
A working paper that assesses ways to address possible data falsification in the European Social Survey (ESS) has been published on our website.
Several measures initiated by the Core Scientific Team (CST) are already in place to ensure the best possible interviewing is undertaken in all participating countries.
These include extensive pretesting of survey questions, guidelines for briefing interviewers, for selecting and recruiting respondents, and for monitoring fieldwork, analyses of the process of obtaining interviews and of the answering process during interviews.
The working paper explores how data falsification can occur and evaluates efforts to combat false data being collected during ESS fieldwork periods.
Data falsification is described by the American Association for Public Opinion Research (AAPOR) as the ‘intentional departure from the designed interviewer guidelines or instructions, which could result in the contamination of the data’.
The ESS Round 9 specifications prescribe that 10% of respondents and 5% of ineligible respondents should be contacted in back-checks conducted by the survey agency in each country.
The working paper confirms that the CST will strengthen the back-checks procedure ahead of Round 10 (2020/21) of the survey, to ensure that the process is more consistent across different countries.
Using only computer-assisted personal interviewing (CAPI) during Round 9 fieldwork allows the ESS to collect time stamps during the interview. This means that the speed at which the questionnaire is answered can give an indication of undesirable interviewer behaviour (speeding, skipping introductions) or data fabrication.
After fieldwork, analyses are also conducted to identify data quality issues - such as straightlining or odd patterns.
An analysis of partial duplicates is also conducted by the ESS Data Archive after fieldwork and, based on the evidence, all cases from interviewers who produced suspicious cases may be removed from the dataset (this has happened in three countries in Round 8).
Looking to the future, the paper suggests that audio recording, GPS tracking of interviewers and continued transparency from the ESS could all help prevent data falsification.
A new work package included in the ESS Work Programme 2019-21 aims to promote desirable interviewer behaviour, to minimise undesirable interviewer behaviour, to prevent falsification as much as possible and to detect it in an early stage.
The paper - Data falsification in the European Social Survey? - was written by ESS Deputy Director - Methodological, Dr. Ineke Stoop (SCP) and CST colleagues, Roberto Briceño-Rosas (GESIS), Achim Koch (GESIS) and Dr. Caroline Vandenplas (KU Leuven).
Other news articles
- 31/10/19 Round 9 data now available
- 14/10/19 Exploring public attitudes, informing public policy
- 25/09/19 Your chance to field questions in our survey
- 19/09/19 Social science event at the European Commission
- 04/09/19 Global partnerships revealed
- 02/09/19 General Assembly meeting next month
- 28/08/19 Online consultation launched
- 25/07/19 Round 8 quality matrix now available
- 24/06/19 Pre-announcement: Call for rotating modules
- 20/05/19 Contextual information for Round 8 now available
- 10/05/19 Sir Roger Jowell Memorial Lecture 2019
- 12/04/19 Measures to prevent data falsification
- 11/03/19 New Horizon 2020 project begins
- 19/02/19 Director visits University of Western Australia
- 23/01/19 Jowell-Kaase Prize winner revealed
- 15/01/19 Upcoming General Assembly meeting
- 17/12/18 Cumulative Data File updated
- 29/11/18 Russian attitudes on climate change and energy explored
- 27/11/18 New report examines Russian attitudes on welfare
- 20/11/18 Registration opens for ESS Conference 2019
- 24/10/18 ESS ERIC General Assembly meeting
- 02/10/18 Europeans accept climate change is happening
- 17/09/18 Europeans receptive to new welfare policy ideas
- 02/08/18 Upcoming events in Brussels
- 11/07/18 Exploring democracy and digital communication