Measures to prevent data falsification
A working paper that assesses ways to address possible data falsification in the European Social Survey (ESS) has been published on our website.
Several measures initiated by the Core Scientific Team (CST) are already in place to ensure the best possible interviewing is undertaken in all participating countries.
These include extensive pretesting of survey questions, guidelines for briefing interviewers, for selecting and recruiting respondents, and for monitoring fieldwork, analyses of the process of obtaining interviews and of the answering process during interviews.
The working paper explores how data falsification can occur and evaluates efforts to combat false data being collected during ESS fieldwork periods.
Data falsification is described by the American Association for Public Opinion Research (AAPOR) as the ‘intentional departure from the designed interviewer guidelines or instructions, which could result in the contamination of the data’.
The ESS Round 9 specifications prescribe that 10% of respondents and 5% of ineligible respondents should be contacted in back-checks conducted by the survey agency in each country.
The working paper confirms that the CST will strengthen the back-checks procedure ahead of Round 10 (2020/21) of the survey, to ensure that the process is more consistent across different countries.
Using only computer-assisted personal interviewing (CAPI) during Round 9 fieldwork allows the ESS to collect time stamps during the interview. This means that the speed at which the questionnaire is answered can give an indication of undesirable interviewer behaviour (speeding, skipping introductions) or data fabrication.
After fieldwork, analyses are also conducted to identify data quality issues - such as straightlining or odd patterns.
An analysis of partial duplicates is also conducted by the ESS Data Archive after fieldwork and, based on the evidence, all cases from interviewers who produced suspicious cases may be removed from the dataset (this has happened in three countries in Round 8).
Looking to the future, the paper suggests that audio recording, GPS tracking of interviewers and continued transparency from the ESS could all help prevent data falsification.
A new work package included in the ESS Work Programme 2019-21 aims to promote desirable interviewer behaviour, to minimise undesirable interviewer behaviour, to prevent falsification as much as possible and to detect it in an early stage.
The paper - Data falsification in the European Social Survey? - was written by ESS Deputy Director - Methodological, Dr. Ineke Stoop (SCP) and CST colleagues, Roberto Briceño-Rosas (GESIS), Achim Koch (GESIS) and Dr. Caroline Vandenplas (KU Leuven).
Other news articles
- 07/07/21 Round 9 Multilevel Data published
- 25/06/21 New report published: The timing of life events
- 10/06/21 Agreement with Japanese General Social Survey
- 03/06/21 Web panel question modules selected
- 16/03/21 Findings from the Human Values Scale
- 08/03/21 Fresh opportunity to field web panel modules
- 03/03/21 Upcoming virtual General Assembly meeting
- 26/02/21 Important update on ESS Round 10 fieldwork
- 11/02/21 5,429 academic publications based on ESS data
- 28/01/21 Upcoming call for online panel questions
- 26/01/21 Round 10 core questionnaire changes
- 11/12/20 New version of Round 9 data now available
- 11/11/20 Questions selected for new online panel
- 30/10/20 Round 10 COVID-19 questions finalised
- 16/10/20 Director becomes Academy of Social Sciences Fellow
- 06/10/20 Justice and fairness analysis now available
- 24/09/20 Virtual General Assembly meeting
- 18/09/20 Justice and fairness: Topline report launch
- 14/09/20 Round 11 rotating modules selected
- 02/09/20 Sir Roger Jowell Memorial Lecture 2020
- 17/08/20 Call for Southern European Ambassador
- 04/08/20 Call for online panel questions now open
- 22/07/20 Coming soon: Call for online panel questions
- 06/07/20 COVID-19 questions selected for Round 10
- 15/06/20 Round 9: Second data release now available