Objectives and expected impact

The European Social Survey currently insists on face-to-face interviewing as its sole mode of data collection.  However, owing to the mounting costs of carrying out face-to-face interviews and the divergent traditions and experiences of survey research across the different countries participating in the survey, there is a growing need to explore alternatives.  The central co-ordinating team of the ESS has, in collaboration with Gallup Europe, begun investigating the feasibility of mixing modes of data collection on future rounds of the survey.  In the near future, the most likely mixed-mode scenario for the ESS, if any, would be for countries meeting appropriately strict quality criteria to switch to telephone interviewing for the survey, either in conjunction with, or instead of face-to-face interviews.  However, longer-term, alternative designs for multiple-mode data collection must be considered, in order to meet the challenges of rising survey costs and falling response rates, as well as to take advantage of the opportunities presented by new technologies.

Even relatively simple mixed mode designs - such as a switch to CATI in a small number of countries - could potentially threaten data quality, disrupting the continuity of the time-series for the countries concerned, as well as affecting the validity of cross-cultural comparisons.  The research we are conducting on the ESS is aimed at exploring the extent and nature of that threat, and different methods of mitigating its impact.  The research we propose here in JRA1 is designed to build on and expand this important methodological work.  Apart from its experimental fieldwork elements, the project is also highly research intensive.  US scholars - eg Don Dillman, Robert Groves, Jon Krosnick and Norman Bradburn - have been grappling with the problem of mixed mode surveys for years and - in a recent meeting in London with the ESS team - expressed admiration for and confidence in the ESS's approach to clarifying and perhaps resolving some of the most intractable difficulties caused by the different measurement properties of face-to face, telephone and self-completion methods of data collection in surveys. We will continue to consult these scholars on an informal basis during this project.

The City research team in JRA1 will continue to work with some of these US scholars, together with their European counterparts if they are available - such as Edith de Leuw, Joop Hox, Willem Saris, Jaak Billiet, Peter Lynn, Robert Manchin, Lars Lyberg, and Eurostat specialists in this field.  They will follow an iterative experimental approach to the problem, aiming for robust strategies for the future.  It is widely acknowledged that this work has the potential to affect survey practice worldwide.

The work proposed in JRA1 presents an ideal opportunity to address the difficulties concerned with mixing data collection modes head-on in an attempt to get further than just measuring mode effects.  The proposed research has three related aims: 1) to improve our understanding of survey practice across Europe, and the most appropriate ways of tailoring data collection strategies by country, as well as for the survey as a whole; 2) to design data collection instruments for use in modes other than face-to-face interviewing specifically so that the likelihood of mode effects is mitigated; and 3) given that we cannot hope to eliminate all types of mode effect, to investigate different methods to correct for remaining mode effects, whether they arise from measurement, coverage or selection errors.  We also wish to look specifically at the potential role of web-based surveys, focusing on how to optimise their design in the context of mixed mode data collection.