In the ESS, data have to be collected via face-to-face interviews (preferably CAPI) in all participating countries.
In each country, the national funding agency appoints a National Coordinator (NC) and a survey organisation to implement the survey according to common ESS specifications. The ESS specifications are set to ensure accuracy of data in each country and to optimise comparability of data across countries. The most important standards on data collection include:
- Response rate target 70% (as a general target; actual target lower in some countries)
- Non-contact rate target of 3% maximum
- Fieldwork period of at least 1 month within the 4 months between September and December of the survey year
- Detailed briefing of interviewers in face-to-face sessions
- Restricted interviewer workload (maximum 48 sample units gross)
- Interviewer call schedule: 4 contacts attempts minimum, among which at least 1 in the evening and 1 at the weekend
- Contact forms to record and document data on fieldwork processes
- Quality control back-checks on completed interviews and non-respondents
- Close monitoring of fieldwork progress
In order to foster compliance with these standards, the ESS Core Scientific Team provides guidelines, training materials, as well as individual feedback and support to countries. The training materials consist of an interviewer manual plus a series of pre-structured slides with movie clips. The slides are developed with a teaching approach which acknowledges the interviewers’ previous experience while identifying and remedying any gaps in their knowledge or skills.
The whole process of preparation and implementation of data collection in each country is monitored by the CST. At various points in the survey life cycle, countries are required to document and discuss the planning and progress of data collection. Important milestones include:
Before data collection:
- Quality report: Country-specific feedback provided by the CST on a wide range of quality issues encountered in the previous round ([see page ‘quality control’]). To be taken into account in the planning of data collection for the upcoming round.
- Meeting of CST members with Field Directors to discuss data collection standards with the survey organisations appointed.
- Fieldwork questionnaire: An instrument to discuss, help decide upon, and document major fieldwork decisions and parameters (timing of fieldwork, number of interviewers, etc.). To be filled in by NCs, discussed by NC and CST, and agreed and signed-off by the CST four weeks before fieldwork starts.
- Fieldwork projections: Forecasts of weekly production/response rates based on experience from previous rounds and current interviewer staffing. To be provided by each country two weeks before fieldwork starts.
During data collection:
- Each country to provide weekly information on fieldwork progress to the CST (number of completed interviews, refusals, etc.). The actual progress of fieldwork is compared with the benchmark data from the fieldwork projections to identify possible problems and a need for action. For troubleshooting survey organisations may be requested to provide more detailed information (like response rates for regions or individual interviewers, response rates for demographic subgroups of target persons, or data on the number and timing of contact attempts).
After data collection:
- Each country to deposit the ESS main data set and all fieldwork documents (briefing materials, advance letters, contact forms, etc.) at the ESS Data Archive.
- Each country to provide metadata (National Technical Summary) and paradata (contact forms data) to the ESS Data Archive.
- CST to analyse and document numerous quality aspects related to data collection (call schedule, refusal conversion, interviewer effects, sample composition, etc.).
Taken together, the ESS aims to achieve a process of continuous improvement in data collection by implementing a coherent set of quality assurance activities. These activities include the definition of standards, the provision of support in implementing the standards, the analysis and documentation of survey procedures and numerous quality indicators, and, finally, the provision of feedback on any observed deficiencies.