Data Collection

In the ESS, data has always been collected via face-to-face CAPI interviews in all participating countries.

It was announced in May 2022 that the ESS General Assembly has endorsed a recommendation from its Core Scientific Team to plan a transition of its data collection. This will see a switch from face to face interviews to a (web first) self-completion design. The design will involve web and paper self-completion surveys. The ESS has appointed four experts to help advise on the transition.

In each country, the national funding agency appoints a National Coordinator (NC) and a survey organisation to implement the survey according to the common ESS Specification. The Specification is set to ensure accuracy of data in each country and to optimise comparability of data across countries. The most important standards on data collection include:

• Response rate target 70% (as a general target; actual target lower in some countries)
• Non-contact rate target of 3% maximum
• Fieldwork period of at least 6 weeks within the 5 months between September of the survey year and January of the following year
• Detailed briefing of interviewers in face-to-face sessions
• Restricted interviewer workload (maximum 48 sample units gross)
• Interviewer call schedule: 4 contacts attempts minimum, among which at least 1 in the evening and 1 at the weekend
• Contact forms to record and document data on fieldwork processes
• Quality control back-checks on completed interviews and ineligible cases
• Close monitoring of fieldwork progress

 

In order to foster compliance with these standards, the ESS Core Scientific Team provides guidelines, training materials, as well as individual feedback and support to countries. The training materials consist of an interviewer manual plus a series of pre-structured slides with movie clips. The slides are developed with a teaching approach which acknowledges the interviewers’ previous experience while identifying and remedying any gaps in their knowledge or skills.

The whole process of preparation and implementation of data collection in each country is monitored by the CST. At various points in the survey life cycle, countries are required to document and discuss the planning and progress of data collection. Important milestones include:

 

Before the data collection:

• Quality report: Country-specific feedback provided by the CST on a wide range of quality issues encountered in the previous round (see Data quality assessment page). To be taken into account in the planning of data collection for the upcoming round.
• Meeting of CST members with Field Directors to discuss data collection standards with the survey organisations appointed.
• Fieldwork questionnaire: An instrument to discuss, help decide upon, and document major fieldwork decisions and parameters (timing of fieldwork, number of interviewers, etc.). To be filled in by NCs, discussed by NC and CST, and agreed and signed-off by the CST four weeks before fieldwork starts.
• Fieldwork projections: Forecasts of weekly production/response rates based on experience from previous rounds and current interviewer staffing. To be provided by each country two weeks before fieldwork starts.

 

During the data collection:

• Each country to provide weekly case level information on fieldwork progress (in the form of a dataset) to the CST. The actual progress of fieldwork is compared with the benchmark data from the fieldwork projections to identify possible problems and a need for action. For troubleshooting, survey organisations may be requested to provide more detailed information or investigate quality problems identified by the CST.

 

After the data collection:

• Each country to deposit the ESS main data set and all fieldwork documents (briefing materials, advance letters, contact forms, etc.) at the ESS Data Archive.
• Each country to provide metadata (National Technical Summary) and paradata (contact forms data) to the ESS Data Archive.
• CST to analyse and document numerous quality aspects related to data collection (call schedule, refusal conversion, interviewer effects, sample composition, etc.).

 

Taken together, the ESS aims to achieve a process of continuous improvement in data collection by implementing a coherent set of quality assurance activities. These activities include the definition of standards, the provision of support in implementing the standards, the analysis and documentation of survey procedures and numerous quality indicators, and, finally, the provision of feedback on any observed deficiencies.