Source Questionnaire Development

The ESS aims to achieve high methodological standards, striving for optimal comparability in the data collected across all countries. This is only possible through the use of high quality questions that are designed to ensure that they are as comparable as possible across countries.

Core module development

The documents in the sidebar detail the development of each section of the core module of the ESS questionnaire, including recommendations made by academic experts who were consulted during the early planning stages of the ESS. Chapter 1 provides an overview of the questionnaire development process.

Whilst a key purpose of the core module is to provide continuity across rounds, the questionnaire has evolved over the life of the ESS to ensure the survey remains fit for purpose, i.e. it measures relevant concepts in a valid way.

• From Round 4, the income variable was changed and a decile approach was applied when measuring income in the ESS.
• From Round 5, new instruments were introduced to provide harmonised educational attainment measures for respondent, partner, father and mother as well as new procedures for bridging of country specific variables. The education variables for Rounds 1 to 4 were updated in line with these new procedures. Further information can be found in the education measurement guidelines for ESS Round 10, as well as in Appendix A1 of the Data Documentation Report of each ESS round since 2010. You can download an Excel file with an overview of all country-specific education categories, their changes over rounds and mapping to the harmonized variable 'edulvlb' across countries since ESS round 5.
• From Round 5, the new ISCO-08 standard for measuring occupation was introduced in the ESS from Round 5, replacing the old ISCO-88COM.
• From Round 5, new measures to distinguish between legal marital and relationship status were introduced. This was done because analysis of data and feedback from Rounds 1 to 4 showed that the phrase ‘legal marital status’ was not easily understood, that ‘cohabiting’ was sometimes regarded as a legal marital status and that ‘civil partnership’ was misunderstood when a cohabitation category was not provided in the questionnaire.
• From Round 5 (on a provisional basis) and from Round 6 (on a permanent basis), some items were dropped from the core module to shorten the length of the interview. The changes are described in detail in this document.
• From Round 7, an item measuring socio-cultural origins via self-reported ancestry, i.e. family origins or descent, was added. The item was fielded using a country-specific showcard and responses were recoded into a newly-developed European Standard Classification of Cultural and Ethnic Groups (ESCEG) to create harmonised variables for comparative analysis.
• From Round 8, several new items were added to improve the measurement of key concepts including media use, national and European identity, attitudes to homosexuality and political efficacy. Documentation on the development of the new items is available from the sidebar.

In advance of ESS Round 10, representatives of the Core Scientific Team (CST) and members of the ESS Scientific Advisory Board (SAB) formed a Core Review Team to assess core questions item by item. Each thematic area of the source questionnaire was assessed for its ongoing social significance and policy relevance, as well as the quality of measurement achieved by the original items. Experts were invited to comment on any perceived thematic gaps in the questionnaire, resulting from changes in the social and political landscape of Europe over two decades. The report - ESS Round 10 Core Questionnaire Review: Summary of changes - includes details of the changes implemented, the rationale behind the decisions and evidence from pre-testing and the consultation of data users.

Questionnaire consultations

Since ESS Round 4, before the start of the data collection the CST undertakes a consultation process with participating countries on a number of items whose answer categories need to be adapted to the national context. For ESS Round 8 these include questions on income, religion, education, marital and relationship status, and ancestry.

The country-specific categories are then mapped and postcoded using either international standards or ESS-specific standards to ensure optimal comparability across countries.

Rotating module development

The ESS uses a carefully designed model for cross-national questionnaire design and pre-testing. A combination of qualitative and quantitative pre-testing strategies are employed during the design process of each rotating module to try and achieve optimal comparability across countries. The source questionnaire is designed in British English, but the process is infused with cross-cultural input (both methodological and substantive).

The design and development process lasts for 20 months – from the appointment of the successful question module design team through to the release of the source questionnaire for the round. It incorporates expert review from members of the CST as well as the national teams, alongside coding item characteristics to predict their validity and reliability using the Survey Quality Predictor (SQP), cognitive interviewing, advance translation and quantitative testing on omnibus surveys and in a two-nation pilot survey. This diagram shows the questionnaire development and pre-testing process for ESS9 rotating modules.

Since ESS Round 4, the design and development of rotating modules has been fully documented through a specially designed questionnaire design template. The final template that is produced shows the concepts underpinning the design of the module as well as the wording of the final questions included in the survey. All templates released so far are available from the sidebar.