Source Questionnaire Development

The ESS aims to achieve high methodological standards, striving for optimal comparability in the data collected across all countries. This is only possible through the use of high quality questions that are designed to ensure that they are as comparable as possible across countries.

Core module development

The documents in the sidebar detail the development of each section of the core module of the ESS questionnaire, including recommendations made by academic experts who were consulted during the early planning stages of the ESS. Chapter 1 provides an overview of the questionnaire development process.

Whilst a key purpose of the core module is to provide continuity across rounds, the questionnaire has evolved over the life of the ESS to ensure the survey remains fit for purpose, i.e. it measures relevant concepts in a valid way.

In advance of ESS Round 10, representatives of the Core Scientific Team (CST) and members of the ESS Scientific Advisory Board (SAB) formed a Core Review Team to assess core questions item by item. Each thematic area of the source questionnaire was assessed for its ongoing social significance and policy relevance, as well as the quality of measurement achieved by the original items. Experts were invited to comment on any perceived thematic gaps in the questionnaire, resulting from changes in the social and political landscape of Europe over two decades. The report - ESS Round 10 Core Questionnaire Review: Summary of changes - includes details of the changes implemented, the rationale behind the decisions and evidence from pre-testing and the consultation of data users.

Questionnaire consultations

Since ESS Round 4, before the start of the data collection the CST undertakes a consultation process with participating countries on a number of items whose answer categories need to be adapted to the national context. For ESS Round 8 these include questions on income, religion, education, marital and relationship status, and ancestry.

The country-specific categories are then mapped and postcoded using either international standards or ESS-specific standards to ensure optimal comparability across countries.

Rotating module development

The ESS uses a carefully designed model for cross-national questionnaire design and pre-testing. A combination of qualitative and quantitative pre-testing strategies are employed during the design process of each rotating module to try and achieve optimal comparability across countries. The source questionnaire is designed in British English, but the process is infused with cross-cultural input (both methodological and substantive).

The design and development process lasts for 20 months – from the appointment of the successful question module design team through to the release of the source questionnaire for the round. It incorporates expert review from members of the CST as well as the national teams, alongside coding item characteristics to predict their validity and reliability using the Survey Quality Predictor (SQP), cognitive interviewing, advance translation and quantitative testing on omnibus surveys and in a two-nation pilot survey. This diagram shows the questionnaire development and pre-testing process for ESS9 rotating modules.

Since ESS Round 4, the design and development of rotating modules has been fully documented through a specially designed questionnaire design template. The final template that is produced shows the concepts underpinning the design of the module as well as the wording of the final questions included in the survey. All templates released so far are available from the sidebar.