Translation Assessment

Translation assessment is a key element for producing high quality translations. The ESS incorporates several steps for assessing the quality of its questionnaire translations:

• The Review/Adjudication steps within the TRAPD scheme (see Translation) are to be considered as a first internal translation quality assessment within each national team.
• Translation verification is an assessment of the linguistic and semantic correctness and quality of all translated national survey instruments, provided by an external service provider (see below).
• Translation assessment looks out for formal differences between the source questionnaire and its translated versions; this step is carried out by the national teams, guided by CST researchers at Universitat Pompeu Fabra (UPF).
• The Pre-testing step within the TRAPD scheme provides a quality check of the final questionnaire translations with a sample of the final target population (see below).

Verification

Translation verification by an external service provider was introduced as an additional stage to the translation process in ESS Round 5. It involves a linguistic quality check of a selection of items for all language versions integrated within the TRAPD scheme, i.e. during the translation process. Translation verification is carried out by cApStAn.

Full Verification Instructions are provided to the national teams to assist them with the verification process. The national teams are asked to confirm acceptance of the comments made during the verification or to explain why they feel that the suggestions made by the verifiers cannot be accepted. The verification step allows a close discussion on translation issues and decisions taken between the national teams, cApStAn and the ESS Translation Team.

The entire translation process, including all steps of the TRAPD scheme, verification and UPF translation assessment, for each language version is documented using a (Translation and) Verification Follow-up Form.

The quality of the verification work is also regularly assessed by members of the ESS Translation Team and Translation Expert Panel.

Translation assessment

The ESS is the only cross-national survey to use the translation assessment by the team at UPF to control for the equivalence between the characteristics of a selection of translated questions and the original wording of the question in British English.

Following the instructions given in the translation assessment guidelines and codebook, all national teams are asked to code a selection of their translated questions using the tool. This allows to compare the characteristics of the source and translated questionnaires in order to detect and prevent differences.

Pre-testing

National pre-tests in the ESS have a dual purpose within the fieldwork preparation process. On the one hand, they allow to test whether the survey instrument works as intended from a technical point of view, e.g. in terms of routing techniques, timestamps or measuring the interview length. On the other hand, they are a full-fledged element within the TRAPD scheme, representing the ‘P’ in this acronym: they allow to test whether items or terms are not easily or incorrectly understood by the target populations; in case of doubt, several translation options may be fielded in the pre-test and compared to each other as a last decision-making point before the actual fieldwork.

Related content

ESS documents on this topic

Publications on this topic

Working papers

Zavala, D. (2014).
A Procedure to Prevent Differences in Translated Survey Items Using SQP, RECSM Working Paper No. 38, Barcelona

Books

Saris, W. E. & Gallhofer, I. N. (2014).
Design, Evaluation, and Analysis of Questionnaires for Survey Research. Hoboken: Wiley