English title: Why are there differences in measurement error across countries?
Author(s): Daniel Oberski - W.E. Saris - Jacques Hagenaars -
Type: Book chapter
The European Social Survey (ESS) has the unique characteristic that in more than 20 countries the same questions are asked and that within each round of the ESS Multitrait-Multimethod (MTMM) experiments are built in to evaluate the quality of a limited number of questions. This gives us an exceptional opportunity to observe the differences in quality of questions over a large number of countries. The MTMM experiments make it possible to estimate the reliability, validity and method effects of single questions (Andrews 1984, Saris and Andrews 1991, Saris et al 2004). The product of the reliability and the validity can be interpreted as the explained variance in the observed variable by the variable one would like to measure. It is a measure of the total quality of a question. These MTMM experiments showed that there are considerable differences in measurement quality across countries. Because these differences in quality can cause wrong conclusions with respect to differences in relationships across countries this paper studies several reasons for these differences. The following explanations are considered: differences in translation, differences in the MTMM design and difference in complexity of the question formulation in the different countries. It turned out that the main reasons for the quality differences are differences in the formulation of the questions in the different countries and differences in the MTMM design. The complexity of the question formulation did not contribute much to the explanation.
From page no: 281
To page no: 300
Anthology: Measuring meaningful data in social research