An Experimental Comparison of Four Modes of Data Collection Within the Eurobarometer Measurement Domain
English title: An Experimental Comparison of Four Modes of Data Collection Within the Eurobarometer Measurement Domain
Author(s): R. Manchin - R. Tortora - E. Peytcheva - R. Groves -
Type: Conference paper/poster
The growing number of methods of data collection and the rising popularity of mixed mode designs to increase timeliness, improve measurement error properties or reduce non-response or cost necessitate studying the effects of different methods of data collection on survey errors. With the increasing interest in cross cultural studies and the difficulties in employing the same method of data collection across countries due to economical or cultural challenges, there is a growing need to compare the major methods of data collection simultaneously and examine mode effects on a variety of question types. An experiment motivated by this need was conducted by Gallup-Europe in conjunction with the European Social Survey (ESS) in May-July, 2003. Four major modes, face-to-face, telephone, selfadministered and web, were compared in two waves of data collection - a convenience quota sample of 1,987 respondents representative of the Hungarian urban population by age, sex and education was selected, randomly assigned to a mode, and then re-interviewed in a different mode. The second interview for some methods of data collection was randomly assigned to be either on location (a public location) or off location (the respondent’s home). The survey instrument included questions from the ESS and the Eurobarometer. The selection was driven by question type and topic with the intention of including socially desirable, sensitive, complex, open-ended, etc. items. Mode effects become visible when the different methods of data collection produce different response distributions. An explanatory model for responses can be employed adopting Lazersfeld’s latent class model (Saris and Hagenaars, 1997). If respondents’ probabilities to give particular responses vary across modes, then the observed response distributions will be different and these differences are not attributable to chance. The within subject design allows estimating the response probabilities from square tables, using the EM algorithm and employing different restrictions. A model assuming equal response probabilities between pairs of modes is tested first and evaluated based on the likelihood ratio test. When such a model does not fit, less restricted models that allow probabilities between modes to vary are tested. A final model is accepted on the basis of parsimony. This procedure is repeated for all questions and across all possible mode comparisons.
Conference name: Sixth International Conference on Social Science Methodology
Start date: Aug 16, 2004