How many countries are needed for multilevel SEM? A simulation study
English title: How many countries are needed for multilevel SEM? A simulation study
Author(s): Bart Meuleman - Tim Reeskens - Jaak Billiet -
Type: Conference paper/poster
In recent years, there has been growing scientific interest for cross-national research. Various scholars have made use of international surveys to link individual characteristics to aspects of the national context. Often, multilevel techniques are applied to explain individual-level variables by means of country-level variables. However, the application of multilevel models in the field of cross-national research is far from unproblematic. Due to budget limitations, the number of participating countries does not exceed 25 for most international surveys. Consequently, the group level sample sizes are often substantially lower than what rules of thumb suggest (at least 30 to 100 units). On the other hand, cross-national surveys typically contain a large number of respondents per country (> 1000). This paper summarizes the results of a Monte Carlo study that was carried out to assess the adequacy of multilevel structural equation modeling (SEM) with small group sample sizes. We simulated a multilevel SEM in which a continuous latent factor with four indicators is explained by both within- and between-level variables. In order to reproduce realistic circumstances as much as possible, the situation of the ESS round 1 (2002-2003) -20 countries and over 35,000 respondents- was taken as a starting point. With a group sample size of 20 units only, the quality of the estimation for the between-level model was poor. Parameter and standard error biases were considerably larger than what is generally considered to be acceptable. Moreover, the power for detecting small (0.10) or average (0.25) context effects was very low at 22 and 38 per cent. A substantially larger number of countries is needed to assure accurate estimation. How much countries are needed exactly depends on the specific purpose of the research and on the expected size of the context effects.
Conference name: Sixth International Amsterdam Conference on Multilevel Analysis
Start date: Apr 16, 2007