Organizational Research Methods

 

Advanced Search

Journal Navigation

Journal Home

Subscriptions

Archive

Contact Us

Table of Contents

Register here to gain access to SAGE's 500+ Journals Online

SAGETRACK

Sign In to gain access to subscriptions and/or personal tools.
This Article
Right arrow Abstract Freely available
Right arrow Free Full Text (Free PDF) Free
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Alert me to new issues of the journal
Right arrow Add to Saved Citations
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Request Reprints
Right arrow Add to My Marked Citations
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Cheung, M. W.-L.
Right arrow Search for Related Content
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati  
What's this?
This version was published on October 1, 2007
Organizational Research Methods, Vol. 10, No. 4, 609-634 (2007)
DOI: 10.1177/1094428106295499

Comparison of Methods of Handling Missing Time-Invariant Covariates in Latent Growth Models Under the Assumption of Missing Completely at Random

Mike W.-L. Cheung

National University of Singapore, mikewlcheung{at}nus.edu.sg

Latent growth models implemented in multilevel models (MLM) or structural equation models (SEM) may be used to analyze longitudinal data with an emphasis on interindividual and intraindividual differences. The main objective of this study is to compare methods of handling missing time-invariant data under the assumption of missing completely at random. Listwise deletion (LD), mean substitution (MS), the expectation-maximization (EM) algorithm, multiple imputation (MI), and full information maximum likelihood (FIML) are compared via a computer simulation study. The findings show that FIML and LD generally perform best, whereas the standard errors of EM and MI are usually underestimated. The results on MS are generally acceptable except when the percentage of missingness is large. Practical implications and directions for future research are discussed.

Key Words: latent growth models • structural equation models • multilevel models • missing data

References

  • Allison, P.D. (2002). Missing data. Thousand Oaks, CA: Sage.
  • Bauer, D.J. (2003). Estimating multilevel linear models as structural equation models. Journal of Educational and Behavioral Statistics, 28, 135-167.[Abstract/Free Full Text]
  • Bentler, P.M. (2004). EQS 6 [Computer software]. Encino, CA: Multivariate Software, Inc.
  • Biesanz, J.C., Deeb-Sossa, N., Aubrecht, A.M., Bollen, K.A., & Curran, P.J. (2004). The role of coding time in estimating and interpreting growth curve models. Psychological Methods, 9, 30-52.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Bliese, P.D., & Ployhart, R.E. (2002). Growth modeling using random coefficients models: Model building, testing, and illustrations. Organizational Research Methods, 5, 362-387.[Abstract]
  • Bollen, K.A., & Curran, P.J. (2006). Latent curve models: A structural equation perspective. Hoboken, NJ: Wiley-Interscience.
  • Bryant, A.L., Schulenberg, J.E., O'Malley, P.M., Bachman, J.G., & Johnston, L.D. (2003). How academic achievement, attitudes, and behaviors relate to the course of substance use during adolescence: A 6-year, multiwave national longitudinal study. Journal of Research on_ Adolescence, 13, 361-397.[CrossRef]
  • Bryk, A.S., & Raudenbush, S.W. (1987). Application of hierarchical linear models to assessing change. Psychological Bulletin, 101, 147-158.[CrossRef][ISI]
  • Bryk, A.S., & Raudenbush, S.W. (1989). Methodology for cross-level organizational research. Research in Sociology of Organizations, 7, 233-273.
  • Chan, D. (1998). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology, 83, 234-246.[CrossRef][ISI]
  • Chan, D. (2002). The conceptualization and analysis of change over time: An integrative approach incorporating longitudinal mean and covariance structure analysis (LMACS) and multiple indicator latent growth modeling (MLGM). Organizational Research Methods, 1, 421-483.[CrossRef]
  • Chan, D., Ramey, S., Ramey, C., & Schmitt, N. (2000). Modeling intraindividual changes in children's social skills at home and at school: A multivariate latent growth approach to understanding between-settings differences in children's social skill development. Multivariate Behavioral Research, 35, 365-396.[CrossRef][ISI]
  • Cheong, J., MacKinnon, & D.P., Khoo, S.T. (2003). Investigation of mediational processes using parallel process latent growth curve modeling. Structural Equation Modeling, 10, 238-262.[CrossRef][ISI]
  • Cheung, M.W.L., & Au, K. (2005). Applications of multilevel structural equation modeling to cross-cultural research. Structural Equation Modeling, 12, 598-619.[CrossRef]
  • Cheung, M.W.L., Leung, K., & Au, K. (in press). Evaluating multilevel models in cross-cultural research: An illustration with social axioms. Journal of Cross-Cultural Psychology.
  • Chou, C., Bentler, P.M., & Pentz, M.A. (1998). Comparisons of two statistical approaches to study growth curves: The multilevel model and the latent curve analysis. Structural Equation Modeling, 5, 247-266.
  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
  • Collins, L.M. (2006). Analysis of longitudinal data: The integration of theoretical model, temporal design, and statistical model. Annual Review of Psychology, 57, 505-528.
  • Collins, L.M., Schafer, J.L., & Kam, C. (2001). A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods, 6, 330-351.[ISI][Medline] [Order article via Infotrieve]
  • Curran, P.J. (2003). Have multilevel models been structural equation models all along? Multivariate Behavioral Research, 38, 529-569.[CrossRef][ISI]
  • Dempster, A.P., Laird, N.M., & Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society, Series B, 39, 1-38.
  • Enders, C.K. (2001). A primer on maximum likelihood algorithms available for use with missing data. Structural Equation Modeling, 8, 128-141.[CrossRef]
  • Enders, C.K., & Peugh, J.L. (2004). Using an EM covariance matrix to estimate structural equation models with missing data: Choosing an adjusted sample size to improve the accuracy of inferences. Structural Equation Modeling, 11, 1-19.[CrossRef]
  • Fichman, M., & Cummings, J.N. (2003). Multiple imputation for missing data: Making the most of what you know. Organizational Research Methods, 6, 282-308.[Abstract]
  • Fredriksen, K., Rhodes, J., Reddy, R., & Way, N. (2004). Sleepless in Chicago: Tracking the effects of adolescent sleep loss during the middle school years. Child Development, 75, 84-95.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Gibson, N.M., & Olejnik, S. (2003). Treatment of missing data at the second level of hierarchical linear models. Educational and Psychological Measurement, 63, 204-238.[Abstract/Free Full Text]
  • Goldstein, H. (2003). Multilevel statistical models (3rd ed.). London: Arnold.
  • Graham, J.W., Hofer, S.M., & MacKinnon, D.P. (1996). Maximizing the usefulness of data obtained with planned missing value patterns: An application of maximum likelihood procedures. Multivariate Behavioral Research, 31, 197-218.[CrossRef][ISI]
  • Gravetter, F.J., & Forzano, L.B. (2003). Research methods for the behavioral sciences. Belmont, CA: Thomson/Wadsworth.
  • Hofmann, D.A. (1997). An overview of the logic and rationale of hierarchical linear models. Journal of Management, 23, 723-744.[Abstract]
  • Hofmann, D.A., & Gavin, M.B. (1998). Centering decisions in hierarchical linear models: Implications for research in organizations. Journal of Management, 24, 623-641.[Abstract]
  • Hoogland, J.J., & Boomsma, A. (1998). Robustness studies in covariance structure modeling: An overview and a meta-analysis. Sociological Methods and Research, 26, 329-367.[CrossRef]
  • Hox, J.J. (2002). Multilevel analysis: Techniques and applications. Mahwah, NJ: Lawrence Erlbaum.
  • Hox, J.J., & Kreft, I.G.G. (1994). Multilevel analysis methods. Sociological Methods and Research, 22, 283-299.[CrossRef]
  • Huang, X., & Van de Vliert, E. (2003). Comparing work behaviors across cultures: A cross-level approach using multilevel modeling. International Journal of Cross Cultural Management, 3, 167-182.[Abstract]
  • Jöreskog, K.G., & Sörbom, D. (1996). LISREL 8: A user's reference guide. Chicago: Scientific Software International.
  • Klein, K.J., & Kozlowski, S.W.J. (2000). From micro to meso: Critical steps in conceptualizing and conducting multilevel research. Organizational Research Methods, 3, 211-236.[Abstract]
  • Klein, K.J., Tosi, H., & Cannella, Jr., A.A. (1999). Multilevel theory building: Benefits, barriers, and new developments. Academy of Management Review, 24, 243-248.
  • Kozlowski, S.W.J., & Klein, K.J. (2000). A multilevel approach to theory and research in organizations: Contextual, temporal, and emergent properties. In K. J. Klein, S.W.J. Kozlowski (Eds.), Multilevel theory, research and methods in organizations: Foundations, extensions, and new directions (pp. 3-90). San Francisco: Jossey-Bass.
  • MacCallum, R.C., Kim, C., Malarkey, W.B., & Kiecolt-Galser, J.K. (1997). Studying multivariate change using multilevel models and latent curve models. Multivariate Behavioral Research, 32, 215-253.[Medline] [Order article via Infotrieve]
  • Mehta, P.D., & Neale, M.C. (2005). People are variables too: Multilevel structural equation modeling. Psychological Methods, 10, 259-284.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Mehta, P.D., & West, S.G. (2000). Putting the individual back into individual growth curves. Psychological Methods, 5, 23-43.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Meredith, W., & Tisak, J. (1990). Latent curve analysis. Psychometrika, 55, 107-122.[CrossRef][ISI]
  • Muthén, B.O. (2002). Beyond SEM: General latent variable modeling. Behaviormetrika, 29, 81-117.
  • Muthén, B.O., & Curran, P.J. (1997). General longitudinal modeling of individual differences in experimental designs: A latent variable framework for analysis and power estimation. Psychological Methods, 2, 371-402.[CrossRef][ISI]
  • Muthén, L.K., & Muthén, B.O. (2002). How to use a Monte Carlo study to decide on sample size and determine power. Structural Equation Modeling, 9, 599-620.[CrossRef][ISI]
  • Muthén, L.K., & Muthén, B.O. (2006). Mplus user's guide (4th ed.). Los Angeles: Muthén, & Muthén.
  • Newman, D.A. (2003). Longitudinal modeling with randomly and systematically missing data: A simulation of ad hoc, maximum likelihood, and multiple imputation techniques. Organizational Research Methods, 6, 328-362.[Abstract]
  • Pinheiro, J.C., & Bates, D.M. (2000). Mixed-effects models in S and S-PLUS. New York: Springer.
  • Ployhart, R.E., & Hakel, M.D. (1998). The substantive nature of performance variability: Predicting interindividual differences in intraindividual performance. Personnel Psychology, 51, 859-901.[CrossRef][ISI]
  • R Development Core Team. (2006). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URL: http://www.r-project.org
  • Rao, C.R. (1958). Some statistical methods for comparison of growth curves. Biometrics, 14, 1-17.[Medline] [Order article via Infotrieve]
  • Raudenbush, S.W. (2001). Comparing personal trajectories and drawing causal inferences from longitudinal data. Annual Review of Psychology, 52, 501-525.
  • Raudenbush, S.W., & Bryk, A.S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Newbury Park, CA: Sage.
  • Raudenbush, S.W., & Willms, J. (1995). The estimation of school effects. Journal of Educational and Behavioral Statistics, 20, 1-41.[Abstract/Free Full Text]
  • Raykov, T. (2005). Analysis of longitudinal studies with missing data using covariance structure modeling with full-information maximum likelihood. Structural Equation Modeling, 12, 493-505.[CrossRef]
  • Roth, P.L. (1994). Missing data: A conceptual review for applied psychologists. Personnel Psychology, 47, 537-560.[CrossRef][ISI]
  • Rovine, M.J., & Molenaar, P.C.M. (1998). A nonstandard method for estimating a linear growth model in LISREL. International Journal of Behavioral Development, 222, 453-473.
  • Rovine, M.J., & Molenaar, P.C.M. (2000). A structural modeling approach to a multilevel random coefficient model. Multivariate Behavioral Research, 35, 51-88.[CrossRef][ISI]
  • Rubin, D.B. (1987). Multiple imputation for nonresponse in surveys. New York: Wiley.
  • Savalei, V., & Bentler, P.M. (2005). A statistically justified pairwise ML method for incomplete for incomplete nonnormal data: A comparison with direct ML and pairwise ADF. Structural Equation Modeling, 12, 183-214.[CrossRef]
  • Schafer, J.L. (1997). Analysis of incomplete multivariate data. London: Chapman & Hall.
  • Schafer, J.L. (2002). Norm: Analysis of multivariate normal datasets with missing values. R package version 1.0-9. http://www.stat.psu.edu/~jls/misoftwa.html#aut
  • Schafer, J.L., & Graham, J.W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7, 147-177.[CrossRef][ISI][Medline] [Order article via Infotrieve]
  • Schafer, J.L., & Olsen, M.K. (1998). Multiple imputation for multivariate missing-data problems: A data analyst's perspective. Multivariate Behavioral Research, 33, 545-571.[CrossRef][ISI]
  • Skrondal, A., & Rabe-Hesketh, S. (2004). Generalized latent variable modeling: Multilevel, longitudinal, and structural equation models. Boca Raton, FL: Chapman & Hall.
  • Tucker, L.R. (1958). Determination of parameters of a functional relation by factor analysis. Psychometrika, 23, 19-23.[Medline] [Order article via Infotrieve]
  • Venables, W.N., & Ripley, B.D. (2002). Modern applied statistics with S (4th ed.). New York: Springer.
  • Wendorf, C.A. (2002). Comparisons of structural equation modeling and hierarchical linear modeling approaches to couples' data. Structural Equation Modeling, 9, 126-140.[CrossRef][ISI]
  • Willett, J.B., & Sayer, A.G. (1994). Using covariance structure analysis to detect correlates and predictors of change. Psychological Bulletin, 116, 363-381.[CrossRef][ISI]
  • Wothke, W. (1993). Nonpositive definite matrices in structural modeling. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp.256-293). Newbury Park, CA: Sage.
  • Wothke, W. (2000). Longitudinal and multi-group modeling with missing data. In T. D. Little, K. U. Schnabel, & J. Baumert (Eds.). Modeling longitudinal and multiple group data: Practical issues: Applied approaches and specific examples (pp. 219-240). Mahwah, NJ: Lawrence Erlbaum.

Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us   Add to Digg Digg   Add to Reddit Reddit   Add to Technorati Technorati    What's this?



This Article
Right arrow Abstract Freely available
Right arrow Free Full Text (Free PDF) Free
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Alert me to new issues of the journal
Right arrow Add to Saved Citations
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Request Reprints
Right arrow Add to My Marked Citations
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Cheung, M. W.-L.
Right arrow Search for Related Content
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us   Add to Digg   Add to Reddit   Add to Technorati  
What's this?