Assessing Measurement Equivalence in Ordered-Categorical Data

  1. Elosua Oliden, Paula
Revista:
Psicológica: Revista de metodología y psicología experimental

ISSN: 1576-8597

Año de publicación: 2011

Volumen: 32

Número: 2

Páginas: 403-421

Tipo: Artículo

DIALNET GOOGLE SCHOLAR lock_openAcceso abierto editor

Otras publicaciones en: Psicológica: Revista de metodología y psicología experimental

Resumen

Assessing measurement equivalence in the framework of the common factor linear models (CFL) is known as factorial invariance. This methodology is used to evaluate the equivalence among the parameters of a measurement model among different groups. However, when dichotomous, Likert, or ordered responses are used, one of the assumptions of the CFL is violated: the continuous nature of the observed variables. The common factor analysis of ordered-categorical data (CFO) has been described in several works, but none evaluate its power and Type I error rate in the evaluation of measurement equivalence (ME). In this simulation study, we evaluated ME under four different conditions: size of group (300, 500 and 1000), type of DIF (thresholds, loadings), amount of DIF (0.25, 0.40), and equality/impact of the distributions. The parameters used for the data generation came from one scale with nine items with three ordered categories. The results were evaluated according to three decision rules: a) the significance of the difference in chi-square values obtained in two nested models, b) the significance of the difference in chi-square values between two nested models with Bonferroni corrections, and c) the difference between the values of the Comparative Fix Index (CFI) obtained in two nested models. The results showed good power as well as good control of the false positives for both the chi-square Bonferroni correction and CFI difference index.

Referencias bibliográficas

  • Beauducel, A., and Herzberg, P.Y. (2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in CFA. Structural Equation Modeling, 13, 186-203.
  • Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238-246.
  • Bock, R. D., and Aitkin, M. (1981). Marginal maximum likelihood estimation or item parameters: Application of an EM algorithm. Psychometrika, 46, 443-459.
  • Bollen, K. A. (1989). Structural equations with latent variables. New York, Wiley.
  • Cattell, H. B. (1989). The 16PF, Personality in depth. Champain, IL, Institute for Personality and Ability Testing, Inc.
  • Cheung, G. W., and Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233-255.
  • Christoffersson, A. (1975). Factor analysis of dichotomized variables. Psychometrika, 40, 5-32.
  • Cohen, A. S., Kim, S. H., and Baker, F. D. (1993). Detection of differenttial item functioning in the graded response model. Applied Psychological Measurement, 17(4), 335-350.
  • Elosua, P., and López, A. (2008). Adapting the AF5 autoconcept scale to Basque: Validity evidences. Individual Differences Research, 6(1), 57-70.
  • Elosua, P., and Wells, C. (July, 2008). A Comparison of MACS and the IRT Likelihood Ratio Test for Identifying DIF. Paper presented at III European Congress of Methodology, Oviedo.
  • Finch, W.H., and French, B.F. (2007). Detection of Crossing Differential Item Functioning. A Comparison of four Methods. Educational and Psychological Measurement, 67(4), 565-582.
  • Finney, S.J., and DiStenfano, C. (2006). Non-normal and categorical data in structural equation modelling. In G.R. Hancock & R. O. Mueller (Eds.), Structural Equation Modeling: A second course (pp. 269-314). Information Age Publishing Inc.
  • Flora, D.B., and Curran, P.J. (2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9, 466-491.
  • French, B.F., and Finch, W.H. (2006). Confirmatory factor analytic procedures for the determination of measurement invariance. Structural Equation Modeling, 13(3), 378-402.
  • French, A. W., and Miller, T. R. (1996). Logistic regression and its use in detecting differential item functioning in polytomous items. Journal of Educational Measurement, 33(3), 315-332.
  • Ihaka, R., and Gentleman, R. (1996). R: A Language for Data Analysis and Graphics. Journal of Computational and Graphical Statistics, 5, 299-314.
  • Jöreskog, K. G. (1971). Simultaneous factor analysis in several populations. Psychometrika, 36, 409-426.
  • Jöreskog, K. G. (1990). New developments in LISREL: Analysis of ordinal variables using polychoric correlations and weighted least squares. Quality and Quantity, 24, 387-404.
  • Jöreskog, K. G. (1994). On the estimation of polychoric correlations and their asymptotic covariance matrix. Psychometrika 59, 381-389.
  • Jöreskog, K. G., and Sörbom, D. (1993). LISREL 8: User's guide. Chicago: Scientific Software International.
  • Kristjansson, E., Aylesworth, R., McDowell, I., and Zumbo, B. D. (2005). A comparison of four methods for detecting differential item functioning in ordered response items. Educational and Psychological Measurement, 65(6), 935-953.
  • Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ, LEA.
  • Lubke, G. H., and Muthén, B. O. (2004). Applying multigroup confirmatory factor models for continuous outcomes to Likert scale data complicates meaningful group comparisons. Structural Equation Modeling, 11(4), 514-534.
  • Maydeu-Olivares, A., and Cai, L. (2006). A cautionary note on using G2 (dif) to assess relative model fit in categorical data analysis. Multivariate Behavioral Research, 41, 55-64.
  • Mellenbergh, G. J. (1989). Item bias and item response theory. International Journal of Educational Research, 13, 127-143.
  • Meade, A. W., and Lautenschlager, G. J. (2004a). A comparison of item response theory and confirmatory factor analytic methodologies for establishing measurement equivalence/invariance. Organizational Research Methods, 7(4), 361-388.
  • Meade, A. W., and Lautenschlager, G. J. (2004b). A monte-carlo study of confirmatory factor analytic tests of measurement equivalence/invariance. Structural Equation Modeling, 11, 60-72.
  • Meade, A. W., and Lautenschlager, G. J. (2004c). Same question, different answer: CFA and two IRT approaches to measurement invariance. 19th Annual Conference of the Society for Industrial and Organizational Psychology, Chicago.
  • Meredith, W. (1964). Notes on factorial invariance. Psychometrika, 29, 177-185.
  • Meredith, W. (1993). Measurement invariance, factor analysis, and factorial invariance. Psychometrika, 58, 521-543.
  • Millsap, R.E. (2005). Four unresolved problems in studies of factorial invariance. In A. Maydeu-Olivares and J. J. McArdle (Eds.), Contemporary psychometrics (pp.153-172). Mahwah, NJ: Lawrence Erlbaum
  • Millsap, R. E., and Yun-Tein, J. (2004). Assessing Factorial Invariance in Ordered-Categorical Measures. Multivariate Behavioral Research, 39(3), 479-515.
  • Mislevy, R. J. (1986). Recent developments in the factor analysis of categorical variables. Journal of Educational Statistics, 11, 3-31.
  • Musitu, G., García, F., and Gutiérrez, M. (1997). AFA. Autoconcepto Forma-A. Madrid: TEA
  • Muthén, B. O. (1984). A general structural equation model with dichotomous, ordered categorical and continuous latent variable indicators. Psychometrika, 49, 115-132.
  • Muthén B. O., du Toit S. H. C., and Spisic D. (1997). Robust inference using weighted least squares and quadratic estimating equations in latent variable modeling with categorical and continuous outcomes. Unpublished manuscript, University of California, Los Angeles.
  • Muthén, B. O., and Kaplan, D. (1985). A comparison of some methodologies for the factor análisis of non-normal Likert variables. British Journal of Mathematical and Statistical Psychology, 38, 171-189.
  • Muthén, L. K., and Muthén, B. O. (2004). Mplus users's guide. Los Angeles: Muthén and Muthén.
  • Oort, F. J. (1998). Simulation study of item bias detection with restricted factor analysis. Structural Equation Modeling, 5, 107-124.
  • Raju, N. S., Laffite, L. J., and Byrne, B. M. (2002). Measurement equivalence: A comparison of methods based on confirmatory factor analysis and item response theory. Journal of Applied Psychology, 87(3), 517-529.
  • Schuerger, J. M. (2001). 16PF-APQ Manual. Champaign, IL, Institute for Personality and Ability Testing.
  • Sörbom, D. (1974). A general method for studying differences in factor means and factor structures between groups. British Journal of Mathematical and Statistical Psychology, 27, 229-239.
  • Stark, S., Chernyshenko, O. S., and Drasgow, F. (2006). Detecting differential item functioning with confirmatory factor analysis and item response theory: Toward a unified strategy. Journal of Applied Psychology, 91(6), 1292-1306.
  • Spray, J. A., and Miller, T. (1994). Identifying nonuniform DIF in polytomously scored test items. (American College Testing Research Report Series 94-1)Iowa City, IA: American College Testing Program.
  • Thissen, D. (1991). MULTILOG user's guide. Item analysis and scoring with multiple category response models (Version 6). Mooresville, IN, Scientific Software.
  • Thissen, D., Steinberg, L., and Wainer, H. (1988). Use of item respones theory in the study of group differences in trace lines. In H. Wainer and H. I. Braun (Eds.) Test validity. (pp. 147-169) Hillsdale, NJ: Lawrence Erlbaum.
  • Zwick, R., Donogue, J. R., and Grima, A. (1993). Assessment of differential item functioning for performance tasks. Journal of Educational Measurement, 30(3), 233-251.
  • Zwick, R., Thayer, D. T., and Mazzeo, J. (1997). Descriptive and inferential procedures for assessing differential item functioning in polytomous items. Applied Measurement in Education, 10(4), 321-344.
Fuente de los datos: Dialnet