Ezike, N., Turner, R. C., Lo, W., Crawford, B.L., & Jozkowski, K.N. (2021, July 20 - July 23). Comparing estimation procedures for omega reliability with non-normal ordinal data. International Meeting of the Psychometric Society (IPMS) Annual Meeting, Virtual.al.
McDonald (1999) proposed the omega coefficient as an estimate of internal consistency reliability when tau-equivalence is not met. Omega total is based on factor analysis and is easily estimated in software such as R. Maximum likelihood estimator is preferred when assumptions such as normality are met. Revelle (2014) recommends minimum residual (MR, Comrey, 1962; Harman & Jones, 1966) factoring method for badly behaved correlation matrices. However, there is little known about how well omega total is estimated for ordinal data with bimodal distributions. van der Eijk and Rose (2015) found bimodal distributions were associated with worst fit for some unidimensional settings. The purpose of this study is to evaluate the performance of three estimation methods (weighted least squares mean and variance [WLSMV], principal axis [PA], and MR in estimating omega reliability with ordinal data having bimodal distributions. Using Monte Carlo simulation with a one-factor structure, we varied the number of items (4, 8, 12), sample size (100, 300, 500), factor loading (0.5, 0.8, 0.5/0.8), and item distributions (normal, uniform, skew, extremely skewed, bimodal, extremely bimodal) extending Green and Yang's (2009) study. Omega total coefficients were estimated using WLSMV, MR, and PA methods using polychoric correlation. Performance was evaluated using relative bias and Type-I error rates of the chi-square model test statistic. Results indicated that omega total estimates were close to population values when using the three factor methods, for all distributions except extreme skewness. However, type I error rates for model fit were lowest when WLSMV was used.