Asamoah, N.A., Turner, R.C., Lo, W., Crawford, B.L., & Jozkowski, K.N. (2022, July 11 - July 15) Evaluating the Rasch Tree Method for Balanced and Unbalanced DIF. International Meeting of the Psychometric Society (IPMS) annual meeting, Bologna, Italy.
The Rasch tree, a differential item functioning (DIF) detection approach, recursively tests all groups that can be defined based on combinations of available grouping variables to identify groups that have different item difficulty parameters. An advantage of the method is that subgroups do not have to be pre-specified. However, as a global DIF detection method, items responsible for DIF are not automatically flagged. A joint Rasch model is fit for all groups and if significant instabilities are detected based on a grouping variable, the sample is split by that variable. A new unpublished study incorporates Mantel-Haenszel effect sizes for identifying DIF items in Rasch tree analyses (Henninger et al., 2022). The simulations compare DIF identification when using statistical testing versus effect sizes as stopping criteria. They further demonstrate the use of effect sizes in identifying DIF magnitude of items.
We build on these prior studies by comparing true and false DIF detection rates using Rasch trees (implementing MH effect size criteria) with previously unstudied conditions. Prior simulations have been restricted to items favoring one group. We are extending conditions to include balanced and unbalanced DIF item proportions that occur in real-world data. Simulation conditions include test length (10, 20 items); sample size (400, 800, 1200); difference in difficulty parameters of selected items (0, 0.21, 0.43, 0.64, 0.85, 1.06); percentage of items with DIF (10%, 20% and 30%); and type of DIF (balanced and unbalanced). The results will provide further guidelines on the use of Rasch trees in empirical DIF studies.