Authors: Hong Jiao; Manqian Liao; Peida Zhan

Introduction

A joint modeling approach of response and response times is often used to understand the trade-off between accuracy and speed (e.g., Thissen, 1983; van der Linden, 2007; van der Linden, Klein Entink, & Fox, 2010). As it is observed in real data analyses, the relationship between accuracy and speed is not invariant across diverse groups of test-takers. Some studies reported positive relation while other studies reported negative correlation between accuracy and speed. Some other studies even observed a curvilinear pattern between the two variables. Such differential or conditional differences have been explored in several studies (e.g., Bolsinova, De Boeck, & Tijmstra, 2016; Bolsinova, Tijmstra, & Molenaar, 2017; Fox & Marianti, 2016; Jiao, Zhan, Liao, & Man, 2017; Meng, Tao, & Chang, 2015; Molenaar, Bolsinova, Rozsa, & De Boeck, 2016; Molenaar, Bolsinova, & Vermunt, 2016; Wang & Xu, 2015) in the joint modeling of responses and response times. Recently researchers (e.g., Klein Entink, Kuhn, Hornke, & Fox, 2009; Minchen, 2017; Minchen & de la Torre, 2016; Zhan, Jiao, & Liao, 2017) have extended joint modeling of responses and response times for cognitive diagnosis.

Objective

However, all the studies for cognitive diagnosis utilizing both responses and response time often assume invariance of model parameters across sub-groups of the examinee population. Thus, this study proposes a multigroup modeling approach jointly for responses and response times which simultaneously take into account of differential item functioning and differential time functioning in cognitive diagnostic modeling.

Design/Methodology

The proposed model is a combination of the hierarchical joint model of response and response time (van der Linden, 2007) and a higher-order cognitive diagnostic model (de la Torre & Douglas, 2004) with multigroup structures embedded in both responses and response time. The proposed joint model will be first fitted with the real data from a large-scale testing program with students from multiple populations. Several alternative joint models will be compared to better understand the nature of the real response and response time data. A follow-up simulation study will mimic the response and response time structures obtained from the real data analysis to investigate item parameter recovery, cognitive diagnosis accuracy and differential item and time functioning.

Results/Conclusion

This study is in progress. The results will be ready in June 2018.

References

Bolsinova, M., De Boeck, P., & Tijmstra, J. (2016). Modelling conditional dependence between response time and accuracy. Psychometrika, 1-23. doi: 10.1007/s11336-016-9537-6.

Bolsinova, M., Tijmstra, J., & Molenaar, D. (2017). Response moderation models for conditional dependence between response time and response accuracy. British Journal of Mathematical and Statistical Psychology, 70(2), 257-279.

de la Torre, J., & Douglas, J. A. (2004). Higher-order latent trait models for cognitive diagnosis.

Psychometrika, 69(3), 333–353.

Fox, J. P., & Marianti, S. (2016). Joint modeling of ability and differential speed using responses and response times. Multivariate Behavioral Research, 51(4), 540-553.

Jiao, H., Zhan, P., Liao, M., & Man, K. (2017, November). A joint multigroup testlet model for

responses and response time accounting for differential item and speed functioning. Presented at the fifth conference on the statistical methods in psychometrics. Columbia University, New York.

Klein Entink, R. H., Kuhn, J. T., Hornke, L. F., & Fox, J. P. (2009). Evaluating cognitive theory: a joint modeling approach using responses and response times. Psychological Methods, 14(1), 54-75.

Meng, X. B., Tao, J., & Chang, H. H. (2015). A conditional joint modeling approach for locally dependent item responses and response times. Journal of Educational Measurement, 52(1), 1-27.

Minchen, N. (2017). Continuous response in cognitive diagnosis models: Response time

modeling, computerized adaptive testing, and Q-matrix validation (Unpublished doctoral dissertation). Rutgers, The State University of New Jersey.

Minchen, N. D., & de la Torre, J. (2016, April). Using response time in cognitive diagnosis

models. Poster presented at the annual meeting of the National Council on Measurement in Education, Washington, DC.

Molenaar, D., Bolsinova, M., Rozsa, S., & De Boeck, P. (2016). Response mixture modeling of intraindividual differences in responses and response times to the Hungarian WISC-IV Block Design test. Journal of Intelligence, 4(3), 10-29.

Molenaar, D., Bolsinova, M., & Vermunt, J. K. (2016). A semi‐parametric within‐subject mixture approach to the analyses of responses and response times. Retrieved from http://members.home.nl/jeroenvermunt/molenaar2016.pdf.

Thissen, D. (1983). Timed testing: An approach using item response theory. In New horizons in testing

(pp. 179–203). Elsevier.

van der Linden, W. J. (2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika, 72(3), 287–308.

van der Linden, W. J., Klein Entink, R. H., & Fox, J.-P. (2010). IRT parameter estimation with response

times as collateral information. Applied Psychological Measurement, 34(5), 327–347.

Wang, C., & Xu, G. (2015). A mixture hierarchical model for response times and response accuracy. British Journal of Mathematical and Statistical Psychology, 68(3), 456-477.

Zhan, P., Jiao, H., & Liao, D. (2017). Cognitive diagnosis modelling incorporating item response times.

British Journal of Mathematical and Statistical Psychology.

11th International Test Commission Conference - Montréal 2018

Event Schedule

There are no sessions available on this day

There are no sessions available on this day

There are no sessions available on this day

There are no sessions available on this day