Psychometric validation of rasch model for critical reasoning on virus material in high school biology

Nani Aprilia; Hani Irawati; Inrayati Atjo

doi:10.21009/biosferjpb.67792

Authors

Nani Aprilia Biology Education, Faculty of Teacher Training and Education, Ahmad Dahlan University, Indonesia
Hani Irawati Biology Education, Faculty of Teacher Training and Education, Ahmad Dahlan University, Indonesia
Inrayati Atjo Student Methodology and Technology of professional education, Faculty Pedagogic, Far Eastern Federal University, Rusia.

DOI:

https://doi.org/10.21009/biosferjpb.67792

Keywords:

Deep learning, Higher order thinking, Partial credit model, Viruses

Abstract

Critical reasoning assessment is crucial in supporting deep learning at the high school level, especially in viruses. However, valid and reliable Rasch-based assessment instruments are still very limited. This study aims to validate a critical reasoning assessment instrument based on the Partial Credit Model (PCM) Rasch on viruses in high school biology. The research used a research and development approach with the Plomp model involving 105 Class X students of state high schools in Banguntapan district through purposive sampling.The instrument consisted of 15 polytomous items (scale 0–3) covering four aspects of critical reasoning. Data analysis was conducted using Winsteps software with a focus on item fit (Infit/Outfit MNSQ), difficulty parameters (eta & beta), threshold, separation reliability, and Wright Map. The analysis results showed a separation reliability of 0.7677 (fairly good category). Most items had good variation in difficulty levels, although many items showed disordered threshold patterns. Items P13, P14, and P15 were the most difficult items, while P6, P7, and P8 were the easiest. The findings showed that the instrument was able to differentiate student abilities but still needed revision in the response categories. Recommendations on instrument improvements through simplification of response categories and item writing are needed before widespread use. The instrument has the potential to be a formative diagnostic tool that supports an in-depth learning approach in Biology learning.

References

Aggraini, M. D., Koto, I., & Gunawan, B. (2024). The Effect of Problem-Based Learning Integrated Numbered Head Together on the Critical Thinking Ability of Grade XI Students. Kasuari: Physics Education …. https://journalfkipunipa.org/index.php/kpej/article/view/780

Akker, J. van den, Bannan, B., Kelly, A. E., Nieveen, N., & Plomp, T. (2013). Educational Design Research Part A: An introduction. In T. Plomp & N. Nieveen (Eds.), Educational design research. Part A : An introduction (1st ed.). Netherlands Institute for Curriculum Development (SLO), Enschede, the Netherlands.

Alnahdi, A. H., Alsubiheen, A. M., & Aldaihan, M. M. (2025). Rasch Measurement Model Supports the Unidimensionality and Internal Structure of the Arabic Oswestry Disability Index. Journal of Clinical Medicine, 14(4), 1259. https://doi.org/10.3390/jcm14041259

Aristizábal Gómez, Y. M., Jiménez Sierra, Á. A., & Ortega Iglesias, J. M. (2025). Students’ Emotions Toward Assessments: A Systematic Review. Social Sciences, 14(11), 652. https://doi.org/10.3390/socsci14110652

Arjana, I. M., Parmiti, D. P., Candiasa, I. M., & Widiartini, N. K. (2025). Unlocking the predictive power of the Rasch model: A systematic literature review on educational instrument calibration and assessment accuracy. Edelweiss Applied Science and Technology, 9(6), 2003–2022. https://doi.org/10.55214/25768484.v9i6.8310

Boone, W. J., & Staver, J. R. (2020). Advances in Rasch Analyses in the Human Sciences. In Advances in Rasch Analyses in the Human Sciences. Springer International Publishing. https://doi.org/10.1007/978-3-030-43420-5

BSKP. (2022). Panduan Pembelajaran dan Asesmen Pendidikan Anak Usia Dini pada Kurikulum Merdeka. In Kemendikbudristek (p. 123). Kepala Badan Standar, Kurikulum, dan Asesmen Pendidikan Kementerian Pendidikan Kementerian Pendidikan, Kebudayaan, Riset, dan Teknologi. https://repositori.kemendikdasmen.go.id/33290/

Calma, A., & Davies, M. (2025). Assessing students’ critical thinking abilities via a systematic evaluation of essays. Studies in Higher Education, 51(2), 422–437. https://doi.org/10.1080/03075079.2025.2470969

Davis, D. R., & Boone, W. (2021). Using Rasch analysis to evaluate the psychometric functioning of the other-directed, lighthearted, intellectual, and whimsical (OLIW) adult playfulness scale. International Journal of Educational Research Open, 2, 100054. https://doi.org/10.1016/j.ijedro.2021.100054

Eneng Sururiyatul Mu’aziyah, S., Edy Nurtamam, M., & Nurmegawati, L. (2025). Mapping secondary school students’ botanical literacy using rasch analysis to support deep learning in science. EPJ Web of Conferences, 344, 01038. https://doi.org/10.1051/epjconf/202534401038

Fitrah, M., Sofroniou, A., Ofianto, Judijanto, L., & Widihastuti. (2024). Reliability and separation index analysis of mathematics questions integrated with the cultural architecture framework using the Rasch model. Journal of Education and E-Learning Research, 11(3), 499–509. https://doi.org/10.20448/jeelr.v11i3.5861

Fu, R., & Koh, K. (2026). Developing and validating teacher formative assessment literacy questionnaire in the high-stakes examination culture: a case of China. Frontiers in Psychology, 17. https://doi.org/10.3389/fpsyg.2026.1771941

Golden, B. (2023). Enabling critical thinking development in higher education through the use of a structured planning tool. Irish Educational Studies, 42(4), 949–969. https://doi.org/10.1080/03323315.2023.2258497

Heim, A. B., Esparza, D., Holmes, N. G., & Smith, M. K. (2023). Comparing study features is easy but identifying next steps is hard: Evaluating critical thinking through the Biology Lab Inventory of Critical Thinking in Ecology. Ecology and Evolution, 13(5). https://doi.org/10.1002/ece3.10071

Hidayati, N., Suryanti, S., & Cahaya, N. (2023). The Potential for the Development of Critical Thinking Tests : An Overview of Educators’ Perceptions. Jurnal Kependidikan: Jurnal Hasil Penelitian Dan Kajian Kepustakaan Di Bidang Pendidikan, Pengajaran Dan Pembelajaran, 9(1), 56. https://doi.org/10.33394/jk.v9i1.6794

Hu, X., & Bi, H. (2025). Exploring and validating the componential model of students’ scientific critical thinking in science education. Thinking Skills and Creativity, 55, 101695. https://doi.org/10.1016/j.tsc.2024.101695

Ibnu Fitrianto, & Al Mukaromah Hidayat. (2024). Critical Reasoning Skills: Designing an Education Curriculum Relevant to Social and Economic Needs. International Journal of Post Axial: Futuristic Teaching and Learning, 245–258. https://doi.org/10.59944/postaxial.v2i4.393

Ismail, S. M., Rahul, D. R., Patra, I., & Rezvani, E. (2022). Formative vs. summative assessment: impacts on academic motivation, attitude toward learning, test anxiety, and self-regulation skill. Language Testing in Asia, 12(1), 40. https://doi.org/10.1186/s40468-022-00191-4

Kassiavera, S., Suparmi, A., Cari, C., & Sukarmin, S. (2024). Application of Rasch Model in Two-Tier Test for Assessing Critical Thinking in Physics Education. Journal of Baltic Science Education, 23(6), 1227–1242. https://doi.org/10.33225/jbse/24.23.1227

Khan, M. S. H., Salele, N., Hasan, M., & Abdou, B. O. (2023). Factors affecting student readiness towards OBE implementation in engineering education: Evidence from a developing country. Heliyon, 9(10), e20905. https://doi.org/10.1016/j.heliyon.2023.e20905

Kusurkar, R. A., Orsini, C., Somra, S., Artino, A. R., Daelmans, H. E. M., Schoonmade, L. J., & Van Der Vleuten, C. (2023). The Effect of Assessments on Student Motivation for Learning and Its Outcomes in Health Professions Education: A Review and Realist Synthesis. Academic Medicine, 98(9), 1083–1092. https://doi.org/10.1097/ACM.0000000000005263

Kwon, J., Raghunandan, R., Nghiem, S. H., Howard, K., Lancsar, E., Huynh, E., Howell, M., Petrou, S., & Smith, S. (2025). Rasch analysis of the self-reported PedsQL TM 4 . 0 Generic Core Scales by Australian children. 4.

Laliyo, L. A. R., Sumintono, B., & Panigoro, C. (2022). Measuring changes in hydrolysis concept of students taught by inquiry model: stacking and racking analysis techniques in Rasch model. Heliyon, 8(3), e09126. https://doi.org/10.1016/j.heliyon.2022.e09126

Mallinson, T., Kozlowski, A. J., Johnston, M. V., Weaver, J., Terhorst, L., Grampurohit, N., Juengst, S., Ehrlich-Jones, L., Heinemann, A. W., Melvin, J., Sood, P., & Van de Winckel, A. (2022). Rasch Reporting Guideline for Rehabilitation Research (RULER): the RULER Statement. Archives of Physical Medicine and Rehabilitation, 103(7), 1477–1486. https://doi.org/10.1016/j.apmr.2022.03.013

Micoloi, M., Ivanjek, L., De Cock, M., Schubatzky, T., Wildbichler, S., Wackermann, R., & Pospiech, G. (2025). Development of a multiple-choice test to assess critical thinking in climate change. Physical Review Physics Education Research, 21(2), 020154. https://doi.org/10.1103/jnbq-1qyk

Pat-El, R. J., de Hoog, N., Segers, M., & Vedder, P. (2024). Exploring the impact of student perceptions of Assessment for Learning on intrinsic motivation. Studies in Educational Evaluation, 83. https://doi.org/10.1016/j.stueduc.2024.101420

Prasetya, W. A., & Pratama, A. T. (2023). Item quality analysis using the Rasch model to measure critical thinking ability in the material of the human digestive system of Biology subject in high school. Jurnal Penelitian Dan Evaluasi Pendidikan, 27(1), 76–91. https://doi.org/10.21831/pep.v27i1.58873

Retnawati, H., Hadi, S., & Nugraha, A. C. (2016). Vocational high school teachers’ difficulties in implementing the assessment in curriculum 2013 in Yogyakarta Province of Indonesia. International Journal of Instruction, 9(1), 33–48. https://doi.org/10.12973/iji.2016.914a

Rodríguez Rincón, Y., Munárriz, A., & Magreñán Ruiz, A. (2024). A new approach to continuous assessment: Moving from a stressful sum of grades to meaningful learning through self-reflection. International Journal of Management Education, 22(3), 101072. https://doi.org/10.1016/j.ijme.2024.101072

Romine, W. L., Schaffer, D. L., & Barrow, L. (2015). Development and Application of a Novel Rasch-based Methodology for Evaluating Multi-Tiered Assessment Instruments: Validation and utilization of an undergraduate diagnostic test of the water cycle. International Journal of Science Education, 37(16), 2740–2768. https://doi.org/10.1080/09500693.2015.1105398

Sari, T. N. I., & Rakhmawati, A. (2024). Analysis the quality of critical thinking and creativity questions in high school biology subjects with the Rasch model. Research and Development in Education (RaDEn), 4(1), 602–616. https://doi.org/10.22219/raden.v4i1.32758

Sartono, B., Sunarno, W., Prayitno, B. A., & Indriyanti, N. Y. (2025). Development and Validation of A Critical Thinking Assessment on Temperature and Heat for Secondary Physics Education. Educational Process: International Journal, 16(1). https://doi.org/10.22521/edupij.2025.16.264

Setiowati, W., Hariyadi, S., & Prihandono, T. (2025). Enhancing student piety and critical reasoning through steamrel-PBL with authentic e-assessment in ecology learning. Biosfer, 18(2), 1–13. https://doi.org/10.21009/biosferjpb.50319

Siswanto, D. H., & Peni, N. R. N. (2023). Publication Trend on the Plomp Development Model in Mathematics Education. Asian Pendidikan, 3, 71–80. https://doi.org/10.53797/aspen.v3i2.9.2023

Sujatmika, S., Sutarno, Masykuri, M., & Prayitno, B. A. (2025). Applying the Rasch model to measure students’ critical thinking skills on the science topic of the human circulatory system. Eurasia Journal of Mathematics, Science and Technology Education, 21(4), em2622. https://doi.org/10.29333/ejmste/16221

Sutiarso, S., Rosidin, U., & Sulistiawan, A. (2022). Developing Assessment Instrument Using Polytomous Response in Mathematics. European Journal of Educational Research, 11(3), 1441–1462. https://doi.org/10.12973/eu-jer.11.3.1441

Suwita, S., Saputro, S., Sajidan, S., & Sutarno, S. (2024). Assessing Lower-Secondary School Students’ Critical Thinking Skills in Photosynthesis: a Rasch Model Approach. Journal of Baltic Science Education, 23(6), 1278–1290. https://doi.org/10.33225/jbse/24.23.1278

Tavares, R., Vieira, R. M., & Pedro, L. (2021). Mobile app for science education: Designing the learning approach. Education Sciences, 11(2), 1–23. https://doi.org/10.3390/educsci11020079

Tesio, L., Caronni, A., Kumbhare, D., & Scarano, S. (2024). Interpreting results from Rasch analysis 1. The “most likely” measures coming from the model. Disability and Rehabilitation, 46(3), 591–603. https://doi.org/10.1080/09638288.2023.2169771

Thalhah, S. Z., Tahrim, T., & Sari, R. E. (2022). Development of Lecturer Performance Assessment Instruments through the Plomp Model in Higher Education. AL-ISHLAH: Jurnal Pendidikan, 14(2), 1527–1540. https://doi.org/10.35445/alishlah.v14i2.1350

Usman, U., Repelita, R., Setiawan, R., & Hidayat, S. (2024). Differentiated Learning and Critical Thinking Skills Development in Biology: A Systematic Literature Review. Indonesian Journal of Multidiciplinary Research, 4(2), 447–454. https://doi.org/10.17509/ijomr.v4i2.78729

Vlachopoulos, D., & Makri, A. (2024). A systematic literature review on authentic assessment in higher education: Best practices for the development of 21st century skills, and policy considerations. Studies in Educational Evaluation, 83. https://doi.org/10.1016/j.stueduc.2024.101425

Wahyuni, V., Ratnawulan, & Mufit, F. (2025). HOTS Assessment Development to Assess High School Physics Problem-Solving Skills. Jurnal Penelitian Pendidikan IPA, 11(10), 536–546. https://doi.org/10.29303/jppipa.v11i10.12880

Wetzel, E., & Carstensen, C. H. (2014). Reversed Thresholds in Partial Credit Models: A Reason for Collapsing Categories? Assessment, 21(6), 765–774. https://doi.org/10.1177/1073191114530775

Wind, S. A. (2023). Detecting Rating Scale Malfunctioning With the Partial Credit Model and Generalized Partial Credit Model. Educational and Psychological Measurement, 83(5), 953–983. https://doi.org/10.1177/00131644221116292

Wuisan, A. C., Simamora, M., & Siboro, M. (2024). Efektifitas Problem Based Learning Terhadap Kemampuan Berpikir Kritis Siswa [The Effectiveness of Problem Based Learning on Students’ Critical Thinking Skills]. Jurnal Teropong Pendidikan. http://ojs.uph.edu/index.php/JTP/article/view/8638

Yang, J., Reheman, Z., Liu, Y., & Zhao, S. (2023). Applying Rasch analysis in refinement and validation of interpersonal skills measure for gifted children. Frontiers in Psychology, 14. https://doi.org/10.3389/fpsyg.2023.1236640

Yokhebed, Karmadi, R. M. D., Silaen, J., Meidina, S., & Nikmah, N. A. (2025). Analysis of the Quality of Sudents’ Critical Reasoning Instruments Using Rasch Model. Jurnal Penelitian Pendidikan IPA, 11(3), 54–60. https://doi.org/10.29303/jppipa.v11i3.9584

Yokhebed, Y., Karmadi, R. M. D., & Nastiti, L. R. (2025). Validity and reliability analysis of a socioscientific issues-based critical thinking self-assessment instrument using the Rasch model. JPBI (Jurnal Pendidikan Biologi Indonesia), 11(1), 73–82. https://doi.org/10.22219/jpbi.v11i1.38902

Zhao, H., Yu, J., & Li, G. (2026). Developing a process-oriented classroom observation protocol for assessing high school students’ computational thinking in science classrooms: a Rasch-based proficiency framework. Frontiers in Psychology, 17. https://doi.org/10.3389/fpsyg.2026.1722422