Polytomous items analysis with gpcm model on development of science process skills performance assessment in biology subject

Aulia Hermawati Ulfah; Supahar

doi:10.21009/biosferjpb.53403

Authors

Aulia Hermawati Ulfah Educational Research and Evaluation, Graduate School, Yogyakarta State University, Indonesia
Supahar Educational Research and Evaluation, Graduate School, Yogyakarta State University, Indonesia

DOI:

https://doi.org/10.21009/biosferjpb.53403

Keywords:

Instrument Analysis, Item Response Theory, Performance Assessment, Science Process Skills

Abstract

Assessment of science process skills can be conducted through tests or non-tests. However, observations, as one non-test technique, often face challenges related to space, time, and evaluator resources. A polytomous-type paper-and-pencil test was developed to address these issues. Item characteristic analysis is an essential aspect of instrument development; thus, this study reports the instrument analysis within the development of this assessment, using the item response theory approach. A total of 415 high school students in Sumedang completed 30 items depeloved. They came from four high schools selected based on the 2022 Asessmen Kompetensi Minimum (AKM) criteria. The test of 30 items had previously undergone successful content validity, construct validity, and reliability tests. Based on the item response theory assumptions, all items were eligible for analysis using this approach. Item fit testing revealed that the instrument suited the Generalized Partial Credit Model (GPCM). The analysis was performed using the R Program application. Results showed that all items (100%) were medium difficulty; 21 items (70%) had good discrimination and items (30%) exhibited high discrimination. 14 items were suggested for category reduction due to minimal delta differences or exceedingly low delta values. These findings emphasize item analysis to achieve high-quality instruments.

References

Agus Kurniawan, D., Putri Wirman, R., Wulan Dari, R., & Yuhanis, E. (2020). Description of student science process skills on temperature and heat practicum. Jurnal Penelitian Dan Evaluasi Pendidikan, 24(1), 88–101. https://doi.org/10.21831/pep

Allen, M. J., & Yen, W. M. (1979). Introduction to measurement theory. Cole Publishing Company.

Andriani, F., Indrowati, M., & Sugiharto, B. (2021). Analysis items of the four-tier immune system multiple choice test instrument using Rasch model. Biosfer, 14(1), 99–119. https://doi.org/10.21009/biosferjpb.18020

Anwar, F., Kanwal, S., Shabir, G., Alkharfy, K. M., & Gilani, A. H. (2015). Antioxidant and antimicrobial attributes of different solvent extracts from leaves of four species of mulberry. International journal of pharmacology, 11(7), 757-765. https://doi.org/10.3923/ijp.2015.757.765

Ardiyanti, D. (2016). Aplikasi model rasch pada pengembangan skala efikasi diri dalam pengambilan keputusan karier siswa. Jurnal Psikologi, 43(3), 248–263. https://doi.org/10.22146/jpsi.17801

Arjaya, I. B. A., & Suma, K. (2023). Problems of biology learning and evaluation analysis at the cipp model-based higher education level. Biosfer, 16(1), 152–167. https://doi.org/10.21009/biosferjpb.26835

Azwar, S. (2022). Reliabilitas dan validitas. Pustaka Belajar.

Baharom, M. M., Atan, N. A., Rosli, M. S., Yusof, S., & Hamid, M. Z. A. (2020). Integration of science learning apps based on Inquiry-Based Science Education (IBSE) in enhancing students' Science Process Skills (SPS). International Journal of Interactive Mobile Technologies, 14(9), 95–109. https://doi.org/10.3991/ijim.v14i09.11706

Bass, J. E., Contant, T. L., & Carin, A. A. (2009). Teaching science as inquiry (8th ed.). Allyn & Bacon.

Bichi, A. A., Ibrahim, R. H., & Ibrahim, F. B. (2019). Assessment of students' performances in biology: Implications for measurements and evaluation of learning. Journal of Education and Learning (EduLearn), 13(3), 301–308. https://doi.org/10.11591/edulearn.v13i3.12200

Biswal, S., & Behera, B. (2023). Enhancing science process skills through inquiry-based learning: a comprehensive literature review and analysis. International Journal of Science and Research (IJSR), 12(8), 1583–1589. https://doi.org/10.21275/sr23817121415

Branch, R. M. (2009). Instructional Design: The ADDIE Approach. Springer Science & Business Media.

Bryce, J., Toole, M. J., Waldman, R. J., & Voigt, A. N. N. (1992). Assessing the quality of facility-based child survival services. Health policy and planning, 7(2), 155-163.

Butler, A. C. (2018). Multiple-choice testing in education: are the best practices for assessment also good for learning? In Journal of Applied Research in Memory and Cognition (Vol. 7, Issue 3, pp. 323–331). Elsevier Inc. https://doi.org/10.1016/j.jarmac.2018.07.002

Chiappetta, E. L., & Koballa, T. R. (2010). Science instruction in the middle and secondary school (7th ed.). Pearson Education.

Chen, Z., Li, J., Wang, B., & Xu, B. (2025). Probabilistic normalization conditions of polytomous knowledge structures. Communications in Statistics-Theory and Methods, 54(15), 4877-4895. https://doi.org/10.1080/03610926.2024.2430735

de Ayala, R. J. (2009). The theory and practice of item response theory. The Guilford Press.

DeMars, C. (2010). Item response theory. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195377033.001.0001

Dogan, E. (2018). An application of the partial credit IRT model in identifying benchmarks for polytomous rating scale Instruments. Practical Assessment, Research & Evaluation, 23(7), 1–10. https://doi.org/10.7275/1cf3-aq56

Eaton, P., Johnson, K., Frank, B., & Willoughby, S. (2019). Classical test theory and item response theory comparison of the brief electricity and magnetism assessment and the conceptual survey of electricity and magnetism. Physical Review Physics Education Research, 15(1). https://doi.org/10.1103/PhysRevPhysEducRes.15.010102

Ekici, M., & Erdem, M. (2020). Developing science process skills through mobile scientific inquiry. Thinking Skills and Creativity, 36. https://doi.org/10.1016/j.tsc.2020.100658

Farach, N., Kartimi, & Mulyani, A. (2021). Application of performance assessment in STEM-based Biological learning to improve student’s science process skills. Journal of Physics: Conference Series, 1806(1). https://doi.org/10.1088/1742-6596/1806/1/012220

Faradillah, A., & Febriani, L. (2021). Mathematical trauma students’ junior high school based on grade and gender. Infinity Journal, 10(1), 53–68. https://doi.org/10.22460/infinity.v10i1.p53-68

Gizaw, G. G., & Sota, S. S. (2023). Improving science process skills of students: a review of literature. Science Education International, 34(3), 216–224. https://doi.org/10.33828/sei.v34.i3.5

Gorgun, G., & Bulut, O. (2021). A polytomous scoring approach to handle not-reached items in low-stakes assessments. Educational and Psychological Measurement, 81(5), 847–871. https://doi.org/10.1177/0013164421991211

Gruijter, D. N. de, & Kamp, L. J. T. van der. (2008). Statistical test theory for the behavioral sciences. Taylor & Francis Group.

Güler, B., & Şahin, M. (2019). Using inquiry-based experiments to improve pre-service science teachers’ science process skills. International Journal of Progressive Education, 15(5), 1–18. https://doi.org/10.29329/ijpe.2019.212.1

Hambleton, R. K., Swaminatan, H., & Rogers, J. H. (1991). Fundamental of item response theory. Sage Publications, Inc.

Harja, M., & Sinaga, P. (2021). Evaluation of science process skills of high school students in Tapaktuan City on static fluid material. Journal of Physics: Conference Series, 1806(1). https://doi.org/10.1088/1742-6596/1806/1/012016

Haryanto. (2020). Evaluasi pembelajaran (konsep dan manajemen). UNY Press.

Hikamudin, E. (2017). Estimasi kemampuan siswa dalam ujian nasional menggunakan metode bayes. Penelitian Kebijakan Pendidikan, 10(3). https://doi.org/10.24832/jpkp.v10i2.171

Ilma, S., Al-Muhdhar, M. H. I., Rohman, F., & Saptasari, M. (2020). The correlation between science process skills and biology cognitive learning outcome of senior high school students. JPBI (Jurnal Pendidikan Biologi Indonesia), 6(1), 55–64. https://doi.org/10.22219/jpbi.v6i1.10794

Im, R., Iwayama, T., & Osa, M. (2024). Assessing the science process skills of chemistry high school teachers: A case study in Cambodia. Universal Journal of Educational Research, 3(3), 235–244. https://doi.org/10.17613/z0c2-8v50

Irwanto, I. (2023). Improving preservice chemistry teachers’ critical thinking and science process skills using research-oriented collaborative inquiry learning. Journal of Technology and Science Education, 13(1), 23–35. https://doi.org/10.3926/jotse.1796

Istiyono, E. (2020). Pengembangan instrumen penilaian dan analisis hasil belajar Fisika dengan teori tes klasik dan modern (2nd ed.). UNY Press.

Kamarudin, N., Wahida, M., & Ahrari, S. (2022). Exploring basic and integrated science process skills and their impact on science achievement among university students. Journal of Public Administration and Governance, 12(4S), 74. https://doi.org/10.5296/jpag.v12i4s.20572

Karoror, I., & Jalmo, T. (2022). Profile of critical thinking ability in ecosystem materials using the Rasch model. Jurnal Penelitian Pendidikan IPA, 8(3), 1599–1604. https://doi.org/10.29303/jppipa.v8i3.1394

Kasuga, W., Maro, W., & Pangani, I. (2022). Effect of problem-based learning on developing science process skills and learning achievement on the topic of safety in our environment. Journal of Turkish Science Education, 19(3), 872–886. https://doi.org/10.36681/tused.2022.154

Khamhaengpol, A., Nokaew, T., & Chuamchaitrakool, P. (2024). Development of STEAM activity “Eco-Friendly Straw” based science learning kit to examine students’ basic science process skills. Thinking Skills and Creativity, 53. https://doi.org/10.1016/j.tsc.2024.101618

Kriswantoro, Kartowagiran, B., & Rohaeti, E. (2021). A critical thinking assessment model integrated with science process skills on chemistry for senior high school. European Journal of Educational Research, 10(1), 285–298. https://doi.org/10.12973/EU-JER.10.1.285

Kunandar. (2013). Penilaian autentik: Penilaian hasil belajar peserta didik Kurikulum 2013. RajaGrafindo Persada.

Li, X., Zhang, Y., Yu, F., Zhang, X., Zhao, X., & Pi, Z. (2024). Do science teachers’ believes related to inquiry-based teaching affect students’ science process skills? Evidence from a multilevel model analysis. Disciplinary and Interdisciplinary Science Education Research, 6(1). https://doi.org/10.1186/s43031-023-00089-y

Ma, H., Liu, W., & Li, G. (2025). Development and application of a five-tier diagnostic test to assess misconceptions on respiration and photosynthesis among senior high school students in Mainland China. Research in Science Education. https://doi.org/10.1007/s11165-025-10232-6

Mardapi, D. (2016). Pengukuran, penilaian, dan evaluasi pendidikan (2nd ed.). Parama publishing.

Martin, D. J. (2009). Elementary science methods a constructivist approach (5th ed.). Wadsworth Cengage Learning.

Minalisa, M., Festiyed, & Ratnawulan. (2019). The development of performance assessment of inquiry-based learning (IBL) to improve student’s science process skill of class XI Senior High School 1 Bayang. Journal of Physics: Conference Series, 1185(1). https://doi.org/10.1088/1742-6596/1185/1/012134

Mulyatiningsih, E. (2011). Riset Terapan Bidang Pendidikan dan Teknik. UNY Press.

Mursali, S., Sri Hastuti, U., Zubaidah, S., & Rohman, F. (2024). Guided inquiry with Moodle to improve students’ science process skills and conceptual understanding. International Journal of Evaluation and Research in Education, 13(3), 1875–1884. https://doi.org/10.11591/ijere.v13i3.27617

Mushani, M. (2021). Science process skills in science education of developed and developing countries: literature review. Unnes Science Education Journal, 10(1), 12–17. https://doi.org/10.15294/usej.v10i1.42153

Nurdini, Y., Wulan, A. R., & Diana, S. (2020, April). Assessment for learning through written feedback to develop 21st-century critical thinking skills on plantae learning. In Journal of Physics: Conference Series (Vol. 1521, No. 4, p. 042019). https://iopscience.iop.org/article/10.1088/1742-6596/1521/4/042019/meta

Paxinou, Evgenia., Kalles, Dimitrios., Panagiotakopoulos, Christos T., Sgourou, Argyro., & Verykios, Vassilios S. (2021). An IRT-based approach to assess the learning gain of a virtual reality lab students’ experience. Intelligent Decision Technologies, 15(3), 487–496. https://doi.org/10.3233/IDT-200216

Pongsophon, P., & Jituafua, A. (2021). Developing and assessing learning progression for botanical literacy using Rasch analysis. Science Education International, 32(2), 125–130. https://doi.org/10.33828/sei.v32.i2.5

Prasetya, A., Rosidin, U., & Herlina, K. (2019). Development of instrument assessment for learning the polytomous response models to train Higher-Order Thinking Skills (HOTS). Journal of Physics: Conference Series, 1155(1). https://doi.org/10.1088/1742-6596/1155/1/012032

Prasetya, W. A., & Pratama, A. T. (2023). Item quality analysis using the Rasch model to measure critical thinking ability in the material of the human digestive system of Biology subject in high school. Jurnal Penelitian Dan Evaluasi Pendidikan, 27(1), 76–91. https://doi.org/10.21831/pep.v27i1.58873

Purnamasari, U. D., & Kartowagiran, B. (2019). Application rasch model using R program in analyze the characteristics of chemical items. Jurnal Inovasi Pendidikan IPA, 5(2), 147–157. https://doi.org/10.21831/jipi.v5i2.24235

Putro, B. L., Waslaluddin, Putra, R. R. J., & Rahman, E. F. (2019). Creative learning model as implementation of curriculum 2013 to achieve 21st century skills. Journal of Physics: Conference Series, 1280(3), 1–7. https://doi.org/10.1088/1742-6596/1280/3/032034

Reeve, B. B., & Masse, L. C. (2004). Item response theory (IRT) modeling for questionnaire evaluation. In methods for testing and evaluating survey questionnaires, ed. Stanley Presser, Jennifer M. Rothgeb, Mick P. Couper, Judith L. Lessler, Elizabeth Martin, Jean Martin, and Eleanor Singer. John Wiley & Sons.

Reise, S. P. (1982). A comparison of item-and person-fit methods of assessing model-data fit in IRT. Applied Psychological Measurement Inc, 14(2), 127–137. https://doi.org/10.1177/014662169001400202

Retnawati, H. (2014). Teori respon butir dan penerapannya. www.nuhamedika.gu.ma

Retnawati, H., Munadi, S., Arlinwibowo, J., Wulandari, N. F., & Sulistyaningsih, E. (2017). Teachers’ difficulties in implementing thematic teaching and learning in elementary schools. New Educational Review, 48(2), 201–212. https://doi.org/10.15804/tner.2017.48.2.16

Rezba, R. J., Sprague, C. R., McDonnough, J. T., & Matkins, J. J. (1995). Learning and assessing science process skills (3rd ed.). Hunt Publishing Company.

Rifatul Mahmudah, I., Makiyah, Y. S., & Sulistyaningsih, D. (2019). Profil keterampilan proses sains (KPS) siswa SMA di Kota Bandung. Diffraction, 1(1), 39–43. https://doi.org/10.37058/diffraction.v1i1.808

Sahin, M. G., & Yildirim, Y. (2018). The examination of item difficulty distribution, test length and sample size in different ability distribution. Journal of Measurement and Evaluation in Education and Psychology, 9(3), 277–294. https://doi.org/10.21031/epod.385000

Samritin, S., & Suryanto, S. (2016). Developing an assessment instrument of junior high school students’ higher order thinking skills in mathematics. Research and Evaluation in Education, 2(1), 92. https://doi.org/10.21831/reid.v2i1.8268

Sarah, Wulan, A. R., & Kusnadi, M. (2020). Needs analysis on developing instruments of interpreting data skills and scientific evidence for biology learning in the 21st century era. ACM International Conference Proceeding Series, 188–192. https://doi.org/10.1145/3416797.3416812

Sarea, M. S., & Ruslan, R. (2019). Karakteristik butir soal: classical test theory vs item response theory? Didaktika Jurnal Kependidikan, 13(1), 1–16. https://doi.org/10.30863/didaktika.v13i1.296

Sari, U., Duygu, E., Şen, Ö. F., & Kirindi, T. (2020). The effects of STEM education on scientific process skills and STEM awareness in simulation-based inquiry learning environment. Journal of Turkish Science Education, 17(3), 387–405. https://doi.org/10.36681/tused.2020.34

Şen, C., & Vekli, G. S. (2016). The impact of inquiry-based instruction on science process skills and self-efficacy perceptions of pre-service science teachers at a university-level biology laboratory. Universal Journal of Educational Research, 4(3), 603–612. https://doi.org/10.13189/ujer.2016.040319

Sholihah, N. A. A., Sarwanto, & Aminah, N. S. (2020). Analysis of science process skill in high school students. Journal of Physics: Conference Series, 1567(3). https://doi.org/10.1088/1742-6596/1567/3/032081

Siahaan, K. W. A., Lumbangaol, S. T. P., Marbun, J., Nainggolan, A. D., Ritonga, J. M., & Barus, D. P. (2020). Pengaruh model pembelajaran inkuiri terbimbing dengan multi representasi terhadap keterampilan proses sains dan penguasaan konsep IPA. Jurnal Basicedu, 5(1), 195–205. https://doi.org/10.31004/basicedu.v5i1.614

Siswono, H. (2017). Analisis pengaruh keterampilan proses Sains terhadap penguasaan konsep fisika siswa. Momentum: Physics Education Journal, 1(2), 83–90. https://doi.org/10.21067/mpej.v1i2.1967

Slepkov, A. D., Van Bussel, M. L., Fitze, K. M., & Burr, W. S. (2021). A baseline for multiple-choice testing in the university classroom. SAGE Open, 11(2). https://doi.org/10.1177/21582440211016838

Soland, J. (2024). Item Response Theory models for difference-in-difference estimates (and whether they are worth the trouble). Journal of Research on Educational Effectiveness, 17(2), 391–421. https://doi.org/10.1080/19345747.2023.2195413

Srirahayu, R. Y., & Arty, I. S. (2019). Development of experiment performance assessment instruments using guided inquiry learning models to assess science process skills. Journal of Physics: Conference Series, 1233(1). https://doi.org/10.1088/1742-6596/1233/1/012075

Subali, B. (2009). Pengembangan tes pengukuran keterampilan proses sains pola divergen mata pelajaran Biologi SMA. Proseding Seminar Nasional Biologi, Lingkungan Dan Pembelajarannya, 581–593.

Supahar, S., & Prasetyo, Z. K. (2015). Pengembangan instrumen penilaian kinerja kemampuan inkuiri peserta didik pada mata pelajaran Fisika SMA. Jurnal Penelitian Dan Evaluasi Pendidikan, 19(1), 96–108. https://doi.org/10.21831/pep.v19i1.4560

Thomas, M. L. (2019). Advances in applications of item response theory to clinical assessment. Psychological Assessment, 31(12), 1442–1455. https://doi.org/10.1037/pas0000597

Tu, D., Gao, X., Wang, D., & Cai, Y. (2017). A new measurement of internet addiction using diagnostic classification models. Frontiers in psychology, 8, 1768. https://doi.org/10.3389/fpsyg.2017.01768

van der Linden, W. J., & Hambleton, R. K. (1997). Handbook of modern item response theory. In Handbook of Modern Item Response Theory. Springer New York. https://doi.org/10.1007/978-1-4757-2691-6

Vo, D. Van, & Simmie, G. M. (2024). Assessing scientific inquiry: A systematic literature review of tasks, tools and techniques. International Journal of Science and Mathematics Education, 23, 871–906. https://doi.org/10.1007/s10763-024-10498-8

Williams-McBean, C. (2025). Factors influencing teachers’ choice and use of assessment. International Journal of Studies in Education and Science, 6(2), 212–239. https://doi.org/10.46328/ijses.129

Yunita, N., & Nurita, T. (2021). Analisis keterampilan proses sains siswa pada pembelajaran daring. Pensa E-Jurnal: Pendidikan Sains, 9(3), 378–385. https://ejournal.unesa.ac.id/index.php/pensa

Zainul, A., & Nasoetion, , Noehi. (1997). Penilaian hasil belajar. Pusat Antar Universitas, Direktorat Jenderal Pendidikan Tinggi: Departemen Pendidikan Dan kebudayaan.

Zi Yan, T. G. B., & Heene, M. (2021). Applying the Rasch model fundamental measurement in the human sciences (4th ed.). Routledge.

Ziraluo, Y. P. B. (2021). Pembelajaran Biologi implementasi dan pengembangan. Forum Pemuda Aswaja.

Zuhera, Y., & Habibah, S. (2017). Kendala guru dalam memberikan penilaian terhadap sikap siswa dalam proses pembelajaran berdasarkan kurikulum 2013 di SD Negeri 14 Banda Aceh. Jurnal Ilmiah Pendidikan Guru Sekolah Dasar, 2(1), 73–87. https://media.neliti.com/media/publications/187406-ID-kendala-guru-dalam-memberikan-penilaian.pdf