Development and Validation of a Higher-Order Thinking Skills-Based Test on Citizen Participation in Consumer Education Using Item Response Theory

Abstract

This study aims to develop and validate a high-order thinking skills (HOTS)-based multiple-choice instrument to assess citizen participation in consumer education. Guided by Bannister and Monsma's framework, the instrument was developed through expert reviews and tested empirically using the Rasch model. The content validity, assessed by Aiken's V, CVR, and I-CVI, showed high expert agreement. Empirical validation involving 250 university students indicated excellent item reliability (0.99), acceptable person reliability (0.79), and appropriate item fit. All 24 items analyzed fell within acceptable MNSQ ranges and showed positive point-measure correlations. While item difficulties ranged from –2.23 to +1.87 logits, most were easier for the sample, suggesting potential refinement for high-ability students. The findings support the instrument's validity and reliability for assessing Higher Order Thinking Skills (HOTS) in citizen participation, providing a robust tool for assessing transformative consumer education.