Development and validation of the critical consciousness scale of environmental issues (CCS-EI): A RASCH model analysis

Supriyadi Supriyadi; Een Yayah Haenilah; Dina Maulina

doi:10.21009/biosferjpb.49559

Authors

Supriyadi Supriyadi Doctoral Program in Education, Faculty of Teacher Training and Education, University of Lampung, Indonesia
Een Yayah Haenilah Doctoral Program in Education, Faculty of Teacher Training and Education, University of Lampung, Indonesia
Dina Maulina Doctoral Program in Education, Faculty of Teacher Training and Education, University of Lampung, Indonesia

DOI:

https://doi.org/10.21009/biosferjpb.49559

Keywords:

Critical consciousness, Environmental issues, RASCH analysis

Abstract

The present study aimed to develop and validate the Indonesian-language version of the Critical Consciousness Scale of Environmental Issues (CCS-EI), specifically for higher education contexts. Employing Rasch analysis, the study rigorously examines the instrument’s reliability and validity through item fit statistics, rating scale functionality, and differential item functioning. The CCS-EI is designed to measure students' critical consciousness in environmental contexts, encompassing dimensions of critical reflection, critical motivation, and critical action. A diverse sample of 579 (male 87%; female 13%) participants from various university in Indonesia completed a 34-item of the CCS-EI. The sampling technique used is convenience sampling. Data were analysed using the Rasch model analysis, it was performed using the Winsteps version 5.6.1.0 software. The results of the Rasch analysis indicated that the reliability of the instrument was excellent (a = 0.91), item quality was excellent (1.00), and person reliability was consistent (0.91). In the validity aspect, all three domains of the CCS-EI exhibited unidimensionality, and a rating scale with four answer choices was deemed appropriate. The study also identifies item difficulty level in each dimension. Furthermore, the CCS-EI showed strong convergent validity, correlating significantly with existing measures of critical reflection, motivation, and action concerning environmental issues. This positions the CCS-EI as a reliable and valid instrument for use in various fields. Its ability to quantify critical consciousness provides valuable insights for designing educational evaluation and shaping policies that promote deeper engagement with environmental sustainability, contributing to both theoretical and practical advancements in the field.

References

Adams, D., Sumintono, B., Mohamed, A., & Mohamad Noor, N. S. (2018). E-Learning Readiness among Students of Diverse Backgrounds in a Leading Malaysian Higher Education Institution. Malaysian Journal of Learning and Instruction, 15(Number 2), 227–256. https://doi.org/10.32890/mjli2018.15.2.9

Aldana, A., Bañales, J., & Richards-Schuster, K. (2019). Youth Anti-Racist Engagement: Conceptualization, Development, and Validation of an Anti-Racism Action Scale. Adolescent Research Review, 4(4), 369–381. https://doi.org/10.1007/s40894-019-00113-1

Anderson, A. J., Baggett, H. C., Andrzejewski, C. E., & Forbes, S. A. (2021). “Why Don’t They Just Move Closer?”: Adolescent Critical Consciousness Development in YPAR About Food Security. Journal of Adolescent Research, 074355842110653. https://doi.org/10.1177/07435584211065343

Andrich, D. (2017). Advances in social measurement: A Rasch measurement theory. In F. Guillemin, A. Leplege, S. Briancon, E. Spitz, & J. Coste (Eds.), Perceived Health and Adaptation in Chronic Disease (1st ed., pp. 66–91). Routledge. https://doi.org/10.1201/9781315155074-7

Autin, K. L., Williams, T. R., Allan, B. A., & Herdt, M. E. (2022). Decent Work Among People of Color: The Moderating Role of Critical Consciousness. Journal of Career Assessment, 30(3), 455–473. https://doi.org/10.1177/10690727211039811

Azizan, N. H., Mahmud, Z., & Rambli, A. (2020). Rasch Rating Scale Item Estimates using Maximum Likelihood Approach: Effects of Sample Size on the Accuracy and Bias of the Estimates. International Journal of Advanced Science and Technology, 29(4). http://sersc.org/journals/index.php/IJAST/article/view/20791

Bond, T. G., Yan, Z., & Heene, M. (2020). Applying the Rasch model: Fundamental measurement in the human sciences (Fourth edition). Routledge. https://doi.org/10.4324/9780429030499

Byrd, C. M. (2017). The complexity of school racial climate: Reliability and validity of a new measure for secondary students. British Journal of Educational Psychology, 87(4), 700–721. https://doi.org/10.1111/bjep.12179

Chan, R. C. H. (2022). Development and validation of the Critical Reflection Scale for youth in China: Factor structure and measurement invariance across age, gender, and sexual orientation. Applied Developmental Science, 1–13. https://doi.org/10.1080/10888691.2022.2059481

Christens, B. D., Winn, L. T., & Duke, A. M. (2016). Empowerment and Critical Consciousness: A Conceptual Cross-Fertilization. Adolescent Research Review, 1(1), 15–27. https://doi.org/10.1007/s40894-015-0019-3

Diemer, M. A., Frisby, M. B., Pinedo, A., Bardelli, E., Elliot, E., Harris, E., McAlister, S., & Voight, A. M. (2022). Development of the Short Critical Consciousness Scale (ShoCCS). Applied Developmental Science, 26(3), 409–425. Scopus. https://doi.org/10.1080/10888691.2020.1834394

Diemer, M. A., Pinedo, A., Bañales, J., Mathews, C. J., Frisby, M. B., Harris, E. M., & McAlister, S. (2021). Recentering Action in Critical Consciousness. Child Development Perspectives, 15(1), 12–17. https://doi.org/10.1111/cdep.12393

Diemer, M. A., Rapa, L. J., Park, C. J., & Perry, J. C. (2017). Development and Validation of the Critical Consciousness Scale. Youth & Society, 49(4), 461–483. https://doi.org/10.1177/0044118X14538289

Diemer, M. A., Rapa, L. J., Voight, A. M., & McWhirter, E. H. (2016). Critical Consciousness: A Developmental Approach to Addressing Marginalization and Oppression. Child Development Perspectives, 10(4), 216–221. https://doi.org/10.1111/cdep.12193

Fischer, L., Rohm, T., Carstensen, C. H., & Gnambs, T. (2021). Linking of Rasch-Scaled Tests: Consequences of Limited Item Pools and Model Misfit. Frontiers in Psychology, 12, 633896. https://doi.org/10.3389/fpsyg.2021.633896

Freire, P. (2021). Education for Critical Consciousness. Bloomsbury Academic. https://doi.org/10.5040/9781350190184

Hope, E. C., Pender, K. N., & Riddick, K. N. (2019). Development and Validation of the Black Community Activism Orientation Scale. Journal of Black Psychology, 45(3), 185–214. https://doi.org/10.1177/0095798419865416

Hughes, S. E., Watkins, A., Rapport, F., Boisvert, I., McMahon, C. M., & Hutchings, H. A. (2021). Rasch Analysis of the Listening Effort Questionnaire—Cochlear Implant. Ear & Hearing, 42(6), 1699–1711. https://doi.org/10.1097/AUD.0000000000001059

Kondratenko, M. Yu., Horodokin, A. D., Shevchenko, N. F., Stolyarenko, A. M., & Doma, I. S. (2024). Approaches to the assessment of the structure and phenomenology of critical awareness in the context of non-psychotic mental disorders. Modern Medical Technology, 16(2), 105–114. https://doi.org/10.14739/mmt.2024.2.301739

Kumar, S. (2023). Comprehensive Analysis of Psychometrics: Techniques, Applications, and Historical Context. International Journal of Science and Research (IJSR), 12(8), 4–6. https://doi.org/10.21275/SR23729121933

Leal, P. (2021). The Development of the Teacher Attitudes to Discrimination in Language Education Scale: A Measurement Tool of Critical Consciousness for Language Teachers. Education Sciences, 11(5), 200. https://doi.org/10.3390/educsci11050200

Liou, G., Bonner, C. V., & Tay, L. (2022). A psychometric view of technology-based assessments. International Journal of Testing, 22(3–4), 216–242. https://doi.org/10.1080/15305058.2022.2070757

Maker Castro, E., Dull, B., Hoyt, L. T., & Cohen, A. K. (2022). Associations between critical consciousness and well‐being in a national sample of college students during the COVID‐19 pandemic. Journal of Community Psychology, 50(2), 760–777. https://doi.org/10.1002/jcop.22678

McClellan, M. J., Montross-Thomas, L. P., Remer, P., Nakai, Y., & Monroe, A. D. (2019). Development and Validation of the Awareness of Privilege and Oppression Scale–2. SAGE Open, 9(2), 215824401985390. https://doi.org/10.1177/2158244019853906

Raman, K., Siegert, R. J., Bharatharaj, J., & Krägeloh, C. U. (2024). Validation of a Tamil Version of the Five Facet Mindfulness Questionnaire Using Rasch Analysis. Indian Journal of Psychological Medicine, 46(2), 147–158. https://doi.org/10.1177/02537176231220564

Rapa, L. J., Bolding, C. W., & Jamil, F. M. (2020). Development and initial validation of the short critical consciousness scale (CCS-S). Journal of Applied Developmental Psychology, 70, 101164. https://doi.org/10.1016/j.appdev.2020.101164

Rapa, L. J., & Geldhof, G. J. (2020). Critical consciousness: New directions for understanding its development during adolescence. Journal of Applied Developmental Psychology, 70, 101187. https://doi.org/10.1016/j.appdev.2020.101187

Seider, S., El-Amin, A., & Kelly, L. L. (2020). The Development of Critical Consciousness. In L. A. Jensen (Ed.), The Oxford Handbook of Moral Development (pp. 202–221). Oxford University Press. https://doi.org/10.1093/oxfordhb/9780190676049.013.11

Singh, S., Javdani, S., Berezin, M. N., & Sichel, C. E. (2020). Factor structure of the critical consciousness scale in juvenile legal system‐involved boys and girls. Journal of Community Psychology, 48(5), https://doi.org/10.1002/jcop.22362

Soeharto, S., Martono, M., Hairida, H., Akhmetova, A., Arifiyanti, F., Benő, C., & Charalambos, C. (2024). The metacognitive awareness of reading strategy among pre-service primary teachers and the possibility of rating improvement using Rasch analysis. Studies in Educational Evaluation, 80, 101319. https://doi.org/10.1016/j.stueduc.2023.101319

Suchishrava Dubey, Prashant Kumar Choudhary, & Megha Sahu. (2023). Development and Validation of Physical Education Awareness Instrument (Pea-I). Journal of Advanced Zoology, 44(4), 527–536. https://doi.org/10.17762/jaz.v44i4.1984

Sulaiman, A., Mashuri, M. F., Sungkono, S. L. L., Zahrah, A., & Royhana, S. F. (2022). Mengukur kesadaran kritis: Konstruksi skala socio-political awareness for indonesian. Cognicia, 10(2), 93–98. https://doi.org/10.22219/cognicia.v10i2.20159

Thomas, A. J., Barrie, R., Brunner, J., Clawson, A., Hewitt, A., Jeremie-Brink, G., & Rowe-Johnson, M. (2014). Assessing Critical Consciousness in Youth and Young Adults. Journal of Research on Adolescence, 24(3), 485–496. https://doi.org/10.1111/jora.12132

Wind, S. A., & Hua, C. (2022). Rasch Measurement Theory Analysis in R (1st ed.). Chapman and Hall/CRC. https://doi.org/10.1201/9781003174660

Yoo, H. C., Gabriel, A. K., Atkin, A. L., Matriano, R., & Akhter, S. (2021). A new measure of Asian American Racial Identity Ideological Values (AARIIV): Unity, interracial solidarity, and transnational critical consciousness. Asian American Journal of Psychology, 12(4), 317–332. https://doi.org/10.1037/aap0000256