Design and psychometric validation of a metacognitive instrument for physics learning: A focus on heat concepts
DOI:
https://doi.org/10.58524/jasme.v5i2.818Keywords:
Metacognitive Knowledge, Physics Education, Instrument Validation, Rasch Model, Metacognitive QuestionnaireAbstract
Background: Metacognitive knowledge, awareness of one's own cognition, tasks, and learning strategies, is critical for independent learning but is often underemphasized in physics education.
Aim: This study aimed to develop and validate a questionnaire to measure high school students’ metacognitive knowledge within the specific context of physics, focusing on the topic of heat.
Method: The research involved content validation by six physics education experts and construct validation through empirical testing with 163 high school students. Content validity was established using Aiken’s V, while construct validity and reliability were evaluated using Rasch model analysis.
Results: The final instrument consisted of 28 items, of which 26 met the Rasch model fit criteria. The analysis confirmed high person reliability (0.82) and item reliability (0.98), indicating strong internal consistency and measurement stability.
Conclusion: The findings support the questionnaire's validity and reliability as a tool for assessing metacognitive knowledge in physics. This validated instrument provides a foundation for future research and instructional practices aimed at enhancing students' metacognitive skills
References
Agnezi, L. A. (2023, September). Development of The Online Assessment Instrument for Fluid, Temperature, and Heat to measure the problem solving skills of high school students. In Journal of Physics: Conference Series (Vol. 2582, No. 1, p. 012053). IOP Publishing. https://doi.org/10.1088/1742-6596/2582/1/012053
Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. Educational and psychological measurement, 40(4), 955-959. https://doi.org/10.1177/001316448004000419
Beck, B., Peña-Vivas, V., Fleming, S., & Haggard, P. (2019). Metacognition across sensory modalities: Vision, warmth, and nociceptive pain. Cognition, 186, 32-41. https://doi.org/10.1016/j.cognition.2019.01.018
Boone, W. J., & Noltemeyer, A. (2017). Rasch analysis: A primer for school psychology researchers and practitioners. Cogent Education, 4(1), 1416898. https://doi.org/10.1080/2331186X.2017.1416898
Carver, R. P. (1974). Two dimensions of tests: Psychometric and edumetric. American Psychologist, 29(7), 512. https://doi.org/10.1037/h0036782
Chan, S. W., Ismail, Z., & Sumintono, B. (2014). A Rasch model analysis on secondary students’ statistical reasoning ability in descriptive statistics. Procedia-Social and Behavioral Sciences, 129, 133-139. https://doi.org/10.1016/j.sbspro.2014.03.658
Cotterall, S., & Murray, G. (2009). Enhancing metacognitive knowledge: Structure, affordances and self. System, 37(1), 34-45. https://doi.org/10.1016/j.system.2008.08.003
Duncan, T., Pintrich, P., Smith, D., & Mckeachie, W. (2015). Motivated strategies for learning questionnaire (MSLQ) manual. National Center for Research to Improve Postsecondary Teaching and Learning.
Efklides, A., & Vlachopoulos, S. P. (2012). Measurement of metacognitive knowledge of self, task, and strategies in mathematics. European Journal of Psychological Assessment. https://doi.org/10.1027/1015-5759/a000145
Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry. American Psychologist, 34(10), 906. https://doi.org/10.1037/0003-066X.34.10.906
Herdman, M., Fox-Rushby, J., & Badia, X. (1998). A model of equivalence in the cultural adaptation of HRQoL instruments: the universalist approach. Quality of life Research, 7(4), 323-335. https://doi.org/10.1023/A:1008846618880
Hikmah, F. N., Sukarelawan, M. I., Nurjannah, T., & Djumati, J. (2021). Elaboration of high school student’s metacognition awareness on heat and temperature material: Wright map in Rasch model. Indonesian Journal of Science and Mathematics Education, 4(2), 172–182. https://doi.org/10.24042/ijsme.v4i2.9488
Jalil, S., Ali, M. S., & Haris, A. (2018, June). Development and validation of science process skills instrument in physics. In Journal of Physics: Conference Series (Vol. 1028, No. 1, p. 012203). IOP Publishing. https://doi.org/10.1088/1742-6596/1028/1/012203
Joshi, A., Kale, S., Chandel, S., & Pal, D. K. (2015). Likert scale: Explored and explained. British journal of applied science & technology, 7(4), 396. https://doi.org/10.9734/BJAST/2015/14975
Krathwohl, D. R. (2002). A revision of Bloom's taxonomy: An overview. Theory into practice, 41(4), 212-218. https://doi.org/10.1207/s15430421tip4104_2
Linacre, J. M. (2002). Optimizing rating scale category effectiveness. Journal of applied measurement, 3(1), 85-106.
Molenda, M. (2003). In search of the elusive ADDIE model. Performance improvement, 42(5), 34-37. https://doi.org/10.1002/pfi.4930420508
Novia, H., Kaniawati, I., Rusli, A., & Rusdiana, D. (2019). The development of metacognitive awareness related to the implementation of metacognitive-based learning. In Journal of Physics: Conference Series (Vol. 1170, No. 1, p. 012034). IOP Publishing. https://doi.org/10.1088/1742-6596/1170/1/012034
Peskin, J., & Astington, J. W. (2004). The effects of adding metacognitive language to story texts. Cognitive development, 19(2), 253-273. https://doi.org/10.1016/j.cogdev.2004.01.003
Pintrich, P. R. (2002). The role of metacognitive knowledge in learning, teaching, and assessing. Theory into practice, 41(4), 219-225. https://doi.org/10.1207/s15430421tip4104_3
Radhakrishna, R. B. (2007). Tips for developing and testing questionnaires/instruments. The Journal of Extension, 45(1), 25.
Retnawati, H. (2016). Proving content validity of self-regulated learning scale (The comparison of Aiken index and expanded Gregory index). REiD (Research and Evaluation in Education), 2(2), 4. https://doi.org/10.21831/reid.v2i2.11029
Samsudin, A., Cahyani, P. B., Rusdiana, D., Efendi, R., Aminudin, A. H., & Costu, B. (2021). Development of a Multitier Open-Ended Work and Energy Instrument (MOWEI) Using Rasch Analysis to Identify Students' Misconceptions. Cypriot Journal of Educational Sciences, 16(1), 16-32. https://doi.org/10.18844/cjes.v16i1.5504
Sari, K. M. (2019, June). Metacognitive Knowledge and Critical Thinking Biology 11th of Public Senior High School in Bogor. In Journal of Physics: Conference Series (Vol. 1241, No. 1, p. 012056). IOP Publishing. https://doi.org/10.1088/1742-6596/1241/1/012056
Sukarelawan, M., Sulisworo, D., Kuswanto, H., & Rofiqah, S. A. (2021). Heat and Temperature Metacognition Awareness Inventory: A Confirmatory Factor Analysis. International Journal of Evaluation and Research in Education, 10(2), 389–395. https://doi.org/10.11591/ijere.v10i2.20917
Sumintono, B. (2018, February). Rasch model measurements as tools in assesment for learning. In 1st International Conference on Education Innovation (ICEI 2017) (pp. 38-42). Atlantis Press. https://doi.org/10.2991/icei-17.2018.11
Suryana, T. G. S., Setyadin, A. H., Samsudin, A., & Kaniawati, I. (2020, February). Assessing multidimensional energy literacy of high school students: an analysis of rasch model. In Journal of Physics: Conference Series (Vol. 1467, No. 1, p. 012034). IOP Publishing. https://doi.org/10.1088/1742-6596/1467/1/012034
Taherdoost, H. (2016). Validity and reliability of the research instrument; how to test the validation of a questionnaire/survey in a research. International journal of academic research in management (IJARM), 5. https://doi.org/10.2139/ssrn.3205040
Teng, M. F. (2025). Metacognition in language teaching. Cambridge University Press.
Van Eck, N., & Waltman, L. (2010). Software survey: VOSviewer, a computer program for bibliometric mapping. scientometrics, 84(2), 523-538. https://doi.org/10.1007/s11192-009-0146-3
Wenden, A. L. (1998). Metacognitive knowledge and language learning. Applied linguistics, 19(4), 515-537. https://doi.org/10.1093/applin/19.4.515
Widhiarso, W., & Sumintono, B. (2016). Examining response aberrance as a cause of outliers in statistical analysis. Personality and Individual Differences, 98, 11-15. https://doi.org/10.1016/j.paid.2016.03.099
Zahran, M., Samsudin, A., Suhandi, A., Aminudin, A. H., Wibowo, F. C., Prahani, B. K., & Kapıcı, H. Ö. (2025). Development of Augmented Reality for Special Education Needs (ARSEN) to Enhance Students' Conceptions of Solar System. Multidisciplinary Science Journal, 7(11), 2025531-2025531. https://doi.org/10.31893/multiscience.2025531
Ziegler, M., & Brunner, M. (2016). Test standards and psychometric modeling. In Psychosocial skills and school systems in the 21st century: Theory, research, and practice (pp. 29-55). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-28606-8_2
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Hera Novia, Muhammad Zahran
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Authors who publish with Journal of Advanced Sciences and Mathematics Education agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-ShareAlike 4.0 International License (CC BY-SA 4.0) that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
Journal of Advanced Sciences and Mathematics Education is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
