AI-Powered Automated Assessment: Aiken Index Analysis of Content Validity Scientific Literacy Asesment

Bela Anisa Putri, Sri Yamtinah, Ari Syahidul Shidiq, Hayuni Retno Widarti, Antuni Wiyarsi

Abstract


This study aims to analyze the content validity of the Scientific Literacy Asesment (SLA) instrument, which integrates ethnochemistry based on artificial intelligence through expert agreement based on the Aiken index so that the instrument can measure what should be measured. The research method used is a qualitative descriptive method based on the results of content validity calculated through the Aiken formula. Content validity data were obtained from 10 experts, namely, chemistry education lecturers from UNS, UNY, and UM and high school chemistry teachers in Surakarta, through focus group discussions (FGDs). Content validity is assessed by a score of 1 – 4, namely, irrelevant (TR) with a score of 1; less relevant (KR) with a score of 2; quite relevant (CR) with a score of 3; and relevant (R) with a score of 4 for each question item with aspects measured, namely, aspects of content, language, and construct.  Based on the calculation, 15 questions were declared valid with an Aiken index value ≥ 0.73.


Full Text:

PDF

References


Aiken, L. R. (1985). Three Coefficients For Analyzing The Reliability And Validity Of Ratings, Educational And Psychological Measurument. Educational and Psychological Measurement, 45(1), 131–142. https://doi.org/10.1177/0013164485451012

Aldiansyah, A., Pasa, J. I., Muttaqin, M. R., Awaliyah, N. N., & Erika, F. (2023). Literatur Review : Keterkaitan Pembelajaran Kimia Terhadap Pendekatan Etnokimia Di Indonesia. CHEDS: Journal of Chemistry, Education, and Science, 7(2), 238–246. https://doi.org/10.30743/cheds.v7i2.8416

Arikunto, S. (2013). Contribution Of Educational Management To Optimal Service Of Educational Development. PSIKOPEDAGOGIA Jurnal Bimbingan Dan Konseling, 2(2), 70. https://doi.org/10.12928/psikopedagogia.v2i2.2573

Azizah, D., Wibawa, A., & Budiarto, L. (2021). Hakikat Epistemologi Artificial Intelligence. Jurnal Inovasi Teknologi Dan Edukasi Teknik, 1(8), 592–598. https://doi.org/10.17977/um068v1i82021p592-598

Azwar, S. (2000). Reliabilitas Dan Validitas. Pustaka Pelajar Offset.

Borg, W And Gall, M. (1983). Educational Research. Longman.

Campino, J. (2024). Unleashing The Transformers: NLP Models Detect AI Writing In Education. Journal of Computers in Education, 0123456789. https://doi.org/10.1007/s40692-024-00325-y

Islamiyah, E. F. N. I., Dewanti, B. A., & Santoso, A. (2024). Development Of Science Literacy Assessment Instruments In Science Learning Using Contextual Learning Models For Junior High School. Quantum: Jurnal Inovasi Pendidikan Sains, 15(1), 75–89. https://doi.org/10.20527/quantum.v15i1.17255

Ismail, M. I., & Vita, P. (2020). Evaluasi Pembelajaran: Konsep Dasar, Prinsip, Teknik dan Prosedur. Rajawali Pers.

Istyadji, M., & Sauqina. (2023). Conception Of Scientific Literacy In The Development Of Scientific Literacy Assessment Tools: A Systematic Theoretical Review. Journal of Turkish Science Education, 20(2), 281–308. https://doi.org/10.36681/tused.2023.016

Neuman, W. L. (1994). Social Research Methods Qualitative And Quantitative Approaches. (2nd ed.). Allyn And Bacon.

PISA. (2022). Notas por país: México. (2024). Perfiles Educativos, 46(183), 188–202. https://doi.org/10.22201/iisue.24486167e.2024.183.61714

Punaji, S. (2010). Metode Penelitian Pendidikan dan Pengembangan. Kencana.

Putri, H., Susiani, D., Wandani, N. S., & Putri, F. A. (2022). Instrumen Penilaian Hasil Pembelajaran Kognitif Pada Tes Uraian Dan Tes Objektif. Jurnal Pendidikan Dan Ilmu Sosial (Jupendis), 2(4), 139–148. https://doi.org/10.54066/jupendis.v2i4.2159

Rahmadani, Y., Fitakurahmah, N., Fungky, N., Prihatin, R., Majid, Q., & Prayitno, B. A. (2018). Profil Keterampilan Literasi Sains Siswa Di Salah Satu Sekolah Swasta Di Karanganyar. Jurnal Pendidikan Biologi, 7(3), 183. https://doi.org/10.24114/jpb.v7i3.10123

Retnawati, H. (2016). Proving Content Validity Of Self-Regulated Learning Scale (The Comparison Of Aiken Index And Expanded Gregory Index). Research and Evaluation in Education, 2(2), 155–164. http://dx.doi.org/10.21831/reid.v2i2.11029

Shwartz, Y., Ben-Zvi, R., & Hofstein, A. (2006). The Use Of Scientific Literacy Taxonomy For Assessing The Development Of Chemical Literacy Among High-School Students. Chemistry Education Research and Practice, 7(4), 203–225. https://doi.org/10.1039/B6RP90011A

Stylos, G., Siarka, O., & Kotsis, K. T. (2023). Assessing Greek Pre-Service Primary Teachers’ Scientific Literacy. European Journal of Science and Mathematics Education, 11(2), 271–282. https://doi.org/10.30935/scimath/12637

Supatmi, S. (2022). Peningkatan Keterampilan Proses Sains Melalui Praktikum Kimia Berbasis Skala Mikro Materi Stoikiometri. Jurnal Riset Pendidikan Kimia, 12(1), 47–58. https://doi.org/10.21009/JRPK.121.07

Triwibowo, S., Parno, P., & Latifah, E. (2021). Kemampuan Literasi Sains Siswa SMA Topik Usaha Dan Energi. Jurnal Pendidikan: Teori, Penelitian, Dan Pengembangan, 6(11), 1787. https://doi.org/10.17977/jptpp.v6i11.15146

Widoyoko, E. P. (2012). Teknik Penyusunan Instrumen Penelitian. Pustaka Pelajar.

Windyariani, S., Setiono, & Sutisnawati, A. (2021). Pengembangan Model Asesmen Literasi Sains Berbasis Konteks Bagi Siswa Sekolah Dasar. Seminar Nasional Kedua Pendidikan Berkemajuan Dan Menggembirakan (The Second Progressive and Fun Education Seminar), 279–286.

Yamtinah, S., Dewi, M. C., Nurhayati, N. D., Susilowati, E., Fakhrudin, I. A., Ramadhani, D. G., Shidiq, A. S., & Saputro, S. (2022). Content Validity In Android-Based Augmented Reality Media For High School Science Students On Covalent Bonds Topic: Rasch Model Analysis. Jurnal Pendidikan Sains Indonesia, 10(2), 240–249. https://doi.org/10.24815/jpsi.v10i2.23280

Yamtinah, S., Saputro, S., & Utami, B. (2016). Content Validity And Scoring Of Two Tier As Measuring Instrument Of Science Process Skills For Knowledge Aspects In Chemistry Learning. Prosiding ICTTE FKIP UNS, 2, 911–916.

Zulfa, L. N., & Haryanto, H. (2021). Pengaruh Media Macromedia Flash Terhadap Literasi Sains Dan Sikap Demokratis Mahasiswa. Jurnal Pendidikan Sains Indonesia, 9(1), 52–64. https://doi.org/10.24815/jpsi.v9i1.18266




DOI: http://dx.doi.org/10.30870/educhemia.v10i1.30640

Refbacks

  • There are currently no refbacks.


Copyright (c) 2025 Sri Yamtinah

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

link alternatif jmtoto

JMTOTO

https://totoberkah.or.id/

EduChemia Indexed by:

   

                                                                                                                                                                                                                                                 More Indecxing...                                                         

Creative Commons License

EduChemia: Jurnal Kimia dan Pendidikan is licensed under a Creative Commons Attribution 4.0 International License

________________________________________________________

EduChemia: Jurnal Kimia dan Pendidikan ISSN 2502-4779 (print) | ISSN 2502-4787 (online)
Published by Department of Chemistry Education - Universitas Sultan Ageng Tirtayasa
Address : Jl. Ciwaru Raya No. 25, Sempu, Kota Serang, Banten 42117, Indonesia
Email  : [email protected]