Email this article 
Using Rasch Model to Detect Differential Person Functioning and Cheating Behavior in Natural Sciences Learning Achievement Test
Abstract
The existence of abberant response showed the inaccuracy of measurement which, in turn, threatens the test validity. This study aims at: (1) Discovering the proportion of students who were having Differential Person Functioning (DPF) in final term assessment test of natural sciences course for 8th grade in odd semester of academic year 2016/2017 in Tegal Regency, Indonesia; (2) Identifying the students suspected of cheating during the final term assessment test of natural sciences course for 8th grade in odd semester of academic year 2016/2017 in Tegal Regency, Indonesia. This research involved 1011 student responses to final term assessment test of natural sciences course for 8th grade in odd semester of academic year 2016/2017 in Tegal Regency, Indonesia. The data were taken from four junior high schools; SMPN I Dukuhturi, SMPN I Suradadi, SMPN 2 Slawi and SMPN 2 Dukuhwaru. The scoring was done using Rasch model and the person fit index used Sijtsma’s Ht person fit statistic (Ht). The result showed: (1) 14% of the students attending final term assessment of natural sciences course for 8th grade in the odd semester of academic year 2016/2017 in Tegal Regency, Indonesia were detected of having DPF. From the four junior high schools involved in the research, three junior high schools have a proportion of students having DPF at a range from 9.6% to 23%, and only 1.1% of the other one’s students were having DPF; (2) Nine pairs of students were suspected of cheating during the final term assessment of natural sciences course for 8th grade in the odd semester of academic year 2016/2017 in Tegal Regency, Indonesia.
Keywords
Full Text:
PDFReferences
Alsmadi, YM & Alsmadi, AA 2009,‘Detecting differential person functioning in emotional intelligence’,Journal of Instructional Psychology, vol. 36,no.4, pp. 284-88.
Ayele, DG, Zewotir, T, and Mwambi, H 2014, ’Using Rasch Modeling to Re-Evaluate Rapid Malaria Diagnosis Test Analyses’, International Journal of Environmental Research and Public Health , vol.11,no.7 ,pp. 6681-91.
Belov, DI & Armstrong, RD 2010, ‘Automatic detection of answer copying via Kullback-Leibler divergence and K-index’,Applied Psychological Measurement, vol.34, no.6, pp. 379-92.
Clauser, BE & Mazor, KM 1998, ‘Using statistical procedures to identify differentially functioning test items’Educational Measurement: issues and practice, vo. 17, no.1, pp. 31-44.
Creswell, JW 2010, Research Design Pendekatan Kualitatif, Kuantitatif dan Mixed, A. Fawaid (Trans), Pustaka Pelajar, Yogjakarta
Dewi, NDL & Prasetyo, ZK 2016,‘Pengembangan instrument penilaian IPA untuk memetakan critical thinking dan practical skill pesertadidik SMP’,Jurnal Inovasi Pendidikan IPA, vol.2 no.2, pp. 213-22.
Emons, WH, Sijtsma, K & Meijer, RR 2005, ‘Global, local, and graphical person-fit analysis using person-response functions’, Psychological Methods,vol.10, no.1, pp. 101.
Engelhard Jr, G 2009, ‘Using item response theory and model—data fit to conceptualize differential item and person functioning for students with disabilities’, Educational and Psychological Measurement, vo. 69, no.4,pp. 585-602.
Feuerherd, M, Knuth, D, Muehlan, H& Schmidt, S 2014, ‘Differential item functioning (DIF) analyses of the Impact of Event Scale-Revised (IES-R): Results from a large European study on people with disaster experiences’,Traumatology, vol. 20, no.4,pp. 313.
Gierl, M, Khaliq, SN &Boughton, K1999,‘Gender differential item functioning in mathematics and science: Prevalence and policy implications’, Inannual meeting of the Canadian Society for the Study of Education, Sherbrooke, Quebec.
Gurel, DK, Eryılmaz, A & McDermott, LC 2015,‘A Review and Comparison of Diagnostic Instruments to Identify Students' Misconceptions in Science’,Eurasia Journal of Mathematics, Science & Technology Education, vo. 11, no.5, pp.989-1008.
Hambleton, RK, Swaminathan, H, & Rogers, HJ1991, Fundamentals of item response theory, Sage, California
Hays, RD, Calderón, JL, Spritzer, KL, Reise, SP, & Paz, SH2018, ‘Differential item functioning by language on the PROMIS® physical functioning items for children and adolescents’,Quality of Life Research, vol. 27, no.1, pp.235-47.
Heckler, AF, Scaife, TM, & Sayre, EC 2010,‘Response times and misconception-like responses to science questions’, In Proceedings of the Annual Meeting of the Cognitive Science Society, vol.32,no.32
Johanson, G & Alsmadi, A 2002,‘Differential person functioning’Educational and Psychological Measurement, vol.62, no.3,pp. 435-43.
Kalaycioğlu, DB & Berberoğlu, G 2011,‘Differential item functioning analysis of the science and mathematics items in the university entrance examinations in Turkey’,Journal of Psychoeducational Assessment, vol.29, no.5, pp. 467-78.
Karabatsos, G 2003,‘Comparing the aberrant response detection performance of thirty-six person-fit statistics’, Applied Measurement in Education, vol.16,no.4, pp. 277-98.
Ministry of Education and Culture 2018, Peraturan Menteri Pendidikan dan Kebudayaan No 20 tahun 2018 tentang Penguatan Pendidikan Karakter Pada Sekolah Formal, Jakarta, Ministry of Education and Culture of Republic of Indonesia.
King, CJH2010,‘An analysis of misconceptions in science textbooks: Earth science in England and Wales’,International Journal of Science Education, vol.32,no.5, pp. 565-601.
Lu, YM, Wu, YY, Hsieh, CL, Lin, CL, Hwang, SL, Cheng, KI, & Lue, YJ 2013, ‘Measurement precision of the disability for back pain scale-by applying Rasch analysis’, Journal of Health and Quality of Life Outcomes , vol.11,no.1, pp. 119.
Luo, S, Liu, Y, Teresi, JA, Stebbins, GT, & Goetz, CG 2017,‘Differential item functioning in the Unified Dyskinesia Rating Scale (udysrs)’. Movement Disorders, vol.32 no. 8, pp. 1244-49.
Mair, P, Hatzinger, R, Maier, MJ, Rusch, T, &Mair, MP2016, Package ‘eRm’, R Foundation,Vienna, Austria
Meijer, R R 1996, ‘Person-fit research: An introduction’,Applied Measurement in Education, vol.9, no.1, pp. 3-8.
Meijer, RR, & Sijtsma, K 2001, ‘Methodology review: Evaluating person fit’,Applied Psychological Measurement, vol.25, no.2, pp. 107-35.
Messick 1996, ‘Validaty and washback in language testing’, Languange Testing, vol.13 no.3, pp. 241-56.
Mok, M. and Wright, B 2004. ‘Overview of RaschModel Families’, In Introduction to Rasch Measurement: Theory, Models and Applications, Jam Press, Minnesota
Perkins, A 2013, Differential Person Functioning, Ph.D thesis, Emory University, Atlanta
Petridou, A & Williams, J 2007, ‘Accounting for aberrant test response patterns using multilevel models’Journal of Educational Measurement, vol.44, no.3, pp. 227-47.
Rahmawati, I, Sutopo, S, & Zulaikah, S 2017, ‘Analysis of Students’ Difficulties about Rotational Dynamic Topic Based on Resource Theory’Jurnal Pendidikan IPA Indonesia, vol.6,no.1, pp. 95-102.
Redhana, IW, Sudria, IBN, Hidayat, I & Merta, LM 2017, ‘ Identification of Chemistry Learning Problems Viewed from Conceptual Change Model’,Jurnal Pendidikan IPA Indonesia,vol.6, no. 2, pp. 356-364.
Rupp, AA 2013, ‘A systematic review of the methodology for person fit research in item response theory: Lessons about generalizability of inferences from the design of simulation studies’Psychological Test and Assessment Modeling, vol. 55, no.1, pp. 3-38.
Salim, AN 2016, Perbandingan metode pendeteksian integritas hasil tes ujian nasional 2015. MPD thesis, Universitas Muhammadiyah Prof Dr. Hamka, Jakarta
Scherbaum, CA 2003, Detecting intentional response distortion on measures of the five-factor model of personality: An application of differential person functioning, Ph.D thesis, Ohio University, Ohio
Sijtsma, K, & Meijer, RR 1992, ‘A method for investigating the intersection of item response functions in Mokken's nonparametric IRT model’,Applied Psychological Measurement, vol.16, no. 2, pp. 149-157.
Sireci, SG 2007, ‘On validity theory and test validation’,Educational Researcher, vol. 36, no.8, pp. 477-481.
Smith, AB, Fallowfield, LJ, Stark, DP, Velikova, G, & Jenkins, V 2010, ‘A Rasch and confirmatory factor analysis of the General Health Questionnaire (GHQ) – 12’, Journal Health and Quality of Life Outcomes ,vol.8, no.45, pp. 45.
Sotaridona, LS & Meijer, RR 2002. ‘Statistical properties of the K-index for detecting answer copying’. Journal of Educational Measurement, vol.39, no.2, pp. 115-32.
Sotaridona, LS, van der Linden, WJ, & Meijer, RR 2006,‘Detecting answer copying using the kappa statistic’,Applied Psychological Measurement, vol.30, no.5, pp. 412-31.
Sotaridona, LS 2003, ‘Statistical Methods for the Detection of Answer Copying on Achievement Test’, Ph.D thesis, Twente University Netherlands, Holland
Strobl, C, Kopf, J &Zeileis, A 2015, ‘Rasch trees: A new method for detecting differential item functioning in the Rasch model’’,Psychometrika, vo.80, no.2, pp. 289-316.
Sumintono, B 2018, ‘Rasch Model Measurements as Tools in Assesment for Learning’, In 1st International Conference on Education Innovation 2017, Atlantis Press.
Sumintono, B & Widhiarso, W 2014, Aplikasi model Rasch untuk penelitian ilmu-ilmusosial. Trim Komunikata Publishing House, Bandung.
Susongko, P 2016, ‘Validation of science achievement test with the Rasch model’, Jurnal Pendidikan IPA Indonesia, vol.5, no.2, pp. 268-77.
Susongko, P & Mardapi, D. 2000, ‘Keberfungsian Butir Diferensial Perangkat Tes Ebtanas Kimia Sekolah Menengah Umum di Jawa Tengah’, Jurnal Penelitian dan Evaluasi Pendidikan, vol. 3, no.4, pp. 1-14.
Tendeiro, JN & Meijer, RR 2014, ‘Detection of invalid test scores: The usefulness of simple nonparametric statistics’, Journal of Educational Measurement, vol.51, no.3, pp. 239-59.
Tendeiro, JN, & Tendeiro, MJN 2016, Package ‘PerFit’, viewed 23 June 2019, http://www.est.colpos.mx/R-mirror/web/packages/PerFit/PerFit.pdf
Tendeiro, JN, Meijer, RR,&Niessen, A S M2016, ‘PerFit: An R package for person-fit analysis in IRT’, Journal of Statistical Software, vol.74, no.5, pp. 1-27.
US Department of Education 2013, Testing integrity symposium Issues and Recommendations for Best Practice, US Department of Education, Institute of Education Sciences National Center for Education Statistics 2013, Washington DC, United States.
Tsai-Wei, H, & Pei-Chen, W 2013, Classroom-based cognitive diagnostic model for a teacher-made fraction-decimal test, Journal of Educational Technology & Society, vol.16, no.3, pp. 347-61.
Widiatmo, H 2009,‘Metode untuk mendeteksi penyontekan jawaban pada tes pilihan ganda: studi kasus SMP di Kabupaten Garut’. Pusat Penelitian Pendidikan, Balitbang Diknas, pp. 219-26.
Wijaya, CP, & Muhardjito, M 2016,‘The diagnosis of senior high school class x mia b students misconceptions about hydrostatic pressure concept using three-tier’. Jurnal Pendidikan IPA Indonesia, vol.5, no.1, pp. 13-21.
Wu, M, & Adams, R 2007, Applying the Rasch model to psycho-social measurement: A practical approach, Educational Measurement Solutions, Melbourne
Yih, JM& Lin, YH 2010, ‘Concept structure based on response pattern detection of SP chart with application in algebra learning’, Learning, vol.100, no.8, pp.847-56.
DOI: http://dx.doi.org/10.30870/jppi.v5i2.5945
Refbacks
- There are currently no refbacks.
Copyright (c) 2019 Jurnal Penelitian dan Pembelajaran IPA
This work is licensed under a Creative Commons Attribution 4.0 International License.
Jurnal Penelitian dan Pembelajaran IPA is licensed under a Creative Commons Attribution 4.0 International License
Copyright © 2023 Jurnal Penelitian dan Pembelajaran IPA. All rights reserved.