.png)
.png)
Holistic vs. Analytic Evaluation in Writing Test of Eighth Grade Students
Abstract
The purpose of this study is to explore the efficacy of holistic and analytic scoring rubrics when applied to the context of writing examinations. Throughout this research, a technique was utilized to compare two grading models based on (a) assessor agreement (reliability) and (b) reasons for the grades given (validity). Teachers were randomly allocated to one of two conditions, where they graded the same student performance using either an analytical or a holistic approach. The design is experimental. Agreement and rank correlation between grades have been compared. A comparison of the students' performance on a writing assignment is made in this study using both holistic and analytic scoring rubrics. The information for this study was gathered from twenty-four students in eighth grade at SMP N 5 Kecamatan Kapur IX. The authors conducted an analysis of the performance of the students on the two rubrics, using psychometric statistics, in order to precisely determine the strengths and weaknesses of the students, and to place them along a continuum of foreign language writing ability. Citation In contrast, analytic scoring rubrics placed examinees on a scale of writing competency that was more specifically defined. As a result, analytic scoring rubrics were more dependable than holistic scoring rubrics when it came to gauging writing for the purpose of achieving goals.
Full Text:
PDFReferences
Al-ghazo, A. (2021). Evaluation and Grading of Students ’ Writing : Holistic and Analytic Scoring Rubrics. 9(1), 77–93. https://doi.org/10.5296/jsel.v9i1.19060
Bachman, L., & Palmer, A. (1996). Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.
Bauer, B. A. (1981). A study of the reliabilities and the cost-efficiencies of three methods of assessment for writing ability. Champaign: University of Illinosis.
Becker, A. (2011). Examining rubrics used to measure writing performance in US intensive English programs. The CATESOL Journal, 22(1), 113-130.
Black, P. (1998). Testing: Friend or foe? London: Falmer Press.
Brown, J. D. (1996). Director of Production: ;Uim Greenblatt Editorial Production ;Design .lanager: Dominick 3losco Interior Design and Electi-onic Production.
Campbell, D. M., Melenyzer, B. J., Nettles, D. H., & Wyman, R. M. Jr. (2000). Portfolio and performance assessment in teacher education. Boston: Allyn and Bacon.
Charney, D. (2009). The Validity of Using Holistic Scoring to Evaluate Writing : A Critical Overview. 18(1), 65–81.
Clapham, C., & McNamara, T. (2000). Measuring Second Language Performance. In TESOL Quarterly (Vol. 34, Issue 2, p. 376). https://doi.org/10.2307/3587967
Crehan, K. (1997, October). A discussion of analytic scoring for writing performance assessments. Paper presented at the Arizona Education Research Association, Phoenix, AZ.
Cumming, A. (1990). Expertise in evaluating second language compositions. Language Testing, 7(1), 31-51.
Eckes, T., Muller-Karabil, A., & Zimmermann, S. (2016). Assessing writing. In Handbook of Second Language Assessment. https://doi.org/10.4324/9781315561301-9
Hamp-Lyons, L. (1990). Second language writing: Assessment issues. In B. Kroll (Ed.), Second language writing: Research insights for the classroom (pp. 69-87). New York: Cambridge University Press.
Hamp-Lyons. (1991). Reliability and validity of rubrics for assessment through writing. In Assessing Second Language Writing in Academic Contexts (Vol. 15, Issue 1). https://doi.org/10.1016/j.asw.2010.01.003
Hamp-Lyons, L. (1995). Rating nonnative writing: The trouble with holistic scoring. TESOL Quarterly, 29, 759-762.
Huot, B. (1990). The Literature of Direct Writing Assessment: Major Concerns and Prevailing Trends (Vol. 60, Issue 2). http://rer.aera.net
Hyland, K. (2002). Teaching and research Writing. Harlow, England: Pearson Education Limited.
Jacobs, H. L., Zingraf, S. A., Wormuth, D. R., Hartfiel, V. F., & Hughey, J. B. (1981). Testing ESL composition: A practical approach. Rowley, MA: Newbury House.
Kim, H. J. (2011). Investigating raters’ development of rating ability on a second language speaking assessment. Dissertation Abstracts International Section A: Humanities and Social Sciences, 72(5-A), 1547. https://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2011-99210-560&%0Alang=ko&site=ehost-live
Knoch, U. (2009). Diagnostic assessment of writing: A comparison of two rating scales. Language Testing, 26(2), 275–304. https://doi.org/10.1177/0265532208101008
Kondo-Brown, K. (2002). A FACETS analysis of rater bias in measuring Japanese second language writing performance. Language Testing, 19(1), 3–31. https://doi.org/10.1191/0265532202lt218oa
Kuo, S. A. (2007). Which Rubric is More Suitable for NSS Liberal Studies ? Analytic or Holistic ? Hong Kong Educational Research Association, 22(2), 179–199.
Legg, S. M. (1998). Reliability and Validity. In W. Wolcott, & S. M. Legg (Eds.), An overview of writing assessment: Theory, research, and practice (pp. 124-142). Urbana, Ill: National Council of Teachers of English.
McNamara, T. (1996). Measuring second language performance. New York: Addison Wesley Longman.
Mendelsohn, D., & Cumming, A. (1987). Professors’ ratings of language use and rhetorical organization in ESL compositions. TESL Canada Journal, 5(1), 9-26.
Mousavi, S. A. (2002). An encyclopedic dictionary of language testing (3rd ed.). Taipei: Tung Hua Publications.
Perkins, K. (1983). On the use of composition scoring techniques, objective measure, and objective tests to evaluate ESL writing ability. TESOL Quarterly, 17(4), 651-671.
Reid, J. (1993). Teaching ESL Writing. Englewood Cliffs, NJ: Regents Prentice Hall.
Peter Newmark. (1998). Textbook Of Translation. 113.
Schafer, L. (2004). Rubric. Retrieved February 9, 2015, from ttp://www.etc.edu.cn/eet/articles/rubrics/index.htm
Stemler, S. E. (2004). A comparison of consensus, consistency, and measurement approaches to estimating interrater reliability. Practical Assessment, Research & Evaluation, 9(4).
Stoddart, T., Abrams, R., Gasper, E., & Canaday, D. (2000). Concept maps as assessment in science inquiry learning—A report of methodology. International Journal of Science Education, 22, 1221-1246.
Shi, L. (2001). Native- and nonnative-speaking EFL teachers” evaluation of Chinese students” English writing. Language Testing, 18(3), 303–325. https://doi.org/10.1177/026553220101800303
Shohamy, E. (1995). Performance Assessment in Language Testing. Annual Review of Applied Linguistics, 15, 188–211. https://doi.org/10.1017/s0267190500002683
Weigle, S. C. (2002). Assessing Writing. Cambridge: Cambridge University Press.
Weir, C. J. (1990). Communicative language testing. Englewood Cliffs, NJ: Prentice Hall Regents.
White, E. M. (1985). Teaching and assessing writing. San Francisco, CA: Jossey-Bass.
Wiseman, C. S. (2012). A Comparison of the Performance of Analytic vs . Holistic Scoring Rubrics to Assess L2 Writing. 2(1), 59–92.
DOI: http://dx.doi.org/10.30870/jels.v7i2.16422
Refbacks
- There are currently no refbacks.
Copyright (c) 2022 Journal of English Language Studies
Journal of English Language Studies is licensed under a Creative Commons Attribution 4.0 International License.
Copyright @ 2024 Journal of English Language Studies.