Using Coh-Metrix to Analyze Chinese ESL Learners’ Writing
Keywords:
ESL essay writing, Textual feature analysis, Automatic Essay Scoring, Computer in educationAbstract
Scoring essays is costly, laborious and time-consuming.
Automated scoring of essays is a promising approach to face this
challenge. Coh-Metrix is a computer tool that reports on cohesion,
sentence complexity, lexical sophistication and other descriptive
features at sentence- and paragraph-level. It has been widely used to
analyze native English speakers’ essay writing. However, few studies
have used Coh-Metrix to analyze essays written by English as a Second
Language (ESL) students. In this study, we analyzed the correlation
between several Coh-Metrix features combined with a set of newly
proposed features and the quality of essays, written by Chinese
university students, both English and non-English majors. This study
shows that each group of students tends to write essays that have their
own signature features. The quality of essays written by English majors
highly correlate to the importance of introduction, conclusion and
cohesion at the sentence level, while the quality of essays written by
chemistry majors are highly related to mechanics errors, sentence
complexity and cohesion at the paragraph level.
References
Crossley, S. A., & McNamara, D. S. (2011). Understanding expert ratings of essay quality:
Coh-Metrix analyses of first and second language writing. International Journal of
Continuing Engineering Education and Life-Long Learning, 21(2/3), 170.
http://doi.org/10.1504/IJCEELL.2011.040197
Crossley, S., & McNamara, D. (2010). Cohesion, coherence, and expert evaluations of
writing proficiency. In The 32nd Annual Conference of the Cognitive Science Society
(pp. 984–989). Austin: TX. Retrieved from
http://csjarchive.cogsci.rpi.edu/Proceedings/2010/papers/0310/paper0310.pdf
Freeman, Y. S., & Freeman, D. E. (1998). ESL/EFL Teaching: Principles for Success.
Heinemann.
Ge, S., & Chen, X. (2007). Automated Essay Scoring for Chinese EFL learners. Foreign
Language World, 122(5), 43–50.
Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z. (2004). Coh-metrix:
analysis of text on cohesion and language. Behavior Research Methods, Instruments, &
Computers, 36(2), 193–202. Retrieved from
http://www.ncbi.nlm.nih.gov/pubmed/15354684
Han, N. (2009). The theory and practice of Automated English Essay Scoring Systems.
China Test, 3, 38–44.
Hongbo Chen, Ben He, Tiejian Luo, B. L. (2012). A ranked-based learning approach to
automated essay scoring. In 2012 Second International Conference on Cloud and Green
Computing (CGC) (pp. 448–455).
Kintsch, W., & van Dijk, T. (1978). Towards a model of text comprehension and
production. Psychological Review, 85, 363–394.
Lafferty, J., Sleator, D., & Temperley, D. (1992). Grammatical Trigrams: A Probabilistic
Model of Link Grammar. In Proceedings of the AAAI Conference on Probabilistic
Approaches to Natural Language.
Landauer, T. K., Foltz, P. W., & Laham, D. (1998). Introduction to latent semantic
analysis. Discourse Processes, 25, 259–284.
Larkey, L. S. (1998). Automatic essay grading using text categorization techniques.
Proceedings of the 21st Annual International ACM SIGIR Conference on Research and
Development in Information Retrieval - SIGIR ’98, (1), 90–95.
http://doi.org/10.1145/290941.290965
Lawrence M. Rudner, & Liang, T. (2002). Automated essay scoring using Bayes’
Theorem. The Journal of Technology, Learning and Assessment, 1(2).
Li, J. (2009). Using latent semantic analysis for automated essay scoring in the Chinese EFL
context. Guangdong University of Foreign Studies.
Liang, M. (2011). Construting a model for the computer assisted scoring of Chinese EFL
learners’ argumentative essays. Foreigh Language Teaching and Research Press.
Liang, M., & Wen, Q. (2007). A critical review and implications of some automated essay
scoring systems. Computer Assisted Foreign Language Education, 18-24.
Liang, Q. (2004). Contrastive study on the objectivity of English and Chinese
argumentative writing: A survey on the objectivity of Chinese college students’
English argumentative writing. Journal of China West Normal University (Philosophy
& Social Sciences), (5).
Liu, X. (2008). A case study of a non-English Major’s Writing: Error analysis. Foreign
Language Research, 2, 140–142.
McNamara, D. S., Crossley, S. a, & Roscoe, R. (2013). Natural language processing in an
intelligent writing strategy tutoring system. Behavior Research Methods, 45(2), 499–
http://doi.org/10.3758/s13428-012-0258-1
Mcnamara, D. S., Crossley, S. A., & Mccarthy, P. M. (2010). Linguistic features of writing
quality. Written Communication, 27(1). http://doi.org/10.1177/0741088309351547
McNamara, D. S., Crossley, S. A., Roscoe, R. D., Allen, L. K., & Dai, J. (2015). A
hierarchical classification approach to automated essay scoring. Assessing Writing,
, 35–59. http://doi.org/10.1016/j.asw.2014.09.002
Page, E. B. (2003). Project essay grade: PEG. In M. D. Shermis & J. Burstein (Eds.),
Automated essay scoring: A cross-disciplinary perspective (pp. 43–54). Mahwah, NJ:
Lawrence Erlbaum Associates.
Rus, V., & Niraula, N. (2012). Automated detection of local coherence in short
argumentative essays based on centering theory. In 13th International Conference on
Computational Linguistics and Intelligent Text Processing (pp. 450–461). New Delhi,
India.
Shermis, M., & Burstein, J. (2003). Automated essay scoring: A cross-disciplinary perspective.
Mahwah NJ: Lawrence Erlbaum Associates.
Shermis, M. D. (2014). State-of-the-art automated essay scoring: Competition, results,
and future directions from a United States demonstration. Assessing Writing, 20, 53–
http://doi.org/10.1016/j.asw.2013.04.001
Shevade, S., Keerthi, S., Bhattacharyya, C., & Murthy, K. (1999). Improvements to the
SMO Algorithm for SVM Regression. IEEE Transactions on Neural Networks.
University, S. (2007). Implementation guidances on the reform of credit system in Southwest
University. Chongqing. Retrieved from agronomy.swu.edu.cn/up/87.doc
Downloads
Published
Issue
Section
License
Copyright (c) 2016 Weiwei Xu, Ming Liu
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
All articles published by IJLTER are licensed under a Creative Commons Attribution Non-Commercial No-Derivatives 4.0 International License (CCBY-NC-ND4.0).
