Written Communication Jan 2010

Linguistic Features of Writing Quality

Danielle S. McNamara University of Memphis ; Scott A. Crossley Mississippi State University ; Philip M. McCarthy University of Memphis

Abstract

In this study, a corpus of expert-graded essays, based on a standardized scoring rubric, is computationally evaluated so as to distinguish the differences between those essays that were rated as high and those rated as low. The automated tool, Coh-Metrix, is used to examine the degree to which high- and low-proficiency essays can be predicted by linguistic indices of cohesion (i.e., coreference and connectives), syntactic complexity (e.g., number of words before the main verb, sentence structure overlap), the diversity of words used by the writer, and characteristics of words (e.g., frequency, concreteness, imagability). The three most predictive indices of essay quality in this study were syntactic complexity (as measured by number of words before the main verb), lexical diversity (as measured by the Measure of Textual Lexical Diversity), and word frequency (as measured by Celex, logarithm for all words). Using 26 validated indices of cohesion from Coh-Metrix, none showed differences between high- and low-proficiency essays and no indices of cohesion correlated with essay ratings. These results indicate that the textual features that characterize good student writing are not aligned with those features that facilitate reading comprehension. Rather, essays judged to be of higher quality were more likely to contain linguistic features associated with text difficulty and sophisticated language.

Journal
Written Communication
Published
2010-01-01
DOI
10.1177/0741088309351547
CompPile
Open Access
Closed
Topics
Export
BibTeX RIS

Citation context

Cited by in this index (47)

  1. Cuberos et al. (2026)
    Text quality and vocabulary in native and non-native Spanish analytical writing…
    Assessing Writing
  2. Caltabellotta et al. (2026)
    Measuring vocabulary use in L2 English and L2 French writing: How methodologica…
    Assessing Writing
  3. Gan et al. (2026)
    From keystrokes to scores: Toward a multidimensional predictive model of writin…
    Assessing Writing
  4. Li et al. (2026)
    Assessing the effects of explicit coherence instruction on EFL students’ integr…
    Assessing Writing
  5. Terrill (2026)
    Tracing the Impact of Writing Center Tutoring on Graduate Dissertation Writing
    Journal of Writing Analytics
Show all 47 →
  1. Zao et al. (2025)
    Comparing the Readability of English-Language CEO Statements in Chinese and Ame…
    IEEE Transactions on Professional Communication
  2. Huang et al. (2025)
    Assessing L2 writing formality using syntactic complexity indices: A fuzzy eval…
    Assessing Writing
  3. Potter et al. (2025)
    Assessing academic language in tenth grade essays using natural language processing
    Assessing Writing
  4. Zappoli et al. (2024)
    A Synchronic and Diachronic Study of Students’ Essays in Italian High Schools: …
    Written Communication
  5. Tate et al. (2024)
    Linguistic Features of Secondary School Writing: Can Natural Language Processin…
    Written Communication
  6. Emmanouil et al. (2024)
    Linguistic factors affecting L1 language evaluation in argumentative essays of …
    Assessing Writing
  7. Wei et al. (2024)
    Effects of task-based language teaching on functional adequacy in L2 writing
    Assessing Writing
  8. Effatpanah et al. (2024)
    A mixed Rasch model analysis of multiple profiles in L2 writing
    Assessing Writing
  9. Wang et al. (2023)
    Writing Quality Predictive Modeling: Integrating Register-Related Factors
    Written Communication
  10. Tabari et al. (2023)
    Exploring new insights into the role of cohesive devices in written academic genres
    Assessing Writing
  11. Qian (2023)
    Use of lexical features in high-stakes tests: Evidence from the perspectives of…
    Assessing Writing
  12. Yang et al. (2023)
    Predicting EFL expository writing quality with measures of lexical richness
    Assessing Writing
  13. Golparvar et al. (2022)
    Unpacking the contribution of linguistic features to graph writing quality: An …
    Assessing Writing
  14. Huang et al. (2022)
    The trajectory of syntactic complexity development in L1 Chinese narrative writ…
    Assessing Writing
  15. Steiss et al. (2022)
    Dimensions of text-based analytical writing of secondary students
    Assessing Writing
  16. Crossley et al. (2022)
    Linguistic Features of Writing Quality and Development: A Longitudinal Approach
    Journal of Writing Analytics
  17. Uzun (2021)
    Performance prediction strengths of noun and verb phrases in L2 writing: Compar…
    Assessing Writing
  18. Riemenschneider et al. (2021)
    Linguistic complexity in teachers' assessment of German essays in high stakes testing
    Assessing Writing
  19. Maamuujav (2021)
    Examining lexical features and academic vocabulary use in adolescent L2 student…
    Assessing Writing
  20. McDonough et al. (2021)
    Instructor Evaluation of Business Student Writing: Does Language Play a Role?
    Business and Professional Communication Quarterly
  21. Jo (2021)
    Exploring General Versus Academic English Proficiency as Predictors of Adolesce…
    Written Communication
  22. Barrot et al. (2021)
    Complexity, accuracy, and fluency as indices of college-level L2 writers’ proficiency
    Assessing Writing
  23. Corrigan et al. (2021)
    Articulating a Sociocognitive Construct of Writing Expertise for the Digital Age
    Journal of Writing Analytics
  24. Kyle (2020)
    The relationship between features of source text use and integrated writing quality
    Assessing Writing
  25. Troia et al. (2019)
    Multidimensional Levels of Language Writing Measures in Grades Four to Six
    Written Communication
  26. Vögelin et al. (2019)
    The influence of lexical features on teacher judgements of ESL argumentative essays
    Assessing Writing
  27. Lim (2019)
    An investigation of the text features of discrepantly-scored ESL essays: A mixe…
    Assessing Writing
  28. Liu et al. (2018)
    Connecting writing assessment with critical thinking: An exploratory study of a…
    Assessing Writing
  29. Kim et al. (2018)
    Modeling second language writing quality: A structural equation investigation o…
    Assessing Writing
  30. Barkaoui et al. (2018)
    The effects of writing mode and computer ability on L2 test-takers' essay chara…
    Assessing Writing
  31. Balling (2018)
    No Effect of Writing Advice on Reading Comprehension
    Journal of Technical Writing and Communication
  32. Banerjee et al. (2015)
    Keeping up with the times: Revising and refreshing a rating scale
    Assessing Writing
  33. Aryadoust et al. (2015)
    Predicting EFL writing ability from levels of mental representation measured by…
    Assessing Writing
  34. Aull (2015)
    Connecting writing and language in assessment: Examining style, tone, and argum…
    Assessing Writing
  35. McNamara et al. (2015)
    A hierarchical classification approach to automated essay scoring
    Assessing Writing
  36. Roscoe et al. (2014)
    The Writing Pal Intelligent Tutoring System: Usability Testing and Development
    Computers and Composition
  37. Crossley et al. (2014)
    What Is Successful Writing? An Investigation Into the Multiple Ways Writers Can…
    Written Communication
  38. Struthers et al. (2013)
    Assessing cohesion in children’s writing: Development of a checklist
    Assessing Writing
  39. Dryer (2013)
    Scaling Writing Ability: A Corpus-Driven Inquiry
    Written Communication
  40. Uccelli et al. (2013)
    Mastering Academic Language: Organization and Stance in the Persuasive Writing …
    Written Communication
  41. Thonney (2012)
    “That's Beyond the Scope of This Paper”: Analyzing the Functions of a Familiar …
    Rhetoric Review
  42. Crossley et al. (2011)
    The Development of Writing Proficiency as a Function of Grade Level: A Linguist…
    Written Communication

References (59) · 3 in this index

  1. Baayen, R.H. (1995)
    The CELEX lexical database
  2. 10.1075/sl.7.3.05bam
     ↗
  3. Biber, D. ( 2003). Variation among university spoken and written registers: A new multidimensional analysis. …
     ↗
  4. Bormuth, J.R. (1969)
    Development of readability analyses
  5. Brace, N. (2006)
    SPSS for psychologists: A guide to data analysis using SPSS for Windows
Show all 59 →
  1. Chall, J. (1995)
    Readability revisited: The new Dale-Chall readability formula
  2. Charniak, E.
    Proceedings of the First Conference on North American Chapter of the Association for Comp…
  3. Collins (1998)
    Strategies for Struggling Writers
    College Composition and Communication
  4. 10.1080/14640748108400805
     ↗
  5. 10.1080/08351818409389208
     ↗
  6. 10.1111/j.1540-4781.2007.00507.x
     ↗
  7. 10.2307/1495373
     ↗
  8. 10.2307/747483
     ↗
  9. DeVillez, R. (2003)
    Writing: Step by step
  10. 10.1017/CBO9780511816796.038
     ↗
  11. 10.4324/9780203506929
     ↗
  12. Field, A. (2005)
    Discovering statistics using SPSS
  13. 10.4324/9780203463932_UC_and_the_SAT
     ↗
  14. 10.21236/ADA221854
     ↗
  15. 10.1075/z.74
     ↗
  16. 10.1075/idj.15.3.02gra
     ↗
  17. Graesser, A.C., McNamara, D.S. & Louwerse, M.M. (2003). What do readers need to learn in order to process coh…
  18. 10.3758/BF03195564
     ↗
  19. Haberlandt, K. (1985)
    Journal of Experimental Psychology: Human Learning and Memory
  20. Hall, C. (2007)
    University of Arizona Working Papers in Linguistics
  21. Halliday, M.A.K. (1976)
    Cohesion in English
  22. Just, M.A. (1987)
    The psychology of reading and language comprehension
  23. 10.1037/0033-295X.99.1.122
     ↗
  24. Kellogg (2008)
    Training writing skills: A cognitive developmental perspective
    Journal of Writing Research
  25. 10.4324/9780203936399
     ↗
  26. Leki, I. ( 1993). Reciprocal themes in ESL reading and writing. In J. Carson & I. Leki (Eds.), Reading in the…
  27. Light, R. (2001)
    Making the most of college
     ↗
  28. Lightman, E.J., McCarthy, P.M., Dufty, D.F. & McNamara, D.S. (2007). The structural organization of high scho…
  29. 10.1016/j.system.2005.02.002
     ↗
  30. Louwerse, M.M. (2001)
    10.1515/cogl.2002.005
    Cognitive Linguistics  ↗
  31. Louwerse, M.M., McCarthy, P.M., McNamara, D.S. & Graesser, A.C. (2004). Variation in language and cohesion ac…
  32. 10.2190/1LN8-7BQE-8TN0-M91L
     ↗
  33. McCarthy, P.M. (2005)
    Dissertation Abstracts International
  34. McCarthy, P.M., Briner, S.W., Rus, V. & McNamara, D.S. (2007). Textual signatures: Identifying text-types usi…
     ↗
  35. 10.1177/0265532207080767
     ↗
  36. McCarthy, P.M.
    Proceedings of the 19th Annual Florida Artificial Intelligence Research Society Internati…
  37. 10.1207/S15326985EP3501_3
     ↗
  38. 10.1037/h0087352
     ↗
  39. 10.4324/9780203810033
     ↗
  40. McNamara, D.S.
    Discourse Processes
  41. 10.3758/BF03195736
     ↗
  42. 10.1093/ijl/3.4.235
     ↗
  43. National Center for Educational Statistics. (2003)
    2002 NAEP Writing 2002 State Snapshot Records (NCES 2003-532)
  44. 10.1080/01638530709336895
     ↗
  45. 10.3102/0002831206298171
     ↗
  46. 10.1207/s1532690xci0102_1
     ↗
  47. 10.1037/0278-7393.31.6.1281
     ↗
  48. Raimes, A. ( 2001). What unskilled ESL students do as they write: A classroom study of composing. In T. Silva…
  49. Ransdell, S. (2003)
    Arob@se
  50. Raynor, A. (1994)
    The psychology of reading
  51. 10.1006/jecp.1996.0054
     ↗
  52. Tabachnick, B.G. (1989)
    Using multivariate statistics
  53. Todd et al. (2007)
    Coherence, cohesion and comments on students’ academic essays
    Assessing Writing
  54. 10.2307/356693
     ↗
CrossRef global citation count: 342 View in citation network → Build reading path →

Related articles

  1. Assessing Writing Oct 2026
    How the complex verb phrase complexity measures predict Chinese L2 writing quality ↗
    Yuxin Hao; Jinshu Li; Xuelin Wang; Haitao Liu
    multilingual writers grammar and mechanics race and writing
  2. Assessing Writing Jul 2026
    From keystrokes to scores: Toward a multidimensional predictive model of writing evaluation by humans and large language models across linguistic, cognitive, and social dimensions ↗
    Qiao Gan; Benjamin Adams
    assessment artificial intelligence grammar and mechanics race and writing gender and writing
  3. Journal of Writing Research Jun 2026
    Good text quality despite deficient transcription skills: An analysis of double dissociation ↗
    Rebecca Maria Kreutz; Julie Wolf; Ann-Kathrin Hennes; Alfred Schabmann
    writing pedagogy grammar and mechanics race and writing
  4. Journal of Writing Research Jun 2026
    The relevance of executive functions for writing competence in skilled writers ↗
    Julie Wolf; Rebecca Maria Kreutz; Ann-Kathrin Hennes; Alfred Schabmann
    grammar and mechanics race and writing
  5. Res Rhetorica Jan 2026
    Review/Recenzja: Nancy Organ. 2024. Data Visualization for People of All Ages. Oxon: CRC Press; and Jen Christiansen. 2023. Building Science Graphics: An Illustrated Guide to Communicating Science Through Diagrams and Visualizations. Oxon: CRC Press ↗
    Ewa Modrzejewska
    modern rhetorical theory rhetorical criticism discourse analysis digital rhetoric multimodality grammar and mechanics race and writing book reviews