Assessing Writing Jan 2025 Open Access

A meta-analysis of relationships between syntactic features and writing performance and how the relationships vary by student characteristics and measurement features

Jiali Wang University of California, Irvine ; Young-Suk G. Kim ; Joseph Hin Yan Lam University of California, Irvine ; Molly Ann Leachman

Abstract

Students’ proficiency in constructing sentences impacts the writing process and writing products. Linguistic demands in writing differ in terms of both student characteristics and measurement features. To identify various syntactic demands considering these features, we conducted a meta-analysis examining the relationships between syntactic features (complexity and accuracy) and writing performance (quality, productivity, and fluency) and moderating effects of both student characteristics and measurement features. A total of 109 studies (effect sizes: 871; the total number of participants: 24,628) met the inclusion criteria. Results showed that there was a weak relationship for syntactic accuracy (r = .25) and complexity (r = .16). Writers' characteristics, including grade level and language proficiency, and measurement features, writing genres, writing outcomes, whether the writing task is text-based or not, and type of syntactic complexity measures, were significant moderators for certain syntactic features. The findings highlighted the importance of writer and measurement factors when considering the relationships between linguistic features in writing and writing performance. Implications were discussed regarding the selection of syntactic features in assessing language use in writing, gaps in the literature, and significance for writing instruction and assessment. • Aimed to depict the relationships between syntactic features and writing performance. • Found weak relationships between syntactic features and writing outcomes. • Relationships vary as a function of student characteristics and measurement features. • Noun phrase complexity might be more valid than some traditional syntactic complexity measures. • Findings have important implications for writing assessments.

Journal
Assessing Writing
Published
2025-01-01
DOI
10.1016/j.asw.2024.100909
CompPile
Open Access
OA PDF Hybrid
Topics
Export
BibTeX RIS

Citation context

Cited by in this index (0)

No articles in this index cite this work.

References (56) · 13 in this index

  1. Andrews (2006)
    The effect of grammar teaching on writing development
    British Educational Research Journal  ↗
  2. Ansarifar (2018)
    Phrasal complexity in academic writing: A comparison of abstracts written by graduate stu…
    Journal of English for Academic Purposes  ↗
  3. Beers (2009)
    Syntactic complexity as a predictor of adolescent writing quality: Which measures? Which genre?
    Reading and Writing  ↗
  4. Beers. (2011)
    Writing development in four genres from grades three to seven: Syntactic complexity and g…
    Reading and Writing  ↗
  5. Berninger (2002)
    Teaching spelling and composition alone and together: Implications for the simple view of…
    Journal of Educational Psychology  ↗
Show all 56 →
  1. Biber (2011)
    Grammatical change in the noun phrase: The influence of written language use
    English Language Linguistics  ↗
  2. Biber (2016)
    Predicting patterns of grammatical complexity across language exam task types and profici…
    Applied Linguistics  ↗
  3. Bulté (2014)
    Conceptualizing and measuring short-term changes in L2 writing complexity
    Journal of Second Language Writing  ↗
  4. Casal (2019)
    Syntactic complexity and writing quality in assessed first-year L2 writing
    Journal of Second Language Writing  ↗
  5. Crossley (2020)
    Linguistic features in writing quality and development: An overview
    Journal of Writing Research
  6. Crossley (2014)
    Does writing development equal writing quality? A computational investigation of syntacti…
    Journal of Second Language Writing  ↗
  7. Crossley et al. (2011)
    The Development of Writing Proficiency as a Function of Grade Level: A Linguist…
    Written Communication
  8. Crowhurst (1980)
    Syntactic Complexity and Teachers’ Quality Ratings of Narrations and Arguments
    Research in the Teaching of English
  9. Crowhurst (1983)
    Syntactic complexity and writing quality: A review
    Canadian Journal of Education/Revue canadienne de l′education
  10. Crowhurst et al. (1979)
    Audience and Mode of Discourse Effects on Syntactic Complexity in Writing at Tw…
    Research in the Teaching of English
  11. Cumming et al. (2005)
    Differences in written discourse in independent and integrated prototype tasks …
    Assessing Writing
  12. Cumming (2016)
    Students' writing from sources for academic purposes: A synthesis of recent research
    Journal of English for Academic Purposes  ↗
  13. Datchuk (2021)
    Accounting for levels of language in narrative and expository writing: A skills analysis …
    The Elementary School Journal  ↗
  14. Ferris (1994)
    Lexical and syntactic features of ESL writing by students at different levels of L2 proficiency
    TESOL Quarterly  ↗
  15. Graham (2018)
    A revised writer(s)-within-community model of writing
    Educational Psychologist  ↗
  16. Grant (2000)
    Using computer-tagged linguistic features to describe L2 writing differences
    Journal of Second Language Writing  ↗
  17. Grobe (1981)
    Syntactic Maturity, Mechanics, and Vocabulary as Predictors of Quality Ratings
    Research in the Teaching of English
  18. Guo et al. (2013)
    Predicting human judgments of essay quality in both integrated and independent …
    Assessing Writing
  19. Hayes (1996)
    A new framework for understanding cognition and affect in the writing process
    The sciences of writing
  20. Hedges (2010)
    Robust variance estimation in meta-regression with dependent effect size estimates
    Research Synthesis Methods  ↗
  21. Hunt (1970)
    Syntactic maturity in school children and adults
    Monographs of the Society for Research in Child Development  ↗
  22. Jagaiah (2020)
    Syntactic complexity measures: variation by genre, grade-level, students' writing abiliti…
    Reading and Writing  ↗
  23. Jones (2013)
    Grammar for writing? An investigation of the effects of contextualised grammar teaching o…
    Reading and Writing  ↗
  24. Kim et al. (2018)
    Modeling second language writing quality: A structural equation investigation o…
    Assessing Writing
  25. Kim (2014)
    Evaluating the dimensionality of first-grade written composition
    Journal of Speech, Language, and Hearing Research  ↗
  26. Kim (2022)
    Expanding the direct and indirect effects model of writing (DIEW): Dynamic relations of c…
    Journal of Educational Psychology  ↗
  27. Kyle (2018)
    Measuring syntactic complexity in L2 writing using fine-grained clausal and phrasal indices
    The Modern Language Journal  ↗
  28. Li, H. (2015). Relationship between measures of syntactic complexity and judgments of EFL writing quality. In…
  29. Loban, W. (1976). Language development: Kindergarten through grade twelve. NCTE Committee on Research Report …
  30. Lu (2011)
    A corpus-based evaluation of syntactic complexity measures as indices of college-level ES…
    TESOL Quarterly  ↗
  31. Lu (2017)
    Automated measurement of syntactic complexity in corpus-based L2 writing research and imp…
    Language Testing  ↗
  32. MacArthur (2019)
    Which linguistic features predict quality of argumentative writing for college basic writ…
    Reading and Writing  ↗
  33. Norris (2009)
    Towards an organic approach to investigating CAF in instructed SLA: The case of complexity
    Applied Linguistics  ↗
  34. Olinghouse (2013)
    The relationship between vocabulary and writing quality in three genres
    Reading and Writing  ↗
  35. Ortega (2003)
    Syntactic complexity measures and their relationship to L2 proficiency: A research synthe…
    Applied Linguistics  ↗
  36. Ouzzani (2016)
    Rayyan—A web and mobile app for systematic reviews
    Systematic Reviews  ↗
  37. Page (2021)
    The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
    BMJ
  38. Puranik, C. S., Lombardino, L. J., & Altmann, L. J. (2008). Assessing the microstructure of written language …
     ↗
  39. Qin (2016)
    Same language, different functions: A cross-genre analysis of Chinese EFL learners' writi…
    Journal of Second Language Writing  ↗
  40. Ravid (2002)
    Developing linguistic literacy: A comprehensive model
    Journal of Child Language  ↗
  41. Schleppegrell (1998)
    Grammar as Resource: Writing a Description
    Research in the Teaching of English
  42. Scott (2000)
    General language performance measures in spoken and written narrative and expository disc…
    Journal of Speech, Language, and Hearing Research  ↗
  43. Staples et al. (2016)
    Academic Writing Development at the University Level: Phrasal and Clausal Compl…
    Written Communication
  44. Steiss et al. (2024)
    U.S. Secondary Students’ Source-Based Argument Writing in History
    Written Communication
  45. Sterne (2005)
    Regression methods to detect publication and other bias in meta-analysis
    Publication bias in meta-analysis: Prevention, assessment and adjustments
  46. Stewart et al. (1979)
    Syntactic Maturity, Mechanics of Writing, and Teachers’ Quality Ratings
    Research in the Teaching of English
  47. Taguchi (2013)
    What linguistic features are indicative of writing quality? A case of argumentative essay…
    TESOL Quarterly  ↗
  48. Troia et al. (2019)
    Multidimensional Levels of Language Writing Measures in Grades Four to Six
    Written Communication
  49. Viechtbauer (2010)
    Conducting meta-analyses in R with the metafor package
    Journal of Statistical Software  ↗
  50. Wagner (2011)
    Modeling the development of written language
    Reading and Writing  ↗
  51. Wang (2024)
    Linguistic features in narrative and opinion genres and their relations to writing qualit…
    Journal of Research in Reading  ↗
CrossRef global citation count: 2 View in citation network → Build reading path →

Related articles

  1. Journal of Response to Writing Apr 2026
    Examining Automated Writing Evaluation Error Coverage in Relation to Uptake and Retention ↗
    Mehrzad, Mohaddeseh; Rahimi, Mohammad; Link, Stephanie
    genre theory writing pedagogy teacher development revision assessment artificial intelligence multilingual writers grammar and mechanics
  2. Pedagogy Jan 2022
    Contributors ↗
    modern rhetorical theory rhetorical criticism genre theory discourse analysis african american rhetorics decolonial rhetorics first-year composition writing pedagogy basic writing writing across the curriculum graduate education teacher development argument collaborative writing transfer assessment portfolios writing program administration writing centers peer tutoring technical communication professional writing archival research digital rhetoric social media grammar and mechanics literacy studies race and writing gender and writing disability studies public rhetoric community literacy literary studies editorial matter
  3. Written Communication Jul 2026
    The Dynamics Between the Biographical Factors, Professional Identities, and Teaching Practices of Second Language Writing Instructors ↗
    Young Wha Lee
    first-year composition writing pedagogy graduate education teacher development assessment qualitative research multilingual writers grammar and mechanics
  4. Writing and Pedagogy Jun 2026
    The Impact of Statutory Assessments on Writing Instruction in English Primary Schools: An Exploration of Teacher Perceptions and Practices ↗
    Rebecca Clarkson
    writing pedagogy teacher development assessment grammar and mechanics
  5. Assessing Writing Jan 2026
    How reliable and valid is peer evaluation in adolescents’ L2 argumentative writing? ↗
    Albert W. Li; Steve Graham
    writing pedagogy teacher development argument assessment multilingual writers grammar and mechanics literacy studies