Written Communication Apr 2023

Prompting Reflection: Using Corpus Linguistic Methods in the Local Assessment of Reflective Writing

Cara Marta Messina Jacksonville State University ; Cherice Escobar Jones Northeastern University ; Mya Poe Northeastern University

Abstract

We report on a college-level study of student reflection and instructor prompts using scoring and corpus analysis methods. We collected 340 student reflections and 24 faculty prompts. Reflections were scored using trait and holistic scoring and then reflections and faculty prompts were analyzed using Natural Language Processing to identify linguistic features of high, middle, and low scoring reflections. The data sets were then connected to determine if there was a relationship between faculty prompts and scores. Additional analysis was completed to determine if there was a relationship between scores and students’ GPAs. The corpus linguistics analysis showed that higher-scoring reflections used words that referred to the self, the writing process, and specific rhetorical terms. Additional analysis showed student GPAs did not correlate with holistic scores but that higher scoring reflections were from faculty who included learning goals on reflective writing prompts. Results suggest that teachers can de-mystify reflective writing by linking learning outcomes to textual tasks and that corpus linguistics methods can provide an understanding of how local learning goals are transmitted to students.

Journal
Written Communication
Published
2023-04-01
DOI
10.1177/07410883221149425
CompPile
Open Access
Closed
Topics
Export
BibTeX RIS

Citation context

Cited by in this index (4)

  1. Toth et al. (2026)
    Generative Artificial Intelligence, Writing Placement, and Principled Decision …
    Journal of Writing Analytics
  2. Hwang (2025)
    The Impact of Subordination Type and Finiteness on Second Language Development …
    Written Communication
  3. Nastal (2024)
    Ungrading in the Ethical Turn as an Assessment Killjoy
    Teaching English in the Two-Year College
  4. Nastal (2024)
    A Sociocognitive Grading Model for First-Year Writing Classes
    Pedagogy

References (54) · 15 in this index

  1. Allan et al. (2014)
    The three-fold benefit of reflective writing: Improving program assessment, stu…
    Assessing Writing
  2. 10.1109/FIE44824.2020.9273975
     ↗
  3. Beaufort A. (2007)
    College writing and beyond: A new framework for university writing instruction
  4. Borsheim-Black (2015)
    Beyond the Language Barrier: Opening Spaces for ELL/Non-ELL Interaction
    Research in the Teaching of English
  5. 10.1080/0969594X.2014.996523
     ↗
Show all 54 →
  1. 10.5172/mra.3.2.140
     ↗
  2. College Board. (2020). AP ® English language and composition: Cou…
  3. Council of Writing Program Administrators. (2019). WPA outcomes statement for first-year composition (3.0). R…
  4. Dewey J. (1909)
    How we think
  5. Driscoll (2011)
    Connected, Disconnected, or Uncertain: Student Attitudes about Future Writing C…
    Across the Disciplines
  6. Driscoll et al. (2020)
    Genre Knowledge and Writing Development: Results From the Writing Transfer Project
    Written Communication
  7. Education Research Center at Texas A&M University. (2015). Student reflection: A mixed method study of “refle…
  8. 10.1037/0003-066X.34.10.906
     ↗
  9. Gallagher C. W. (2010)
    WPA: Writing Program Administration
  10. 10.18608/jla.2016.32.3
     ↗
  11. 10.37514/PER-B.2016.0797.2.08
     ↗
  12. Government of Alberta. (2022). Oral language: Skills and procedures. Learn Alberta. https://curriculum.learna…
  13. 10.1016/B978-012330595-4/50003-2
     ↗
  14. Householder et al. (2020)
    Writing Analytics and Program Assessment: How Novice Writers Use Rubric Termino…
    Journal of Writing Analytics
  15. 10.2307/358601
     ↗
  16. 10.7330/9781607325161.c007
     ↗
  17. International Baccalaureate. (2022). Learner profile. https://www.ibo.org/benefits/learner-profile/
  18. Johnson et al. (2020)
    Revealing the Human and the Writer: The Promise of a Humanizing Writing Pedagog…
    Research in the Teaching of English
  19. Kelly-Riley D. (2011). Validity inquiry of race and shared evaluation practices in a large-scale, university-…
  20. Kelly-Riley D. (2017). A case study exploring the connections between locally defined writing and student eng…
  21. Kelly-Riley et al. (2014)
    The WPA Outcomes Statement, validation, and the pursuit of localism
    Assessing Writing
  22. Lindenman et al. (2018)
    Revision and Reflection: A Study of (Dis)Connections between Writing Knowledge …
    College Composition and Communication
  23. 10.1016/j.chb.2021.106733
     ↗
  24. Massachusetts Department of Elementary and Secondary Education. (2017). Massachusetts curriculum framework fo…
  25. Moore (2021)
    Feature: Lexical Analysis of Reflection Texts in First-Year Composition
    Teaching English in the Two-Year College
  26. National Research Council (2012)
    Education for life and work: Developing transferable knowledge and skills in the 21st century
  27. Northeastern University. (2021). Facts and figures. Retrieved November 30, 2021, from https://facts.northeast…
  28. Northeastern University Writing Program. (2021). Learning goals. Retrieved November 30, 2021, from https://cs…
  29. Perkins D. N. (1992)
    The International Encyclopedia of Education
  30. Player (2021)
    “My Color of My Name”: Composing Critical Self-Celebration with Girls of Color …
    Research in the Teaching of English
  31. 10.5070/W414155459
     ↗
  32. Reiff et al. (2011)
    Tracing Discursive Resources: How Students Use Prior Genre Knowledge to Negotia…
    Written Communication
  33. Robertson L. (2012)
    Composition Forum
  34. 10.1353/csd.2015.0063
     ↗
  35. Schön D. A. (1987)
    Educating the reflective practitioner: Towards a new design for teaching and learning in …
  36. 10.1007/BF02212307
     ↗
  37. 10.1145/2883851.2883955
     ↗
  38. 10.37514/JBW-J.2011.30.1.05
     ↗
  39. Taczak K. (2011). Connecting the dots: Does reflection foster transfer? [Doctoral dissertation, Florida State…
  40. Trimble et al. (2019)
    Using Taxonomies of Metacognitive Behaviors to Analyze Student Reflection and I…
    Pedagogy
  41. 10.1145/3027385.3027394
     ↗
  42. 10.1007/s40593-019-00174-2
     ↗
  43. 10.1037/11193-000
     ↗
  44. Wang (2020)
    Becoming Multilingual Writers through Translation
    Research in the Teaching of English
  45. Wardle E. (2007)
    WPA: Journal of Writing Program Administrators
  46. White (2005)
    The Scoring of Writing Portfolios: Phase 2
    College Composition and Communication
  47. 10.2307/j.ctt46nsh0
     ↗
  48. 10.7330/9781607325161
     ↗
  49. Ybarra (2020)
    “We Have a Strong Way of Thinking … and It Shows through Our Words”: Exploring …
    Research in the Teaching of English
CrossRef global citation count: 8 View in citation network → Build reading path →

Related articles

  1. Journal of Writing Analytics Jan 2018
    Placing Writing Tasks in Local and Global Contexts: The Case of Argumentative Writing ↗
    Hannah Ringler
    modern rhetorical theory rhetorical criticism genre theory first-year composition writing pedagogy graduate education argument assessment writing program administration quantitative research book reviews
  2. Journal of Writing Analytics Jan 2018
    Is this Too Polite? The Limited Use of Rhetorical Moves in a First-Year Corpus ↗
    Thomas Peele
    rhetorical criticism writing pedagogy assessment quantitative research book reviews
  3. Journal of Academic Writing Nov 2016
    A Genre–based Study of Case Response Writing on an MBA Programme ↗
    Philip Bernard Nathan
    rhetorical criticism genre theory writing pedagogy assessment peer tutoring quantitative research
  4. Writing and Pedagogy May 2016
    English for Specific Academic Purposes (ESAP) Writing ↗
    John Flowerdew
    rhetorical criticism genre theory discourse analysis assessment qualitative research quantitative research
  5. IEEE Transactions on Professional Communication Jun 2012
    Different Approaches to Similar Challenges: An Analysis of the Occupational Cultures of the Disciplines of Technical Communication and Training Tutorial ↗
    Saul Carliner
    rhetorical criticism assessment technical communication qualitative research quantitative research