Journal of Writing Research Feb 2026 OA PDF

Using AI to understand students’ self-assessments of their writing

Madeleine Sorapure Santa Barbara City College ; Seth Erickson Santa Barbara City College ; Sarah Hirsch Santa Barbara City College ; Kenny Smith Santa Barbara City College

Abstract

This study focuses on a generative AI approach to facilitate qualitative analysis in Writing Studies research. We gathered 13,336 one-sentence to one-paragraph responses written by 3,334 incoming students in a directed self-placement program administered at a large R1 U.S. university. In these responses, students describe their high school writing experience and college writing expectations. In stage one of the project, we pilot the use of Retrieval-Augmented Generation to expedite the selection of relevant responses for a topic—in this case, students’ positive self-assessments as writers. The selected responses were then compared to a random sample and rated by three faculty with writing expertise. In stage two, these faculty generated codes and themes from a subset of the responses, incorporating ChatGPT-4 through the stages of thematic analysis. Results show that the use of AI expedites and enhances qualitative analysis, but human participation in the process is still essential. We suggest a machine-in-the-loop framework with which Writing Studies researchers can more readily integrate generative AI to study large corpora of student writing.

Journal
Journal of Writing Research
Published
2026-02-17
DOI
10.17239/jowr-2026.17.03.07
CompPile
Open Access
OA PDF Diamond
Topics
Export
BibTeX RIS

Citation context

Cited by in this index (1)

  1. Jiang et al. (2026)
    When writing students design AI chatbots: Integrating design thinking as a crit…
    Computers and Composition

References (0)

No references on file for this article.

CrossRef global citation count: 0 View in citation network → Build reading path →

Related articles

  1. Written Communication Jul 2026
    The Dynamics Between the Biographical Factors, Professional Identities, and Teaching Practices of Second Language Writing Instructors ↗
    Young Wha Lee
    first-year composition writing pedagogy graduate education teacher development assessment qualitative research multilingual writers grammar and mechanics
  2. The Peer Review Sep 2025
    The Impact of Working at a Writing Center in Brazil: Perspectives of Student Tutors ↗
    Ana Raquel Fialho Ferreira Campos; João Tiago Gaspar Cozechen; Elaine Pereira Lustosa; Marcos Angel De Carvalho Eing; Leonardo Schimiloski
    first-year composition writing pedagogy graduate education argument writing centers professional writing qualitative research artificial intelligence multilingual writers literacy studies race and writing
  3. Business and Professional Communication Quarterly May 2025
    “Don’t Ban, Teach”: Two Pilot Studies on AI Instruction in Business Communication ↗
    Emily Barrow DeJeu; Beth A. Walter; Michael C. Melville
    assessment professional writing qualitative research artificial intelligence affect and writing
  4. Journal of Academic Writing Apr 2025
    A Case Study in Mindfully Integrating AI Tools into Writing Classes ↗
    Gea Dreschler; Abby Gambrel; Jens Branum
    writing pedagogy assessment qualitative research artificial intelligence
  5. Journal of Academic Writing Feb 2025
    Generative AI for Academic Writing: Case Studies Beyond Simple Chatbot Interactions ↗
    Thomas Lancaster
    teacher development assessment peer tutoring qualitative research artificial intelligence