Assessing Writing

Although previous research has explored how L2 writers allocate their time and how time limits influence writing scores, many of these studies focus solely on final texts, providing only a partial understanding of the writing process. A more comprehensive investigation is essential for enhancing the construct validity of writing assessments. Drawing on empirical research on timed writing and on theoretical models of the writing process, this study explores how time constraints affect L2 learners' written texts and fluency behaviors. A total of 123 participants (60 high-intermediate and 63 advanced EFL learners) completed an argumentative writing task with keystroke logging software under either a 30-minute or 60-minute condition, and stimulated recalls were conducted with a subset. The texts were analyzed for syntactic and lexical complexity, fluency, and overall quality. Results indicate that time constraints did not significantly affect linguistic complexity or writing quality, although fluency behaviors differed across conditions. Time constraints thus affected learners' writing processes more than the linguistic quality of their texts. The core implication for L2 writing assessment is that, for test takers experienced with timed writing, shortening the time allotted to an argumentative task changes composing processes without compromising the product-based scores on which assessment decisions rest.

As automated writing evaluation (AWE) systems proliferate, it is important to assess them for the extent to which they serve an authentic formative assessment purpose. We developed eRevise as an AWE system to engage upper elementary students in text-based writing, receive automated feedback, and revise their essay based on that feedback. In past work, we presented a validity argument for a response-to-text formative assessment. Here, we replicate evidence for the mediational processes we identified with a second response-to-text formative assessment. Beyond replication, we expand the validity argument in several ways: We examine multiple writing outcomes to understand whether the relationships in the data generalize. We also explore patterns of relationships to better understand which students are most likely to benefit from eRevise . Furthermore, we expand the investigation from feature score improvement alone to also consider students’ conceptual development over time. Our findings provide further evidence for sociocultural mechanisms supporting students’ conceptual development of evidence use in writing. We discuss the implications of our findings for future design of eRevise and AWE systems, in general. We also discuss the need for a recursive process where our findings also contribute to refined theory development and design of future AWE systems. (1). Formative Assessment; (2) Argumentative Writing; (3) Adaptive Expertise; (4) Conceptual Change; (5) Validity Argument.

Second language (L2) writers’ capacity to reason about language choices during revision is theoretically central to advanced writing development, yet few studies trace how this linguistic reasoning emerges in repeated feedback-revision cycles that integrate teacher and AI suggestions. This multi-case, longitudinal mixed-methods study examined how ten L2 learners enacted metalinguistic strategies and developed linguistic reasoning across a 12-week online Academic English program. Triangulated data sources included 1246 feedback revision episodes, pre-post reasoning-task responses, and semi-structured exit interviews. We operationalized linguistic reasoning as the processual construct of interest and treated metalinguistic strategies as observable indicators; reasoning responses were scored with a three-dimensional analytic rubric while revisions and interaction logs were coded thematically and by strategy. Results show a systematic shift from early, correctness-focused edits toward later revisions characterized by coordinated structural comparison, more explicit self-explanation, and integrative argumentation; composite reasoning scores increased, with the largest gains observed on depth of explanation. Process analyses identified recurrent patterns in which alternative-rich feedback coincided with comparison moves, partially divergent suggestions often coincided with more explicit self-explanation, and integrative argumentation involving discourse-level revision became more visible over time. Pedagogical implications are also discussed.

Existing scholarship abounds in pedagogical efforts to leverage the advantages of GenAI to support argumentative writing. Yet, these efforts have mainly positioned learners as consumers of GenAI feedback and support on writing, which is often associated with concerns of cognitive atrophy and metacognitive laziness. The emergence of AI-assisted vibe coding makes it possible for learners to play a more active role in creating personalized AI solutions. The repositioning of learners’ roles in relation to GenAI may induce deeper cognitive processing and greater metacognitive engagement. This article introduces AI-assisted vibe coding, a novel no-code development approach that enables learners as technology creators by designing and developing their own personalized argumentative writing tools. We present Base44, a full-stack, AI-powered platform that facilitates this process, and illustrate how students can leverage it to create personalized tools for addressing authentic argumentative writing challenges through concrete workflow examples. We explore the potential of integrating AI-assisted vibe coding into argumentative writing instruction, examine its limitations, and offer directions for future teaching practice and research.

While previous studies on online resource use in L2 writing have focused on the overall writing quality, limited attention has been paid to its effects on linguistic complexity and real-time writing processes. Addressing this gap, the present study explored how online resource use influences both the processes and products of L2 writing. Forty-nine intermediate L2 learners completed two computer-mediated argumentative writing tasks, either with or without the use of online resources. Writing behaviors were captured via keystroke logging and screen recording, and analyzed for search activity, fluency, pausing, and revision quantity. Cognitive processes were examined through stimulated recall interviews, and written products were evaluated for both quality and linguistic complexity. The results showed that participants spent an average of 14 % of task time using online resources, with considerable individual variation. Mixed-effects modeling revealed that resource use facilitated the production of more sophisticated words, with marginal influence on writing quality or syntactic complexity. Resource use was also associated with longer between-word pauses, fewer within-word pauses, and reduced revisions. These findings highlight the potential of online resource use to enhance the authenticity of L2 writing assessment tasks without compromising test validity, while encouraging the use of more advanced vocabulary in writing. • Learners spent 14 % of the total writing task time using online resources. • Online resource use had no significant impact on L2 writing quality. • Online resource use improved lexical sophistication, not syntactic complexity. • Online resource use reduced within-word pauses and aided spelling retrieval. • Online resource use led to fewer revisions but did not affect fluency.

Peer evaluation is widely recognized for its educational benefits; however, its reliability and validity, particularly among adolescent second-language (L2) writers at the early stages of English language and literacy development, remain insufficiently explored. This explanatory sequential mixed-methods study investigated the reliability and validity of peer evaluation in English argumentative writing among 35 Grade 10 and 37 Grade 12 students from a public high school in Beijing, China. Twelve of the participating students (six at each grade) were interviewed about the validity, reliability, and value of peer evaluation. The findings indicated that peer evaluations demonstrated high levels of reliability and validity, with peer-assessed writing scores closely aligning with inter-teacher assessments. Notably, variations were observed among Grade 10 students, particularly in the evaluation of lower-order writing skills, such as grammar and vocabulary, which exhibited reduced validity. These results underscore the potential of peer evaluation in assessing higher-order content-level writing across varying levels of L2 English writing proficiency. The study also highlights areas where adolescent L2 writers may require additional support to enhance the effectiveness of peer evaluation practices in English argumentative writing. Implications for improving English argumentative writing instruction and refining peer evaluation strategies in high school L2 English classrooms are discussed. • Peer evaluation shows high reliability, similar to inter-teacher rating. • Peer evaluation works well for higher-order skills in L2 argumentative writing. • 10th graders struggled with evaluating lower-order skills like grammar. • 12th graders evaluate lower- and higher-order skills with greater validity than 10th graders.

Perfectionism, a personality trait characterized by the pursuit of flawlessness and high personal standards, and motivational regulation, the strategies through which individuals manage their motivational states, have received limited attention in second language (L2) writing. Framed within social cognitive theory, this study examines how two dimensions of perfectionism—perfectionistic strivings and perfectionistic concerns—relate to writing performance (syntactic complexity, accuracy, lexical complexity, and fluency) and how motivational regulation sub-strategies (interest enhancement, self-talk, and emotional control) mediate these relationships. Data from 689 university students in China were analyzed using questionnaires and argumentative writing samples. Results indicated that perfectionistic strivings positively predicted syntactic complexity, accuracy, and lexical complexity, while perfectionistic concerns negatively predicted these dimensions; neither dimension significantly affected fluency. Crucially, motivational regulation sub-strategies partially mediated the relations between perfectionism and writing performance. These findings underscore the importance of distinguishing perfectionism dimensions and targeting motivational regulation strategies to improve L2 writing. Implications for instruction and directions for future longitudinal research are discussed. • Perfectionistic strivings and concerns affect writing via motivational regulation. • Strivings improve syntax, accuracy, and lexical complexity; concerns hinder them. • Most motivational regulation sub-strategies mediate perfectionism’s impact on CALF. • Perfectionism influences writing through motivational regulation.

To meet the current trends in higher education, there is accountability on EAP programmes to prepare and assess students’ access to higher education. Thus, multimodal tasks including integrated writing (IW) assessments have seen a resurgence because they arguably closely mirror academic writing. However, test practicality constraints and variability in the use and format of these assessments mean rating scales often fall short in substantiating the central claims of IW assessment. We developed an integrated reading-writing scale taking into account reading-writing requirements and empirical research on IW tests designed to assess readiness for first-year humanities and social science courses. We approached test development as part of the ongoing validation efforts, detailing the considerations involved in the scale development process. We argue that alignment with academic writing requirements should guide the development of IW tests, thereby acknowledging and comprehending nuances of academic writing. The paper demonstrates considerations and decisions in scale design as the validation process from the start, which is a reminder that assessment is not just a quantitative exercise but a multifaceted process. • The design of a rating scale for first-year undergraduate academic writing is detailed. • Emphasis is placed on the role of reading in integrated writing scales. • Academic argumentation, rather than solely source-use mechanics, is considered. • Implications for construct operationalisation in academic evaluations are offered.

Verbatim source use (VSU) in integrated argumentative writing tasks may enhance linguistic complexity of writing performance. This assistance might present an unequal advantage for test-takers across levels of writing proficiency, engendering validity and fairness concerns. While previous research has mostly examined the relationships between source use characteristics and proficiency levels, the relationship between VSU and linguistic complexity remains underexplored. To further unpack these relationships, this study examined both the direct impact of VSU on linguistic complexity of writing performances and its indirect impact through interaction with writing proficiency. Using natural language processing tools and techniques, we examined 34 linguistic complexity features and three VSU features of 3250 argumentative writing performances on a university-level English Placement Test (EPT). We performed exploratory factor analysis to identify linguistic complexity dimensions and applied mixed-effect models to examine how VSU features and proficiency level impacted these dimensions. Post-hoc analyses suggested weak direct impacts of different VSU features on linguistic complexity, which might reflect different essay writing strategies. However, no meaningful indirect impact was found. The findings help unravel the impact of VSU on argumentative writing and provide empirical evidence for validity arguments for integrated writing assessments.

This research methods article introduces the open source PERSUADE 2.0 corpus. The PERSUADE 2.0 corpus comprises over 25,000 argumentative essays produced by 6th-12th grade students in the United States for 15 prompts on two writing tasks: independent and source-based writing. The PERSUADE 2.0 corpus also provides detailed individual and demographic information for each writer. The goal of the PERSUADE 2.0 corpus is to advance research into relationships between discourse elements, their effectiveness, writing quality, writing tasks and prompts, and demographic and individual differences.

The study explored EFL students' development of syntactic complexity by employing the Latent Class Growth Analysis (LCGA) approach. A total of 214 tertiary EFL students from Southwest China were invited to write four argumentative essays over an academic semester. The unconditional models of LCGA were utilized to explore the optimal latent classes of students' development trajectories of syntactic complexity. The conditional models of LCGA were employed to investigate the predictive effect of English proficiency on the optimal latent classes. Results of the unconditional models revealed different latent classes of development trajectories for six indices of syntactic complexity rather than the remaining ones, which offers tentative evidence for the heterogeneity of L2 development trajectories. Results of the conditional models showed that English proficiency did not predict the membership in these latent classes. These results are discussed and implications for L2 instruction are attempted.

This paper introduces the Persuasive Essays for Rating, Selecting, and Understanding Argumentative and Discourse Elements (PERSUADE) corpus.The PERSUADE corpus is large-scale corpus of writing with annotated discourse elements. The goal of the corpus is to spur the development of new, open-source scoring algorithms that identify discourse elements in argumentative writing to open new avenues for the development of automatic writing evaluation systems that focus more specifically on the semantic and organizational elements of student writing.

The objectives of this paper are to identify the metadiscourse devices used by English learners at the different levels of language acquisition established by the Common European Framework of Reference and to categorise them to facilitate the assessment and learning of textual and interpersonal devices. First, a learner corpus of essays written by English learners was compiled. Then, the metadiscourse devices were classified in different levels and categories. The results showed the lists and frequencies of metadiscourse devices. The examples aim to make additional and explicit connections between levels of language proficiency and assessment of metadiscourse devices. It can be stated, as a conclusion, that metadiscourse devices portray specific ways of argumentation in essay writing in different levels of EFL proficiency.