Experienced but detached from reality: Theorizing and operationalizing the relationship between experience and rater effects

Iasonas Lamprianou University of Cyprus ; Dina Tsagari OsloMet – Oslo Metropolitan University ; Nansia Kyriakou Frederick University
Journal
Assessing Writing
Published
2023-04-01
DOI
10.1016/j.asw.2023.100713
CompPile
Open Access
Closed
Topics
Export

Citation context

Cited by in this index (0)

No articles in this index cite this work.

References (60) · 6 in this index

  1. A comparison of newly-trained and experienced raters on a standardized writing assessment
    Language Testing  ↗
  2. What makes marking reliable? Experiments with UK examinations. Assessment in education
    Assessment in Education: Principles, Policy & Practice
  3. Individual differences in rater decision-making style: An exploratory mixed methods study
    Language Assessment Quarterly  ↗
  4. Do ESL essay raters’ evaluation criteria change with experience? A mixed‐methods, cross‐s…
    TESOL Quarterly  ↗
  5. Variability in ESL essay rating processes: The role of the rating scale and rater experience
    Language Assessment Quarterly  ↗
Show all 60 →
  1. Effects of marking method and rater experience on ESL essay scores and rater performance
    Assessment in Education: Principles, Policy & Practice
  2. A many-facet Rasch analysis of the second language group oral discussion task
    Language Testing  ↗
  3. Effect of genre on the generalizability of writing scores
    Language Testing  ↗
  4. Application of an automated essay scoring engine to english writing assessment using many…
    Language Testing  ↗
  5. Expertise in evaluating second language compositions
    Language Testing  ↗
  6. The influence of training and experience on rater performance in scoring spoken language
    Language Testing  ↗
  7. Examining Rater Effects in TestDaF Writing and Speaking Performance Assessments: A Many-F…
    Language Assessment Quarterly  ↗
  8. Evaluating rater responses to an online training program for L2 writing assessment
    Language Testing  ↗
  9. The effect of standardization-training on rater judgements for the IELTS writing module
    IELTS Collected Papers: Research in speaking and writing assessment
  10. Investigating rater severity/leniency in interpreter performance testing: A multifaceted …
    Interpreting International Journal of Research and Practice in Interpreting  ↗
  11. Raters’ perceptions of rating scales criteria and its effect on the process and outcome o…
    Language Testing in Asia  ↗
  12. Assessment in higher education: The potential for a community of practice to improve inte…
    Accounting Education  ↗
  13. The temporal experience today: A phenomenological study
    The American Journal of Psychoanalysis  ↗
  14. Validation of empirically derived rating scales for a story retelling speaking test
    Language Assessment Quarterly  ↗
  15. Assessing Writing
  16. Three approaches to qualitative content analysis
    Qualitative Health Research  ↗
  17. Evaluating CEFR rater performance through the analysis of spoken learner corpora
    Language Testing in Asia  ↗
  18. The influence of holistic scoring procedures on reading and rating student essays
    Validating holistic scoring for writing assessment: Theoretical and empirical foundations
  19. Rater experience, rating scale length, and judgments of L2 pronunciation: Revisiting rese…
    Language Assessment Quarterly  ↗
  20. Writing scale effects on raters: an exploratory study
    Language Testing in Asia  ↗
  21. Examining rater accuracy and consistency with a special education observation protocol
    Studies in Educational Evaluation  ↗
  22. Assessing Writing
  23. The stability of marker characteristics across tests of the same subject and across subjects
    Journal of Applied Measurement
  24. High stakes tests with self-selected essay questions: Addressing Issues of Fairness
    International Journal of Testing  ↗
  25. Comparability of examination standards between subjects: An international perspective
    Oxford Review of Education  ↗
  26. Unintended consequences of forced policy-making in high stakes examinations: The case of …
    Principles, Policy & Practice
  27. Investigation of rater effects using Social Network Analysis and Exponential Random Graph…
    Educational and Psychological Measurement  ↗
  28. Measuring and visualizing coders’ reliability: New approaches and guidelines from experim…
    Sociological Methods and Research  ↗
  29. Accuracyof measurement in the context of mathematics national curriculum tests in England…
    Journal of Educational Measurement  ↗
  30. The longitudinal stability of rating characteristics in an EFL examination: Methodologica…
    Language Testing  ↗
  31. A comparison of EFL raters’ essay-rating processes across two types of rating scales
    Language Assessment Quarterly  ↗
  32. Assessing Writing
  33. The development and maintenance of rating quality in performance writing assessment: A lo…
    Language Testing  ↗
  34. Assessing Writing
  35. Assessment criteria in a large-scale writing test: What do they really mean to the raters?
    Language Testing  ↗
  36. Validity considerations ensuing from examinees’ perceptions about high-stakes national ex…
    Policy & Practice
  37. High-stakes language testing in the republic of Cyprus
  38. R Core Team. (2018). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Co…
  39. The effects of different rater training procedures on ESL essay raters’ rating accuracy
    Social Sciences and Humanities
  40. Robitzsch, A. , Kiefer, T. , & Wu, M. (2022-08-28). TAM: Test Analysis Modules (R package version 3.7–16). 〈h…
  41. Dichotomizing continuous predictors in multiple regression: A bad idea
    Statistics in Medicine  ↗
  42. Do experience and text quality matter for raters’ decision-making behaviors?
    Language Testing  ↗
  43. The assessment of writing ability: Expert readers versus lay readers
    Language Testing  ↗
  44. Intervention in teachers’ differential scoring judgments in assessing L2 writing through …
    Studies in Educational Evaluation  ↗
  45. The assessment of complex tasks: A double reading
    Studies in Higher Education  ↗
  46. The effect of raters’ background and training on the reliability of direct writing tests
    The Modern Language Journal  ↗
  47. Teachers as raters: Investigation of a long term writing assessment program
    L1 Educational Studies in Language and Literature  ↗
  48. What goes through an examiner’s mind? Using verbal protocols to gain insights into the GC…
    British Educational Research Journal  ↗
  49. Assessing Writing
  50. Temporal experience
    Psychological Bulletin  ↗
  51. Using FACETS to model rater training effects
    Language Testing  ↗
  52. Communities of practice: Learning, meaning and identity
  53. Willey, K., & Gardner, A. (2011). Getting tutors on the same page. AAEE Conference, Fremantle, Western Australia.
  54. Assessing Writing
  55. A methodologically improved study on raters’ personality and rating severity in writing a…
    SAGE Open  ↗