Written Communication Oct 1989

The Reliability and Validity of Protocol Analysis

PETER SMAGORINSKY

Abstract

Rhetoricians and researchers have vigorously debated the reliability and validity of protocol analysis findings. Social science scholars have contended the value of verbal data since their original use in the 1920s. This article reviews the history of verbal data in a variety of fields, places protocol analysis in its historical context, and examines more recent claims and criticisms regarding protocol analysis, concluding that protocol analysis, when conducted according to certain principles, can be an important addition to the repertoire of tools for researching the composing process.

Journal
Written Communication
Published
1989-10-01
DOI
10.1177/0741088389006004003
CompPile
Open Access
Closed
Topics
Export
BibTeX RIS

Citation context

Cited by in this index (10)

  1. Khuder et al. (2019)
    L2 Writing Task Representation in Test-Like and Non-Test-Like Situations
    Written Communication
  2. Harwood (2018)
    What Do Proofreaders of Student Writing Do to a Master’s Essay? Differing Inter…
    Written Communication
  3. Takayoshi (2018)
    Writing in Social Worlds: An Argument for Researching Composing Processes
    College Composition and Communication
  4. Leijten et al. (2013)
    Keystroke Logging in Writing Research: Using Inputlog to Analyze and Visualize …
    Written Communication
  5. Krahmer (2004)
    Thinking About Thinking Aloud: A Comparison of TwoVerbal Protocols for Usabilit…
    IEEE Transactions on Professional Communication
Show all 10 →
  1. Mix (2003)
    Evidencing Nonstandard Feature Dynamics: “Speak Aloud and Write” Protocols by A…
    Written Communication
  2. Waes (2000)
    Thinking aloud as a method for testing the usability of Websites: the influence…
    IEEE Transactions on Professional Communication
  3. Weigle (1999)
    Investigating rater/prompt interactions in writing assessment: Quantitative and…
    Assessing Writing
  4. DEJONG et al. (1997)
    Reader-Focused Text Evaluation: An Overview of Goals and Methods
    Journal of Business and Technical Communication
  5. CUMMING (1990)
    Metalinguistic and Ideational Thinking in Second Language Composing
    Written Communication

References (66) · 12 in this index

  1. Applebee, A.
    Writing in the secondary schools
  2. 10.1207/s1532690xci0202_2
     ↗
  3. Berkenkotter (1981)
    Understanding a Writer's Awareness of Audience
    College Composition and Communication
  4. Berkenkotter et al. (1983)
    Decisions and Revisions: The Planning Strategies of a Publishing Writer, and Re…
    College Composition and Communication
  5. 10.2307/3150297
     ↗
Show all 66 →
  1. 10.2307/357637
     ↗
  2. Claparede, E. (1934)
    Archives de Psychologie
  3. Clarkson, G.
    Portfolio selection—A simulation of trust investment
  4. 10.2307/376913
     ↗
  5. 10.2307/358259
     ↗
  6. 10.1037/h0025519
     ↗
  7. 10.2307/377377
     ↗
  8. 10.1037/h0040017
     ↗
  9. 10.1080/08856559.1926.10533052
     ↗
  10. Duncker, K. (1945)
    Psychological Monographs
  11. Durkin, H. E.
    Archives of Psychology
  12. Durst (1987)
    Cognitive and Linguistic Demands of Analytic Writing
    Research in the Teaching of English
  13. Emig, J.
    The Composing Processes of Twelfth Graders
  14. Ericsson, K. A.
    Retrospective verbal reports as data
  15. Ericsson, K. A.
    Thinking-aloud protocols as data: Effects of verbalization
  16. 10.1037/0033-295X.87.3.215
     ↗
  17. Ericsson, K. A.
    Protocol analysis: Verbal reports as data
  18. 10.2307/376707
     ↗
  19. Feldman, J. (1959). An analysis of predictive behavior in a two-choice situation …
  20. 10.2307/375768
     ↗
  21. Flower, L.
    Cognitive processes in writing: An interdisciplinary approach
  22. 10.2307/356630
     ↗
  23. 10.2307/356600
     ↗
  24. Flower, L.
    Writing: The nature, development, and teaching of written communication
  25. Flower et al. (1981)
    The Pregnant Pause: An Inquiry Into the Nature of Planning
    Research in the Teaching of English
  26. FLOWER et al. (1984)
    Images, Plans, and Prose: The Representation of Meaning in Writing
    Written Communication
  27. 10.1037/h0048703
     ↗
  28. Goodlad, J. I.
    A place called school: Prospects for the future
  29. 10.2307/358257
     ↗
  30. Hayes, J.
    Cognitive processes in writing: An interdisciplinary approach
  31. Hayes, J.
    Research in Writing: Principles and Methods
  32. Irmscher, W.
    College Composition and Communication
  33. Karpf, D. A. (1972). Thinking aloud in human discrimination learning …
  34. Kaufer et al. (1986)
    Composing Written Sentences
    Research in the Teaching of English
  35. 10.3758/BF03334190
     ↗
  36. Langer, J. (1986)
    Journal of Reading
  37. Langer, J.
    Children reading and writing: Structures and strategies
  38. Langer, J.
    How Writing Shapes Thinking: A Study of Teaching and Learning
  39. 10.1037/h0067016
     ↗
  40. Marshall (1987)
    The Effects of Writing on Students’ Understanding of Literary Texts
    Research in the Teaching of English
  41. Mischel (1974)
    A Case Study of a Twelfth-grade Writer
    Research in the Teaching of English
  42. Montgomery, H. A.
    Decision making and change in human affairs
  43. Murray, D. (1983)
    College Composition and Communication
  44. Newell, A.
    Human problem solving
  45. Newell (1984)
    Learning from Writing in Two Content Areas: A Case Study/Protocol Analysis
    Research in the Teaching of English
  46. 10.1037/0033-295X.84.3.231
     ↗
  47. Odell et al. (1982)
    Writing in a Non- Academic Setting
    Research in the Teaching of English
  48. Penner, B. C.
    Problem solving skills in the social sciences: Methodological considerations
  49. Perl (1979)
    The Composing Processes of Unskilled College Writers
    Research in the Teaching of English
  50. 10.2307/1419339
     ↗
  51. Rose (1980)
    Rigid Rules, Inflexible Plans, and the Stifling of Language: A Cognitivist Anal…
    College Composition and Communication
  52. Scardamalia, M.
    The psychology of written language: A developmental approach
  53. 10.1207/s1532690xci0201_1
     ↗
  54. 10.2307/256146
     ↗
  55. 10.1037/h0093765
     ↗
  56. 10.2307/377376
     ↗
  57. Swarts, H.
    Research in Language Arts Workshop
  58. 10.1016/0010-0285(74)90013-9
     ↗
  59. 10.1037/h0026750
     ↗
  60. 10.1016/0010-0285(73)90014-5
     ↗
  61. 10.2307/357634
     ↗
CrossRef global citation count: 51 View in citation network → Build reading path →