Locating the Semiotic Power of Multimodality

Glynda A. Hull ; Mark Evan Nelson University of California, Berkeley

Abstract

This article reports research that attempts to characterize what is powerful about digital multimodal texts. Building from recent theoretical work on understanding the workings and implications of multimodal communication, the authors call for a continuing empirical investigation into the roles that digital multimodal texts play in real-world contexts, and they offer one example of how such investigations might be approached. Drawing on data from the practice of multimedia digital storytelling, specifically a piece titled “Lyfe-N-Rhyme,” created by Oakland, California, artist Randy Young (accessible at http://www.oaklanddusty.org/videos.php), the authors detail the method and results of a fine-grained multimodal analysis, revealing semiotic relationships between and among different, copresent modes. It is in these relationships, the authors argue, that the expressive power of multimodality resides.

Journal
Written Communication
Published
2005-04-01
DOI
10.1177/0741088304274170
CompPile
Open Access
Closed
Topics
Export

Citation context

Cited by in this index (38)

  1. Written Communication
  2. Writing and Pedagogy
  3. Research in the Teaching of English
  4. Written Communication
  5. Written Communication
Show all 38 →
  1. Written Communication
  2. Written Communication
  3. Research in the Teaching of English
  4. Research in the Teaching of English
  5. Journal of Writing Analytics
  6. Journal of Writing Analytics
  7. Computers and Composition
  8. Written Communication
  9. Written Communication
  10. Research in the Teaching of English
  11. Written Communication
  12. Rhetoric Review
  13. Written Communication
  14. Computers and Composition
  15. Written Communication
  16. Communication Design Quarterly
  17. Written Communication
  18. Technical Communication Quarterly
  19. Journal of Business and Technical Communication
  20. Computers and Composition
  21. Communication Design Quarterly
  22. Written Communication
  23. Written Communication
  24. Computers and Composition
  25. Computers and Composition
  26. Computers and Composition
  27. Written Communication
  28. Written Communication
  29. Written Communication
  30. Computers and Composition
  31. Technical Communication Quarterly
  32. Written Communication
  33. Written Communication

References (55) · 2 in this index

  1. Rewriting literacy: Culture and the discourses of the other
  2. Writing space: The computer, hypertext, and the history of writing
  3. The psychology of literacy
  4. Doing documentary work
     ↗
  5. The social construction of literacy
Show all 55 →
  1. Multiliteracies
  2. Other people’s children
  3. Course in general linguistics
  4. 10.3102/0013189X032001005
     ↗
  5. 10.1598/RRQ.39.1.3
     ↗
  6. Discourse and social power
  7. Critical discourse analysis
  8. Communicating: The multiple modes of human interconnection
  9. Cultural studies
  10. What video games have to teach us about learning and literacy
  11. The ecological approach to visual perception
  12. Forms of talk
  13. Computers and Composition
  14. Global literacies and the World Wide Web
  15. Sound identities: Popular music and the cultural politics of education
  16. Blurring boundaries: Research and Teaching Beyond a Discipline
  17. Crafting an agentive self: Case studies on digital storytelling
  18. National Council of Teachers of English Research Assembly
  19. Before writing: Rethinking paths into literacy
  20. 10.4324/9780203201220_chapter_3
     ↗
  21. 10.4324/9780203164754
     ↗
  22. Reading images: A grammar of visual design
  23. Multimodal discourse. The modes and media of contemporary communication
  24. 10.1075/jnlh.7.49som
     ↗
  25. Digital storytelling: Capturing lives, creating community
  26. Reading science
  27. Handbook of literacy and technology: Transformations in a post-typographic world
  28. Understanding media: The extensions of man
  29. An introduction to visual culture
  30. Conference on Visual Culture, University of California
  31. National assessment of adult literacy
  32. The neglected“R”: The need for a writing revolution
  33. Reading at risk: A survey of literary reading in America
  34. 10.17763/haer.66.1.17370n67v22j160u
     ↗
  35. Living narratives: Creating lives in everyday storytelling
  36. Selected philosophical writings
  37. Selected philosophical writings
  38. 10.1525/9780520940888
     ↗
  39. 10.3102/0013189X032001025
     ↗
  40. 10.1177/0002764204266234
     ↗
  41. 10.1017/CBO9781139524834.006
     ↗
  42. Literacy in theory and practice
  43. The visual display of quantitative information
  44. Envisioning information
  45. Visual explanations: Images and quantities, evidence and narrative
  46. 10.1007/978-1-349-27700-1
     ↗
  47. Mind in society: The development of higher psychological processes
  48. Thought and language
  49. Written Communication
  50. invited lecture at the University of Wisconsin-Madison