Welcome to Your Guide on Multimodal Listening!

In the MYP Language Acquisition course, Criterion A: Listening is about more than just hearing words. It is about understanding spoken multimodal texts. This means you are looking at how what we hear and what we see work together to tell a story or share information.

In these notes, we will explore the visual and spatial "clues" that help you understand a speaker's message, feelings, and even their secret attitudes!

1. What is a "Spoken Multimodal Text"?

A multimodal text is simply a message that uses more than one "mode" (way) of communication at the same time. In Criterion A, the primary mode is spoken language, but it is supported by:

  • Visuals: Things you can see (images, symbols, colors).
  • Spatial features: How things are organized in a space (layout, distance, position).

Real-world examples: A weather report on TV, a TED Talk video, an animated movie clip, or even a teacher giving a presentation with a slideshow.

Quick Review: If you only listen to the audio with your eyes closed, you are missing half the message! Multimodal listening requires you to use your eyes and ears together.

2. Visual Features: What Are You Seeing?

Visual features are the elements on the screen or in the room that give extra meaning to the spoken words. Here are the most important ones to look for:

Body Language and Facial Expressions

The way a speaker moves tells you how they feel.
Example: If a speaker says "I am so excited" but has a frown and crossed arms, the visual feature (their body language) tells you they might actually be bored or lying!

Graphics and Symbols

Watch for icons, charts, or maps. These often highlight the main ideas or supporting details of the talk.
Example: A red "X" symbol appearing next to a word might mean that the speaker is talking about something they disagree with.

Color and Lighting

Colors can set the mood. Bright colors might indicate a happy, energetic topic, while darker lighting might suggest a serious or sad subject.

On-Screen Text (Captions and Titles)

Sometimes important vocabulary words or names appear as text on the screen. This is a visual feature designed to help you recognize basic facts.

3. Spatial Features: Where Are Things Placed?

Spatial features refer to the "space" in the text. This sounds complicated, but it’s actually quite simple once you know what to look for:

Proximity (Distance)

How close is the speaker to the camera or the audience?
- Very Close (Close-up): Often shows intimacy or strong emotion.
- Far Away (Wide shot): Often shows the context or environment, making the speaker seem less personal.

Placement and Layout

Where is the information placed?
- The Center: Usually the most important thing.
- The Background: Provides "clues" about the setting or the speaker's values and attitudes (e.g., a speaker standing in front of a library might value education).

Key Takeaway: Spatial features help you understand the relationship between the speaker and the subject.

4. The "Interplay": Putting it All Together

The IB syllabus mentions the interplay between images and oral text. "Interplay" is just a fancy word for "how they work together."

To succeed in Criterion A, you must explain how the visual/spatial features support or change the meaning of the spoken words:

  • Reinforcement: The visual shows exactly what the speaker is saying (e.g., the speaker mentions a "dog" and a picture of a dog appears). This helps you identify facts.
  • Contrast: The visual shows the opposite of the words (e.g., the speaker says "the city is beautiful" while showing a picture of trash). This helps you draw conclusions about sarcasm or criticism.
  • Emphasis: A spatial feature makes one word more important (e.g., the camera zooms in right as the speaker says a key phrase).

5. Tips for Different Phases

Don’t worry if this seems tricky at first! Depending on your level, you will be asked to do different things with these features:

  • Emergent Level (Phases 1-2): Focus on identifying. "I see a picture of a car while the speaker talks about transport."
  • Capable Level (Phases 3-4): Focus on understanding conventions. "The speaker uses bright colors and a smile to show they have a positive attitude toward the topic."
  • Proficient Level (Phases 5-6): Focus on analysing. "The interplay between the dark background and the speaker’s formal register suggests a serious global perspective on climate change."

Common Mistakes to Avoid

1. Ignoring the "silent" parts: Even when a speaker stops talking, the visuals might still be giving you information. Pay attention to the transitions!

2. Focusing ONLY on words: If you are asked about the "multimodal text," you must mention at least one visual or spatial feature in your answer. Don't just quote what was said.

3. Forgetting the purpose: Always ask yourself: "Why did the creator choose THIS image for THIS spoken sentence?"

Note: For more information on identifying the specific messages within a text, check out the chapter on "Identifying facts, messages and main ideas."