An eye-tracking study on the influence of audiovisual cues on shadowing performance

The materials that learners use for the shadowing practice serve as important source of input, and they have different variants, such as the materials only with audio cues and those with audio and visual cues. The shadowing practice requires learners to perceive the audio cues and repeat them as soon as possible. It has been known that when a visual cue, such as a moving image of speaker's face including his or her lip movements, is given to a listener, the lip movement affects the listener's perception of the speaker's utterances. For instance, if there is a mismatch between a given speech and a lip movement, the speech is incorrectly perceived-the McGurk effect. Therefore, it is conceivable that if both audio and visual cues, such as moving images of a speaker's face including lip movements, are given, the visual cues may influence the performance of shadowing. However, it has been reported that the integration of visual and audio cues varied across languages. For instance, although magnitudes of McGurk effect have been reported for native speakers of English and some other languages, the magnitudes of McGurk effect for Japanese native speakers reported so far have been smaller due to cross-linguistic cultural differences (Sekiyama and Tohkura, 1991). This means that different perceptions of audiovisual cues do not necessarily influence the shadowing performance of Japanese learners of English. Since the influence of audiovisual materials on the shadowing performance has not yet been studied well, this study compared the audiovisual materials for shadowing audio cues either with a still image or with a moving image of a speaker' face through an eye-tracking experiment with L1 Japanese speaking learners of English. (Sekiyama, K. and Tohkura, Y. 1991. McGurk effect in non-English listeners. Journal of Acoustic Society of America 90: 1797-1805.)