If we consider interlocutors' video conferencing interaction (where multiple modes and semiotic resources are simultaneously available) as a multimodal orchestration, then how do interlocutors 'conduct' the orchestration to maintain to the harmony of their communication? Our participants managed to solve this problem by explicitly negotiating the use of mode(s).
Different modes of communication afforded by different types of technology can affect the way people communicate, and thus affecting the process of second language acquisition. As technology develops, multimodal videoconferencing communication (or video CMC) has become a popular tool for online interaction and second language learning. In such a multimodal technology-mediated communication environment, where text, audio, video, and many other semiotic resources are simultaneously available, a huge challenge for interlocutors is how they can make good use of modes for their communication. In this study, four dyads of Chinese postgraduate English language learners performed two lexically seeded information gap tasks in a synchronous video CMC environment. Episodes of negotiation for meaning were selected for the multimodal analysis to examine how interlocutors make use of different multimodal/semiotic resources to negotiate meaning in video CMC. The main modes and semiotic resources available include audio, video, text, and a hard copy of the task sheet. The multimodal analysis focuses on verbal interactions, gaze, gestures, facial expressions, and the multimodal 'orchestration' among these aspects. It was found that when multiple modes and semiotic resources are available, some students were able to negotiate the use of mode(s) intentionally and explicitly so that they can focus on the same mode(s) at a particular point of their interaction. For example, one student showed her hand to the webcam, wanting to demonstrate the meaning of the word 'nail', but her peer did not look at her video image. Then, the first student repeatedly asked her peer 'Can you see me? Can you see my hand?' to remind her peer to focus her attention on the video image. Such negotiation of modes reflects students' awareness of the multimodal nature of their communication. More importantly, through negotiating mode, they become the 'conductor' of this multimodal 'orchestration' during videoconferencing interactions.
