Abstract Summary
MiKo is a multimodal corpus triangulating data involved in the complex task of note-taking in university lectures. It comprises transcriptions, video and audio streams, and students’ notes of first-semester lectures. This poster presentation focuses on introducing the corpus and discussing methodical challenges in the processing of multimodal data.
Abstract :
MiKo is a multimodal corpus triangulating data involved in the complex task of note-taking in university lectures. This poster presentation focuses on introducing the corpus and, on a more general level, discussing methodical challenges in the processing of multimodal data.
Note-taking, involving receptive and productive language skills, is a demanding task (Steets, 2003), particularly for L2 students. However, research on note-taking has primarily focused on its cognitive aspects, while the linguistic challenges involved have, as yet, not been well understood (author forthcoming). Note-taking in lectures happens in complex settings where both visual and audio input is decisive (Steets, 2003). Therefore, a multimodal corpus is particularly suited for its investigation.
MiKo comprises transcriptions and aligned video and audio streams of first-semester lectures (currently N=7, 73.600 tokens) as well as students’ notes taken during these lectures (L2: N=89; L1: N=57). Thus, it includes more than one “sensory modality” (Allwood, 2008, p. 208) and both spoken and written language.
A variety of quantitative and qualitative methods is used for the gathering, processing, and analysis of the data. The audio and video files are processed in EXMARaLDA (Schmidt, 2002) using a modification of the HIAT transcription guidelines (Rehbein et al., 2004). Annotations focus, among others, on the relationship of spoken and written language, e.g. on lecture slides.
Students’ notes are included in MiKo in a digitized, but not machine-readable format, augmented by information on various characteristics.
For a comprehensive analysis of the data, student questionnaires on note-taking, lecturer interviews, and longitudinal data (on academic success and language proficiency) are also made available.
MiKo will provide a resource for the corpus-based analysis of spoken academic language as well as further areas of research such as modality interdependencies (Reimer et al., 2015) and their impact on note-taking, or verbal and nonverbal relevance markers used by lecturers.
