Inter-rater reliability of a task-based assessment tool designed for practicality in the classroom

This submission has open access
Abstract Summary

This presentation introduces a universal rubric for teachers to more easily grade speaking, writing, or mixed-skills task performance in the classroom. The tool emphasizes practicality of use while conforming to TBLT assessment principles, has "good enough" validity and inter-rater reliability for the classroom, and allows for guided instructional feedback based on adapted Gricean maxims. It is compatible with language descriptor systems, and can be used with a wide range of communicative tasks and proficiency levels.

Submission ID :
AILA2430
Submission Type
Abstract :

Skehan (2018, p.255) argues that task-based testing practices must be made commonplace in order for task-based language teaching techniques to be more widely implemented. This goal needs to be achieved both with high-stakes proficiency testing and classroom-based achievement testing. In other words, it is not enough to simply provide communicative opportunities in tasks, if requirements of formal testing and classroom grading inevitably cause teachers to revert to tests that are based only on the correct usage of taught linguistic forms, and thus undermine TBLT principles.

While formal proficiency tests today are increasingly based on TBLT-compatible descriptor systems such as the CEFR, and thus are appropriate for communicative assessment, one limitation of such tests is that they require rater training and reliability calibration (Vedder & Kuiken, 2019). While this may be appropriate for research purposes and global proficiency measurement, formal testing procedures are not easily scaled for day-to-day use in the language classroom. This presentation introduces an instrument that non-expert teachers who have not undergone rater training can use to reliably score speaking, writing, and mixed-skills tasks in a principled TBLT way. The instrument prioritizes ease of use in the classroom and is compatible with language descriptor systems while not being dependent on long lists of proficiency descriptors. It aims to make task-based assessment intuitive, easy to perform, and sufficiently valid and reliable for everyday classroom use.


References:

Kuiker & Vedder. (2019). "From CAF to CAFFA: Measuring linguistic performance and functional adequacy in TBLT". Plenary at 2019 International Conference on TBLT. (2019- 08-20).

Skehan, P. (2018). Second Language Task-based Performance: Theory, Research, Assessment. London: Routledge.

Pre-recorded video :
If the file does not load, click here to open/download the file.
Handouts :
If the file does not load, click here to open/download the file.
Associate Professor
,
J. F. Oberlin University
Instructor
,
J. F. Oberlin University

Abstracts With Same Type

Submission ID
Submission Title
Submission Topic
Submission Type
Primary Author
AILA1060
AILA Symposium
Standard
Dr. Yo-An Lee
93 visits