Assessment of English Language Oral Production Using CALF: an Automatic Analytic Tool for Complexity, Accuracy, Lexis and Fluency

Abstract

This paper shows the evident lack of ICT based oral language production assessment tools and how the English language assessment in the State school education system is limited to testing the listening, reading and writing language skills. This paper projects four important factors which are proven to be effective indicators of an individual’s oral language proficiency. This research paper explores the use of ICT in language assessment and explains the usage of CALF tool in depth which is an automatic analytic tool developed by Gavin Bui and Peter Skehan (2016) to analyse the complexity, accuracy and fluency of spoken language for research purposes. The study was conducted primarily to pilot the CALF tool and test its functionality in the context of interest for a research project. The CALF tool was piloted in a Government school in Tamil Nadu and the participants were learners of English as a second language from VI standard. A total of 15 students were present for all the sessions. The speech data of the learners were recorded, transcribed, coded and uploaded in the CALF system for obtaining the indicative measures of complexity, accuracy, lexical and fluency in English language production. The recorded speech data was analysed twice and the Intraclass Correlation Coefficient (ICC) was obtained using SPSS to establish the reliability of the CALF tool. The obtained ICC value of 0.955 proved an excellent level of conformity in the Complexity, Accuracy, Lexical and Fluency measures derived from the CALF system. The study most importantly exhibits the potential of the CALF tool to be added as a significant online language production assessment tool with further advancement research on the transcription and coding stages.