Diverse population, homogenous ability: The development of a new receptive vocabulary size test for young language learners in England using Rasch analysis

With mandatory second language learning in primary education becoming the norm worldwide, research investigating young language learners’ (YLLs) linguistic development has increased. However, designing language tests appropriate for YLLs poses unique challenges due to population characteristics and variability in national and institutional contexts. In this article, we present a new vocabulary test designed to track the rate of progression in receptive vocabulary size of primary school children learning French, German or Spanish in England. Test content was selected after an analysis of programmes of study commonly used in primary schools in England. The test required two validation phases using Rasch analysis. The initial tests were administered to 1662 students from Year 3 (7–8 years old) to Year 5 (9–10 years old). All tests showed poor person reliability, which was driven by a mismatch between item difficulties and participant abilities. Various actions were taken in relation to vocabulary identification, test format and length, and sampling procedures, and the revised tests were re-administered to 2202 students from Year 3 to Year 6 the following year. As a result, person reliability considerably improved, and all test versions showed good fit to the Rasch model. Drawing on the lessons learnt, we discuss some of the key population- and context-related challenges of designing robust language tests for beginner YLLs learning a language other than English in input-poor, instructed contexts. Further, we provide recommendations on suitable approaches for test-item identification, test format and length, and data analysis.