Können Misfits vermieden werden? Eine quantitative und qualitative Analyse von nicht integrierbaren items in Aufgaben zum hörverstehen

Abstract

The Association of Language Testers in Europe has established a set of common standards for language exams which cover all stages of the language testing process. The international ECL language examination system works on meeting these so-called ‘minimum standards`.

In the course of this quality assurance process special attention is paid to the quantitative and qualitative analysis of the test results. After each exam, ECL experts carry out statistical analyses in order to compute the difficulty, discrimination and reliability of the task items. For this purpose, the experts deploy the measurement methods of the Classical Test Theory on the one hand, and the Rasch model, on the other hand. Unlike the psychometric tests of the Classical Test Theory, the Rasch measurement model indicates misfits, or in other words test items having a poorer quality compared to the rest of the items and, therefore, do not fit the model.

The present paper deals with the issue of misfit items in German language tests measuring listening skills at level B2 according to CEFR. Those items which were indicated as misfits by the Rasch model in the course of ten exam terms are carefully examined; the reasons for their `malfunction` are then thoroughly searched for. The findings of these analyses are to be applied by item writers in the course of future test developments.