6 Assessing productive and interactive skills Personal refl ection

parts so that the developer has a large collection of isolated statements 
such as ‘makes frequent errors with simple grammatical structures’ or 
‘can give directions to places around town’. Groups of expert judges 
fi rst decide whether these descriptions are meaningful in classifying 
learners. If they are not, they are edited to make them more meaningful 
or they are rejected. The experts then rank the statements according to 

---

154
 Part 
II
Figure 6.5 
Example of a simple four-level EBB scale for essay writing based on 
Upshur and Turner (1995, p. 7)
diffi culty. They may also classify them according to the aspect of 
language being described. Statements of similar diffi culty are grouped 
into sets to form a level and the level descriptions are arranged into a 
scale. The resulting scales are taken to refl ect a consensus view of the 
characteristics of different levels of language ability. This is, in outline, 
the approach that was taken to building the scales associated with the 
Common European Framework of Reference for Languages (CEFR) 
(Council of Europe, 2001; North, 2000). 
Fulcher 
et al. 
(2011, p. 9) objected to such quantitative ‘scaling 
descriptors’ approaches, arguing that ‘Measurement-driven scales 
suffer from descriptional inadequacy. They are not sensitive to the 
communicative context or the interactional complexities of language 
use.’ Both the ordering of descriptors into levels and the categories 
used have been questioned because they seem inconsistent with second 
language acquisition research (Green, 2012a). Areas of concern 
include the trade-off discussed above between accuracy and range. As 
learners expand their repertoire of language, or as they become more 
fl uent, they often lose a degree of accuracy. As a result, learner language 
does not necessarily or consistently become more accurate in the way 
some scales seem to suggest. However, defenders of quantitative scales 
point out that they are concerned with broad characterisations of 
functional language ability, while second language acquisition research 
tends to focus on discrete elements of language form (North, 2000). 
Although learners become less accurate in their use of certain structures, 
their overall ability to communicate may nonetheless improve.
It has also been argued that scales sometimes misrepresent the nature 
of language in use. As noted in the overview of speaking abilities, highly 
competent interactions between skilled speakers of a language are 
rarely consistently ‘complex’ (CEFR, C2: grammatical accuracy scale) 
with a ‘high degree of grammatical accuracy’ (CEFR, C1: grammatical 
accuracy scale), displaying ‘a very broad lexical repertoire’ (CEFR, C2: 
vocabulary range scale), but are often quite simple syntactically and 
may involve a rather restricted range of vocabulary (Hughes, 2010).

---