6 Assessing productive and interactive skills Personal refl ection

 Part 
II
Table 6.6
 
Comparison between holistic and analytic scoring approaches
Holistic scoring
Analytic scoring
Straightforward and intuitive
More time consuming for raters
Refl ects a natural tendency to make 
holistic judgements of learners’ language 
abilities
Not clear that raters are really able to 
differentiate between several different 
criteria: four or fi ve may be the maximum 
manageable number
Raters may interpret the scale selectively, 
focusing only on features that they value 
or that they fi nd easy to understand
Helpful as a training tool to encourage 
raters to focus on the variety of features 
highlighted by the scale developers – these 
represent the intended test construct
Not clear how the overall score is arrived 
at – if the assessee is strong on one 
criterion, but weak on another, this is not 
revealed by the score
Differential performance across criteria can 
be captured in a score profi le
Less reliable as only one score is 
awarded
More reliable as multiple observations are 
collected
Less informative – fails to identify 
strengths and weaknesses
More informative – potential for useful 
feedback to teaching and learning
Differences between holistic and analytic scale formats are summarised 
in 
Table 6.6
. Whether holistic or analytic scales are chosen, when 
assessees undertake a series of tasks as part of a test, a separate score 
should be awarded for each task. Ideally, different raters should award 
scores to each. In other words, if a writing test involves an essay and a 
letter, each should be scored by different raters. If a speaking test 
includes a presentation, an interview and a discussion, the recording 
should be split and the parts sent to different raters to score. This will 
substantially improve the reliability of the fi nal result as the assessee 
will be given multiple opportunities to show what they are able to do 
and the impact of each rater’s tendency towards harshness or leniency 
of judgement is balanced by the others’.