rater
training
can be carried out. Raters look at and discuss a set of previously
scored performances to help them understand the scale. In order to ensure
that their judgements are consistent with the consensus interpretation of
the scales, they then rate some performances. If their judgements
consistently match the consensus view, they are accepted as raters. If not,
they are rejected or have to undertake further training. As the effect of
training can be transient, raters have to be monitored over time to ensure
that they remain consistent, perhaps by requiring them periodically to
score additional performances that have already been assigned scores or
by comparing their judgements with those of other raters.
Where scores have serious consequences for the assessee, it is essential
that the results should not depend on the views of a single rater, even if
that rater is an experienced expert. It is good practice to obtain scores
from at least two raters who are given no opportunity to consult or to
see each other’s scores. If they are in agreement, that suggests that the
156
Do'stlaringiz bilan baham: |