|
2.2 Criteria and context
Look at these two examples of differences in the selection and weighting of assessment criteria. Note how these two examples of assessment criteria sets vary according to the situation, and try to list the factors in the testing situation which can affect such selection and weighting.
Placement test
A placement test for University students (to decide which level they go into for their General English Communication course. This is an interview - basically answering questions - with a teacher who is both interviewer and scorer at the same time. It is taken with a single written gap-fill test, which assesses knowledge of grammatical structures, vocabulary, and functionally appropriate structures.
In informal terms, the qualities we felt were important for success in a class were the ability to understand the teacher, knowledge of vocabulary, structures and functions, confidence and a willingness to take chances and try things out, as well as the ability to distinguish (productively) basic phonemes in English. The category of ‘range of grammar and vocabulary’ aims to capture people with a wide experience of English.
|
Numbers on main list
|
Weighting
|
Range of grammar and vocabulary
|
1 a and 2 a
|
3
|
Accuracy of grammar and vocabulary
|
1 b and 2 b
|
2
|
Phonemic distinctions
|
3 a
|
2
|
Hesitation
|
4 b and c
|
4
|
Initiative, topic development, and conversational control
|
5 a, b and d
|
4
|
Listening comprehension
|
Not listed
|
5
|
End-of-term ESP test
This test was given at the end of one term of a course for receptionists in international hotels, to see how much they had progressed and what they needed to work on the following term. The speaking test was a role-play with two students, and the teacher only observed and marked the score sheet. There were several other tests - gap-fill and multiple-choice tests of grammar, vocabulary, and functions, and a listening comprehension test.
|
Numbers on main list
|
Weighting
|
Grammar and vocabulary
|
1 and 2
|
3
|
Pronunciation
a. individual sounds
b. stress and rhythm
c. intonation and linking
|
3
|
1
1
1
|
Fluency
a. hesitation before speaking
b. hesitation while speaking
|
4
|
1
1
|
Conversational skill
a. cohesion
b. conversation maintenance
|
5
|
1
1
|
Sociolinguistic skill
a. distinguishing register and style
b. use of cultural references
|
6
|
2
1
|
Non-verbal
a. eye contact and body posture
b. gestures and facial expressions
|
7
|
1
1
|
Content (relevance)
|
8
|
2
|
Context variables (15 minutes)
By looking at the examples and thinking of their own experience, the teachers are asked to abstract the variables in the context which may affect the choice and weighting of the criteria. The variables could include the following:
The purpose of the test:
- achievement, proficiency, predictive or diagnostic?
- and depending on that: the course objectives; the underlying theory (formal or informal) of what constitutes language proficiency; the future situations the students are being tested for; the types of feedback the students in question would understand and could benefit from.
The circumstances of eliciting the sample of language being assessed.
- the degree of freedom or control over what the student could say and do.
- the number of participants and their roles in the situation.
Observation restrictions
- extent to which assessor participates in speaking situation (e.g. interviewer or observer).
- whether recorded or not (on audio or video cassette).
The other tests in the battery
(e.g. the selection or weighting of a criterion for grammatical accuracy may depend on how much it has been assessed in accompanying written tests).
d. Using the different criteria sets (15 minutes)
(Optional if short of time.)
Teachers watch another video clip and assess the student’s oral skills using first one of the example criteria sets and then the other. This is to demonstrate how different criteria sets (appropriate in different contexts) can produce different assessments of the same performance. Different criteria sets will not always produce differing results, and so care needs to be taken to use a clip which will make the point clearly.
Task (20-30 minutes)
Teachers are given details of a testing situation (preferably compatible with their own) and are asked to decide on the criteria they would use and the weighting for those criteria. There follows an example of such a task:
TASK (Selecting Assessment Criteria.)
You should imagine you are responsible for the oral testing of 100 students who have applied to study in U.S. colleges for one year (various subjects). You have to judge whether they have sufficient speaking skills to survive and do well there. You can conduct a 10-minute interview for each (with only one interviewer/assessor for each interview). The interviewers are all experienced and trained EFL teachers. The other tests in the battery will be multiple-choice tests of grammar and vocabulary, of academic reading and lecture listening comprehension, and essay-type questions.
Decide on the criteria you would use in assessing their spoken English, and the relative weighting of each.
|
The purpose of this task is:
- for teachers to think more concretely about the points raised so far, to let them see how referring to a particular context can reduce the differences in opinion when talking more generally.
- to provide an intermediary stage between the thinking in the earlier part and the need for application of those thoughts to their own situation after the workshop.
f. Conclusion (5 minutes)
The presenter can ask students for their comments on the workshop - how useful it was, how it could have been more useful, whether they think they would change the way they assess their students’ speaking skills, and so on.
Do'stlaringiz bilan baham: |
