|
Ensuring Reliability
There are practical constraints in achieving perfect reliability in tests. This is particularly true in the case of language testing, especially when it involves testing of productive skills such as writing and speaking. However, the problem cannot be simply ignored. It is possible to attempt to maintain higher degree of reliability in all testing situations. The following measures will be helpful in this:
Include as many test items as possible
This is based on the fact that reliability is higher when enough sample of language is elicited. For example, suppose in one question, we get students to write an essay of 100 words and in another we ask them to write one sentence only. Which provides more accurate picture of the performances of candidates? The more language you elicit the better and more reliable the information will be. However, the constant pressure of tester is to maintain reliability against practicality. There is limit to what we can do!
Include test items which are independent
When related test items are given, if a student does not perform well in one, it may affect the other one too: the true performances of candidates will not be revealed. Therefore, it is useful to give a fresh start in each test item.
Pay attention to the length of test items and the test paper
If the test items are too long, the students will get bored and as a result, their true performances will not be revealed. If the test paper is too short, the tester may not be able to elicit language of the student sufficient. It is important to elicit true language ability of the learner – otherwise in two sittings, scores will be completely different and this means the reliability will be less.
Do not allow candidates too much of freedom
If the test paper requires candidates to select a topic from a wide range of selection options for writing an essay, for example, different candidates will react in different ways. Some candidates have their own preferences and they will score better irrespective of their true language ability. In marking too, different examiners are likely to judge the contents differently and the score received by the candidate will not be a true indicator of his language performances. Further, some essays may require a lot of thought, imagination and creativity and some students have special potential in developing imagination to compensate language deficiency. When such candidates are given a better score, this score does not reflect his true language proficiency.
Get students to be familiarize with the format of the paper, testing techniques and the so on
It is important to discuss past papers and help learners to be familiar with testing techniques. There are some students who demonstrate better test-taking abilities. They may also be better aware of testing techniques. If a candidate receives a better score since he possesses such abilities, it means the marks are offered not for the true language abilities of the candidate. The reliability is low!
Provide uniform testing conditions of administration
Timing, acoustic conditions, resources should be equal in every respect. If such conditions are different, candidates perform differently and again their true abilities will not be revealed. We can achieve a better degree of reliability only if true abilities of the learner are revealed in the particular testing situation. Then only the marks will be consistent. It is also important to identify the candidates by an index number, not by their names.
Include objective test items
These test items do not require the judgment of the examiner. Whoever marks it, only one possible answer is considered. In this way a higher degree of reliability is assured. The problem in language testing is that receptive skills such as reading and listening can be tested objectively. In writing and speaking tests, it is inevitable that subjective test items are used.
Provide guidelines in subjective test items
The following test item was included in an Achievement Test:
During your University vacation, you went on a trip with your family members. Write a letter to your best friend, describing your journey. Give the following details about the trip:
This is a subjective test item to assess the writing skills of the students. However, students will write answers of different lengths, including various types of information. In these types of writing test items, those who have better skills in imagination will score better. This is not valid and the scores may not be given to true writing performances of the learner. Thus reliability is less. This test item can be improved in the following way:
During your University vacation, you went on a trip with your family members. Write a letter to your best friend, describing your journey. Give the following details about the trip:
- who went with you
- the places you visited
-what you could see in those places
-where you stayed at night
-any interesting experience
-any other relevant details
Use about 150 words. (10 marks)
In this modified version, guidelines are given and there will be a uniform pattern in answers. The comparison between answers is direct. In this way the tester helps the examiner to make his judgment about the writing performances. The marking principle is that the longer the required judgment the lower the reliability will be.
Prepare a detailed marking scheme
This should include the following details:
All acceptable answers
Whether partial credit is offered to any answers
Number of marks for each part
Clear instructions as how to make required judgment
Detailed band schemes for marking subjective test items
When this is done, there will be uniform pattern in assessment and evaluation.
Train the scorers
This is very important in large scale examinations, where many examiners are involved.
Reliability refers to the consistency of test scores. If the same test is taken twice or if two examiners mark the same test paper, marks should be more or less the same. In language tests, it is difficult to maintain higher degree of reliability because they require subjective test items to test the productive skills. Therefore, effective measures should be taken to sustain reliability at a higher rate.
|
Do'stlaringiz bilan baham: |
