An Introduction to Applied Linguistics

6.1.2 Social (and its interface with the linguistic and sociolinguistic) factors
ELTS was predicated on a committed view of language variety, itself deriving from
the more theoretical notions in sociology of role and status, and in sociolinguistics
(or sociology of language) of the significance of situation in determining language
use (situation being understood to include all aspects of setting, including the
speakers and their relationships to one another), as well as the more practical com -
muni cative competence view that language learning (and therefore its assessment)
should be communicative, by which was understood being appropriate to its
purpose.
The theoretical and practical reasons for the design of ELTS appeared convincing
at the time of its construction. But when it came to the evaluation it was necessary
to look closely at the rationale for its design, and in particular at the validity of the
notion of discrete language varieties. For this part of the investigation, again it was
necessary to call on subject specialists but also to undertake an empirical investi -
gation which compared performance on the same subject modules by students with
different specialist backgrounds, and the opposite, different subject modules by
students from the same specialist backgrounds.
It was also necessary to examine the claims of communicative competence and
estimate the extent to which a test could ever be truly communicative, given the
requirements of the communicative competence idea, that it be spontaneous and
non-predictable.
Here too it was necessary to look at differential performance on ELTS by various
groups (for example males and females, younger and older students) as well as by
speakers from different language backgrounds. A continuing controversy (which we
come back to under politics below) was whether those who could lay claim to an
education in their own countries in English medium should be excused the test.
A related issue, which has become increasingly important, is whether
international tests such as TOEFL, ELTS or its successor IELTS are biased against
speakers of non-metropolitan varieties of English, such as Indian English (Davies,
Hamp-Lyons and Kemp 2003).
6.1.3 Psychological (and its interface, the psycholinguistic) factors
Levels of proficiency beyond early and intermediate acquisition have not been
prominent in second-language acquisition research as a topic worth pursuing. And
yet studies of advanced writing, of vocabulary acquisition, of pragmatic interference
and of discourse handling, these are all relevant to the study and the assessment
of advanced proficiency. The ELTS evaluators needed to analyse how far the test
designers had looked at these different areas of language studies and had inco r -
porated these ideas in the ELTS test. What became clear during the study was how
78
An Introduction to Applied Linguistics
02 pages 001-202:Layout 1 31/5/07 09:31 Page 78

---

little is known about levels of proficiency and what it is that distinguishes one level
from another in terms of language function. It was also necessary to take note of how
much English was used in different countries, the question being whether those from
settings with poor resources should be tested separately, on the grounds that more
important than the present proficiency of candidates from those settings was their
aptitude for gaining future proficiency within a resource-rich environment, for
example in a UK university.
6.1.4 Anthropological factors ( for insights on cultural matters)
Since the ELTS test was being used throughout the world and therefore open to
scrutiny across cultures it was important to examine whether any of its content could
be thought offensive.
Medicine, for example, might be regarded as neutral in this regard since the whole
point of overseas students attending medical courses in the UK was (presumably)
to learn Western medicine. But in the West itself there is disagreement as to what
should be studied and what is legitimate medical practice. There are severe differ -
ences of opinion among those involved in medicine about such topics as abortion
and euthanasia. In some countries, mention of contraception is unacceptable and
should probably be avoided in such a widely used test.
The attitude that if these students actually come to the UK to study they will
not be able to escape from discussions about contraception as well as medical
explanations of its uses and patients’ enquiries about its efficacy, these are all true but
not the point. Culture is context bound and what the test must try to achieve is a
judicious balance between a valid sampling of what students in a particular discipline
need to do once started on a UK university course and an avoidance of giving offence
to the students and their home communities. This may be censorship but it is
sensitive self-censorship since it recognises local norms and does not try to change
them. If that is thought to be desirable, then the ELTS test is not its vehicle.
Another use of anthropological skills is in the exercise of ethnographic techniques
in researching how students study in a foreign language medium and the extent to
which they do so in distinct ways. Evidence from this type of study is confirmatory
or not of the test scores and may show that what the test is testing is not exactly
(perhaps not even remotely) what the students themselves know and are capable of.
This is another approach to the issue of test validity.