|
2.3 Rating Scales and Scale Descriptors
Assessment of speaking requires assigning numbers to the characteristics of the speech sample in a systematic fashion through the use of a scale. A scale represents the range of assessment of speaking 3 values that is associated with particular levels of performance, and scaling rules represent the relationship between the characteristic of interest and the value assigned. The use of a scale for measurement is more intuitively clear in familiar domains apart from language ability. For example, we can measure weight very accurately. While the use of pounds or kilograms is usually sufficient for measuring the weight of adults, when measuring babies, we move to smaller and more precise units—ounces or grams. The characteristics of the objects measured and our need for accuracy determine the units of measurement selected, and, in turn, scaling rules describe the units of measurement we employ.
Measurement of a speaking performance, however, requires a different kind of scale, such as those used in rating performance in sports competitions. For example, the quality of a figure skater’s performance in the Olympics is based on rank; there is no equal-interval unit of measurement comparable to ounces or pounds that allows precise measurement of a figure skating performance. Likewise, assessing speaking is generally considered an endeavor that ranks students into ordinal categories (often referred to as vertical categories) similar to bronze, silver, or gold; short, medium, and tall; third, second, or first—more familiar in instructional contexts involving language as beginning, intermediate, and advanced.
Such a global assessment of performance would result from the use of a holistic scale. To clarify what such a global assessment means, the abilities associated with scale levels are represented by level descriptors which represent a qualitative summary of the raters’ observations. In order to facilitate description, benchmark performances are selected to exemplify the levels and their descriptors. Such descriptors are typically associated with, but are not limited to, descriptions of the following components of a speaking performance at different levels of the scale: pronunciation, focusing on segmentals; phonological control, focusing on suprasegmentals; grammar/accuracy (morphology, syntax, and usage); fluency (speed and pausing); vocabulary (range and idiomaticity); coherence; and organization. If the assessment involves evaluation of interaction, the following may also be included: turn-taking strategies, cooperative strategies, and asking for or providing clarification when needed.
Holistic vertical indicators, even when accompanied by scale descriptors and benchmarks, may not be sufficient for making instructional or placement decisions. In such cases, an analytic rating is done to produce scores on components of the examinee’s performance. The specific components chosen, which can include any of the same aspects of performance used in holistic scale descriptors, will depend on the purpose of the test and the needs of the score users.
Score users are of central concern in a distinction among three types of scales: user-oriented, assessor-oriented, and constructor-oriented. The language used to describe abilities tends to focus on the positive aspects of performance in user-oriented and constructor-oriented scales where the former may focus on likely behaviors at a given level and the latter may focus on particular tasks that may be associated with a curriculum or course of instruction. Assessor-oriented scales shift the focus from the learner and the objectives of learning toward the rater; the scale descriptors are often negatively worded and focus on the perceptions of the rater. These scales tend to be more useful for screening purposes.
From another perspective, scales for speaking assessments can be theoretically oriented, empirically oriented, or a combination of both. The starting point for all assessments is usually a description or theory of language ability. These broad theoretical orientations are then narrowed down to focus on a particular skill and particular components of skills. Empirical approaches to the development and validation of speaking assessment scales involve identification of characteristics of interest assessment of speaking for the subsequent development of scale levels and/or explications of characteristics of assigned ability levels.
Specific-purpose scales are often derived from general guidelines and frameworks. For example, The ACTFL Proficiency Guidelines serve as a starting point for the ACTFL OPI scale. Another influential framework is the Common European Framework of Reference for Languages. The CEFR is a collection of descriptions of language ability, ranging from beginning to advanced, across and within the four main skills. The document is comprehensive and formidable in scope, but in spite of its breadth, the CEFR has been used to construct scales for assessing language performance, communicate about levels locally and nationally, and interpret test scores.
The most familiar part of the framework presents six stages of proficiency from A1 (Breakthrough) to C2 (Mastery) in general terms. Descriptors are provided in a series of tables for each skill and then each skill is additionally broken down into subskill descriptions that can be adapted for use in the creation of scales for specific purposes within specific contexts. For example, with respect to speaking, the framework presents separate tables for overall oral production standards, for understanding a native speaking interlocutor, for conversation, for informal discussion with friends, for formal discussions and meetings, for goal-oriented cooperation, for transactions to obtain goods and services, for information exchange, and for interviewing and being interviewed. Perhaps of greatest interest here are the “qualitative aspects of spoken language use” in which spoken abilities are additionally divided into CEFR levels in which aspects of range (vocabulary), accuracy, fluency, interaction, and coherence are described.
Do'stlaringiz bilan baham: |
