C e n t e r o n h I g h e r e d u c a t I o n r e f o r m a m e r I c a n e n t e r p r I s e I n s t I t u t e

The Validity of the Test Instrument. Before educa-
tors can use an assessment to award credit or degrees 
for demonstrated competencies, they must determine 
that the assessment is a valid measure of those com-
petencies. That means that (1) the test must fully 
measure the competency, (2) the processes students 
use to complete the assessment tasks must be an 
authentic reflection of the competency, and (3) stu-
dents would receive the same test results if they were 
to take a different form of the test scored by differ-
ent raters. These ideas correspond with validity evi-
dence based on test content, response processes, and 
internal structure, respectively. Each will be briefly 
described in the following paragraphs.
Evidence Based on Test Content. Once program design-
ers have clearly defined relevant competencies, they 
should collect evidence that their test content fully 
reflects those competencies. Specifically, providing 
validity evidence based on test content means showing 
the relationships between test questions or tasks and the 
defined competencies. Test developers must consider 
how well the breadth and depth of their test relates to 
defined competencies. 
The Advanced Placement (AP) program is one exam-
ple. Its test content validity evidence is grounded in a 
process known as evidence-centered design (ECD).
25
Using ECD, assessment designers connect three com-
ponents: (1) the intended claims about students’ con-
tent knowledge and skill proficiency, (2) the observable 
evidence a student would need to produce to support 
those claims, and (3) the tasks or situations that would 
elicit that evidence. By designing assessment tasks that 
enable students to demonstrate the relevant knowledge 
and skills, test developers provide evidence of validity.
An example from the AP Chemistry assessment 
starts with the following claim: “Students can apply 
mathematics in which they evaluate the reasonableness 
of quantities found in stoichiometric calculations.”
26