8127/frame/fm

 
—Norah Rudin
A forensic examination is not a controlled experiment. In fact, it is not an
experiment at all. By definition, a scientific experiment requires not only
known conditions, but also controlled conditions. Ideally, one variable at a
time is altered while the others are held constant. This allows the scientist to
determine specifically what is causing the change, if any, in the final results.
Case samples have a whole history of which we are unaware. They are
the element in the analysis over which we have no control. We test them for
certain attributes, but a controlled experiment is not an option and, contrary
to NASA directive, failure is frequently an option. We accept this as a limi-
tation of the testing and interpret our results accordingly. Because of this,
however, all of the other elements of the test system must be both 
 
optimized
and 
 
validated
to establish maximum reliability. 
 
Standards
and 
 
controls
must
also be used when appropriate.
It is important to realize that optimization, validation, standards, and
controls may apply differently to analytical testing than to comparison test-
ing. These concepts first arose from test systems in which a reaction occurs
between the sample and the test reagents, or some instrumentation is used
to probe the nature of a sample. It is more difficult to understand how these
ideas might be useful for comparison evidence where the examination is a
simple visual exercise, perhaps with the aid of a microscope at most, and any
interaction or analysis takes place in the head of the analyst.
Nonetheless, particularly in the current climate of intense scrutiny, it is
necessary to demonstrate that any test system is working as expected and
providing reliable results. It is also good scientific practice to do so. Histor-
ically, this has neither been offered nor expected with any consistency. In
some cases, a great amount of knowledge has been amassed, but never sys-
tematized For any particular test system, we must either address these con-
cerns or show that they are not concerns.