The Research Process: Step 6 (Research Design for Experiments Part B)
Lecture 23
Lecture Topics Covered Previously in the Last Lecture
Introduction to Experimental Designs
Control and Manipulation of Independent Variable
Techniques for Controlling Exogenous Variables
Internal and External Validity
What we are going to Cover in this Lecture
Factors Effecting the Internal Validity of Experiments
THE RESEARCH PROCESS
(1).
Observation
The Broad Problem Area
(2).
Preliminary Data Gathering
Interviews and Library Search
(3).
Problem Definition
(4).
Theoretical
Framework
Variables
Identification
(5)
Generation of
Hypothesis
(6).
Scientific
Research
Design
(7).
Data Collection and Analysis
(8)
Deduction
(9).
Report Writing
(10).
Report Presentation
(11).
Managerial Decision Making
THE ELEMENTS OF RESEARCH DESIGN
2. Type of
Investigation
Establishing:
Causal
Relationship
or
Co-relational
1. Purpose of
Study
Exploratory
Descriptive
Hypothesis
Testing
Case Study
3. Extent of
Researcher
Interference
Minimal
Moderate
Excessive
4. Study Setting
Contrived
Non-Contrived
10. Test
Application
Feel for
Data
Goodness
of Data
Hypotheses
Testing
6.Unit of Analysis
(Population to be
studied)
Individuals
Dyads
Groups
Organizations
Machines
etc.
7. Sampling
Design
Probability
Non-probability
Sample Size (n)
8. Time Horizon
One-Shot
(Cross-Sectional)
or
Longitudinal
9. Data
Collection
Methods
Observation
Interviews
Questionnaire
Physical
Measurement
5. Measurement
& Measures
Operational Definition
Scaling
Categorizing
Coding
Problem Statement
FACTORS AFFECTING INTERNAL VALIDITY
Even the best designed lab studies could be influenced by factors that might affect the internal validity of the lab experiment. These possible confounding threats pose a threat to internal validity. The seven major threats to internal validity are:
HISTORY EFFECTS
MATURATION EFFECTS
TESTING EFFECTS
INSTRUMENTATION EFFECTS
SELECTION EFFECTS
STATISTICAL REGRESSION EFFECTS
MORTALITY EFFECTS
History Effects
Certain events or factors that would have an impact on the independent variable - dependent variable relationship might unexpectedly occur while the experiment is in progress, and this history of events would confound the cause and effect relationship between the two variables, thus affecting the internal validity.
Example
Independent variable Dependent variable
Sales Promotion
Sales
Dairy
Farmer’s
Advertisement
Uncontrolled
variable
Maturation Effects
Cause and effect inferences can also be contaminated by the effects of the passage of time - another uncontrollable variable. Such contamination is called maturation effects. Examples of maturation effects processes could include growing older, getting tired, feeling hungry, and getting bored.
Maturation
effects
Example
Independent variable Dependent variable
Maturation
effects
Enhanced Technology
Efficiency Increase
Gaining experience
and doing the job faster
Testing Effects
Frequently, to test the effects of a treatment, subjects are given pretest (e.g., a short questionnaire eliciting their feelings and attitudes). That is, first a measure of the dependent variable is taken (the pretest), then the treatment is given, and after that a second test, called the posttest, is administered. The difference between the posttest and pretest is then attributed to the treatment. However, the very fact that respondents were exposed to the pretest might influence their response on the posttest, which would adversely impact on internal validity.
Testing Effect = 27-24 = 3
We have to minus this score from posttest values of Group A and B to adjust for testing effects.
Example of Testing Effect:
Instrumentation Effects:
Instrumentation effects are another source of threat to internal validity. These effects might arise because of a change in the measurement instrument between pretest and posttest, and not because of the treatment’s differential impact at the end.
Selection Bias Effects: The threat to internal validity could also come from improper or unmatched selection of subjects for the experimental and control groups.
Statistical Regression Effects:Statistical regression occurs when members chosen for the experimental group have extreme scores on the dependent variable to begin with.
Mortality
Another confounding factor on the cause and effect relationship is the mortality or attrition of the members in the experimental or control group or both, as the experiment progresses. When the group composition changes overtime across the groups, comparison between the groups become difficult, because those who dropped out of the experiment may confound the result.
The shorter the time span of the experiments, the less the chances of encountering the history, maturation, and mortality effects. Experiments lasting an hour or two do not usually encounter many of these problems. It is only when experiments take place over an extended period (e.g., several months) that the possibility of encountering more of the confounding factors increases.
How to Avoid the Confounding Factors
Summary
Factors Effecting the Internal Validity of Experiments
