Scientific Method

Sampling and Non Sampling Errors

Sampling Errors

Random Sampling Error

• random error- the sample selected is not representative of the population due to chance
• the level of it is controlled by sample size
• a larger sample size leads to a smaller sampling error.

Systematic Errors – These errors result from factors such as an improper research design that causes response error or from errors committed in the execution of the research, errors in recording responses and non-responses from individuals who were not contacted or who refused to participate

• Systematic Errors – These errors result from factors such as an improper research design that causes response error or from errors committed in the execution of the research, errors in recording responses and non-responses from individuals who were not contacted or who refused to participate

Graphical Depiction of Sampling Errors

Total Population

Sampling Frame Error

Random Sampling Error

Sampling Frame

Planned

Sample

Non-Response Error

Respondents

(actual

sample)

Non-Sampling Errors (I)

• The basic types of non-sampling error
• Non-response error
• Response or data error
• A non-response error occurs when units selected as part of the sampling procedure do not respond in whole or in part

Non-sampling Error

• the level of it is NOT controlled by sample size.

Non-Sampling Errors (II)

• A response or data error is any systematic bias that occurs during data collection, analysis or interpretation
• Respondent error (e.g., lying, forgetting, etc.)
• Interviewer bias
• Recording errors
• Poorly designed questionnaires

Questions for sampling design

• Pre-sampling choices
• What is the nature of the study: exploratory, descriptive, analytical?
• What are the outcomes of interest?
• What are the target populations?
• Do you want estimates for subpopulations or just for the entire population?
• How will the data be collected?
• Is sampling appropriate?

Questions for sampling design

• Sampling choices
• What listing will be used as the sampling frame?
• What is the desired precision?
• What type of sampling will be done?
• Will the probability of selection be equal or unequal?
• What is the sample size?

Questions for sampling design

• Postsampling choices
• How can the effect of nonresponse be assessed?
• What are the confidence limits?

But…

Result from survey is never

exactly the same as

the actual value in the population

WHY?

Components of total error

0%

100%

True

population

value

50%

Point

estimate

from survey

40%

Total error

Nonsampling

bias

Sampling bias

Sampling

error

Nonsampling bias

Source of bias

Sampling frame out of date

Non-response

Measurement error

Use current sampling frame

Limit generalizations

Minimize non-response

Use various statistical methods to weight data

Standardize instruments

Write clear & simple questions

Train survey workers

Supervise survey workers

Prevention or cure

Sampling bias

• Selection of non representative sample
• Failure to weight analysis of unequal probability sample

In sum, you have not sampled people with equal probability and you have not accounted for this in your analysis!

Sampling bias

• Examples
• Non-representative sample
• Selecting youngest child in household
• Using a different sampling fraction in different states
• Failure to do statistical weighting

Sampling bias

Source of bias

Nonrepresentative sampling

Failure to do weighting

ALWAYS ask yourself "Will this choice enhance representativeness or reduce it"?

Calculate the probabilities of selection

Apply appropriate statistical weights if selection probabilities unequal

Prevention or cure

Sampling error

• Measures of sampling error:
• Confidence limits
• Standard error
• Coefficient of variance
• P values
• Others

Bias and sampling error

Nonsampling bias

Sampling bias

Sampling error

Bias

Sampling error