Internal validity: chance, bias, confounding

What is internal validity

• Chance
• Bias
• Confounding

Chance

• is the observed association explained due to chance alone?
• Study finds those with high concentrations of arsenic in water have skin diseases
• Is it possible that this finding could arise by chance?
• Chance is POSSIBLE, so
• Rule out play of chance

Null hypothesis

• To rule out the play of chance
• Use Null Hypothesis
• Null Hypothesis is effect of NO DIFFERENCE

An example of Null Hypothesis

• Suppose we know that exposure to inorganic arsenic in drinking water causes skin disease
• Risk of Skin disease equal between those with and without high Arsenic exposure
• Null Hypothesis can be TRUE of FALSE
• Null Hypothesis should be rejected or failed to be rejected

Alpha and beta errors

Study	H0 TRUE	H0 FALSE
Reject H0	Type I error (alpha)
Fail to Reject H0		Type II error (beta)

Before planning the study

• Set a value for the Type I error (alpha error)
• Usually type 1 error set at 5%
• Set a value for Type II erro (beta error)
• Usually set at 20%

After completion of study

• What is the probability of the findings, if
• Null Hypothesis (H0) were true?
• If that probability is LOW,
• Reject the null hypothesis
• That probability is "p-value"

Interpretation of p-value

• If H0 were true:
• out of 100 iterations of the study,
• We would find the findings p times

How do reject the null

• if p is very low
• the probability is low
• we rule out the chance factor

Alternative approach

• Construct a 95% confidence interval
• If the study were to be conducted 100 times
• 95 out of 100 times, the findings
• Would be between the lower and upper value

You rule out the play of chance

• Before the study you set the values for Type I and Type II error
• Decide on the effect size you want to see as "significant"
• Estimate sample size

Hands-on practice with sample size calculator

• Visit
• http://www.openepi.com/SampleSize/SSCohort.htm

Example

Bias

• Systematic error
• The compared groups are unequal in different ways
• These impact their outcomes

Selection Bias

• You want to study effect of X on Y
• You will select different values of X in a way that
• That will favour your conclusions

Example of selecion bias

• Suppose you want to study association between indoor smoking and respiratory illnesses
• You know that indoor smoking is common in poor households
• You also know that many elderly people in poor households suffer from respiratory illnesses
• For your case control study you select
• Cases from poor neighbourhoods
• This will stack the results in your favour

Response Bias

• When the information collected is
• different for different groups that
• distort the direction of association

Example of Response Bias

• You want to study association association between indoor smoking and respiratory illnesses
• In your case control study,
• Cases if they know the purpose of the study could provide
• More accurate information about smoking than controls
• This can DISTORT relationships between smoking and lung disease

Steps to eliminate bias

• Objective measurement of exposure and outcome
• If using subjective tools such as interviews,
• Train interviewers and use checks and balances
• Blinding and concealment of information from all parties
• Do everything at the design stage of the study

Confounding

• Associated with Exposure
• Associated with Outcome
• Does not come in the causal pathway connecting the two

Illustration of confounding

Example of confounding

• You want to study association between indoor smoking exposure and heart disease
• Male spouse of smokers are both at increased risk of exposure
• Males are also more likely to suffer from heart disease
• Yet maleness DOES NOT come in the causal pathway
• Hence "gender of the spouse" is a confounder

Control for confounding

• Randomisation (works for randomised controlled trial)
• Matching for observational studies
• Stratified analysis
• Multivariate modelling and analysis

Summary

• Chance, bias and confounding are three important factors
• Chance can be ruled out with adequate sample size estimation
• Bias can be eliminated with design
• Confounding can controlled with several strategies
• Next up: Causal inference