# Hypothesis Testing

In statistics, hypothesis testing is a method for evaluating hypotheses. A hypothesis is a proposed explanation for an observed phenomenon.

The way hypothesis testing works is in a way analogous to how proof by contradiction works [1].

Unlike proof by contradiction though, the conclusions obtained via hypothesis testing do not underline some *absolute* truth. This is by nature of statistics, which is inherently inferential and approximate.

In proof by contradiction,

- we are interested in a
*candidate statement*which we want to*demonstrate to be false* - we start by assuming the statement to be true
- we then seek for a contradiction to this statement
- if we demonstrate a contradiction, then we will have proven the candidate statement to be false
- if we fail to find a contradiction, then we will have
*failed to prove*the statement to be false.

Analogously, in hypothesis testing,

- we have a
*null hypothesis*, which we seek to reject so to instead accept an opposing*alternative hypothesis* - we start by assuming the null hypothesis to be true
- we then make a number of observations, collecting data on the outcomes so to estimate the probability of the observed outcomes, assuming that nothing observed is extraordinary
- if what we observe has a probability lower than some
*significance threshold*which we define in advance, then we conclude that this is evidence that our assumptions (null hypothesis) are incorrect (false) and we must reject them, accepting the alternative hypothesis instead.^{1} - otherwise, we fail to reject our null hypothesis. This does not mean we accept the null hypothesis.
- after all, the null hypothesis is nothing but a “tool” for producing a “proof” by contradiction, or better, for coming to statistical conclusions.

- if what we observe has a probability lower than some

Here is a table that may be useful in seeing the analogy

Proof By Contradiction |
Hypothesis Testing |
---|---|

Candidate Statement (CS) | Null Hypothesis (NH) |

we want to prove CS to be false | we want to reject NH |

assume CS to be true | assume NH to be true |

seek for contradiction | collect data on phenomenon |

contradiction found -> prove CS false | probability < \(\alpha\) -> reject NH |

otherwise -> fail to prove CS false | otherwise -> fail to reject NH |

## References

[1] Goldman, D. S. (2018). *The Basics of Hypothesis Tests and Their Interpretations*. Center for Open Science.

Rejecting the null hypothesis does not necessarily mean accepting the alternative hypothesis.

- This is because the null hypothesis also assumes an underlying distribution, which is assumed in the alternative hypothesis too.
- As such, rejecting the null hypothesis either means that indeed there is something unusual that warrants an acceptance of a new (alternative hypothesis)
*or*that the original distribution which we chose was actually wrong.