If you have a distribution, such as rolling dice (uniform distribution), and you repeatedly sample that and make a new distribution from the means of those samples, then that new distribution will eventually approach a normal distribution with a sufficiently large sample size.
Useful because if you have two funky-looking distributions, you can sample them a bunch of times, and now you have two nice normal distributions that you can compare.
Criteria for samples:
- Picked at random
- Representative of population
- Big enough to draw conclusions (>=30)
- Include less than 10% of the population, if you’re sampling without replacement