Randomization Tests

Most non-parametric tests are based on the principle of randomization. The idea behind this is that the probability of a particular value of a test statistic is calculated by evaluating all possible values of the test statistic. Since the number of possible values of a test statistic can be huge, even for small samples sizes, it is often impractical (or even impossible) to calculate the probability directly.

A possible strategy to reduce the amount of required computations is called resampling: instead of calculating all possible combinations, one could draw numerous random samples and calculate the probability from these samples.