Fundamentals of Statistics contains material of various lectures and courses of H. Lohninger on statistics, data analysis and chemometrics......click here for more.


Interpreting p values

For statistical tests there is one value which specifies the strength of its evidence:
 
p-value The p-value defines the probability that a test statistic equal to or greater than the one obtained in the test, will be observed for the given population if the null hypothesis is true.

A low p-value for a statistical test should lead to the rejection of the null hypothesis. Thus, it is important to know the null and the alternative hypotheses. P-values provide a sense of the strength of the evidence against the null hypothesis.
 

An example should clarify the point:

Suppose you have to decide whether there are any differences in the wear and tear of truck tires between two different brands. The null hypothesis will be that the wear does not differ, the alternative hypothesis is that they do differ. Assuming that the data (18 samples each) is normally distributed and the means (2.03 and 2.69 mm) and standard deviations (1.30, and 1.11, respectively) are known, we can calculate the test statistic t=1.762.

Using the t-distribution, we can find the corresponding p-value of 0.086. This means that in 86 out of 1000 cases, the test statistic will exceed the value of 1.762, although the null hypothesis is true. Or, in other words, if you reject the null hypothesis, you commit a mistake in 8.6 percent of the cases.