Variable Selection - Stepwise Regression

Stepwise regression is a special case of forward selection: in addition to the steps performed in the forward selection algorithm, all variables are tested if their contribution is significant after a new variable has been added. This may lead to the elimination of an already selected variable if this variable has become superfluous because of its relationship to the other variables.

Algorithm:
 

1. Calculate the correlations of all independent variables, X_i, with the response variable Y. Use the variable with the highest correlation as the starting variable.
2. Add the variable with the highest partial F value.
3. Check all variables of the current model for their partial F values and remove any variable which falls below a predefined threshold.
4. Repeat the procedure with step 2 until some stopping criterion is met.

Note that the list of variables obtained by stepwise regression may be different from the set of variables obtained by forward selection.