Tune Your Models

Evaluate model tuning strategies and compare settings for your prediction model.

Learning objectives

At the end of this session you should be able to

  • fit a generalized additive model (GAM) with a smooth response curve,
  • compare one flexibility setting with others on the same held-out observations, and
  • distinguish training fit, tuning scores and independent final evaluation.

Basic idea of model tuning

Schematic comparison of spline fits with increasing flexibility.

The graphic illustrates how a smooth curve can become more flexible. In the worked example, k limits how complex the curve can be; an optional penalty discourages unnecessary wiggles. We introduce both controls beside the first smooth fit.

Closer agreement with training observations does not establish better prediction. Use held-out scores to tune settings and keep final evaluation separate from that choice. The worked example distinguishes these roles and defines the repeated-run metrics needed for the assignment.

For a reminder of how training R squared, held-out RMSE, and AIC differ, see which measure answers which question in Unit 07.

Comments?

You can leave comments below if you have questions or remarks about any of the text or code in this unit. Please copy the corresponding line into your comment to make it easier to answer your question.

Updated: