Unmarked Assignment: Model Tuning
How much curve flexibility helps predict beech-group harvest from oak-group harvest? Compare GAM settings on the same held-out years.
Things you need for this worksheet
- R — the interpreter can be installed on any operating system.
- RStudio — we recommend using RStudio for (interactive) programming with R.
- Git installed and available to RStudio. GitHub Desktop is an optional graphical client.
- The file hessen_holzeinschlag_2003-2025.csv, saved in
Data/and also used in Assignment 03.
Model tuning
Reuse the Hesse wood-harvest data from Assignment 03: 23 annual records from 2003 to 2025. Import them without the four introductory lines or the final notes.
The aim is to predict harvest in the beech and other deciduous trees group (Beech_other_deciduous) from harvest in the oak and red oak group (Oak_red_oak). Both columns are measured in 1,000 m³ without bark; they represent wood type groups, not individual species.
Write an R Markdown document tuning a Gaussian additive model for Beech_other_deciduous from Oak_red_oak. Use all 23 complete pairs and check that each year occurs once. Keep Year to identify the observations; use only Oak_red_oak as a predictor.
Fit and compare basis dimensions
Use mgcv::gam() with one thin-plate regression smooth, s(Oak_red_oak, bs = "tp", k = ..., fx = TRUE), and the default Gaussian identity-link family. Here fx = TRUE makes the smooth unpenalized so that the exercise isolates basis dimension k, from 3 through 13, as the tuning setting. k is not a literal number of knots; the worked explanation contrasts this with penalized smoothing.
Create 100 random splits, each with 18 training years and five held-out years (approximately 80/20). Set the seed once and record it with your R and mgcv versions. Keep the training-row list and year IDs, and reuse the same splits for every k. Fit on the training rows and predict the five held-out observations.
Check that each training set has enough distinct predictor values for k and leaves residual degrees of freedom. Here the 18 distinct training values support k = 3:13; at k = 13, only five residual degrees of freedom remain. Diagnose failed fits or nonfinite predictions before comparing settings; do not silently omit runs.
Summarize and visualize the tuning scores
For every k, calculate one RMSE per run from its held-out predictions, and record summary(fit)$r.sq from that run’s training fit. Plot these summaries as a function of basis dimension k:
- The mean of the 100 run-level held-out RMSE values.
- That same mean plus/minus the SD across those 100 RMSE values.
- The mean adjusted training R squared across the 100 fitted models. This is a fitted measure and can be negative; it is not a predictive R squared on the five held-out observations.
For the shared-scale visualization, divide the mean RMSE and both SD bounds by one common divisor: the maximum mean RMSE across k. Leave adjusted R squared on its own scale, label the normalization, and keep the original harvest unit when reporting numerical errors. Ensure the axes show all three curves and both SD bounds, including any negative values. The scaled error and R squared have different meanings despite sharing a plot.
Interpret the result
Identify the setting with the smallest mean held-out RMSE and describe its sensitivity to the splits. The SD band describes that variation, not a confidence interval. The minimum is a tuning score, since it helped choose k; a separate final-test submission is not required.
These splits assess held-out years within the historical series, not future-year forecasts. Trends, temporal dependence and collection changes limit transfer. Check whether any held-out oak-harvest values lie outside their training range and therefore require extrapolation. The worked example shows how an independent final assessment differs.
Save your R Markdown (.Rmd) document in your course repository, knit it, update (i.e. commit) your local repository and publish (i.e. push) it to GitHub. Make sure that the created HTML file is also part of your course repository on GitHub.