Generalized Models: GLMs and GAMs

Linear regression has a few problems. It handles continuous, roughly symmetric outcomes well. The real world also gives us counts, categories, bounded rates, waiting times, and income distributions with long tails.

A generalized linear model (GLM) keeps the weighted sum $\eta=\beta_0+\beta^Tx$ , chooses a probability distribution for the outcome, and connects the expected outcome to $\eta$ through a link function.

Choose the Outcome, Then the Link

The weighted sum can stay familiar while the outcome changes. Pick what you need to predict and watch the inverse link map the score onto a valid scale.

Feature value x3.0Let a GAM learn a curved feature function

Distribution

Poisson

Link

log

Predicting

support tickets

1. Linear predictor η = β₀ + β₁x

η = −0.5 + 0.55(3.0) = 1.150

2. Inverse link μ = e^η (log)

μ = e^(1.150) = 3.158

Expected support tickets: 3.16 tickets

A GAM changes the predictor side by learning a smooth function of a feature. A GLM changes the outcome distribution and link. You can combine both ideas.

◆

Match the family to the measurement

Normal + identity: delivery time measured in days.
Bernoulli + logit: whether a borrower defaults.
Poisson + log: number of support tickets in a week.

The link keeps predictions on a sensible scale. Exponentiating a Poisson score prevents a negative ticket count.

A generalized additive model (GAM) addresses a different limitation: nonlinear relationships between features and the outcome. GAMs replaces some coefficients (\beta) with smooth functions (\f) called splines:

$g(E[y])=\beta_0+f_1(x_1)+f_2(x_2)+\cdots+f_p(x_p)$

Suppose cardiovascular risk rises slowly with age until 50 and then climbs quickly. A single age coefficient misses that shape. A GAM can learn and plot $f_{age}(age)$ while keeping the contributions additive.

Spline Functions in a GAM

A GAM replaces each coefficient with a smooth function f(x). That function is built from overlapping basis pieces — a spline. Drag the knots along the curve to reposition them, or click the curve to add one.

Risk rises slowly until middle age, then climbs steeply — a bend a linear term would miss.

Interior knots (2)

knot 1 — 35%

knot 2 — 60%

Drag handles on the curve, or click the curve to add a knot (max 6).

Show individual basis functions

GAM term

f(x) =+++++

Enable "Show basis functions" to highlight individual terms.

Composite spline f(x) Knot (drag to move)

Each knot is a join point between polynomial pieces. Moving a knot reshapes the curve in its neighbourhood — each basis function spans several knot intervals, so the effect is local but not confined to a single point. More knots give more flexibility but require a stronger smoothing penalty to prevent overfitting.

ℹ

"Additive" doesn't mean the curves are straight

The word linear appears in both GLMs and GAMs, but it refers to the model's structure, not the shape of the curves.

A GLM is linear in its parameters — each feature is multiplied by a single coefficient β.
A GAM is linear in its additive structure — the feature contributions are summed without interaction terms. Each f(x) is itself a smooth, nonlinear curve built from piecewise cubic polynomials.

The basis functions you see in the explorer are cubic bumps, not lines. What makes a GAM interpretable is that you can plot and inspect each f separately — not that each f is a straight line.

⚠

More flexible means more to explain

A GLM coefficient must be interpreted through its link. A GAM contribution is a curve rather than one number. Both can remain inspectable, but the stakeholder now needs to understand the distribution, link, or smooth function. Use the extra machinery when the outcome or relationship demands it.

Checkpoint

You need to predict the number of emergency-room visits per patient next year. Which starting point is most defensible?

Checkpoint

A GAM is substantially more accurate than a linear model, but the age effect bends twice. How would you present that effect to a clinical stakeholder?

←PreviousLogistic RegressionRegression Next→Decision TreesTrees and Rules