Medicine: predicting time-to-event outcomes

Survival analysis is the discipline of predicting when an event happens — not just whether — and of being explicit that predicting the right rank order is a different achievement than predicting the right absolute time.

The core idea

Clinical researchers rarely have a full record of every patient's outcome — some patients are still alive when a study ends, some drop out. This partial information is called censoring, and survival analysis is built around handling it correctly rather than discarding incomplete cases. The two workhorse tools are a non-parametric way to estimate the overall survival curve from censored data (Kaplan-Meier) and a regression framework for how covariates shift that curve (Cox proportional hazards).

The formula

Kaplan-Meier estimator

The non-parametric estimate of the survival function — the probability of surviving past time t:

where dᵢ is the number of events at time tᵢ and nᵢ is the number of subjects still at risk (alive and uncensored) just before tᵢ. Its variance, used to build confidence intervals, is given by Greenwood's formula:

Cox proportional hazards model

The Cox model expresses each subject's hazard as a baseline hazard multiplied by an exponentiated linear combination of covariates:

The coefficients β are fit by maximizing the partial likelihood — a clever construction that cancels out the unknown baseline hazard λ₀(t) entirely, so it never has to be estimated directly:

An alternative parametric approach, the accelerated failure time (AFT) model, instead models log-survival-time directly as a linear function of covariates:

AFT requires committing to a distribution for the error term ε (extreme-value, normal, logistic), which determines whether survival time is modeled as Weibull-, lognormal-, or log-logistic-distributed — but in exchange it is simpler to interpret and cheaper to fit than Cox.

How this field validates its predictions

Survival analysis's headline discrimination metric is Harrell's concordance index (C-index), introduced in 1982 and adapted for censored survival data in 1996:

In words: among all pairs of subjects where the earlier time is a genuine observed event (not a censoring), the C-index is the fraction of pairs the model correctly ranked — did it predict the subject who actually failed first as the higher-risk one? A C-index of 0.5 is equivalent to a coin flip; 1.0 is perfect discrimination. The commonly cited convention is that roughly 0.7 is "generally acceptable" discrimination and above 0.8 is "strong." Best practice corrects for the fact that a model's C-index on its own training data is optimistically biased — bootstrap internal validation (on the order of 200 resamples) is used to report an optimism-corrected figure before a model is considered validated.

Critically, the field is explicit that the C-index has a real blind spot: it measures rank-ordering only. A model can have a strong C-index while still producing systematically wrong absolute survival probabilities — so calibration has to be checked separately.

Calibration — the companion check

• Calibration-in-the-large compares the Kaplan-Meier-observed survival at a given time point to the model's average predicted survival probability at that same point — flagging systematic over- or under-prediction.
• Calibration slope comes from plotting observed vs. predicted risk: a slope of 1 and intercept of 0 (the 45° line) is perfect calibration; a slope under 1 means predictions are systematically too extreme (overconfident).
• Brier score, a time-dependent and censoring-adjusted error measure, ranges 0–1 (lower is better) and is mainly used to compare models relative to each other rather than against a fixed universal threshold.