Maintenance: counting down to failure

Predictive maintenance is the most directly transferable field to "how accurate is a predicted time-to-death," because Remaining Useful Life prognostics is explicitly a countdown-to-failure problem with a mature, named, percent-based accuracy standard.

The core idea

Industrial equipment — turbine blades, bearings, batteries, aircraft engines — is instrumented with sensors that produce a degrading signal over time. A prognostics algorithm's job is to estimate Remaining Useful Life (RUL): how much operating time is left before the component reaches its end-of-life threshold. Because a maintenance decision hangs on that number, the field doesn't just report a single RUL estimate — it reports a full probability distribution over RUL and asks a very specific question: how tightly, and how early, does that distribution cluster around the true answer as more data comes in?

The formula

The canonical reference for how this field measures itself is Saxena, Celaya, Saha, Saha & Goebel's 2009 paper "Metrics for Offline Evaluation of Prognostic Performance" (International Journal of PHM), which defines four metrics.

α-λ Performance — the ±% RUL accuracy cone

This is the field's literal answer to "what accuracy is good enough": a binary pass/fail test of whether a prediction falls within an α-percent band of the true RUL, evaluated at a fractional point λ between the first prediction and end-of-life:

α−λ Accuracy = 1 if π[r(i_λ)]^{α+}_{α−} ≥ β, else 0

where λ ∈ [0,1] locates the evaluation time as t_λ = t_P + λ(t_EoL − t_P), and the α-bounds are expressed as a percentage of the true RUL at that point:

α+ = r*(i_λ) + α·r*(i_λ) α− = r*(i_λ) − α·r*(i_λ)

The paper gives a worked example directly: "this metric would determine whether a prediction falls within 10% accuracy (α = 0.1) of the true RUL halfway to failure from the time the first prediction is made (λ = 0.5)." α = 0.10 (±10%) and α = 0.20 (±20%) are the two values conventionally used across the field's benchmark literature — for example, RUL papers using the CMAPSS turbofan-engine dataset routinely report "α-λ accuracy at α=10%" and "α=20%" as their headline numbers. The accuracy cone narrows as λ approaches 1 — predictions are held to a tighter percentage tolerance the closer the system gets to actual failure. In plain terms: predictions are expected to tighten as failure approaches.

Prognostic Horizon and Relative Accuracy — the companion metrics

Prognostic Horizon (PH) asks how far in advance the algorithm first predicts within the accuracy band and stays there:

PH = t_EoL − t_{i_αβ}

Relative Accuracy (RA) quantifies the error magnitude (not just pass/fail) at a given λ:

RA = 1 − |r*(i_λ) − ⟨r(i_λ)⟩| / r*(i_λ)

ranging 0 to 1, with 1 being perfect. A Cumulative Relative Accuracy (CRA) variant weights predictions closer to end-of-life more heavily. Finally, Convergence measures how fast the accuracy metric improves as more data arrives — defined geometrically as the distance from the first-prediction point to the centroid of the area under the accuracy-vs-time curve, with a smaller distance meaning faster "lock-in" as failure nears.

How this field validates its predictions

A second, independently used benchmark is the PHM08 Data Challenge's asymmetric scoring function (still the standard for the CMAPSS turbofan-engine benchmark dataset), which penalizes early and late predictions differently:

score(d) = e^(−d/13) − 1 for early predictions (d = predicted RUL − actual RUL < 0) score(d) = e^(d/10) − 1 for late predictions (d ≥ 0)

The penalty grows exponentially with error magnitude and is deliberately asymmetric: overestimating remaining life (a late prediction, missing the failure window) is penalized more heavily than underestimating it (an early prediction, which just means scheduling maintenance sooner than strictly necessary) — because underestimation is operationally safer. This asymmetric-cost framing has no clean analog in the other fields surveyed on this site and is unique to safety-critical maintenance scheduling.

The field's honesty norm: there is no single universal numeric bar the way survival analysis has "C-index ≥ 0.7." The NASA paper itself states that required accuracy is cost-benefit-risk derived per application — how much time is needed for corrective action, the cost of a false positive versus a false negative, mission criticality — and that α, λ, and β must be chosen per deployment. The de facto community convention across benchmark papers is to report α-λ accuracy at the ±10% and ±20% bands, plus RMSE/MAE on raw RUL cycles, plus the PHM08 asymmetric score. Recent published benchmark results on CMAPSS run 13.4–14.4 cycles RMSE with 10.5–10.7 cycles MAE, with the best current methods claiming up to 52% MAE / 64% RMSE reduction over older baselines.