Fifteen concepts that decide whether a prediction can be trusted.

A model is calibrated when its stated probabilities match observed frequencies: of all the cases it scored at 0.70, about 70% actually resolved that way. Calibration is what turns a score into a price — an uncalibrated probability cannot be underwritten, reserved against, or allocated on, because it does not correspond to anything measurable.

Area Under the ROC Curve measures discrimination: how often the model ranks an actual favorable outcome above an actual unfavorable one. 0.5 is a coin flip; 1.0 is perfect ordering. Our public promotion gate requires AUC ≥ 0.70 on temporally held-out cases. AUC and calibration are different properties — a model needs both, which is why the gate reviews both.

Instead of randomly splitting data into train and test sets, split by time: train on earlier filings, test on later outcomes the model never saw. Random splits let a model be flattered by memorizing period-specific patterns; a temporal split evaluates it the way production will — against cases decided after everything it learned.

The background frequency of an outcome for a jurisdiction and case type. Base rates differ enormously across venues, which is why pooled national statistics mislead: the national average settlement rate is not the settlement rate in your district. Jurisdiction-specific models exist to anchor every prediction to the right base rate first.

Training-data leakage is when information that would not be available at prediction time sneaks into training — a feature that encodes the outcome, a record duplicated across train and test. Symptoms include implausibly perfect metrics. Suspiciously perfect models in our registry are held out of production for exactly this reason, and leakage findings are treated as governance events, not embarrassments to hide.

The chain of custody from a model back to real source records: which system a record came from, its source identifier, when it was ingested, how it was deduplicated. Provenance is what makes an audit possible — and its absence is disqualifying: models whose training data cannot be traced do not hold production status here.

A model overfits when it performs brilliantly on data it trained on and poorly on data it has not seen — it memorized specifics rather than learning structure. Defenses are evaluation design (temporal holdout), minimum training-data thresholds, and gates that simply refuse to promote models that cannot demonstrate generalization.

A statistical model trained to estimate the probability of defined outcomes from structured features. Deterministic, versionable, auditable, and evaluable against real frequencies — the properties underwriting requires. Contrast with generative models, which are trained to produce plausible sequences and are the right tool for language tasks, not for calibrated frequencies.

Joining two records only when a key other than the name agrees, so spelling variants never merge two parties and one party is never counted twice.

The dates a relationship actually held in the world, recorded separately from when it was observed, so a question asked as of a date sees only what was true then.

A label on every relationship stating whether it came from a source record, a customer’s statement or a model, so an inference can prompt a review but never stands as a fact.

Concentration measured with each group of positions that share a firm, judge, provider or insurer counted as one exposure, compared with the same book measured position by position.

Tracing a change on one counterparty along currently valid links for a fixed number of steps, so every affected position is found without the change spreading across the whole record.

One specific entitlement to cash from a recovery, such as a fee, a lien or an advance, allocated in priority order so the rights together never exceed the recovery.

A cohort figure stated only when the data rights behind every row permit the use and coverage against a stated population is known.