Why every model carries a card
A reviewer looking at a Criterica report sees figures: a duration band, a resolution probability, a settlement range. Behind each figure is a model, and the reviewer’s question is always the same: what is this model, what was it tested on, and how did it do? The model card is the standing answer. It is written when a model is promoted, updated whenever the model changes, and kept with every earlier version.
Cards follow one format across the production predictive fleet.2 The card is designed for the person who has to sign off on reliance: a model-risk officer, an investment committee, a court-appointed expert or a regulator.
What a card contains
| Field | What the reviewer learns |
|---|---|
| Identity and version | A stable identifier and version, so any figure can be traced to the exact model that produced it. |
| Estimate | The measure, its unit and its horizon, for example probability of resolution before trial within 24 months. |
| Population and scope | The case type, venue and stage the model is valid for, and the populations it is not used for. |
| Training window and provenance | The filing period the model learned from and the class of source. |
| Validation design | The time cut and the later period the model was scored on. |
| Discrimination | AUC on that later period. |
| Calibration | ECE and the reliability diagram on that later period. |
| Gates | Each promotion gate, the result and the date it was cleared. |
| Status and intended use | Production or otherwise, and the decisions the model is meant to inform. |
| Known limits | Thin sub-segments, recent regime changes and other conditions a reader should weigh. |
| Version history | Every earlier version, with what changed and why. |
| Monitoring and retraction | How the model is watched and the conditions under which it is withdrawn. |
The card states results and design. It does not publish the model’s internal construction; that is reviewed under NDA where a reviewer needs it. TN-003 explains how to read the calibration fields.1
Sample card
The sample below follows the card format exactly. In place of a single model’s results it carries fleet-level figures cleared for publication, so the format can be shown in full.
| Field | Sample entry |
|---|---|
| Identity and version | SAMPLE-FLEET, card format version 1.0 |
| Estimate | Binary outcome probabilities for production predictive models across the fleet |
| Population and scope | Production predictive fleet; each model valid only for its own case type, venue and stage |
| Training window and provenance | Real records; each model trained on matters filed before its time cut |
| Validation design | Temporal holdout: scored on matters filed after the time cut |
| Discrimination | Median AUC 0.739; upper-quartile AUC 0.853 (July 2026 registry extract) |
| Calibration | Every production classifier clears ECE ≤ 0.05 on its holdout |
| Gates | AUC ≥ 0.70 and ECE ≤ 0.05 on the temporal holdout; both required |
| Status and intended use | Production; informs pricing, reserving and strategy alongside duration and settlement bands |
| Known limits | Segments without enough resolved matters are held at stub status and not issued as production figures |
| Version history | Each model version kept; changes recorded in the public changelog |
| Monitoring and retraction | Monitored against new resolutions; withdrawn to experimental status on the record if it stops clearing its gates |
Reading the sample
A median AUC of 0.739 means that, for the typical production model, a randomly chosen matter with the outcome is ranked above a randomly chosen matter without it about 74% of the time, on matters from a period the model never saw. A quarter of the fleet does better than 0.853. Every model in the fleet has also cleared the 0.70 floor and the 0.05 calibration gate; the card for each one states its own figures.
Versioning and retraction
- A retrained model is a new version with its own card. The earlier card is kept, so a figure issued under it can still be traced.
- A model that stops clearing its gates on new resolutions is withdrawn from production and the withdrawal is recorded with its reason.
- Every promotion, demotion and retirement appears in the public changelog maintained with the model registry (MC-001).
Corrections are treated the same way as improvements: recorded with a date and a reason. A registry that only ever shows additions would not be telling the reader what changed. See the model registry and changelog.
How to request cards
Institutional reviewers receive the cards for every model that contributes to a report they are relying on, under NDA, as part of the method pack. A request names the report or the matters concerned; the research team returns the cards with their version history.
The question, as the reader has it
A model-risk officer, an investment committee member or a court-appointed expert is asked to rely on a figure from a model they did not build. Their problem is the standard one in model governance: what is this model for, what population does it describe, how was it tested, and under what conditions would it be withdrawn? A single headline accuracy number cannot answer that. It is silent on population, on the period it was tested over and on calibration, so it supports reliance for no particular decision.
A card answers it in a fixed order, the same for every model, so reviewers can compare cards and a missing field is visible as a gap.
What a card measures
| Quantity | Unit | Population |
|---|---|---|
| Discrimination | Area under the receiver operating characteristic curve, between 0 and 1 | Matters filed after the time cut, scored by a model trained only on matters filed before it |
| Calibration | Expected calibration error and a reliability diagram | The same later-period matters |
| Interval coverage | Share of outcomes inside the stated band, for models that state one | The same later-period matters |
| Estimate | Probability over a stated horizon, a duration in months or a range in US dollars | Closed matters of one case type, venue and stage, from filed court records with a recorded termination |
Inclusion is stated on the card at the level of a recorded termination: only matters whose record shows how they ended contribute to an observed outcome. The card names the case type, venue and stage a model is valid for and, as importantly, the ones it is not.
How a number travels, and what the card records at each step
- Record to structured event. The card states the class of source the model learned from, and the window, so the reviewer can match the model to the population it is used for. Entities are resolved on a knowledge graph before any model sees them.
- Event to model class. The card names the class: survival analysis for durations (Kaplan-Meier and proportional-hazards framing), calibrated probabilistic classification for event probabilities, and hierarchical partial pooling where a venue is sparse. It states why that class fits the estimate.
- Model class to calibrated output. The card records that scores were recalibrated on held-out matters and that bands are quantile or conformal intervals with a tested coverage.
- Output to band shown. The card ties the band a reader sees to the version that produced it, so any figure can be traced to its card.
The card publishes design and results. It does not publish a model’s internal construction, which a reviewer sees under NDA when it is needed.
Validation design
Every card describes the same design. What is frozen: the model and its recalibration at a time cut. What is compared: each stated probability or band with how the matter closed in the later period. How leakage is prevented: the point-in-time boundary applies to every input, so a field recorded after the cut is not visible to an earlier estimate. The metric families are discrimination, calibration and interval coverage, each reported with the date of the holdout beside it.
The public promotion gates are stated in the sample card and in TN-003. Figure 1 is evidence of what a reviewer will find on a card: twelve fields in a fixed order. Figure 2 is evidence of the card’s place in the life of a model: issued at promotion, updated at every later step, and retained at retirement.
Scope and precision
A card describes a model that estimates what the court record supports: probabilities, durations and ranges for comparable closed matters. It does not describe a prediction of the realized dollars of one matter, and none of the cards states who prevails from a procedural disposition code.
- Data lag: the card states the date of its record and is updated as new resolutions arrive.
- Censoring: open matters are not outcomes; the duration models account for them.
- Venue sparsity: a segment without enough resolved matters is held at stub status and the card says so.
The sample card reports fleet-level figures, which describe the fleet as a whole and not any one model.
What a reader can reproduce
The card format is public, so a reader can check any report against it: a card is complete when every field is present. The reading of the fleet-level sample, what a median AUC means and how to interpret a gate, can be reproduced from the definitions in TN-003. The results for a specific model can be reproduced by an institutional reviewer from the held-out evaluation described on its card, which is supplied under NDA with the version history, because it depends on matters held on the platform.