Criterica Intelligence — production models trained on real court records, not synthetic data
Models vs. Generative AI

"Why not just use GPT?" is the right question. Here is the complete answer.

We are not anti-AI — language models are genuinely good at parts of legal work, and we say which parts below. But an institution deploying capital against an outcome needs a number that is true at a measured frequency, and that is a different machine. This page is the architectural argument, in full.

0.250.250.500.500.750.751.001.00CALIBRATED — SAYS 70%, RIGHT ~70%OVERCONFIDENT — SAYS 70%, RIGHT FAR LESSSTATED PROBABILITY →OBSERVED FREQUENCY →
The only test that matters

When the system says 70 percent, is it right about 70 percent of the time?

That single question separates prediction infrastructure from everything that merely sounds predictive. A calibrated model's stated probabilities track observed frequencies — the diagonal on this chart. Systems that were never trained against real outcome frequencies drift off the diagonal, most often into overconfidence, and no amount of fluency fixes that.

Calibration is not a nice-to-have. It is what makes a probability usable as a price. An uncalibrated 70 percent cannot be underwritten, reserved against, or allocated on — because it does not mean anything.

Six dimensions, side by side
CALIBRATED STATISTICAL MODELS
GENERATIVE LANGUAGE MODELS
What it outputs

A calibrated probability. 0.73 means this configuration of jurisdiction, case type, and posture resolves favorably about 73% of the time, measured against real outcomes.

Fluent text that reads like an answer. Any number inside it is a token the system judged likely to appear — not a measured frequency of anything.

What it learned from

Real filed court records with known outcomes, deduplicated and provenance-keyed. The training target is what actually happened.

The public internet — including commentary, advocacy, fiction, and marketing about litigation. Court outcomes at the jurisdiction level are largely absent from that corpus.

How it is evaluated

Temporal holdout: scored on later real outcomes it never saw. Public promotion gate at AUC ≥ 0.70, calibration reviewed, per model.

Benchmarks of general capability — reasoning puzzles, exams, preference ratings. None of them measure whether a stated litigation probability is true.

Reproducibility

Deterministic and versioned. The same case profile scored twice returns the same number, and the registry records which model version returned it.

Stochastic by design. The same question asked twice can produce different answers, and neither is anchored to a measurable frequency.

Auditability

Inspectable structured features. You can ask why the number moved and get an answer in litigation terms — venue, posture, assignment, timing.

Post-hoc explanations generated the same way the answer was. The explanation is more fluent text, not a trace of a calculation.

Failure mode

Quantified uncertainty. When data is thin the model fails the gate and never ships — the registry holds thousands of models that were never promoted for exactly this reason.

Confident hallucination. Thin data does not lower the confidence of the prose; it lowers the truth of it.

Where language models genuinely help

Drafting. Summarization. Document extraction. First-pass research. Interface — turning a question into a query and a result into prose. These are language tasks, language models are the right tool for them, and the industry building them on legal workflows is doing useful work.

Where the stack meets

The clean division of labor: language models handle the words, calibrated models handle the numbers. A workflow tool that drafts your motion still cannot tell you the probability it works in your venue — that number has to come from a system trained and gated on real outcomes. We are the actuarial layer underneath; workflow AI sits on top of it. Complementary machines, not competitors.

The full technical version of this argument — training-data absence, stochasticity, and why the limitation is architectural rather than temporary — is on the Architectural Limitation page.

The architectural argument →