Accuracy: how forecasts are checked against outcomes
Accuracy is two numbers under Past on Passes: how often what a pass delivered landed inside its range, and how far it typically missed what was expected. Both come from passes that have a forecast and a logged result.
In range and the typical miss
| On screen | What it is | How to read it |
|---|---|---|
| In range | The share of compared passes whose delivered GB fell inside their range (p10 to p90: the low and high ends, with 8 in 10 inside) | Aim for 80%. A range holds 8 in 10, so well above 80% means the ranges run wide, and well below means they run narrow. A pass on p10 or p90 counts as inside. A range with no width, p10 equal to p90, holds only a pass that delivered exactly that amount, so it almost always counts as a miss. |
| Below its range, above its range | How many passes landed on each side | A lean to one side is a biased forecast, even when the in-range share looks right. |
| Typical miss | The average distance between delivered GB and expected GB (p50, the middle forecast), in GB | Smaller is better. Read it against the size of your passes. |
Past groups both by model or by ground station, so a ground station that delivers less than forecast, or a model that runs wide, stands out. A pass with nothing logged, or with no forecast, is left out and not counted as a miss.
Where the outcomes come from
The audit-ledger template measures them. For each pass over a past window it prices what the pass could have delivered with the same chain a forecast uses, and sets that against the contacts you logged:
| Output | Meaning |
|---|---|
possible_gb | What the pass could have delivered, counted at full lock where your telemetry shows the link lost lock |
delivered_gb | What your log says it delivered |
status | The pass's outcome: delivered, failed, missed or unlogged. A pass with nothing logged whose telemetry never showed lock is missed, and loses all its possible GB |
totals | Sums per ground station and overall. An antenna tracks one pass at a time, so passes that overlap on it count its best schedule once: possible GB totals can be less than the sum of the passes |
unmatched_logs | Logged contacts that matched no computed pass |
The logged contacts come from the dataset the run names: a pass log you uploaded (audit), or your live telemetry. See Audit passes against forecasts and Data modes.
What the API reports today
The API does not report accuracy per model or per ground station yet. Passes works it out from the audit's items: the in-range share from each pass's range and delivered GB, the typical miss from its expected and delivered GB. The same numbers can be computed from GET /predictions?run_id= on an audit run.
How each model is trained or calibrated, and what it reads from your telemetry, is on its model card. See Models and their tiers.
Related
- Reading a result: what expected, the range and the low case mean
- Free audit: a first report from a week of pass logs