Model validation
What the stored forecasts actually did against the next bars: directional accuracy, absolute error, interval coverage, and the calibration spread across the probability bins.
Read the sample size before the percentages. A 70 % hit rate on nine settled forecasts is not a result; it is nine forecasts.
Every number below is the ratio over rows that exist in this database right now — no benchmark is imported and no figure is carried over from a model card.
DEMO DATASET last stored bar 2026-10-23 · scores computed 2026-10-23 17:35:24 · every figure below is read from these rows, nothing is computed in the browser
Settled forecasts0.00rows joined in prediction_results
Directional accuracynot storedshare where direction_correct = 1
Mean absolute errornot storedversus the stored p50 path
Base-case coveragenot stored0 inside the prediction interval
Outcome mix
Each settled forecast is bucketed by where the bar closed relative to the bands.
Nothing settled yet.
Buckets appear once php bin/pipeline.php --cmd=settle has matched expiring forecasts to the bars that followed them. Until then this page is honestly empty — it will not show you a placeholder score.
What each scoreboard row means
directional accuracy
Share of forecasts whose predicted sign matched the sign of the realized move. The direction, not the size.
mean absolute error
Average distance between the stored p50 path and the realized return, in percentage points. Penalises a right call at the wrong size.
coverage
How often the realized return landed inside the announced p10–p90 band. A model that is always right but always narrow fails this test.
base case vs. prediction interval
Two nested questions: did the close fall in the base case, and did it fall anywhere in the announced range at all.
calibration by probability bin
Group forecasts by the probability the model attached, then compare that with the observed hit rate in the bin.
Read alongside the model version and the sample size. A version with a better score on fewer rows is not a better model.
Validation runs
Table: model_validation_runs, last 30, each one frozen against a window of settled samples.
0 run(s)
No validation runs recorded.
Run php bin/pipeline.php --cmd=settle to score expired forecasts against the bars that followed them; the scoreboard above reads the same rows, which is why it is empty too.
Settled forecasts
Rows from prediction_results joined to the forecast each one scores.
0 row(s)
No settled predictions.
A forecast settles when the horizon ends and the bar after it exists; then the actual return is written back here.
Model registry
Table: ai_models with the counts this install actually holds for each one.
| Model | Key | Version | Family | Status | Trained | Features | Scores | Forecasts | Released | Notes |
|---|---|---|---|---|---|---|---|---|---|---|
| QuantMind Scenario Forecaster | qm-forecast | fore-1.0.0 | parametric_normal | ACTIVE | rolling 3y realised vol + score tilt | 7 | 0 | 5,850 | 2026-10-23 13:46 | Ranges, not point forecasts. Probabilities are model estimates, never… |
| QuantMind Composite Scoring (rule engine) | qm-scoring | score-1.0.0 | rule_engine | ACTIVE | n/a — transparent rules, no fitted weights | 41 | 975 | 0 | 2026-10-23 13:46 | Deterministic and explainable by design: every score decomposes into … |
Scores and forecasts are counts of rows in this database for that model id — they say nothing about quality, which is what the tables above are for.