This scoreboard answers one question: how reliably does a model operate the Cast harness?
It is not a general intelligence ranking. A model is certified once it clears 80%
of the full behavior suite across exactly three fresh attempts per case, where every attempt on a case must
agree (a case that only passes sometimes doesn't count). Core/Chain break
the same score down by single-turn tool contracts vs. multi-turn stateful workflows (see
Behavior Evals).
Tokens are per-attempt averages; the time and turns columns are avg/median/p75/p95/p99 over every
individual attempt. See
Eval Methodology
for the full methodology.
| Model | Reasoning | Score | Passed | Certified | Core | Chain | Time (avg/p50/p75/p95/p99) | Turns (avg/p50/p75/p95/p99) | Input tokens (avg/p50/p75/p95/p99) | Output tokens (avg/p50/p75/p95/p99) | Provider URL | Last updated |
| openai/gpt-5.6-luna | medium | 92.1% | 35/38 | ✓ certified | 92% (11/12) | 92% (24/26) | 11.4s / 8.5s / 11.7s / 30.1s / 77.1s | 3.5 / 3.0 / 4.0 / 10.0 / 13.0 | 28.7k / 19.7k / 29.4k / 78.6k / 150.4k | 478 / 255 / 485 / 1.4k / 4.6k | https://openrouter.ai/api/v1 | 2026-08-09 |
| deepseek/deepseek-v4-flash-0731 | high | 86.8% | 33/38 | ✓ certified | 92% (11/12) | 85% (22/26) | 13.2s / 7.7s / 14.1s / 34.3s / 126.1s | 3.1 / 3.0 / 4.0 / 6.0 / 9.0 | 29.4k / 23.6k / 34.0k / 69.6k / 107.7k | 717 / 350 / 763 / 1.7k / 8.5k | https://openrouter.ai/api/v1 | 2026-08-09 |
| deepseek/deepseek-v4-pro | high | 84.2% | 32/38 | ✓ certified | 100% (12/12) | 77% (20/26) | 17.3s / 9.6s / 15.3s / 34.4s / 240.0s | 3.1 / 3.0 / 4.0 / 7.0 / 10.0 | 29.4k / 23.1k / 31.1k / 67.7k / 116.5k | 873 / 309 / 661 / 1.6k / 16.8k | https://openrouter.ai/api/v1 | 2026-08-09 |
Per-signal breakdown
openai/gpt-5.6-luna — signal breakdown
| Signal | Passed |
| argument-grounding | 9/10 |
| background-lifecycle | 3/3 |
| delegation | 3/4 |
| filesystem-safety | 9/9 |
| mcp-discovery | 2/2 |
| mcp-tool-chain | 1/1 |
| no-unneeded-tools | 11/11 |
| parallel-tools | 1/2 |
| plan-lifecycle | 5/6 |
| plan-safety | 1/1 |
| required-tool | 7/8 |
| skill-discovery | 2/2 |
| state-persistence | 5/5 |
| state-transition | 2/2 |
| tool-chain | 10/10 |
| tool-error-recovery | 5/6 |
| tool-result-integrity | 8/8 |
deepseek/deepseek-v4-flash-0731 — signal breakdown
| Signal | Passed |
| argument-grounding | 9/10 |
| background-lifecycle | 3/3 |
| delegation | 3/4 |
| filesystem-safety | 6/9 |
| mcp-discovery | 2/2 |
| mcp-tool-chain | 1/1 |
| no-unneeded-tools | 10/11 |
| parallel-tools | 2/2 |
| plan-lifecycle | 5/6 |
| plan-safety | 1/1 |
| required-tool | 7/8 |
| skill-discovery | 2/2 |
| state-persistence | 4/5 |
| state-transition | 1/2 |
| tool-chain | 7/10 |
| tool-error-recovery | 5/6 |
| tool-result-integrity | 8/8 |
deepseek/deepseek-v4-pro — signal breakdown
| Signal | Passed |
| argument-grounding | 10/10 |
| background-lifecycle | 2/3 |
| delegation | 4/4 |
| filesystem-safety | 6/9 |
| mcp-discovery | 2/2 |
| mcp-tool-chain | 1/1 |
| no-unneeded-tools | 11/11 |
| parallel-tools | 2/2 |
| plan-lifecycle | 3/6 |
| plan-safety | 1/1 |
| required-tool | 8/8 |
| skill-discovery | 2/2 |
| state-persistence | 3/5 |
| state-transition | 2/2 |
| tool-chain | 7/10 |
| tool-error-recovery | 4/6 |
| tool-result-integrity | 7/8 |