Results · limited preview · October 2026
A benchmark for real-world constraint-based coding.
General AI model reasoning across embedded systems, sensor integrations, BSPs, and more areas targeted by real-world engineering constraints.
You’re on the list.
We’ll write to when the full results are published.
Where models fail
GPT-6.1 Sol100%85 failed
Claude Opus 5.5Opus 5.588%12%98 failed
Claude Sonnet 5.5Sonnet 5.599%131 failed
Muse Spark 1.370%30%170 failed
Kimi K387%222 failed
Qwen3.8 Max25%57%18%229 failed
GLM 5.326%29%41%232 failed
Nemotron 3 Ultra75%23%232 failed
Pass rate by area
| Model | Embedded systems19 tasks | Sensor configuration faults6 tasks | Driver and peripheral bring‑up3 tasks | Control logic and PCB2 tasks |
|---|---|---|---|---|
| GPT-6.1 Sol | 76 | 33 | 54 | 62 |
| Claude Opus 5.5Opus 5.5 | 58 | 44 | 75 | 94 |
| Claude Sonnet 5.5Sonnet 5.5 | 43 | 35 | 67 | 62 |
| Muse Spark 1.3 | 27 | 33 | 38 | 25 |
| Kimi K3 | 3 | 17 | 4 | 19 |
| Qwen3.8 Max | 1 | 8 | 0 | 25 |
| GLM 5.3 | 1 | 8 | 0 | 19 |
| Nemotron 3 Ultra | 0 | 8 | 0 | 0 |