Programming

There's a useful split lurking in this. For narrow agentic work like retrieval over internal docs, structured classification, test scaffolding, deterministic refactor passes, a self-hosted 30B-class model can be fine and the inference economics work out at team scale. For multi-step planning and the harder agent loops, the frontier gap still shows up in the number of retries and the time-to-correct-answer.

The honest test is to pick the prompt category that's costing you the most and benchmark something like Qwen 2.5 Coder 32B or DeepSeek V3 against whatever you're paying for now. If the gap is small you've found your candidate. If it isn't, you've at least costed the gap accurately rather than guessing at it.

The two costs people underestimate are the GPU box (plus a second one for the eval/staging path) and the maintenance overhead. Model picks go stale fast and someone on the team has to own that, or you end up shipping a Llama 3.1 stack into 2026 because nobody rebuilt the harness for whatever's current.