Skip to content

feat(models): route normal to DeepSeek V4 Flash, classifier to gpt-oss-120b - #272

Merged
winlp4ever merged 3 commits into
mainfrom
feat/model-routing-deepseek-base
Sep 4, 2026
Merged

feat(models): route normal to DeepSeek V4 Flash, classifier to gpt-oss-120b#272
winlp4ever merged 3 commits into
mainfrom
feat/model-routing-deepseek-base

Conversation

@winlp4ever

Copy link
Copy Markdown
Contributor

What

Alternative to the GLM base in #268, which regressed in testing: GLM 5.3 Flash via OpenRouter did not emit real tool_calls (the agent hallucinated the whole tool loop as text, so nothing hit the board) and was too slow for the 2s classifier timeout (it took ~2030ms, so the classifier always fell back to medium and complex routing died).

DeepSeek V4 Flash tests clean for tool calling. This routes:

  • normal (lite) -> DeepSeek V4 Flash (deepseek/deepseek-v4-flash:nitro, OpenRouter) when an OpenRouter key is present; falls through to gpt-5.4-mini with no key.
  • complex (pro) -> gpt-5.5 (unchanged).
  • classifier -> gpt-oss-120b (openai/gpt-oss-120b): open-weight, native structured output, ~$0.03/$0.17 per M tokens. Pinned by id so it stays independent of the base model.

How

  • models.yml: deepseek-v4-flash becomes the first lite entry (auto base); add gpt-oss-120b (lite, no :nitro since the openai/ route is single-provider). gpt-5.5 stays the first entry overall, so the no-tier fallback and complex default are unchanged (the no-tier-default lesson from feat(models): route normal to GLM 5.3 Flash, complex to gpt-5.4 #268's review).
  • catalog.resolved_by_id(id): resolve a specific model by canonical id.
  • auto_model.py: pin the classifier to gpt-oss-120b, falling back to the lite default then any model. Bump AUTO_MODEL_TIMEOUT_SECONDS 2 -> 5 for OpenRouter first-token latency headroom.

BYOK fallback

DeepSeek and gpt-oss are OpenRouter-only, so with no OpenRouter key the base falls to gpt-5.4-mini and the classifier to the lite default, both native OpenAI. No one is stranded.

Test plan

  • ruff check topix test/unit: clean
  • pytest test/unit: 706 passed (added resolved_by_id tests)

Notes

…s-120b

Alternative to the GLM base (#268), which regressed: GLM 5.3 Flash via OpenRouter
did not emit real tool_calls and was too slow for the 2s classifier timeout.

- Make deepseek-v4-flash the first lite entry so normal complexity resolves to it
  on OpenRouter (falls through to gpt-5.4-mini with no OpenRouter key). It tests
  clean for tool calling.
- Pin the complexity classifier to gpt-oss-120b (cheap, fast, native structured
  output) via a new catalog.resolved_by_id, decoupled from the base model. Bump
  AUTO_MODEL_TIMEOUT_SECONDS 2 -> 5 for OpenRouter first-token latency headroom.
- gpt-5.5 stays first overall, so the no-tier fallback and the complex/pro default
  are unchanged (5.5 > 5.4).
gpt-oss is OpenAI-branded but open-weight and multi-provider on OpenRouter, so
:nitro routes the classifier to the fastest provider (it isn't single-provider
like proprietary openai/* models). Refine the nitro-rule tests to treat gpt-oss
as multi-provider so it requires :nitro rather than forbidding it.
@winlp4ever
winlp4ever merged commit e00198b into main Sep 4, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant