fix(routing): resolve model-keyed lookups the way the runtime resolves them

[lidge-jun]

Summary

Two evidence surfaces read per-model config with a bare map lookup while the runtime resolves the same keys through modelRecordValue. Family and case overrides were invisible to them, and a prototype-shaped id resolved an Object.prototype member instead of missing.

Credit: both patches are @ntdatt812's work, from #2100 and #2077, applied unchanged. They are combined here because they are one thesis — model-keyed lookups must use the runtime's own resolution rules — on two disjoint files.

• src/routing/capability.ts (fix(routing): resolve capability evidence the way the resolver resolves it #2100): gpt-oss:120b inherited the provider-wide 8k window instead of the gpt-oss family's 131072, and noVisionModels was ignored. These values select routing candidates, not just log lines.
• src/routing/compatibility/behavior.ts (fix(lab): read per-model overrides in the report the way the runtime reads them #2077): the same lookup bug in the Lab behavior report, where constructor resolved to Object.prototype.constructor, made jcsStringify throw and silently dropped Lab subjects.

Exact-own maps (modelPreferHostedTools, modelOpenRouterRouting) deliberately do not family-spread, and that boundary is pinned by its own tests.

Verification

• Both patches applied cleanly to current dev and are proven RED: reverting only the two source files fails 13 of the new tests.
• bun test --isolate tests — 13,556 pass, 0 fail, 10 skip (856 files).
• tests/core-lab-boundary.test.ts green: behavior.ts already imported Lab types, and this does not newly puncture the core/Lab boundary.
• bun run typecheck — clean.
• bun run privacy:scan — passed.

Supersedes

Closes #2100 and #2077 once merged. Both are @ntdatt812's; I will close them with attribution rather than leaving duplicates open.

Checklist

• Scope stays focused and avoids unrelated cleanup.
• Docs or release notes were updated when needed. (Fixes lookups to match documented runtime behavior; nothing user-facing changes shape.)
• Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults. (No auth/credential surface. The prototype-id hardening removes an Object.prototype leak, which is a small safety improvement.)

Closes #2100
Closes #2077

Summary by CodeRabbit

• Bug Fixes
  • Improved model capability detection for tagged and case-variant model IDs.
  • Corrected context-window and capability fallback behavior for model families and provider defaults.
  • Improved vision detection so explicitly text-only models are classified correctly.
  • Prevented ambiguous model names from matching unrelated compatibility settings.
  • Preserved exact model-specific settings over broader family-level defaults.
• Tests
  • Added coverage for model-family matching, fallback behavior, vision handling, and routing compatibility.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

[github-actions]

✅ Deterministic PR hygiene checks passed.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: a537655c-b72b-4c26-9058-5011912fa14f

📥 Commits

Reviewing files that changed from the base of the PR and between 8d828bd and f9c224b.

📒 Files selected for processing (4)

• src/routing/capability.ts
• src/routing/compatibility/behavior.ts
• tests/routing-capability-model-matching.test.ts
• tests/routing-compatibility-model-matching.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 3 remain after this review.

---

📝 Walkthrough

Walkthrough

Routing capability evidence and compatibility behavior now resolve model overrides through safe, family-aware matching. Vision exclusions take precedence over modality metadata. New tests cover family, exact, case-insensitive, fallback, and prototype-shaped model IDs.

Changes

Model-aware routing resolution

Layer / File(s)	Summary
Capability evidence resolution src/routing/capability.ts, tests/routing-capability-model-matching.test.ts	Capability evidence now uses modelRecordValue for context windows, modalities, and reasoning efforts. noVisionModels forces text-only evidence. Tests cover family matching, exact precedence, registry fallback, provider fallback, and prototype-shaped IDs.
Compatibility override resolution src/routing/compatibility/behavior.ts, tests/routing-compatibility-model-matching.test.ts	Behavior reports use family-aware, case-folded lookups. OpenRouter routing and hosted-tool preferences use safe exact own-property lookups. Tests verify runtime agreement and fingerprint stability.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to f9c22

This focused change aligns model-keyed lookups with runtime resolution rules, with the reported tests and checks passing; no actionable merge-blocking risk remains.

Possibly related PRs

• lidge-jun/opencodex#2100: Covers the same capability-resolution changes and tests.
• lidge-jun/opencodex#2077: Covers the related compatibility behavior lookup changes.
• lidge-jun/opencodex#2124: Uses the same model-keyed provider configuration matching approach.

Suggested labels: review-ready

Suggested reviewers: ingwannu

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: aligning routing model-keyed lookups with runtime resolution.
Linked Issues check	✅ Passed	The implementation addresses both issues: family-aware lookups, exact-own boundaries, prototype safety, capability evidence, behavior reports, and targeted tests.
Out of Scope Changes check	✅ Passed	All source and test changes directly support the linked routing and Lab model-resolution objectives; no unrelated changes are evident.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/absorb-capability-evidence

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[lidge-jun]

리뷰 · 우선순위 54 / 80

모델 키 lookup이 modelRecordValue를 안 타고 맵을 그냥 보던 거임. 패밀리/대소문자 오버라이드가 안 보이고, constructor 같은 프로토타입 id가 Object.prototype.constructor로 붙어서 Lab jcsStringify가 터짐. capability.ts는 라우팅 후보를 고름. gpt-oss:120b가 프로바이더 8k를 먹고 gpt-oss 131072를 무시함. noVisionModels도 무시. 로그가 아니라 후보 선택임.

#2100이랑 #2077을 한 테제로 합친 거임. 둘 다 ntdatt812 작업. exact-own 맵(modelPreferHostedTools, modelOpenRouterRouting)은 패밀리 스프레드 안 함. 그 경계를 테스트로 고정함. 소스 두 파일만 되돌리면 새 테스트 13개가 빨갬. Lab 바운더리는 원래 behavior.ts가 Lab 타입을 임포트함. 새로 뚫은 거 아님.

점수는 54임. 맞는데 지금 dev 불은 아님. #2137 bearer 401, #2138 gpt-5.6 400이 먼저임. types.ts/config.ts 안 건드림. 닫고 리베이스하라는 쪽 아님. #2100 #2077은 이 PR 머지되면 닫으면 됨. 중복으로 열어둘 이유 없음.

해결방안: 머지 후 원 PR 두 개 attribution 달고 닫기. 2.27 필수 아님. #2137 #2138 뒤에 넣으면 됨.

이 댓글은 grok-bot이 작성했습니다

[ntdatt812]

Thanks for consolidating these, and for the write-up — the single-thesis framing is better than my two PRs were. "Model-keyed lookups must use the runtime's own resolution rules" is the right way to state it, and folding both files under one resolverVersion decision is cleaner than reviewing them apart. No objection to closing #2100 and #2077.

One mechanical thing, and only because the PR is still open so it costs you nothing now:

f9c224b7 is authored by bitkyc08-arch <bitkyc08@gmail.com> with the credit in prose, and there is no Co-authored-by: trailer. Prose credit reads fine to a human, but git log, git blame, the contributor graph and /repos/.../contributors all key off the author field and that trailer — so as it stands the two patches will show as entirely yours in the history.

Your earlier merges did carry it through, which is how I noticed the difference: #2085 and #2086 landed as merge commits, so my commits survived intact and the API shows contributions=8. A squash drops that unless the trailer is present.

If you're willing, one line at the end of the commit body is all it takes:

Co-authored-by: Nguyen Thanh Dat <ntdat812@gmail.com>

That is the identity GitHub already links to my account, so it will resolve.

Entirely your call, and I would rather have the patches land as you have them than have them back. Just flagging it while it is still a one-line amend rather than after the merge.

[Ingwannu]

I cannot approve the current head because it is 18 commits behind the current dev tip (caf20353f). The existing diff and CI results do not validate the change against the current integration state.

Please rebase onto the latest dev, resolve conflicts in the affected runtime paths, rerun the focused checks plus the full exact-head suite, and re-request review. This is a review-state blocker, not a rejection of the fix direction.

[ntdatt812]

@Ingwannu's rebase blocker is fair, and since the diff here is my patch from #2100/#2077 I ran the validation against the current tip so the rebase does not have to be done blind. I cannot push to this branch, so this is data rather than a fix.

The patch cherry-picks onto caf20353f with no conflicts.

git checkout --detach origin/dev        # caf20353f
git cherry-pick f9c224b70               # clean, 4 files changed, 345 insertions(+), 12 deletions(-)

Nothing in the 18 intervening commits touches src/routing/capability.ts or src/routing/compatibility/behavior.ts in a way that collides.

Results on that exact rebased head:

bun run typecheck                                             exit 0

bun test routing-capability-model-matching
         routing-compatibility-model-matching                 29 pass / 0 fail

bun test routing-capability-catalog · route-explainability
         service-tier-capability · fastwire-characterization-routing
         catalog-vision-sidecar-modalities · routing-policy-fallback
         routing-compatibility · openrouter-provider-routing
         fastwire-observability · lab-live-review-regressions  189 pass / 0 fail

On the full suite, and I would rather state this plainly than round it up: my bun run test run on this rebased head was interrupted before it printed a total, and at the point it stopped it had 7 failures — 5 in tests/codex-auth-api.test.ts, 2 in tests/codex-auth-modal-status.test.ts. Every one of them took ~5000 ms, which is the timeout signature rather than an assertion failure.

Run in isolation on the same tree, those two files are 192 pass / 0 fail. They exercise OAuth pool login and GUI status copy; this patch touches two routing files and neither is in their import graph. So they are load-dependent on my Windows box, not consequences of the change — but I have not produced a clean full-suite total on the rebased head, and that is the one thing @Ingwannu asked for that I cannot hand you.

Worth knowing when you read a full-suite result from this repo on Windows: I have run it four times today on four different heads and it reddened a different subset each time, always tests with multi-second timeouts. Separately, tests/server-auth.test.ts > OpenAI option auth matrix keeps direct, pool, and API credentials independent fails reproducibly on a clean dev tree — 2/2 with the working tree checked out wholesale to origin/dev — so a full-suite run there is not expected to be green even before this patch.

No action needed from me on the credit question in my earlier comment; it can wait until after the rebase.

[Ingwannu]

Thanks @ntdatt812 — the clean cherry-pick and focused results are useful evidence that the patch still applies. They do not update the PR head, though: f9c224b70 remains 18 commits behind current dev (caf20353f), so the repository CI shown here still validates the old ancestry. The commit also still lacks the requested Co-authored-by: Nguyen Thanh Dat <ntdat812@gmail.com> trailer.

My existing changes-requested review therefore remains in place. @lidge-jun, please rebase/amend this branch, preserve the contributor trailer, and let exact-head CI run. The isolated Windows timeout reruns support treating those local failures as load-related, but they are not a substitute for updating the actual PR head.