feat(provider): make complete/stream overridable for in-process providers - #4
Merged
Merged
Conversation
…ders Promotes complete(_:) and stream(_:) to LLMProvider protocol requirements (existing HTTP impls become the defaults), so an on-device provider — MLX, llama.cpp, etc. — can override them and be dispatched correctly through `any LLMProvider`. Previously they were extension-only (static dispatch), so an override was never called via the existential and the HTTP path won. Also adds default (throwing/empty) impls for the HTTP-shaped hooks (prepareRequest/parseStreamLine/parseResponse) so an in-process provider only implements complete/stream + name/configuration. Existing HTTP providers are unchanged. Regression: an in-process EchoLocalProvider overrides complete/stream and wins dispatch through the existential; HTTP hooks throw by default. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Groundwork for on-device inference (Phase 8 / MLX): make
LLMProvider.complete(_:)andstream(_:)overridable.They were defined only in an
extension, so throughany LLMProvider(howAgent/LLMServicecall providers) they were statically dispatched to the HTTP implementation — an in-process provider's override would never run. This promotes them to protocol requirements (the existing HTTP code becomes the default impl), so a provider that runs locally (MLX, llama.cpp, a mock) can overridecomplete/streamdirectly.Also adds default (throwing/empty) implementations for the HTTP-shaped hooks (
prepareRequest,parseStreamLine,parseResponse) so an in-process provider only implementscomplete/stream+name/configuration.Existing HTTP providers (Ollama/OpenAI/Anthropic/Gemini) are unaffected — they implement all three hooks and rely on the default
complete/stream.Tests
New
InProcessProviderTestswith anEchoLocalProvider(no HTTP):completeoverride wins dispatch throughany LLMProvider.streamoverride wins dispatch.All 38 tests pass.
Why
Unblocks a future
MLXProviderthat plugs into the sameAgent/UI with no changes — the on-device stretch for Naseem.🤖 Generated with Claude Code