Skip to content

Llm proxy - #54

Open
iidsample wants to merge 2 commits into
mainfrom
llm_proxy
Open

Llm proxy#54
iidsample wants to merge 2 commits into
mainfrom
llm_proxy

Conversation

@iidsample

Copy link
Copy Markdown
Member

Added LLM Proxy for replacing OpenAI, Anthropic and Bedrock calls.

@coderabbitai

coderabbitai Bot commented Aug 26, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 47ff5cc7-b14b-4907-bb27-100d855efd6b


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Comment thread llm_proxy/app.py
return proxy_request(prov, subpath, request)
except Exception as exc: # surface upstream/adapter errors as 502
log.exception("proxy error for %s/%s", provider, subpath)
return jsonify(error="proxy_error", detail=str(exc)), 502
Comment thread llm_proxy/README.md
- **OpenAI / Anthropic** — straight HTTP reverse-proxy: rewrite host, swap in the
real key, forward with `requests`, return the response.
- **Bedrock** — re-issued through the proxy's own `boto3` client (handles SigV4
signing + URL-encoding correctly). Only `invoke` is wired up.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No .converse functionality? Would we be changing all the current calls to use .invoke?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me see how to add converse functionality

@nickhuo

nickhuo commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

i'm thinking about how to adapt the llm proxy to the agent skill, some questions:

  • skill adaptation happens before the Ventis build, but it seems the base URL can only be obtained after deployment, so the agent cannot fill in the base URL for the user, while the controller set it up. Is my understanding correct?
  • if it's right, the agent skill can identify if this project can use this version of llm proxy (bedrock, openai etc)

@nickhuo

nickhuo commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

i added requests to the dependency of ventis to coverage the llm proxy's

@Saaketh0

Saaketh0 commented Sep 5, 2026

Copy link
Copy Markdown
Collaborator

#73 Has this code integrated, may be better to review the code over there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants