Skip to content

feat: support llm_config BYOK on jobs and retrieval#33

Merged
suguanYang merged 5 commits into
mainfrom
feat/suguanYang/byok-llm-credentials
Jul 15, 2026
Merged

feat: support llm_config BYOK on jobs and retrieval#33
suguanYang merged 5 commits into
mainfrom
feat/suguanYang/byok-llm-credentials

Conversation

@suguanYang

@suguanYang suguanYang commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add LLMConfig / LLMProviderConfig / LLMModelsConfig and thread llm_config through jobs / parse / retrieval.
  • Flat root {api_key, model, base_url} for multimodal; models for same-endpoint different model ids; text / vision for different endpoints.

Depends on: Ontos-AI/knowhere#217

Usage

# One multimodal model
llm_config = {
    "api_key": "sk-...",
    "model": "gpt-4o",
    "base_url": "https://api.openai.com/v1",
}

# Same endpoint, different models
llm_config = {
    "api_key": "sk-...",
    "base_url": "https://api.openai.com/v1",
    "models": {"text": "gpt-4o-mini", "vision": "gpt-4o"},
}

# Two different endpoints
llm_config = {
    "text": {"api_key": "sk-...", "model": "gpt-4o-mini", "base_url": "https://api.openai.com/v1"},
    "vision": {"api_key": "sk-ali-...", "model": "qwen-vl-max", "base_url": "https://dashscope.aliyuncs.com/compatible-mode/v1"},
}

client.parse(url="https://example.com/doc.pdf", llm_config=llm_config)
client.retrieval.query(query="...", use_agentic=True, llm_config=llm_config)

Test plan

  • pytest tests/test_jobs.py tests/test_retrieval.py tests/test_parse.py

Expose optional text/vision OpenAI-compatible credentials on job create,
parse, and retrieval.query so callers can drive KNOWHERE with their own keys.

Co-authored-by: Cursor <cursoragent@cursor.com>
suguanYang and others added 4 commits July 15, 2026 21:39
Document that missing text/vision slots keep server defaults instead of
acting as a unified multimodal fallback.

Co-authored-by: Cursor <cursoragent@cursor.com>
Document and type the shared provider slot for single multimodal BYOK.

Co-authored-by: Cursor <cursoragent@cursor.com>
Document flat multimodal shorthand and split text/vision endpoints.

Co-authored-by: Cursor <cursoragent@cursor.com>
Support shared auth with different text/vision model ids.

Co-authored-by: Cursor <cursoragent@cursor.com>
@suguanYang
suguanYang merged commit 51adb5a into main Jul 15, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant