Skip to content

Tech debt: centralize OpenAI-compatible endpoint URL building + unify model-discovery error handling #30

Description

@soyasis

Follow-up from #28.

Two related bits of duplication around the OpenAI-compatible endpoint:

  1. LLMService.availableModels() returns [] silently on a non-200 / missing
    data, while checkStatus() (post-Custom OpenAI-compatible provider does not detect models behind HTTP + Nginx + API Key #28) fails loudly. No user impact today —
    checkStatus surfaces the error next to the dropdown on the same triggers —
    but the two parse the same /v1/models response with divergent contracts
    that have to be kept in sync by hand.
  2. \(baseURL)/v1/... is hand-built at 4 sites (checkStatus, availableModels,
    generateWithMessages, streamWithCloudProvider), each repeating the
    !isEmpty && hasPrefix("http") && URL(string:) guard. Same class as Cloud provider (OpenRouter/LM Studio/Ollama) responses truncated — hardcoded max_tokens=1024, config ignored #26/Custom OpenAI-compatible provider does not detect models behind HTTP + Nginx + API Key #28.

Proposed: one CaiSettings.endpointURL(path:) (or LLMService helper) that
normalizes + joins + guards once; route all 4 sites through it; share the
response-parse/error path between availableModels and checkStatus.

Severity: low / internal. No user-facing symptom.

Metadata

Metadata

Assignees

No one assigned

    Labels

    🔁 refactorCode restructuring with no user-visible change

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions