diff --git a/AUDIT-cybernetic-remediation.md b/AUDIT-cybernetic-remediation.md new file mode 100644 index 0000000..8b2bd72 --- /dev/null +++ b/AUDIT-cybernetic-remediation.md @@ -0,0 +1,143 @@ +# 系统工程审核 + 控制论修复方案 + +> 产出方式:5 路并行 Sonnet 子 agent 审计(架构/数据链路/安全/前端契约/测试运维), +> P0 级 + 锚定发现由主模型逐条读码复核。日期 2026-07-02。 +> 分支 `refactor/thin-channel-thick-runner`。 +> +> 标注:✅ = 已读码亲验 / ▫ = agent 报告,未独立复核。 + +## 部署信任模型(已定 2026-07-02) + +**局域网通信走 NetBird / WireGuard mesh,不做直接公网端口映射。** + +- **入站边界 = WG 层**:只有 fleet overlay 上已认证的 peer 能触达 API/MCP/odp-ingest。 + 应用层"无鉴权"由此获得真实网络边界兜底 → 相关发现(轮子 7)从 P1 降为 **P2 纵深防御备案**。 +- **出站威胁不受 WG 保护**:SSRF 类威胁是"服务器被恶意内容(爬到的网页/RSS/被改的 provider + `base_url`)骗着主动发出请求"。WG 挡入站不挡出站——出站可打 fleet overlay(100.80.x.x)、 + 本机 localhost 服务(redis 6379 等),或把真实 LLM key 外泄到公网。**轮子 4 仍必修(P1)。** +- **LLM agent 是可信 mesh 内的不可信因子**:MCP 把 `create_source`+`trigger_task` 暴露给 agent, + 而 agent 处理不可信内容(爬网页/RSS),prompt injection 可触发 cli 渠道任意二进制 → + **轮子 4 的 binary 白名单仍值做(P1 硬化),不因可信 mesh 而免**。 + +--- + +## 一、控制论诊断:这不是一堆孤立 bug,是四类回路病 + +维纳四原则拿来当透镜,本次绝大多数发现归到前两类——**反馈回路断裂**和**信息丢失**。 +这解释了为什么它们分散在 Python/Rust/React 三层却"长得一样"(同构性),也决定了修法: +不是逐点打补丁,是**装轮子**——一处机制,全层复用,让系统自己纠错(自组织)。 + +| 控制论原则 | 系统里对应的东西 | 违反它的发现 | +|---|---|---| +| **① 反馈是生命线**(感知差异→调整→再执行) | 重试分类、DLQ、错误上浮、CI 门禁 | #1 重试不触发、#3 DLQ 吞消息、#8 主循环 panic、shadow 错误没人读、前端缺 onError、CI 不测关键路径 | +| **② 信息即控制力**(信息丢=控制力衰减) | accepted 计数真实性、可观测性、凭证保密 | #2 摄入黑洞(假 accepted)、#4 毒消息与 trim 混同、明文 key 外泄、无 healthcheck、shadow 计数只进日志 | +| **③ 同构性**(同一模型跨领域复用) | 错误处理模式在三层重复出现 | 缺 onError(React)≡ 重试漏分类(Celery)≡ DLQ 丢信号(Rust)——**同一个病,该用同一个轮子治** | +| **④ 目的性/自组织**(局部规则+全局反馈→有序) | 校验器、契约、不变量在调用点就地强制 | SSRF 无统一校验器、env 开关散落、鉴权靠"部署层但愿"、游标非原子推进 | + +--- + +## 二、发现台账(记录) + +### P0 — 会真丢数/坏/被攻破 + +| # | 位置 | 问题 | 原则 | 验 | +|---|---|---|---|---| +| P0-1 | `backend/pipeline/error_taxonomy.py` + ODP sink 路径 | `httpx.HTTPStatusError` 不在 `_RETRYABLE`,ODP 回 5xx→判永久→`pipeline.py` 吞成失败结果正常返回→Celery `autoretry_for` 不触发。配 `odp_only` 策略=无处落、无自动重试。**机制现成**:有 `is_retryable_http_status()`+`RetryableHTTPStatus` sentinel,只是 ODP 路径没用 | ① | ✅ | +| P0-2 | `odp-rs/.../odp-ingest/src/state.rs:24` + `handlers.rs:99-101` | 漏配 Redis→`bus=None`→`else { accepted += 1 }`,返回 202 但没入流。纯黑洞,运行期零信号。(注:NDJSON 入口对解析失败是 400,黑洞只在 batch+无 bus) | ② | ✅ | +| P0-3 | `odp-rs/.../odp-store/src/reap.rs:56-62`(**本仓 `ef4828d` 引入**) | 毒消息(JSON 反序列化失败)走 DLQ 路径时 `read_entries_by_id` 静默丢弃→不进 `found_ids`→被误判"已 trim"直接 XACK,不写 DLQ 行=静默永久丢,且与"真被 trim"安全分支无法区分 | ①② | ✅ | +| P0-4 | `backend/channels/cli_channel.py:40,46` | `binary`+`command` 全来自 channel_config,`create_subprocess_exec(*full_cmd)`=任意二进制执行。配无鉴权 API/MCP=RCE。exec-form 无 shell 注入 | ④ | ✅ | + +### P1 — 严重债/潜伏 bug + +| # | 位置 | 问题 | 原则 | 验 | +|---|---|---|---|---| +| P1-1 | `backend/pipeline/storer.py:23,38` | 默认 `forward_to_odp=True`,裸 `ODP_INGEST_URL` 触发,绕开 `write_strategy` 状态机→legacy 源意外泄漏进 ODP + 双入口迷惑。(idempotency_key 去重挡掉字面重复,故非重复灌数)。`HANDOFF-strangler-fig.md` 标了"PR3 必收口" | ④ | ✅ | +| P1-2 | `backend/schemas/provider.py:34` + `api/v1/providers.py:15` | `GET /providers` 明文返回所有 LLM api_key;`ModelProvider.api_key` 明文存库,没走 `SourceCredential` 的 Fernet | ② | ✅ | +| P1-3 | `backend/main.py`(全局) | 全 API 无 AuthN/AuthZ(`api_key_enabled` 定义了没 `Depends` 校验);odp-ingest 无 auth 且默认 bind `0.0.0.0`;CORS debug 下 `["*"]`+`allow_credentials=True` | ④ | ✅ | +| P1-4 | 见下方 SSRF 枚举 | 响应回显型 SSRF(rss/web_scraper/api/crawl4ai 抓用户 URL→`GET /records` 读回=内网读原语);无 scheme 白名单/私网 IP 拦截/元数据防护;多处 `follow_redirects=True`;provider `base_url` 可改指外部主机→连真实 Bearer key 一起外泄 | ②④ | ▫ | +| P1-5 | `odp-rs/.../odp-store/src/main.rs:54,59,60` | 主循环裸 `?`,Redis/PG 瞬断→整进程 panic 退出(reap 有 log-continue,主循环没有) | ① | ✅ | +| P1-6 | `backend/pipeline/cursor_store.py` + `runner.py` | SELECT-then-write 无原子 UPSERT/行锁;只有按域名进程内信号量,非 per-source→并发触发同源丢更新(漏抓中间段) | ①④ | ▫ | +| P1-7 | `backend/pipeline/sinks/dual_sink.py:57` + `pipeline.py` | shadow 写失败塞进 `SinkResult.errors` 但 `pipeline.py` 从不读→影子模式 ODP 长期挂,任务仍显 completed,唯一线索是 worker 日志一行 warning | ② | ▫ | +| P1-8 | `.github/workflows/ci.yml` | 只 `pytest -m "not live" --no-cov`——无 `alembic upgrade head` 冒烟、无 `cargo test`(odp-rs 连已有单测都不跑)、coverage gate 关掉。迁移链坏能一路绿灯合并 | ① | ▫ | +| P1-9 | `tests/unit/pipeline/test_pipeline_errors.py:34,58,153` | 调用签名跟真实 `run_pipeline(task_id, source, parameters=None,...)` 对不上:传 `run_pipeline(db_session, source, task.id)`,`db_session` 当了 task_id(仅进日志 `%s` 侥幸不炸),`task.id` 当了 parameters(UUID 非 dict)。rss 源无 `session_affinity` capability 跳过 `params.get` 分支→侥幸通过=测了假东西 | ① | ✅ | +| P1-10 | `frontend/src/pages/BrowsersPage.tsx:778,528` | 重启 API 按钮 + 实例 agent_url 保存两个 mutation 缺 onError,失败全静默 | ① | ▫ | + +### P2 — 硬化项 + +- ▫ odp-ingest `dedup.rs` 无上限/TTL→投毒(自选幂等键让合法事件被当 duplicate 丢)+ 内存 DoS;`handlers.rs:71` 整批持单写锁 +- ▫ `handlers.rs:30` NDJSON 无显式请求体大小上限(仅靠 axum 默认 2MB) +- ▫ `odp-store/main.rs:46` 无优雅关闭(裸 `loop`,无 SIGTERM 处理)→部署强杀撞未 flush 批次 +- ▫ `docker-compose.yml` odp-ingest/odp-store 无 healthcheck;下游 `depends_on: service_started` 非 `service_healthy`→起容器时序竞争 +- ▫ `.env.example` 与实际 env 读取脱节:`ODP_INGEST_REQUIRED`(fail-open/closed 语义关键开关)、`ODP_INGEST_TIMEOUT`、odp-rs 侧 `ODP_DATABASE_URL/ODP_STORE_BATCH_SIZE/ODP_BUS_*` 全没进主文档 +- ▫ `frontend/src/pages/SourcesPage.tsx:301` 6 种完工渠道被硬编码标"(开发中)"=误导文案 +- ▫ `frontend/src/pages/SkillsPage.tsx` RecordWizard 修了点取消泄漏,但没堵路由跳转/后退卸载(缺 unmount cleanup)→录制中离开页面照旧泄漏 pool mutex +- ▫ `odp-store/src/reap.rs:38` `p.delivery_count > MAX_DELIVERIES`(=5)实为 6 次才进 DLQ,与注释"5"字面不符(应 `>=`) +- ▫ `external_http_processor.py:78` `os.path.expandvars(auth_header)`→配置者设 `$OPENAI_API_KEY` 可把宿主 env 注入出站 header +- ▫ `config.py:19,91` 弱默认密钥(`change-me-*`);本机 `.env` 已真实覆盖且已 gitignore,无提交泄密 +- ▫ `frontend` `connectivity_ok/connectivity_errors` 后端有、前端类型直接丢弃 + +### SSRF 完整出站枚举(20 点,详见安全 agent 原始报告) + +需统一校验器覆盖的用户/DB 供给 URL 出站点: +- 回显型(→records 可读回内网):rss_channel(2)、web_scraper_channel、api_channel、crawl4ai_channel、external_http_processor +- 探测 oracle:source_service `discover-feed`(2) +- 盲打:webhook/feishu/wecom notifier(3) +- **凭证外泄型**(base_url 可改+附真实 key):distill、skill_channel、crawl4ai LLMConfig、openai_processor +- health_check 变体(仅 bool):web_scraper/api/crawl4ai(3) + +--- + +## 三、修复方案 = 8 个"轮子"(可复用机制,非逐点补丁) + +排序 = 性价比。每个轮子标:治哪些发现 / 控制论原则 / 大致工作量。 + +### 轮子 1 — DLQ 毒消息可区分【P0-3】① ② · 小 +`reap.rs` 必须把"解析失败"与"真被 trim"分开:`read_entries_by_id` 对解析失败的条目保留原始 bytes 放进 DLQ payload(或单独计数+告警),绝不走"没 entry 就直接 ack"分支。 +**这是本仓上轮自己引入的丢数 bug,码在手边,最先修。** + +### 轮子 2 — 重试分类闭环【P0-1, P1-5】① · 小 +(a) ODP sink/`odp_client` 捕获 `httpx.HTTPStatusError`,对 5xx/429 走已有的 `is_retryable_http_status()`→设 `RetryableHTTPStatus` error_type,让负反馈回路真正闭合。 +(b) `odp-store/main.rs` 主循环三个裸 `?` 改 log-and-continue(沿用 reap 的模式),瞬断不再 panic 退出。 + +### 轮子 3 — 摄入信号真实性【P0-2】② · 小 +odp-ingest `bus=None` 时要么 fail-fast 拒绝启动,要么响应显式标 `degraded/no-op` 且**不计入 accepted**。假 accepted = 被污染的信息 = 假的控制力。 + +### 轮子 4 — 统一 URL 校验器【P0-4 部分, P1-4, 多个 P2】④ · 中 +一个 `safe_url(url)` 模块(仅 http/https;解析后拦 RFC1918/loopback/link-local/元数据 169.254;禁跳转到私网;pin 已解析 IP),在全部 20 个出站点就地调用。**同构性**:一处规则全层复用;**自组织**:每个调用点就地强制不变量,不需中央防火墙。cli_channel 额外加 binary 白名单。 + +### 轮子 5 — 错误上浮契约【P1-7, P1-10, 重试耗尽告警】① ② · 中 +定一条契约:任何失败都必须浮到可观测通道。 +- Python:`DualSink` shadow 错误接入 `events.emit(level=warning)` 或写回 `TaskRun`;Celery 重试耗尽发 dedicated 信号而非只写 DB 状态字段。 +- React:所有 `useMutation` 缺 onError 的补齐 toast(BrowsersPage 两处起)。 +**同构性**:三层同一个"别吞错误"模式,一份契约拉齐。 + +### 轮子 6 — CI 反馈门禁【P1-8, P0-3/P1-5 回归防护】① · 中 +这是元反馈回路——**捕捉断裂反馈回路的系统**。加: +- `alembic upgrade head`(fresh db)+ `downgrade -1 && upgrade head` 冒烟 job +- `cargo test --workspace`(odp-rs)job +- odp-store writer/reap 的 testcontainers 集成 job(redis+postgres,先 2-3 个关键用例:dead_letter 失败不 ack、reap 重复 claim 幂等、savepoint 隔离单条坏消息) +- 恢复 coverage gate + +### 轮子 7 — 鉴权边界【P1-3 → 降 P2 备案;P1-2 部分保留】④ · 视情 +**已定:WG mesh 承担入站边界**,故应用层鉴权降为纵深防御备案(非急):全局 API-key/session +`Depends`、CORS 修正、bind 收到 WG 接口/127.0.0.1(而非 0.0.0.0,防 split-tunnel 误配扩大暴露面)。 +**但 P1-2 的 provider key 加密+响应 mask 保留(挪进轮子 4 一起做)**——因为 key 外泄的真实路径 +是 SSRF 出站 exfil + 日志 + DB 备份,不是入站,WG 救不了。 + +### 轮子 8 — 游标原子推进 + strangler 收口【P1-1, P1-6】① ④ · 中 +(a) `source_cursors` 用 `INSERT ... ON CONFLICT (source_id) DO UPDATE` 或 `SELECT FOR UPDATE`;游标只在"这批已确认落盘"后推进。 +(b) `storer.py` 默认 `forward_to_odp=False`,删掉裸 env var 兜底路径,ODP forward 只由显式 `write_strategy` 触发(收口 HANDOFF 标注的 PR3)。 + +--- + +## 四、落地批次 + +| 批 | 内容 | 理由 | +|---|---|---| +| **B1(先)** | 轮子 1+2+3 | 三个数据链路 P0,全在既有职责内,码已读清,小改+补测,直接堵静默丢数 | +| **B2** | 轮子 6 | 装 CI 门禁,给后续所有改动兜底(含 B1 的回归) | +| **B3** | 轮子 4+5 | SSRF 校验器 + 错误上浮,面大但同构,一次机制多点收 | +| **B4** | 轮子 8 | strangler 收口 + 游标原子性,需设计确认 | +| **B5(降级备案)** | 轮子 7 | 部署面已定 WG mesh → 应用层鉴权降 P2 纵深防御,不进主线 | + +> 部署面已定(WG mesh),轮子 7 降级备案。B3 的 SSRF 校验器现含 provider key 加密+mask。 diff --git a/GOAL-2.md b/GOAL-2.md new file mode 100644 index 0000000..696b07e --- /dev/null +++ b/GOAL-2.md @@ -0,0 +1,29 @@ +# GOAL-2 — opencli-admin Phase 2/3: AuthManager + session affinity 泛化 + +> `/loop` 自驱（接 strangler-fig GOAL 完成 `09e4860` 后的新里程碑）。同纪律： +> 每轮读本文件 → 下个未完 PR → 端到端做+测绿 → 自检 staged → auto-commit → 勾掉。 +> 命中真分叉 → 停问，别 big-bang。 +> 用户 2026-07-01 授权**每 PR 绿即 auto-commit**（仅限此 goal，显式 add 路径，push 等用户）。 + +## 坐标 +- repo `D:\projects\opencli-admin` 分支 `refactor/thin-channel-thick-runner`（接 `09e4860`） +- 测试闸 `uv run pytest tests/unit --no-cov -q`（基线 **379**）；PowerShell 跑 +- ⚠️ 永不 stage：`backend/api/v1/chat.py`、`PR-DESCRIPTION.md`、`HANDOFF-strangler-fig.md`、`GOAL.md`、`GOAL-2.md` + +## Track 1 — session affinity 泛化（Phase 3，低风险） +- [x] **PR-A** — pipeline 绑定 gate 改读 `capabilities.session_affinity`；opencli+skill 声明 `Capabilities(session_affinity=True)`（`073d391`，382 passed，行为零变）。 +- [x] **PR-B** `6e08d41`（404 passed）— 按域名并发上限 = **task 层进程内**（option ②，用户拍板）：`domain_limiter.domain_of(source)` 取 host + `domain_slot()` per-domain semaphore(`PER_DOMAIN_CONCURRENCY` 默认 3，registry 按 `(loop,domain)` 键)，runner Phase3 外套。全渠道覆盖（含 opencli/skill）。跨 worker 严格限 = 换 Redis（同插入点，缓）。 + +> ✅ **GOAL-2 完成**（2026-07-01）：Track-1(PR-A+PR-B) + Track-2(PR-C) 全落，379→**404 passed**，零回归，**已 push fork**（`09e4860..6e08d41`）。 + +## Track 2 — AuthManager + 加密凭据（Phase 2）✅ **DONE** +- [x] **PR-C** `d4aa324`（397 passed）— Fernet 加密凭据存储(`source_credentials` 表 + migration `q7l8m9n0o1p2`) + `AuthManager`(store/resolve/`resolve_context→AuthContext`) + channel_runner 注入 AuthContext(替占位) + api_channel 内联明文 **deprecation warning**。`cryptography` 提为直接依赖。已选方案=Fernet + env `CREDENTIAL_ENCRYPTION_KEY` + 表；接线深度到 AuthManager（未强迁 api_channel→`fetch()`=follow-up）。 +
原决策记录（已决议）： + 1. 加密方案：Fernet/AES（`cryptography` 依赖在否待查） + 2. master key 来源：env var + 3. 存储：新 `source_credentials` 表 / `DataSource` 字段 + 4. 迁移现有内联 secret + 5. 接线深度：api_channel 还没上 `fetch()`，`AuthContext` 走 runner 注入需 api_channel 迁厚契约（run_channel 已建 `AuthContext(kind=cap.auth_kind)` 占位） + +## 每 PR 验收 / 停止条件 / 提交策略 = 同 `GOAL.md` +行为零变（旧路径）、全 tests/unit 绿、显式 stage、auto-commit、真分叉停。 diff --git a/GOAL-3.md b/GOAL-3.md new file mode 100644 index 0000000..dc97437 --- /dev/null +++ b/GOAL-3.md @@ -0,0 +1,53 @@ +# GOAL-3 — api_channel 厚契约迁移代码审查修复环 + +> `/loop` 自驱(接 GOAL-2 `6e08d41` 后,来自这轮 `/code-review xhigh` 的 14 条已验证 finding)。 +> 每轮读本文件 → 下个未 [x] 项 → 端到端做+测绿 → 自检 staged → auto-commit → 勾掉。 +> 命中真分叉/需重新设计游标语义等敏感决策 → 停问,别自己拍板架构。 +> 用户 2026-07-01 授权:每 PR(每条 finding)绿即 **auto-commit**(仅限此 goal,显式 `git add <路径>`,绝不 `add -A`;**push 仍等用户**)。 + +## 坐标 +- repo `D:\projects\opencli-admin` 分支 `refactor/thin-channel-thick-runner` +- 测试闸:`uv run pytest tests/unit tests/integration tests/skills --no-cov -q`(当前基线:unit 412 / integration 77 / skills 97,零失败) +- 跑测试用 PowerShell(`cd D:\projects\opencli-admin; uv run ...`);Bash 被 RTK hook 改写易炸("z: command not found") +- ⚠️ **永不 stage**:`backend/api/v1/chat.py`、`PR-DESCRIPTION.md`、`HANDOFF-strangler-fig.md`、`GOAL.md`、`GOAL-2.md`、`GOAL-3.md`(用户 dock WIP + 控制文件) +- 本 goal 源头改动(api_channel 厚契约 + credential endpoint,尚未 commit)已跑通:后端 tests 全绿 + 真实 uvicorn+encryption key 活验证过 store/list/delete 全链路 + `npx tsc -b` 前端类型检查干净。**这些改动本身先 commit 一刀(PR0),再逐条修 finding。** + +## PR0 — commit 本轮 api_channel 厚契约迁移源头改动(先做,别跳过) +把这些已验证但未提交的文件 add 并提交(排除上面永不 stage 清单): +`backend/api/v1/sources.py backend/auth/manager.py backend/channels/api_channel.py backend/channels/base.py backend/pipeline/channel_runner.py backend/pipeline/collector.py backend/schemas/credential.py frontend/src/api/endpoints.ts frontend/src/components/ChannelConfigForm.tsx frontend/src/pages/SourcesPage.tsx tests/integration/test_sources_api.py tests/unit/auth/test_manager.py tests/unit/channels/test_api_channel.py tests/unit/channels/test_rss_fetch.py tests/unit/pipeline/test_channel_runner.py tests/unit/pipeline/test_collector.py tests/unit/pipeline/test_collector_incremental.py` +测试闸跑一遍确认绿,再 commit。commit message 示例:`feat(api-channel): thick-contract fetch() + encrypted credential store endpoints`。 + +## 状态机(每轮更新) + +- [x] **PR0** — 见上,提交源头改动(`d15e4fd`,593 passed 7 skipped) +- [x] **PR1** — `api_channel.py` fetch() 转发 `timeout`(`8d0c6ad`,594 passed):`client.request(...)` 调用加 `timeout=timeout` kwarg(两处:owns_client 分支的 httpx.AsyncClient 已经隐式带了 client 级 timeout,但 `.request()` 显式传更保险且修的是 ctx.http 分支——共享 client 硬编码 30s,必须靠 per-request `timeout=` 覆盖)。加测试:mock ctx.http 记录 kwargs,断言 `timeout=` 被传入。 +- [x] **PR2** — `fetch()` 补 `except Exception` 兜底(`7de0191`,595 passed)(镜像 `collect()` 的 `except Exception as exc: ... "API request failed: {exc}"` 文案),包成 `ChannelFetchError` 抛出(别学 collect() 返回 ChannelResult.fail——fetch() 契约是抛异常)。加测试:mock client.request 抛 `OSError("connection refused")`,断言抛 `ChannelFetchError` 且消息含 "connection refused"。 +- [x] **PR3** — basic auth 两处实现合一(`2beefe8`,608 passed):抽一个共享 helper(建议 `backend/auth/header_builder.py` 或就近放 `backend/auth/manager.py` 顶层函数,如 `build_basic_auth_header(username, password) -> dict|None`——两者都空返回 None/不发头,而不是发空 Basic 头),`AuthManager.resolve_context()` 和 `ApiChannel._resolve_auth_headers()` 都改调用它。顺带把 bearer/api_key 的 header 构造也一并抽成共享 helper(消掉三处硬编码 key 名的问题:token/key/username+password 约定收敛到一处)。加测试覆盖"两条路径行为一致"的场景(空 creds → 都不发 Basic 头)。 +- [x] **PR4** — `AuthManager.store()` 防并发撞唯一约束(`79fc2e4`,609 passed):`session.commit()` 外包 `try/except IntegrityError`,冲突时 rollback + 重新 select + UPDATE(不是插第二行)。加测试:模拟并发双写同 `(source_id, key_name)`,断言最终只有一行、值是后写的。 +- [x] **PR5** — `CredentialCreate.key_name` 的 `max_length`(`91edcc0`,610 passed;有一次全量跑偶发单测flaky重跑绿,非本改动引入) 从 100 改成 64,和 `SourceCredential.key_name` 的 `String(64)` 对齐。加测试:65 字符 key_name 触发 Pydantic 422。 +- [x] **PR6** — `source_service.delete_source()`(`96511d9`,611 passed) 同一 session 里级联删对应 `source_credentials` 行(delete 前先 `DELETE FROM source_credentials WHERE source_id = ...` 或用 SQLAlchemy `delete(SourceCredential).where(...)`,和删 source 同一事务提交)。加测试:存凭据→删源→断言 `AuthManager().resolve(source_id)` 返回空字典(不再是孤儿)。 +- [x] **PR7** — 收窄 `run_channel()`/`collector.py` 在非增量渠道上的多余开销(`93c67db`,614 passed;RSS增量/限速路径零回归;isinstance检查换成`owns_client`布尔位,顺带更好mock):①`collector.py` 的 `_collect_via_runner` 在调用 `DBCursorStore().load()` 前先判 `channel.capabilities.incremental`,非增量渠道整段 db_cursor/staging 逻辑跳过(等价于 PR5b 之前的行为,只是路由仍统一走 `run_channel`);②`channel_runner.py` 的 `run_channel()` 只在 `chan.fetch is not AbstractChannel.fetch`(即渠道真正覆写了 fetch())时才构建 `RateLimitedClient`,否则 `client=None` 传给默认适配器(反正它不读 `ctx.http`)。**这条要仔细验证不破坏 PR5a/PR5b 的增量渠道行为**——RSS 仍要正常走 cursor+rate-limit。全量跑 tests/unit + tests/skills + tests/integration 确认零回归。 +- [x] **PR8** — `ApiChannel.collect()` 改成薄包装(`eafef17`,614 passed;老13个collect()测试原样过,顺带把fetch()的owns_client分支也改回`async with`保mock兼容+抽了个`_send`小helper去重),委托给 `fetch()`(构造一个不带 `http`/`source_id` 的 `FetchContext`,catch `ChannelFetchError` 转回 `ChannelResult.fail(str(exc))`,成功则 `ChannelResult.ok(result.items, **result.metadata)`)。**`tests/unit/channels/test_api_channel.py` 里所有 `test_collect_*` 断言必须原样通过不改**(这是这条 PR 的验收标准——旧接口行为零变,只是实现委托了)。 +- [x] **PR9** — `CredentialField` name 属性改派生自 `keyName`(已唯一+ASCII,没另加`fieldId` prop——`keyName`已经满足这个要求,加会是纯重复)(`1ee0cb0`,tsc干净+614 passed;⚠️本仓无React组件测试框架,只能typecheck验证)。 +- [x] **PR10** — `CredentialField` 的 `listSourceCredentials` 失败态(`5e0a344`,tsc干净):加 `loadError` state,`.catch()` 里 `setLoadError(true)`,placeholder 逻辑区分"确认未存储" vs "状态获取失败"(后者显示类似"⚠ 无法获取存储状态"而不是伪装成未配置)。 +- [x] **PR11** — `collector.py`/`pipeline.py` 的 `cursor_pending`/`cursor_source_id` 键名(`f5c3f76`,615 passed)改成防撞的保留前缀(如 `__cursor_pending__`/`__cursor_source_id__`),两处出现(`collector.py` 写入 + `pipeline.py` 的 `pop`)同步改。 +- [x] **PR12** — `channel_runner.py` 分页 metadata 合并策略加注释文档化(`b8b4e57`,615 passed)(明确"后页覆盖前页同名键"是有意行为,不是 bug),不改代码逻辑,只补 docstring/inline comment——因为目前无真实渠道 exercise 这条路径,别在没有真实用例前臆造合并语义。 +- [x] **PR13** — `run_channel()` 分页循环(`75db78a`,616 passed) + +> ✅ **GOAL-3 完成**(2026-07-01):PR0→PR13 全落,593→**616 passed**,零回归。`d15e4fd..75db78a` 共 14 个 commit,分支 `refactor/thin-channel-thick-runner`。**未 push**(push 等用户)。 `chan.fetch(ctx)` 外包 try/except,失败时 `log.warning(...)` 打出"第 N 页失败,已丢弃 M 条已抓 items,cursor 可能已推进到 X"再重新抛出(**不改变异常传播行为和 cursor 提交时机**——只加可观测性,cursor/一致性语义的真正修复留给以后,那是 GOAL.md 自己都判过的敏感决策点)。 + +## 每 PR 验收(DoD) +1. 全 `tests/unit` + `tests/integration` + `tests/skills` 绿(≥ 基线,PR0 后基线按新数字算) +2. 旧路径行为零变(尤其 PR7、PR8——这两条改的是既有生产路径,必须零回归) +3. commit(仅码+测路径,自检 staged 集,`git status --porcelain` 核对无 chat.py/GOAL*.md/HANDOFF*.md/PR-DESCRIPTION.md) +4. 更新本文件状态框 + 一行进度 + +## 停止条件(任一 → 停+报,别瞎猜) +- 全 PR 完 +- pytest 红且 2 轮内修不动 +- **真分叉**:PR7 发现会破坏 RSS 增量行为、PR13 发现必须动 cursor 提交时机才能修对、或任何一条修法出现多条不等价路径 +- 需要 push(push 永远等用户) + +## 参考 +- 本轮 finding 全文出自 `/code-review xhigh`(10 finder angles + 13 verify + 3 sweep,当前会话 transcript)。P1(PR1-4)/P2(PR5-9,含 PR0)/P3(PR10-13)按严重度排。 +- 已知刻意不修的架构级问题(记在这,别在 loop 里自己动):`ApiChannel` 从不声明 `Capabilities.auth_kind`,`ctx.auth` 对它架构性地永远无用——真正修法是把 `Capabilities.auth_kind` 泛化成可按 source 动态解析,这是设计级决策,不进本 goal。 diff --git a/GOAL-4.md b/GOAL-4.md new file mode 100644 index 0000000..3c867d9 --- /dev/null +++ b/GOAL-4.md @@ -0,0 +1,67 @@ +# GOAL-4 — 采集管线高可靠化(接 GOAL-3 之后) + +> `/loop` 自驱。每轮读本文件 → 下个未 [x] 项 → 端到端做+测绿 → auto-commit → 勾掉。 +> 命中真分叉 → 停问,别自己拍板架构。 +> 用户已批:每 PR 绿即 auto-commit(显式 `git add <路径>`,绝不 `add -A`,绝不碰 `backend/api/v1/chat.py`/`GOAL*.md`/`HANDOFF*.md`/`PR-DESCRIPTION.md`);push 仍等用户。 + +## 坐标 +- repo `D:\projects\opencli-admin` 分支 `refactor/thin-channel-thick-runner` +- 测试闸:`uv run pytest tests/unit tests/integration tests/skills --no-cov -q`(PowerShell,`cd D:\projects\opencli-admin`;Bash 工具在此环境被 RTK hook 改写会炸) +- 基线(PR0 已完成,commit `2e58cc3`):616 passed 7 skipped + +## 已完成背景(不用再做) +- PR0(本文件外,已提交 `2e58cc3`):`ChannelResult.error_type` 打标 + 6 渠道 catch-all 补齐 + `rss_channel.py:98` timeout 转发漏洞。 + +## 已锁定的架构决策(别重新问,直接照做) +1. **重试接入 celery**:`pipeline.py` 对 retryable 异常 re-raise,celery task 声明的 `max_retries=3` 借此自动生效——不改 `tasks.py`,不加显式 `self.retry()`。 +2. **调度器**:全套接通 redbeat 当 celery beat backend;`CronSchedule` CRUD(创建/改/删)时同步写/删 redbeat entry(不再是"改了要重启 beat 才生效");本地 `backend/scheduler.py` 那条轮询 loop 是否保留由你在做 PR-C 时看情况定(local executor 模式下可能还需要它,celery 模式下应该完全让位给 redbeat)——但两套调度语义不能同时活着互相打架,拿主意时记录在本文件里说明取舍,不用停下来问。 + +## 状态机 + +- [x] **PR-A — 错误分类法(taxonomy)+ 幂等性验证**(`e0c196f`,639 passed)。`error_taxonomy.py`落地;幂等性:序列重跑靠content_hash已经对(既有测试证实),但发现真gap——check-then-insert非原子,并发写同content_hash会IntegrityError丢整批,已修(rollback+recheck+逐条插survivors)+补测试。PR-B激活重试后这个race会变真实,不再是纯理论。 + 在 `backend/channels/base.py` 或新文件(建议 `backend/pipeline/error_taxonomy.py`)定义 `is_retryable(error_type: str) -> bool`: + - retryable:`TimeoutException`、`TimeoutError`、`ConnectionError`、`ConnectError`、`ReadError`、`RemoteProtocolError`、`OSError`(网络/subprocess 层瞬时故障) + - permanent:`ValueError`、`KeyError`、`FileNotFoundError`(二进制/配置缺失,重试没用)、`json.JSONDecodeError`、`ChannelFetchError` 本身(已经是包装过的语义错误,看 `__cause__` 才能细分——如果 cause 是 retryable 类型则 retryable,否则 permanent) + - `httpx.HTTPStatusError`:4xx(除 429,已经在 `RateLimitedClient` 层重试过、到这里说明重试也没用)→ permanent;5xx/429 理论上不该漏到这层(`RateLimitedClient` 已处理),如果漏到了按 retryable 处理兜底 + - 加单测覆盖每类判断。 + **幂等性验证**(不是新写,是确认现状):读 `backend/pipeline/sinks/`,确认 `collect()` 被重跑一次(同一批 items 再来一遍)时 `write_batch` 不会产生重复记录(应该靠 `identity()`/内容 hash 去重)。写一个测试用例证明"同一 task 的 collect→persist 跑两遍,`records_collected` 不翻倍"。如果发现不幂等,记录在本文件里,是否要在本 goal 内修还是记成已知风险——这是真分叉,停问。 + +- [x] **PR-B — pipeline.py 对 retryable 异常 re-raise,激活 celery 真重试**(`30ac9ae`,645 passed)。查清楚了:光re-raise不够,`tasks.py`必须加`autoretry_for=(Exception,)`(此前max_retries=3是死的,没人调self.retry());runner.py Phase 4只在run_pipeline正常返回时跑,re-raise会跳过它把TaskRun卡在running——加了except统一收口标failed再往上抛。`run_scheduled_collection`(cron那条兄弟task)现状没retry,不在这条PR范围,记了没动。 + `pipeline.py` 的 step1/collect 与 step2-3/sink 两处 `except Exception as exc:` 改成:先判 `is_retryable(type(exc).__name__)`(或 `channel_result.error_type` 那条路径),retryable 就 re-raise(让 `run_collection` celery task 函数本身抛出,`autoretry_for` 或 `self.retry()` 生效——先确认 celery task 装饰器需不需要加 `autoretry_for=(Exception,)` 或类似,`bind=True` 已经有了,可能只需要在 catch 到异常处显式 `raise self.retry(exc=exc, countdown=...)`,去 `worker/tasks.py` 确认清楚再改,别瞎猜);permanent 才照旧转 `PipelineResult(success=False)` 吞掉。 + **验证**:写测试模拟一次 retryable 失败,断言 celery task 走了 retry 路径(用 celery 的 eager/test 模式或 mock `self.retry`);模拟一次 permanent 失败,断言不重试、直接 `PipelineResult(success=False)`。 + +- [x] **PR-C — redbeat 接通(全套,见上面锁定决策)**(`77b4afb`,657 passed)。挖到底:celery beat压根没接过(`build_beat_schedule()`零调用方,已删,只留`parse_cron_expression`复用)。全套落地:celery-redbeat依赖+`beat_scheduler`配置+`redbeat_sync.py`(sync_entry/remove_entry/populate_all)+schedule CRUD同步(gate在`task_executor=="celery"`,fail不炸请求)+main.py启动populate_all。本地`scheduler.py`**保留不动**——local模式下唯一调度器,跟redbeat靠task_executor互斥不打架。踩坑并修:populate_all首版抄了tasks.py那套`new_event_loop().run_until_complete()`,但调用方main.py lifespan本身就在跑着的loop里,会炸"already running"——改成async函数直接await。本仓无真实redis,新测试走mock redbeat库边界。 + 1. `pyproject.toml`/`uv add celery-redbeat` + 2. `celery_app.py`:`beat_scheduler = "redbeat.RedBeatScheduler"`,`redbeat_redis_url` 配置(复用 `settings.redis_url` 或 `celery_broker_url`) + 3. 起始 populate:进程启动时(或一次性脚本)把现有 `CronSchedule` 表全量写成 redbeat entries(用 `worker/beat_schedule.py` 里已有的 `_get_enabled_schedules`/`parse_cron_expression` 复用,别重写) + 4. `backend/api/v1/schedules.py`(或对应 CRUD 文件——先找到)的创建/更新/删除/enable-toggle 端点里,同步调用 redbeat 的 `RedBeatSchedulerEntry(...).save()` / `.delete()`,不再只写 DB + 5. `backend/worker/beat_schedule.py` 的 `build_beat_schedule()` 现在有实际调用方了(populate 脚本)——如果它的逻辑跟 redbeat entry 构造重复,提取共享的 cron-parse 部分,别留两份平行实现 + 6. `backend/scheduler.py` 的本地 loop:决定去留(见上面架构决策),照实现,在本文件补一行说明取舍 + 7. **验证**:起真实 redis(本仓测试环境已有 fixture 大概率),写集成测试:建一个 schedule → 断言 redbeat 里有对应 entry;删 schedule → entry 消失;不需要真等 cron 触发(那是 celery beat 自己的事,不用集成测出"真的到点跑了") + +- [x] **PR-D — web_scraper 迁 fetch();opencli/cli/skill 评估后跳过**(`bbfd5b9`,660 passed)。web_scraper真迁了(拿限速+backoff),16个老collect()测试原样过。opencli:非HTTP client(subprocess+browser pool),迁了ctx.http也用不上,跳过。cli/skill:评估中发现PR-B的celery重试已经靠`error_type`taxonomy覆盖它们了(所有渠道failure都走`ChannelResult.fail(error_type=...)`,不只fetch()渠道)——单独retry wrapper纯重复,不加。 + 参照 GOAL-3 PR8 api_channel 的迁法:`collect()` 变薄包装委托给新写的 `fetch()`,**老的 `test_collect_*` 断言必须原样通过不改**(验收标准)。`web_scraper` 走 `ctx.http`(拿 `RateLimitedClient` 的 429/backoff);`opencli` 是 subprocess+浏览器池,不是 HTTP 请求,`fetch()` 迁移对它意义有限(它本来就没有走 HTTP client 这条路)——**先判断 opencli 值不值得迁,若"迁了但 ctx.http 完全用不上"就没必要,只把它记成"评估过、不迁,原因是 XXX"跳过,别为了凑数硬迁**。 + `cli`/`skill` 渠道如果评估后确实需要重试(比如 opencli/skill 的浏览器 flaky 场景),再加独立的 retry wrapper(不是 fetch() 迁移,是 collect() 外面包一层"失败重试 N 次"的装饰器)——先看有没有真实需求再动,没有就跳过记录原因。 + +- [x] **PR-E — 真健康探针(cheap liveness ≠ deep readiness,分两档)**(`a86c216`,675 passed)。`health_check()`签名从`()->bool`拓成`(config=None, source_id=None)->bool`(向后兼容,老0参调用不变),`source_service.py`透传。api真HEAD/GET+真auth头(走`_resolve_auth_headers`包括加密store);web_scraper两档(lxml驱动能用+目标真可达);opencli两档(二进制在+真打CDP `/json/version`,agent/bridge模式跳过深探针,pool未初始化兜底老行为)。health→dispatch gating按计划没做(feature不是fix)。 + - `api_channel.health_check()`:对 `config.get("base_url")` 发一个轻量 HEAD/GET(带 `_resolve_auth_headers`,真的带认证探活,不是随便connect一下),超时给短(如 5s),网络错误/4xx5xx → False + - `web_scraper_channel.health_check()`:探目标 URL 可达(HEAD/GET) + BeautifulSoup/lxml driver 能正常 import/实例化(这个基本不会挂,但按用户原话"driver 活着"补上) + - `opencli_channel.health_check()`:现在只查 `_OPENCLI_BIN` 存在;补上真正打 CDP endpoint(`GET {cdp_endpoint}/json/version`)确认浏览器起得来、够得着——注意这依赖 browser_pool 已 acquire 一个 endpoint,看现有 `pool.acquire()` 怎么用,别为了 health_check 常驻占一个浏览器槽位 + - **明确不做**:health_check 接入 dispatch gating(不健康就跳过任务)——那是 feature 不是 fix,本 goal 不做,写清楚原因 + +> ✅ **GOAL-4 完成**(2026-07-01):PR0(session外,`2e58cc3`)→PR-A→PR-E 全落,616→**675 passed**,零回归。`2e58cc3..a86c216` 共 6 个 commit,分支 `refactor/thin-channel-thick-runner`。**未 push**(push 等用户)。挖到的额外发现:celery beat 之前压根没接通(PR-C)、`run_scheduled_collection` 仍无重试(PR-B 范围外,已记录)、cli/skill 靠 PR-B 的 taxonomy 已间接拿到重试(PR-D 评估结论)。 + +## 每 PR 验收(DoD) +1. `tests/unit` + `tests/integration` + `tests/skills` 全绿(≥ 616 基线) +2. 老路径行为零回归(尤其 PR-B、PR-D——改的是生产路径) +3. commit 仅码+测路径,`git status --porcelain` 自检无 chat.py/GOAL*.md/HANDOFF*.md/PR-DESCRIPTION.md +4. 勾掉本文件对应项 + 一行进度(commit hash + 测试数) + +## 停止条件(真分叉才停,别瞎猜) +- 全 PR 完 +- pytest 红且 2 轮内修不动 +- PR-A 幂等性验证发现真的不幂等,要不要本 goal 内修 +- PR-B 里 celery 重试到底该用 `autoretry_for` 还是显式 `self.retry(exc=exc)`,两种语义不等价(前者装饰器声明式、每次都重试同样逻辑;后者能按错误类型定制 countdown/次数)——如果 `worker/tasks.py` 现状明显该用哪种就直接用,不明显再停问 +- PR-C 步骤 6(本地 scheduler.py 去留)如果发现 local executor 模式还有活人在用(不只是测试覆盖),别直接删,记下来问 +- PR-D 判断某渠道"迁不迁"本身有分歧(不确定算不算真分叉,判断错了也就是白评估一次,不算严重后果,不用为这个停) +- 需要 push(push 永远等用户) diff --git a/GOAL-5.md b/GOAL-5.md new file mode 100644 index 0000000..72f8c4a --- /dev/null +++ b/GOAL-5.md @@ -0,0 +1,92 @@ +# GOAL-5 — Agent 接入(SKILL.md + RSS + REST)+ 内容分类 taxonomy(仿 AIHOT / Dify) + +> `/loop` 自驱。每轮读本文件 → 下个未 [ ] 项 → 端到端做+测绿 → auto-commit → 勾掉。 +> 命中真分叉 → 停问,别自己拍板架构。 +> 每 PR 绿即 auto-commit(显式 `git add <路径>`,绝不 `add -A`,绝不碰 `backend/api/v1/chat.py`/`GOAL*.md`/`HANDOFF*.md`/`AUDIT*.md`/`GRILL*.md`/`PR-DESCRIPTION.md`);push 仍等用户。 + +## 背景 + +对标 https://aihot.virxact.com/agent 的"Agent 接入"模式(Skill/RSS/REST 三轨,匿名免 token,按用户意图分流端点)。opencli-admin 现状(GOAL-5 之前): +- 内容模型 `CollectedRecord`(`backend/models/record.py`)全 JSON blob(raw_data/normalized_data/ai_enrichment),无真实分类/标签字段 +- `DataSource.tags`(`backend/models/source.py`)是自由 JSON list,不是本次要做的内容分类 +- 整个 `/api/v1/*` 无鉴权(admin 工具 style),无 public/private 区分 +- RSS 只有摄入(`rss_channel.py`),没有对外发布 +- 无 SKILL.md 生成能力(fork 上游有个同名"skill"概念但语义不同——浏览器录制→蒸馏的数据源渠道,和这次"agent 可装的 SKILL.md 包"是两回事,注意别混) + +## 坐标 +- repo `D:\projects\opencli-admin`,新分支 `feat/agent-access-taxonomy`(从当前 `main` 切出) +- 测试闸:`uv run pytest tests/unit tests/integration tests/skills --no-cov -q`(PowerShell,`cd D:\projects\opencli-admin`;Bash 工具在此环境被 RTK hook 改写会炸) +- 部署目标未定 —— 本 goal 只出设计落地的代码,不碰部署/域名 + +## 已锁定的架构决策(别重新问,直接照做) + +1. **鉴权**:公开接口匿名免 token(仿 AIHOT),限流用 IP-based 轻量中间件(内存 token bucket,不引入 Redis 依赖),不做 API Key 发放机制。 +2. **暴露范围**:`DataSource` 加 `public`(bool,默认 `False`)开关,只有显式打开的源的内容才可能进公开接口。 +3. **分类机制**:抄 Dify 的 `Tag` + `TagBinding` 模式,去掉 `tenant_id`(opencli-admin 无多租户概念)。`Tag(id, type, name, created_at)`,`type ∈ {"category", "subtag"}`;`TagBinding(id, tag_id, target_id, created_at)`,`target_id` 指向 `collected_records.id`,不建 DB 级 FK(照抄 Dify 做法,完整性靠服务层)。业务不变量:每条 record 最多绑 1 个 `type=category` 的 tag,`type=subtag` 可绑多个。 +4. **顶级分类闭集种子值**(占位提议,未被用户改,直接按此写 seed):`模型能力` / `产品动态` / `行业资讯` / `研究论文` / `工程实践` / `其它`。分类名不照抄 AIHOT(那套是仿 Dify 机制,不是仿 AIHOT 命名)。 +5. **分类来源**:`DataSource.default_category` 兜底(source 级默认);AI enrichment 阶段跑完后可用 LLM 输出覆盖式细化绑定。LLM 分类调用失败/超时 —— 不阻断 pipeline,直接落回 source 默认值,record 状态复用现有 `status` 枚举(`raw|normalized|ai_processed|notified|error`),不新增状态。 +6. **curated 精选**:`CollectedRecord` 加 `curated`(bool,默认 `False`),**v1 只做人工打标**(走现有 admin API 手动 PATCH)。自动/规则化 curate 是二期,本 goal 不做 —— 这是 YAGNI 裁决不是遗漏,别加。 +7. **PublicContentService**(`backend/services/public_content_service.py`)是唯一查询入口 —— 给定 `mode`(`selected`|`all`)/`category`/`since`/`q`/`take` 返回过滤后的 record 集合,REST 和 RSS 都调它,不重复写"哪些内容可对外"这条过滤逻辑(`source.public=True` 是硬性前提,`mode=selected` 再加 `curated=True`)。 +8. **RSS**:新增对外发布方向(现有 `rss_channel.py` 只是摄入,方向不同,不复用其摄入逻辑,只复用 `PublicContentService` 的查询结果),用 `feedgen` 库序列化成 Atom。 +9. **SKILL.md**:不手写。从 `taxonomy.py`(分类闭集)+ 路由定义脚本生成后 commit 进仓库,按静态文件路由 served。CI 加一致性检查:重新生成结果必须等于已提交文件,防止改分类忘同步文案。 +10. **Daily digest**:独立定时任务(复用现有 pipeline 调度基建,没有就加最简单 cron 任务),把当日 `public=True AND curated=True` 的 record 快照进新表 `daily_digests`(date, record_ids, 可选 LLM 摘要文案),不是实时计算。同一天重跑必须幂等(upsert,不重复插入)。 +11. **响应白名单**:新增 `PublicRecordRead` schema,只含 `id/title/url/summary/source_name/published_at/category/subtags`,显式排除 `raw_data`/`normalized_data`/内部 source 配置 —— 防止 admin 内部字段随手泄露到公开接口。 + +## 状态机 + +- [x] **PR-A — 数据模型:Tag/TagBinding + DataSource 扩展字段 + taxonomy 闭集定义**(`0f48495`,398→414 passed,零回归)。 + 新增 `backend/models/tag.py`(`Tag`/`TagBinding`,建表 migration,索引 `(type, name)`、`target_id`、`tag_id`)。`DataSource` 加 `public: bool = False`、`default_category: str | None` 两列(migration)。新增 `backend/taxonomy.py` 定义闭集分类常量(见架构决策 #4)+ 校验函数 `is_valid_category(name) -> bool`。 + 验收:migration 跑通,新表/新列存在,`taxonomy.py` 单测覆盖合法/非法分类名判断。 + +- [x] **PR-B — TagService**(`adaf55b`,414→424 passed,零回归)。 + `backend/services/tag_service.py`:`bind_category(record_id, category_name)`(校验闭集、强制"最多 1 个"覆盖式绑定,非法分类名报错)、`add_subtags(record_id, names: list[str])`(去重、允许新建)、`get_tags(record_id)`、`list_by_category(category_name)`。 + 验收:单测覆盖"重复绑定 category 是覆盖不是叠加"、"非法 category 名拒绝"、"subtags 去重"、"list_by_category 只返回该分类下的 record_id 集合"。 + +- [x] **PR-C — Enrichment 阶段接入分类**(`25d48f0`,424→439 passed,零回归)。走全LLM驱动+兜底:现有enrichment(`ai_processor.py`)本来就没有专门分类调用,只是把用户配置prompt的LLM JSON输出存进`ai_enrichment`——没造新LLM调用架构,机会性读该JSON里的`category`/`subtags`键(有校验),没有/非法/enrichment未跑/失败都落回`source.default_category`;`default_category`也空则记警告跳过分类,subtags仍照跑,不阻断pipeline、不新增status。 + 扩展现有 AI enrichment 阶段(找到 `backend/pipeline/` 里对应步骤):跑完常规 enrichment 后,调 `TagService.bind_category`(优先 LLM 输出,失败/超时/未跑则用 `source.default_category` 兜底)+ `TagService.add_subtags`(LLM 输出的细粒度标签)。 + 验收:LLM 分类成功路径、LLM 失败兜底路径、`source.default_category` 为空时的行为(记录警告,不崩)都有测试;确认 pipeline 整体状态机不受影响(现有 `status` 流转测试不回归)。 + +- [x] **PR-D — PublicContentService**(`68993be`,439→457 passed,零回归)。补了`CollectedRecord.curated`列(架构决策#6要但之前PR都没加,migration`n4i5j6k7l8m9`)。`since`过滤用`created_at`原生列不是`normalized_data.published_at`(那字段格式不统一,查了会错不只是慢)。`q`用`lower(cast(normalized_data as String)).contains()`(跟`record_service.py`现有写法一致,dialect通用非Postgres专属ILIKE)。`take`默认50上限200。安全测试(`public=False`永不泄露,含"给我全部"式恶意参数组合)已覆盖。 + `backend/services/public_content_service.py`:核心过滤(`source.public=True` 硬前提 + `mode`/`category`/`since`/`q`/`take`)。 + 验收(安全关键):`source.public=False` 的 record 无论调用方传什么过滤参数都不能出现在结果里 —— 这条测试必须覆盖"恶意/异常参数"场景,不只是正常路径。 + +- [x] **PR-E — REST API + 限流**(`b343146`,457→474 passed,零回归)。限流60次/分钟/IP内存token bucket,只挂public router非全局middleware。响应壳复用现有`ApiResponse[T]`(`backend/schemas/common.py`),白名单字段用显式mapper函数(非`model_validate`裸转)。`summary`取`ai_enrichment.summary`优先,没有则退`normalized_data.content`(合理外推非锁定决策原文)。泄露测试断言`raw_data`/`normalized_data`/`source_id`等键不在响应里,非仅信任schema。 + `backend/api/public/`(`router.py`/`items.py`/`schemas.py`/`throttle.py`),挂载 `/api/public/*`(与现有 `/api/v1/*` 并列,`main.py` 加一行 `include_router`)。`GET /api/public/items?mode=&category=&since=&q=&take=`。IP-based 内存限流中间件,只挂在 public router 上。非法 `category` 参数返回 400 + 合法值列表。 + 验收:端点集成测试(public/private 混合数据不泄露私有源)、限流触发 429+Retry-After、`PublicRecordRead` 字段白名单测试(响应体绝不含 `raw_data`/`normalized_data`)。 + +- [x] **PR-F — RSS 发布**(`457499b`,474→479 passed,零回归)。路由`/api/public/rss`,线路格式实为Atom(feedgen序列化),路径名"rss"照AIHOT命名对齐,docstring写清楚。新加`feedgen==1.0.0`依赖。复用PR-D查询+PR-E白名单mapper,不重写过滤逻辑。序列化失败兜底走手搭空Atom壳(不靠feedgen本身,防它自己炸时兜底也炸)。 + `backend/api/public/rss.py`:`GET /api/public/rss` 复用 `PublicContentService` 结果集,`feedgen` 序列化 Atom。序列化异常兜底吐合法空 `` 壳,不 500。 + 验收:产出 XML 用 `feedparser`(现有摄入用的同一库)反解 round-trip 测试通过;异常兜底路径有测试。 + +- [x] **PR-G — Daily digest**(`ce86256`,479→509 passed,零回归)。调度:本分支实际有`scheduler.py`+celery/beat但`CronSchedule.source_id`非空FK跟digest(不挂单一source)不契合,改动太侵入不值——改走混合:celery beat静态entry(`daily-digest-snapshot`,00:10 UTC,`task_executor=celery`时免费生效)+独立入口`backend/worker/digest_job.py`(`task_executor=local`时给外部OS调度器调),不新造进程内调度器。`PublicContentService`加`until`上界参数(可选,不改现有调用行为)。 + 新表 `daily_digests`(migration)+ `backend/services/digest_service.py` + 定时任务(复用现有调度基建;GOAL-4 已接通 redbeat,优先复用而不是新开一套调度)+ 端点 `GET /api/public/daily`、`/api/public/daily/{date}`、`/api/public/dailies?take=N`。 + 验收:同一天重跑两次不重复(幂等测试);空数据日的行为(无 curated 内容时不报错,返回空)。 + +- [x] **PR-H — SKILL.md 生成**(`cde20bf`,509→521 passed,零回归)。生成脚本`backend/scripts/generate_skill_md.py`直接内省PR-E/F/G的真实`APIRouter`(参数/默认值/description),不是手抄文档,配`--check`漂移模式。输出`backend/skills/agent_access/SKILL.md`,额外加`main.py`挂`StaticFiles`到`/skills`,真正served出`GET /skills/agent_access/SKILL.md`(spec原文要求的"静态文件路由 served")。CI真接了一步进`.github/workflows/ci.yml`的`backend-test`job,另加pytest漂移测试当本地权威闸(手动验证过篡改SKILL.md/taxonomy分类名都能触发失败)。**发现但没动的遗留问题**:`backend-test`job的`working-directory: backend`跟`pyproject.toml`/`tests/`实际在仓库根不一致,job本身像是已经跟别处改动脱节坏了,跟本PR无关,没有顺手改,记录在此。 + +> ✅ **GOAL-5 完成**(2026-07-08):PR-A→PR-H全落,398→**521 passed**(414/424/439/457/474/479/509/521 逐PR过点),零回归。`0f48495..cde20bf`共8个commit,分支`feat/agent-access-taxonomy`。**未push**(push等用户)。遗留:`.github/workflows/ci.yml`的`backend-test`job working-directory疑似已脱节坏了(见PR-H),不在本goal修。 + 生成脚本(从 `taxonomy.py` + 路由定义读取)产出 `backend/skills/agent_access/SKILL.md`,静态文件路由 served。CI 步骤:重新跑生成脚本,diff 已提交文件,不一致则失败。 + 验收:生成脚本单测 + CI 一致性检查文档化(写清楚怎么跑、什么时候会红)。 + +## 每 PR 验收(DoD) +1. `tests/unit` + `tests/integration` + `tests/skills` 全绿(不低于 PR-A 前的基线) +2. 老路径行为零回归(尤其 PR-C —— 改的是生产 pipeline 路径) +3. commit 仅码+测路径,`git status --porcelain` 自检无 `chat.py`/`GOAL*.md`/`HANDOFF*.md`/`AUDIT*.md`/`GRILL*.md`/`PR-DESCRIPTION.md` +4. 勾掉本文件对应项 + 一行进度(commit hash + 测试数) + +## 停止条件(真分叉才停,别瞎猜) +- 全 PR 完 +- pytest 红且 2 轮内修不动 +- 顶级分类种子值(架构决策 #4)如果实现中发现现有已采集内容明显套不进这 6 类,要不要现在改分类表 —— 停问,别自己加类目 +- 限流阈值具体数值(次数/窗口)没有强共识,给个保守默认(比如 60 req/min/IP)先落地,除非明显不合理不用为这个停 +- PR-G 复用哪套调度基建(redbeat vs 本地 scheduler.py)如果 GOAL-4 那两套还在互斥期,按 GOAL-4 已锁定的 `task_executor` gate 走,不重新纠结 +- 需要 push(push 永远等用户) + +## 后续排队(不在本 goal 内,已定顺序,未来各开一个 GOAL-N) +参照 https://github.com/langgenius/dify 的组件功能(后端+前端都要,不是只抄后端),按依赖链顺序: +1. 模型 Provider 管理(多 LLM provider 抽象) +2. Workflow 编排引擎 +3. 插件市场 +4. App 发布机制 + +这四项跟本 goal(Agent 接入 + 分类)相互独立,规模各自都够开一轮完整 brainstorming + spec,本文件不展开。 diff --git a/GOAL-6.md b/GOAL-6.md new file mode 100644 index 0000000..4012bca --- /dev/null +++ b/GOAL-6.md @@ -0,0 +1,89 @@ +# GOAL-6 — 模型 Provider 管理(仿 Dify model-runtime,自研) + +> `/loop` 自驱。每轮读本文件 → 下个未 [ ] 项 → 端到端做+测绿 → auto-commit → 勾掉。 +> 命中真分叉 → 停问,别自己拍板架构。 +> 每 PR 绿即 auto-commit(显式 `git add <路径>`,绝不 `add -A`,绝不碰 `GOAL*.md`/`HANDOFF*.md`/`AUDIT*.md`/`GRILL*.md`/`PR-DESCRIPTION.md`);push 仍等用户。 + +## 背景 + +Dify 队列 item 1(见 GOAL-5 末尾排队)。main 上已有胚胎版,本 goal 升级成真子系统: + +- 已有:`ModelProvider` 表(name / provider_type∈{claude,openai,local} / base_url / api_key **已 Fernet 加密**(`backend/auth/crypto.py`,env `CREDENTIAL_ENCRYPTION_KEY`)/ default_model / enabled)+ CRUD API(`backend/api/v1/providers.py`,纯 CRUD 无 test/sync)+ 响应 masking(`has_api_key`/`api_key_preview`)+ Next.js providers 页(`frontend/app/(app)/providers/page.tsx`) +- 痛点 1:四个消费点各自为政拼 client —— `chat.py` agent 坞(私有 `_build_client`)、`skill_channel`、processors(openai/claude/local,config dict + env fallback)、`crawl4ai_channel`(litellm 间接) +- 痛点 2:`DataSource.ai_config` JSON 是绕开 ModelProvider 的平行凭证通道(双真相源) +- 痛点 3:无模型目录、无"系统默认模型"概念、无 test connection、无 failover +- 轮子边界:model-hotel(5080 自研网关)管跨 provider 聚合/凭证池/quota;app 内**只做** role 级候选顺序 failover,不重复造 + +## 坐标 + +- repo `D:\projects\opencli-admin`,新分支 `feat/model-provider-mgmt`,**从 main 切** +- **前置(用户手动)**:GOAL-5 分支 `feat/agent-access-taxonomy` 由用户 push + merge 进 main 后再开工;若开工时 main 里还没有 taxonomy 提交,停问 +- 测试闸:`uv run pytest tests/unit tests/integration tests/skills --no-cov -q`(PowerShell,`cd D:\projects\opencli-admin`;Bash 工具在此环境被 RTK hook 改写会炸) +- 基线测试数以切分支后 main 实测为准,PR-A 前先跑一遍记进本文件 + +## 已锁定的架构决策(别重新问,直接照做) + +1. **运行时自研,不引 litellm**。新包 `backend/llm/`:`base.py`(`ProviderAdapter` ABC:`chat()` / `list_models()` / `test_connection()`)、`openai_compat.py`、`anthropic.py`、`factory.py`、`resolver.py`、`catalog.py`(anthropic 硬编码模型目录常量)。 +2. **`provider_type` 枚举不动**(`openai|claude|local`),语义=adapter 族:`openai`/`local` → `OpenAICompatAdapter`,`claude` → `AnthropicAdapter`。不破坏 crawl4ai 的 litellm 前缀映射和存量数据。 +3. **`provider_models` 表**(模型目录):`id, provider_id(真 FK→model_providers, ondelete CASCADE), model_id, model_type(str, default "llm",闭集校验,v1 只有 "llm",列留 embedding/rerank 扩展位), capabilities(JSON 可空: tools/vision/context_window), source∈{discovered,manual}, enabled(bool default True), created_at`。唯一约束 `(provider_id, model_id)`。sync 是 upsert,`source=manual` 条目绝不被 sync 覆盖或删除。 +4. **`model_defaults` 表**(系统默认模型,按消费角色):`role`(闭集 `chat|executor|enrichment`,唯一)、`candidates`(JSON 有序列表 `[{provider_id, model_id}]`)。首位=主选,后位=failover 候选。role 对应:agent 坞对话 / skill_channel 便宜执行 / pipeline enrichment 兜底。 +5. **模型发现**:OpenAI-compat 打 `GET {base_url}/v1/models`(ollama/model-hotel/deepseek 等全通);anthropic 返回 `catalog.py` 常量。发现失败不崩,返回错误详情供前端展示,可手动登记兜底。 +6. **SSRF/key 外泄防护**:factory 建带 api_key 的 client 前必须过 `backend/security/url_guard.py`(`avalidate_public_url_and_ip` + `PinnedAsyncHTTPTransport`),与 main 现有 openai_processor/skill_channel 做法一致。本地地址(ollama/model-hotel)按 url_guard 现有豁免机制走,没有豁免机制则停问。加密/masking 复用现有,不重做。test connection 的错误响应绝不回显 api_key。 +7. **failover 语义**(resolver):`resolve(role)` 返回首位候选;`resolve_with_fallback(role)` 按 candidates 顺序试。**只有连接级错误(连不上/超时/5xx)才降级**;4xx 业务错误(如 401 key 错)不降级——那是配置错,降级会掩盖问题。坏 provider 进内存 cooldown(简单时间窗,进程内 dict,不引 Redis),窗口内跳过。 +8. **消费点收编范围**:`chat.py`(替 `_build_client`)、`skill_channel`、processors(openai/claude/local)三处走 factory;agent 级 `processor_config` 覆盖能力保留(provider 供底,agent config 覆盖)。**crawl4ai 例外**:client 是 crawl4ai 内部造的(litellm),只收编 provider/model/key 的**解析**走同一 resolver,litellm 调用保留,docstring 写明例外原因。 +9. **双轨收敛(软)**:`DataSource.ai_config` 支持 `provider_id` 引用;存量 inline `api_key`/`base_url` 继续能跑但记 deprecation 警告日志;新前端只给 provider 下拉。v1 不硬迁移、不删字段。`ai_agents.provider_id` 维持松散字符串列,不在本 goal 升 FK(改动收益比不值)。 +10. **API 面**(挂现有 providers router):`POST /providers/{id}/test`、`POST /providers/{id}/models/sync`、`GET|POST|PATCH|DELETE /providers/{id}/models`、`GET|PUT /model-defaults`。响应壳复用 `ApiResponse[T]`。 +11. **前端**:扩展 Next.js providers 页(shadcn/ui + react-query hooks 现有模式,不新增 zustand 用途):provider 行展开模型目录表格 + sync 按钮、test connection 按钮 + 状态徽章(延迟/失败原因)、defaults 卡(三 role 各配 candidates,可排序)、preset 列表加 model-hotel(prefill base_url)。 + +## 状态机 + +- [x] **PR-A — 数据模型**(基线 1504→**1530 passed**,+26 新测,零回归;GOAL-6 从 main `e60c473` 切 `feat/model-provider-mgmt`,该 main 已含 GOAL-7 browser-act 但**无 GOAL-5 taxonomy**——用户显式指示此序,taxonomy 非 GOAL-6 代码依赖)。`backend/models/provider_model.py`(`ProviderModel` 表 `provider_models`:**真 FK** `provider_id→model_providers.id ondelete CASCADE`+index、`model_id`、`model_type` default `llm`、`capabilities` JSON、`source`、`enabled`、唯一约束 `(provider_id, model_id)`)+ `backend/models/model_default.py`(`ModelDefault` 表 `model_defaults`:`role` 唯一、`candidates` JSON 有序)+ 注册进 `backend/models/__init__.py`。`backend/llm/__init__.py`(闭集 `VALID_MODEL_TYPES/ROLES/SOURCES`+校验 helper)+ `backend/llm/catalog.py`(`ANTHROPIC_CATALOG` 硬编码常量,决策 #5 无 /v1/models discovery:`claude-opus-4-8`/`claude-sonnet-5`/`claude-haiku-4-5-20251001`,ctx 200000)。Pydantic schema `provider_model.py`/`model_default.py`(field_validator 用 backend.llm helper 校闭集)。migration `d8e9f0a1b2c3`(down_revision=`a7v8w9x0y1z2`,scratch db 全链 base→head + downgrade round-trip 验通,repo db 被别进程锁故用 scratch)。闭集校验只在 Pydantic/backend.llm 层(SQLAlchemy 无 @validates,匹配现有 provider_type/channel_type 约定)。测试 `tests/unit/llm/` + `tests/unit/test_provider_model.py`/`test_model_default.py`(26 测:闭集 helper+schema 双层、唯一约束 IntegrityError、role 唯一、FK cascade、catalog 完整性)。 + ~~验收:migration 跑通;唯一约束生效测试;model_type/role 闭集校验单测;基线测试数记录进本文件。~~ 全达成。 + ⚠️ **PR-C/E 注意**:sqlite 默认不强制 FK,本 repo runtime(`backend/database.py`)从不发 `PRAGMA foreign_keys=ON`(已 grep 确认)——DB 级 cascade 只在 pragma 开时生效,生产删 `ModelProvider` 不会自动级联删 `provider_models`;PR-C 删 provider 时要显式清理 catalog 行或开 pragma,别假设 FK cascade 自动触发。 + +- [x] **PR-B — `backend/llm/` 运行时**(1530→**1551 passed**,+21 新测,零回归)。`base.py`(`ProviderAdapter` ABC:`chat(messages,*,model)->str` / `list_models()->list[str]`(失败 raise) / `test_connection()->ConnectionTestResult` TypedDict `{ok,latency_ms,error,models_sample}`(不 raise);`LlmAdapterError` 不含 secret;`redact_secret` helper)。`openai_compat.py`(`OpenAICompatAdapter` for openai|local,`avalidate_public_url_and_ip` 在建 `AsyncOpenAI` 前跑 + `PinnedAsyncHTTPTransport`,照 skill_channel/openai_processor)。`anthropic.py`(`AnthropicAdapter` for claude,`list_models`=`anthropic_catalog()` 决策 #5)。`factory.get_adapter(provider)` 按 provider_type 派发。**决策 #6 local-address 解**(关键):url_guard **无既有** localhost/私有豁免机制(读全模块+38测确认),故扩展 `backend/security/url_guard.py` 加 keyword `allow_private=False`(全层穿透:is_ip_blocked→_check_host_and_ips→validate*→PinnedAsyncHTTPTransport→guarded_async_client),**default False 全部现有调用行为不变**(38 url_guard 测 + 全量 1551 零回归验证);`unspecified/multicast/reserved` 恒 blocked、DNS-rebind pin 恒生效,`allow_private=True` 只放行 loopback/private/link-local/CGNAT;**唯一传 True 者=OpenAICompatAdapter 且仅 provider_type=="local"**(openai/claude 全守卫)。测试 `tests/unit/llm/test_adapters.py`(21:两 adapter chat/list_models/test_connection、url_guard 拒绝(恶意 base_url 建 client 前拒+SDK 从不被调)、api_key 不入 error(5测 redact 断言)、factory 派发、local 放行 loopback vs openai 拒同 URL + local 仍 pin)。 + ~~验收:...api_key 不出现在任何异常消息断言。~~ 全达成。 + ⚠️ **url_guard 是共享审计模块(AUDIT B3)**,本 PR 加了 `allow_private` 扩展——向后兼容(default False),但 review/合并时留意此跨切改动。 + +- [x] **PR-C — API:test + sync + 目录 CRUD + defaults**(1551→**1575 passed**,+24 新测,零回归)。`backend/services/provider_model_service.py`(薄端点,DB 逻辑在此,单一 mock 缝=`get_adapter`):`sync_models`(决策 #3:manual 行永不覆盖/删=`kept_manual`,discovered 已存=去重不违唯一约束,新=added,**stale discovered prune**=本 PR 设计选择已文档化,manual 永不 prune,幂等)、catalog CRUD(`add_manual_model` 结构上强制 source=manual)、`delete_provider_models`、`put_default`(role 闭集 + 每 candidate provider 存在 + model 在该 provider catalog,清晰错误无 key)、`test_connection`(转 adapter 结果已 sanitize)。端点(providers.py 扩展 + 新 `model_defaults.py` router 挂 __init__):`POST /providers/{id}/test`、`.../models/sync`(LlmAdapterError→502)、`GET|POST|PATCH|DELETE /providers/{id}/models[/{row}]`、`GET /model-defaults`、`PUT /model-defaults/{role}`(role 入 path,比 spec 的 GET|PUT 更 RESTful,判断改)。**provider-delete 清理**:现有 `DELETE /providers/{id}` 先调 `delete_provider_models`(PR-A 坑:sqlite 无 cascade)。测试 `tests/integration/test_provider_models_api.py`+`test_model_defaults_api.py`(24:test 成功/失败+api_key 不在 body、sync 幂等+manual 存活+stale prune、CRUD、defaults 校验 bad-role/nonexistent-provider/model-not-in-catalog、provider-delete 无孤儿)。 + ~~验收:...错误响应无 key 泄露断言。~~ 全达成(现有 providers 测试零回归)。 + +- [x] **PR-D — resolver + failover**(1575→**1599 passed**,+24 新测,零回归)。`backend/llm/resolver.py` `ProviderResolver`(注入式 monotonic clock、进程内 `_cooldown_until` dict 无 Redis、模块单例 `resolver`)。`resolve(db,role)->ResolvedModel|None`(首候选,无配置/空/provider 已删=None,不 failover)。`resolve_with_fallback(db,role,operation)`:顺序试候选——cooled→跳过不建 adapter、missing provider→跳过无 cooldown、成功→立即返、`LlmAdapterError retryable=True`(连接级)→cooldown+下一个、**`retryable=False`(4xx 业务)→立即 re-raise 不 cooldown 不 fallthrough(决策 #7 核心)**、全竭→`ResolverError`带 tried/cooled 计数无 key。`_set_cooldown` 同步读写无 await 协程安全。**错误分类**(改 PR-B 3 文件,向后兼容):`LlmAdapterError` 加 `retryable=False` kw、`base.classify_retryable(exc)` 三层(openai/anthropic APIConnection/Timeout/InternalServer→True,4xx/auth→False,`status_code>=500` 兜底,余 False),adapter chat/list_models except 传 `retryable=classify_retryable(exc)`。测试 `tests/unit/llm/test_resolver.py`(24:顺序降级+cooldown 记录、4xx 不降级+不 cooldown+B 不试、cooldown 窗口跳过+过期重试、全竭 ResolverError 无 key、20 并发一致、classify_retryable 15 例);adapters 测重跑 21/21 无回归。 + ~~验收:...并发调用下 cooldown dict 不炸。~~ 全达成。 + +- [x] **PR-E — 消费点收编**(1599→**1617 passed**,+18 新测,**零回归**——高危生产路径 PR,全量已跑验)。核心=去重 client 构造走 factory,**不改行为**。新 factory 助手 `build_openai_compat_adapter`/`build_anthropic_adapter`/`litellm_prefix_for` + `_provider_view`(SimpleNamespace,解两难:chat.py 的 live ORM provider 直接写 env-key 会被 autoflush 持久化进 DB→用抛弃视图规避;skill_channel/processors 是 dict 配置需属性访问)+ 各 adapter 加 `get_client()` 交出守卫 client(tool-loop 要裸 client)。收编:**chat.py**(`_build_client` 走 factory,`OPENAI_API_KEY` env fallback + `_pick_provider` + tool-loop 全保留;**原来无 SSRF 守卫→现在有**,决策 #6 顺带补洞,无测试覆盖故不回归)、**skill_channel**(走 factory,qwen3:4b + guard 保留)、**openai/claude processor**(走 factory + 各自 env fallback/usage 日志/JSON-mode 保留,裸 SDK response 仍自取)、**crawl4ai**(litellm 调用+LLMConfig 未动,只 `litellm_prefix_for` 收编映射)。**runner agent-override 未动**+新测锁定(provider 供底 agent processor_config 覆盖)。dead-code grep 净(无残留重复 client 构造)。测试 `tests/unit/llm/test_pr_e_consumers.py`+runner 测(18)。 + ~~验收:零回归闸...各消费点删掉的私有 client 拼装代码不残留。~~ 全达成,无改现有测试。 + ⚠️ **留白/偏差**(诚实记):(1)**resolver 未接入任何消费点**——PR-D 建好测好(24测)但 PR-E 未 wire,agent 保守判断:决策 #8 只要求"走 factory"非"走 resolver",接入会引入 role/model_defaults 选择轴改现有 provider 选择行为+风险回归;resolver 可 import 待未来 opt-in 消费点用(或另开收尾接入)。(2)**local_processor 未改**——ollama 原生 `/api/generate` 协议 + `timeout` 配置旋钮与冻结的 OpenAICompatAdapter 不匹配,强收会改 wire 行为/丢旋钮,判为不安全跳过。(3)chat.py 补了 SSRF 守卫(原缺)。 + +- [x] **PR-F — 双轨收敛(软)**(1617→**1622 passed**,+5 新测,零回归)。seam=`backend/pipeline/ai_processor.py` 新 `_resolve_llm_config(ai_config, source_id)`(process_with_ai 调):无 provider_id→`return ai_config` **字节不变**(仅 inline api_key/base_url 存在时 log deprecation 警告);provider_id 解析→`dict(ai_config)` copy 覆盖 processor_type/api_key/base_url/model(inline 也给时警告"provider_id 优先");provider_id 不解析→警告+回落 ai_config 不崩(fail-soft 同现有 posture)。DB session 仅 provider_id 存在时开(常路不碰 DB)。`ai_agents.provider_id` 未动(仍松散字符串,决策 #9)。判断:加 `resolve_provider=True` kwarg,pipeline.py 调用传 `resolve_provider=agent_config is None`(agent 级 config 经 runner 另路解析 provider_id,不走本 deprecation 逻辑避免每次 agent 运行误报)。测试 `tests/unit/pipeline/test_ai_processor.py`(5:provider_id 路径、inline 字节相同+warn(断言 `passed_config is ai_config`)、both→provider 赢、provider_id 不存在回落、resolve_provider=False 门控)。 + ~~验收:...警告日志断言。~~ 全达成(inline 路径行为字节相同)。 + +- [x] **PR-G — 前端**(决策 #11)。现状=providers 页原为纯只读 Card 网格(`createProvider/update/delete` 是死代码)。新建 `frontend/components/providers/`(`provider-form-dialog.tsx` 增改弹窗+3 预设 Claude/OpenAI/**model-hotel**、`provider-catalog-panel.tsx` 展开目录表+sync+手动添加、`model-defaults-card.tsx` 三角色候选卡可增删排序按角色保存)+ `types.ts`(修 `ModelProvider` 去假 api_key 加 `has_api_key/api_key_preview`,加 `ProviderModelRead/ConnectionTestResult/ModelRole/ModelDefault*` 等)+ `endpoints.ts`(8 端点)+ `hooks.ts`(11 react-query hooks 含 queryKey 失效)+ `page.tsx` 接线。四交互(目录展开/sync、test+徽章、defaults、preset)完整实现;**额外补了 create/edit/delete 弹窗**(超决策 #11 原文——preset 需 add-provider 表单才有落点,且后端一直有 provider CRUD 端点前端未接,接上使页面真可用)。api_key 全程 masked(密码框写入态、编辑只显 preview、page 只碰 has_api_key/api_key_preview,无处渲染原始 key)。 + ~~验收:...api_key 在 UI 全程 masked。~~ tsc `--noEmit` exit 0(独立重跑)+ eslint 净 + `next build` 成功 + 无碰 backend/tests。⚠️ **弹窗提交/mutation toast/Select 联动/惰性请求 需真实后端跑起来才能点击验证**——本轮无法起全栈(后端+DB+dev server),编译/构建/类型接线已证,合并前建议接后端手点一遍。 + +> ✅ **GOAL-6 完成**(2026-07-09):PR-A→PR-G 全落,1430→**1622 passed**(A~F 后端 1448/1467/1484... 至 1622;PR-G 前端 tsc/eslint/build 绿),零回归(既有 12 failed 全 main 遗留:opencli/workflow/nodes-install + 4 `*_live.py` 需真 Chrome)。分支 `feat/model-provider-mgmt`(从 main `e60c473` 切,7 commit `6cf3358..`)。**未 push**(push 等用户)。数据模型(provider_models+model_defaults 真 FK)+ 自研 LLM 运行时(OpenAICompat+Anthropic 双 adapter+factory,**不引 litellm**)+ url_guard `allow_private` local 豁免 + API(test/sync/目录 CRUD/defaults)+ resolver+failover(4xx 不降级+cooldown)+ 消费点收编(factory 去重 client 构造)+ ai_config 软收敛 + Next.js 前端。 +> **⚠️ 关键留白**(合并前须知):(1)**resolver+failover(PR-D)建好测好但未接入任何消费点**(PR-E 保守判断:接入改 provider 选择行为+风险回归)——model_defaults 表+API+resolver 全在,但目前无代码路径真正调用 `resolve_with_fallback`;要激活 failover 需另开收尾把消费点接上 resolver(或明确它是 opt-in 基建)。(2)**url_guard 加了 `allow_private`**(共享审计模块 AUDIT B3,向后兼容 default False,但跨切改动 review 留意)。(3)local_processor 未收编(ollama 协议+timeout 旋钮不匹配 adapter)。(4)前端交互需真实全栈点击验证。(5)`BROWSER_ACT_API_KEY`... 属 GOAL-7 无关。 + +## 每 PR 验收(DoD) + +1. `tests/unit` + `tests/integration` + `tests/skills` 全绿(不低于 PR-A 前基线) +2. 老路径行为零回归(尤其 PR-E —— 改的是生产 chat/pipeline 路径) +3. commit 仅码+测路径,`git status --porcelain` 自检无 `GOAL*.md`/`HANDOFF*.md`/`AUDIT*.md`/`GRILL*.md`/`PR-DESCRIPTION.md` +4. 勾掉本文件对应项 + 一行进度(commit hash + 测试数) + +## 停止条件(真分叉才停,别瞎猜) + +- 全 PR 完 +- 开工时 main 没有 GOAL-5 taxonomy 提交(前置未满足) +- pytest 红且 2 轮内修不动 +- url_guard 对本地地址(ollama/model-hotel)没有既有豁免机制(决策 #6) +- main 合并后发现 provider 相关结构与本设计冲突(比如别的分支也动了 ModelProvider) +- 需要 push(push 永远等用户) + +## 后续排队(不在本 goal 内,已定顺序,未来各开一个 GOAL-N) + +参照 https://github.com/langgenius/dify 组件功能,依赖链顺序: +1. ~~模型 Provider 管理~~(本 goal) +2. Workflow 编排引擎 +3. 插件市场 +4. App 发布机制 diff --git a/GOAL-7.md b/GOAL-7.md new file mode 100644 index 0000000..019c686 --- /dev/null +++ b/GOAL-7.md @@ -0,0 +1,97 @@ +# GOAL-7 — 内置 browser-act 采集包(vendor SKILL.md 包 + browser-act channel) + +> `/loop` 自驱。每轮读本文件 → 下个未 [ ] 项 → 端到端做+测绿 → auto-commit → 勾掉。 +> 命中真分叉 → 停问,别自己拍板架构。 +> 每 PR 绿即 auto-commit(显式 `git add <路径>`,绝不 `add -A`,绝不碰 `GOAL*.md`/`HANDOFF*.md`/`AUDIT*.md`/`GRILL*.md`/`PR-DESCRIPTION.md`);push 仍等用户。 + +## 背景 + +内置 https://github.com/browser-act/skills(MIT)—— BrowserAct 的浏览器自动化 CLI + ~30 个站点采集 SKILL.md 包(taobao/amazon/google-maps/youtube/reddit/微信/知乎 等,分 ecommerce/lead-generation/search-research/social-listening/video-platforms 五类)。作为 opencli-admin 的一个新采集 channel + vendored 包目录。 + +**上游仓三块**(参考克隆在 scratchpad,已看过): +- `browser-act` CLI —— 外部工具(`uv tool install browser-act-cli --python 3.12`),提供 session 制原语 `navigate`/`wait`/`eval`/`state`/`click N`/`input N`;`browser-act get-skills core` 出环境态。**本 goal 不 vendor CLI 本体**(它是外部 PyPI 工具),只 vendor 包 + 写 channel 壳调它。 +- `solutions/` ~30 个包 —— 每个 = `SKILL.md`(散文运行手册,给 agent 读)+ `scripts/*.py`(纯 JS 发射器:argparse → `print(js字符串)`,无 LLM/无网络/无文件读写)。 +- `browser-act-skill-forge`(包生成器)—— **本 goal 不碰**,YAGNI。 + +**命名雷(GOAL-5 已警告"别混")**:opencli-admin main 上现有 `backend/skills/` + `Skill(domain, capability)` DB 表是**完全不同的东西**(record→distill→执行环,DB 存储,无文件包)。browser-act 的 SKILL.md 是**文件包**。本 goal 的 vendored 包与现有 DB Skill 子系统**完全隔离**,不共用表、不共用 `skill_channel`,不写 pack→DB 导入器(架构决策 #2)。 + +**执行契约(已实测确认)**:SKILL.md 的确定性部分 = `navigate {url}` → `wait stable` → `eval "$(python scripts/x.py {params})"` → 收 JSON。散文里的 `$(...)` 是 shell 替换语法(注入风险);本 channel **绝不走 shell**,两跳都 argv-only(见架构决策 #6)。登录/反爬是散文里的 LLM/人工部分,本 channel 故意不做,遇到就 fail loudly(架构决策 #4)。 + +## 坐标 + +- repo `D:\projects\opencli-admin`,新分支 `feat/browser-act-channel`,**从 main 切**(main 有 channel 子系统全套:`AbstractChannel`/registry/`cli_channel`/`opencli_channel`/`crawl4ai_channel`;本 worktree 的 `feat/agent-access-taxonomy` 没有,别在这切) +- 排期:GOAL-6(模型 Provider)先跑;本 goal 正交于 Provider,选了 script-runner 不依赖 GOAL-6,可并行/后排 +- 上游参考克隆:scratchpad `browser-act-skills/`(实施时重新 `git clone --depth 1` 拿最新,别信旧副本) +- 测试闸:`uv run pytest tests/unit tests/integration tests/skills --no-cov -q`(PowerShell,`cd D:\projects\opencli-admin`;Bash 工具在此环境被 RTK hook 改写会炸) +- 基线(feat/browser-act-channel 从 main edbb1ca 切,PR-A 前实测):**1430 passed, 12 failed, 6 skipped**。12 failed 全是 main 既有(opencli channel/workflow/nodes-install + 4 个 `tests/skills/*_live.py` 需真 Chrome),与本 goal 无关,DoD"全绿"=不新增失败 + +## 已锁定的架构决策(别重新问,直接照做) + +1. **执行模型 = 确定性 script-runner,不引 LLM**。channel 按 manifest 步骤序列驱动 browser-act 子进程:`navigate` → `wait` → 跑 `python scripts/x.py ` 拿 JS → `browser-act ... eval ` → 解析 JSON → 收 items。无 perceive/gate/act 环,不碰 ModelProvider。(LLM-runner 是未来可选二期,本 goal 不做。) + +2. **与现有 DB Skill 子系统完全隔离**。vendored 包是**文件**,不进 `skills` 表,不复用 `backend/skills/`(那是 DB 的 record→distill 系统)。新建独立包目录 + `PackCatalog`(扫目录)。不写 pack→Skill 导入器(两种格式强映射阻抗大,YAGNI)。命名统一用 "pack"/"browser_act_pack" 而非 "skill",避免与现有概念撞名。 + +3. **vendored 包目录**:`backend/browser_act_packs///`,内含**原样** `SKILL.md` + `scripts/*.py`(不改上游内容,作出处 + 人类参考 + 上游 `git pull` 刷新)+ **我们新写的** `channel.manifest.json`(机器可读执行契约,见 #5)。保留上游 `LICENSE`(MIT)+ 顶层 `backend/browser_act_packs/VENDOR.md` 记来源 commit/URL/署名(MIT 合规)。 + +4. **登录/反爬 = fail loudly**,不自动化。scripts 吐的 JSON 若含 `{error: true, message: "...login..."}` 或页面判据失败,channel 返回 `ChannelResult(success=False, error_type="needs_human", error=<原因>)`,不吞、不猜、不重试绕过。这是采集边界=用户手动能看到的数据(照抄上游 SKILL.md 的"operational boundary"声明),不越权破鉴权。 + +5. **`channel.manifest.json` schema**(我们新写,每包一份):`{domain, capability, param_schema:[{name, required, default, enum?}], steps:[{op: "navigate"|"wait"|"eval_script"|"click"|"input", ...}], pagination:{mode, url_template?, page_param?, stop_when?}, success:{min_count, required_field?}}`。channel = 这份 manifest 的通用解释器,不为每包写死代码。手写 SKILL.md 散文→manifest 的翻译过程记进 VENDOR.md。 + +6. **子进程安全**(复用 `cli_channel`/`opencli_channel` 的 `asyncio.create_subprocess_exec` + `wait_for(timeout)` + `TimeoutError→kill()` 模式,别新造): + - **两跳全 argv-only,绝不 shell**:`python scripts/x.py ...`(argv 列表)拿 stdout JS;再 `browser-act ... eval `(argv)。用户参数经 argv 传,永不字符串插值进 shell/JS 模板。 + - browser-act 二进制走专属 env `BROWSER_ACT_BIN`(默认 `browser-act`),照 `opencli_channel` 的 `OPENCLI_BIN` 做法(固定二进制,无需 `CLI_CHANNEL_ALLOWED_BINARIES` 那种任意二进制 allowlist)。 + - vendored 包 = 信任边界(vendor 时人工审 + git 钉死);v1 用户**不能**上传/新增任意包(那是 skill-forge,不在本 goal)。 + - 超时:navigate/eval 每步 env `browser_act_timeout`(默认 120s,进 `backend/config.py` Settings,与现有 `opencli_timeout` 同款)。 + +7. **browser mode / 凭证**:server 端默认 `chrome-direct`(CDP,免 signup);`stealth` 模式需 BrowserAct API key,存 `SourceCredential`(加密,复用 `AuthManager.store/resolve`,别明文别新造凭证系统),key_name=`browser_act_api_key`。channel `collect()` 前经 `AuthManager.resolve` 取 key 注入子进程 env,错误响应绝不回显 key。 + +8. **channel 契约**:实现 `AbstractChannel`(`channel_type="browser_act"`,`collect(config, parameters)` + `validate_config(config)` + `health_check`)。`channel_config` schema:`{pack: "/" 或 domain+capability, params: {...}, mode: "chrome-direct"|"stealth", max_pages?}`。`validate_config` 校验 pack 存在于 PackCatalog + 必填 param 齐 + mode 合法。`health_check` = `browser-act --version` 子进程探活。`capabilities = Capabilities(paginated=True, session_affinity=True)`(browser-act session 有状态)。 + +9. **前端上架**(照 main 现有硬编码模式,别新造 API):`browser-act` 加进 `frontend/lib/api/types.ts` 的 `channel_type` union + `frontend/app/(app)/sources/page.tsx` 的 `CHANNEL_LABEL`(中文标签"浏览器采集/BrowserAct");preset 从 `PackCatalog`(新 `GET /api/v1/browser-act/packs` 端点)拉包列表填一键配置。 + +10. **CLI agent 化摩擦(记录,不硬解)**:上游 browser-act SKILL.md 要求"每条命令前先 `get-skills core`、别截断输出"—— 那是给 agent 的指令。本 channel 把 CLI 当受控子进程(固定 session、我们管生命周期),按需在 session 开头调一次 `get-skills core` 取环境/browser 选择态,不把它当交互 agent。若实测发现 CLI 强依赖交互确认(browser 创建确认门)无法非交互跑通 —— **停问**,别硬灌 yes。 + +## 状态机 + +- [x] **PR-A — vendor 包 + PackCatalog**(1430→**1448 passed**,+18 新测,零回归)。上游 `a23131e`(browser-act/skills)`solutions/**` 原样 vendor 进 `backend/browser_act_packs///`(**78 个包** 194 文件,Get-FileHash 逐文件核对 0 mismatch,非"~30"—— 上游实际 78,决策已用 `>=20` 断言不写死)+ `LICENSE`(MIT)+ `VENDOR.md`(commit/URL/署名)+ `SOLUTIONS-README.md`。`catalog.py`=`PackCatalog`(rglob SKILL.md,YAML frontmatter 取 name/description,domain=category/capability=pack 目录名从布局派生;**发现并修**:`social-listening/reddit-warmup/SKILL.md` 带 UTF-8 BOM 致 `startswith("---")` 漏包,改 `utf-8-sig` 读,只动 catalog.py 非 vendored 字节)。`manifest.py`=`PackManifest` schema(决策 #5)+`load_manifest`,无 manifest 内容(留 PR-D)。测试 `tests/unit/browser_act_packs/`(catalog 扫 ≥20 + BOM/坏 frontmatter skip + get_pack + manifest 校验,18 测)。 + ~~验收:catalog 扫出 ~30 包(数量断言)、frontend 解析不崩、非法/缺 frontmatter 包被跳过并记警告的单测;`VENDOR.md` 存在且含 commit hash;基线测试数记录进本文件。~~ 全达成。 + +- [x] **PR-B — browser-act CLI 封装**(1448→**1467 passed**,+19 新测,零回归)。新包 `backend/browser_act/`(≠现有 `backend/cli.py` opencli HTTP client、≠ `browser_act_packs/`):`cli.py` = `_run`(`create_subprocess_exec` + `wait_for(timeout)` + `TimeoutError→kill()+wait()`,照 `cli_channel.py`)、`version()`/`get_skills()`、`session(name, env)` async ctx mgr → `BrowserActSession`(navigate/wait/eval/state/click/input/run,每条前置 `--session `)。binary 走 `BROWSER_ACT_BIN` env 调用时读(照 opencli `OPENCLI_BIN`),`browser_act_timeout=120` 进 Settings。**全 argv-only**(#6):仅 `create_subprocess_exec` 无 shell,用户值(URL/JS/input)恒单 argv 元素;`BrowserActError` 错误文本不含 env(secret 走 env 不入 argv/日志)。session `__aexit__` no-op(#10:browser open/close 归 PR-C,不对称清理会误拆调用方仍需的 session,已注释)。CLI 命令面已对上游 `docs/commands.md` 核实。测试 `tests/unit/browser_act/test_cli.py`(19 测:argv 精确断言、注入安全断言 `create_subprocess_shell` 从不被调+危险串单 argv 逐字、timeout kill、非零退出错误不含注入的 api_key、`BROWSER_ACT_BIN` 覆盖、真 `sys.executable` 往返)。 + ~~验收:mock 子进程...参数含 shell 元字符时不注入的测试(断言走 argv 非 shell)。~~ 全达成。 + +- [x] **PR-C — BrowserActChannel + manifest 解释器**(1467→**1484 passed**,+17 新测,零回归,全量已跑验)。`backend/channels/browser_act_channel.py`=`BrowserActChannel(AbstractChannel)` `channel_type="browser_act"` `@register_channel`,`Capabilities(paginated=True, session_affinity=True)`;通用 manifest 解释器(无 per-pack 码,#1/#5):resolve pack(PackCatalog)→`load_manifest`→逐 step 驱动(navigate/wait/eval_script/click/input),`eval_script`两跳=`run_pack_script`(新 `backend/browser_act/scripts.py`,`create_subprocess_exec(sys.executable,...)` argv-only #6,`ScriptError`)拿 JS→`sess.eval`拿 JSON。**登录/反爬**(#4):JSON `{error:true,message}` 经 `_classify_error` 命中 auth 关键词(login/captcha/verify/登录/验证/人机…)→`error_type="needs_human"`立即停不重试;其余 error 归 `"error"`。分页仅解释 `url_page`+`result_count ✅ **GOAL-7 完成**(2026-07-08):PR-A→PR-E 全落,1430→**1504 passed**(1448/1467/1484/1492/1504 逐 PR 过点),零回归(既有 12 failed 全 main 遗留:opencli/workflow/nodes-install + 4 `*_live.py` 需真 Chrome,与本 goal 无关)。`d72c0e8..` 共 5 commit,分支 `feat/browser-act-channel`(从 main `edbb1ca` 切)。**未 push**(push 等用户)。vendor 78 包(byte 不改)+ browser_act CLI 封装 + BrowserActChannel manifest 解释器 + 2 seed manifest + 凭证/端点/前端。**留白**:~76 包无 manifest(schema 是扩展点,增量补)、api-skill 包异形态(脚本直打 API 无 browser session)本 interpreter 不建模、google 多页需 `start=(page-1)*num` 算术当前不支持、interpreter 不 URL-encode 参数、前端无 source 创建 UI 故 packs hook 暂未消费、`BROWSER_ACT_API_KEY` env 名是假设(上游无文档)。 + +## 每 PR 验收(DoD) + +1. `tests/unit` + `tests/integration` + `tests/skills` 全绿(不低于 PR-A 前基线) +2. 老路径行为零回归(尤其 PR-C/E —— 动 channel registry + API 路由) +3. commit 仅码+测+vendored 包路径,`git status --porcelain` 自检无 `GOAL*.md`/`HANDOFF*.md`/`AUDIT*.md`/`GRILL*.md`/`PR-DESCRIPTION.md` +4. 勾掉本文件对应项 + 一行进度(commit hash + 测试数) +5. vendored `SKILL.md`/`scripts` 内容零改动(git diff 自检:只新增,不改上游文件字节) + +## 停止条件(真分叉才停,别瞎猜) + +- 全 PR 完 +- 从 main 切分支时 channel 子系统结构与本设计冲突(别的分支也动了 registry/AbstractChannel) +- browser-act CLI 强依赖交互确认门,非交互跑不通(#10) +- pytest 红且 2 轮内修不动 +- seed 包全都需登录、无一能在无凭证下端到端验证(PR-D 选包卡住) +- 需要 push(push 永远等用户) + +## 明确不做(YAGNI 裁决,别加) + +- 不 vendor browser-act CLI 本体(外部 PyPI 工具) +- 不做 skill-forge(用户上传/生成任意包) +- 不做 LLM-runner(登录/反爬自愈) +- 不写 pack→DB Skill 表导入器 +- 不改上游 SKILL.md/scripts 字节 +- 不给全 ~30 包写 manifest(seed 2-3 个验证管道,余量增量补) diff --git a/GOAL-agent-runtimes.md b/GOAL-agent-runtimes.md new file mode 100644 index 0000000..94775d3 --- /dev/null +++ b/GOAL-agent-runtimes.md @@ -0,0 +1,146 @@ +# GOAL: Pluggable Agent Runtimes on Fleet Edge Nodes + +Status: PROPOSAL (2026-07-03) — research done, design settled, awaiting implementation green-light. +Owner intent (user, 2026-07-03): the edge agent (Docker agent / agent_server.py) should be able to +run agentic workflows built on multiple frameworks — LangGraph, VoltAgent, pi — behind ONE +decoupled, reusable abstraction. Framework choice must never leak past the adapter boundary. +Reference patterns: OpenAlice `feature/openalice-dev` (`src/workspaces/` CLI-adapter layer). + +--- + +## 1. Why process-level, not library-level + +The three frameworks span two runtimes: + +| Framework | Lang | Native external-invocation surface | Weight | +|---|---|---|---| +| LangGraph (langchain-ai) | Python | `langgraph dev` local HTTP server — Assistants/Threads/Runs API + SSE (open-source, not Platform-gated) | heavy (langchain-core stack) | +| VoltAgent | TS/Node | embedded REST server (Hono/Elysia): `POST /agents/:id/stream` SSE, OpenAPI 3.1 spec | moderate (Node + ai-sdk) | +| pi (earendil-works) | TS/Node | `--mode rpc`: stdio JSONL RPC, purpose-built for subprocess embedding; `--mode json`; `-p` one-shot | lightest (pinned npm shrinkwrap) | + +A Python `import`-based abstraction can only ever cover LangGraph. Therefore the adapter contract +is a **process/protocol contract**: each runtime runs as a subprocess or local sidecar in its own +env (venv / node_modules), and the adapter translates its native stream (SSE / JSONL) into one +normalized event set. This is exactly OpenAlice's proven split: subprocess CLI adapters +(`workspaces/`) kept separate from in-process SDK providers (`ai-providers/`) — different failure +modes, different contracts. We build the subprocess layer. + +## 2. Core contract (new module `backend/agent_runtimes/`) + +Mirrors `backend/channels/` conventions (AbstractChannel / Capabilities / registry decorator): + +```python +@dataclass(frozen=True) +class RuntimeCapabilities: + transport: str # "stdio" | "http" + streaming: bool = True + resume_by_id: bool = False # can reopen a session by launcher-assigned id + checkpoint: str = "none" # none | memory | sqlite | postgres + concurrent_sessions: bool = True + +@dataclass +class AgentTask: + task_id: str + workflow: str # runtime-native workflow/agent identifier + input: dict[str, Any] + config: dict[str, Any] # runtime-specific (model, tools, cwd, ...) + session_id: str | None = None # resume handle + +# Closed tagged-union event set — adapters normalize INTO this, callers never +# see framework-native shapes. (OpenAlice lesson: normalize the protocol, not +# the output; keep the set tiny and closed.) +RuntimeEvent = {"type": "started" | "text" | "tool_call" | "tool_result" + | "state" | "done" | "error", ...} + +class RuntimeAdapter(ABC): + runtime_type: str # "pi" | "langgraph" | "voltagent" + capabilities: RuntimeCapabilities # data flags — callers branch on these, never isinstance + + @abstractmethod + async def invoke(self, task: AgentTask) -> AsyncIterator[dict]: ... # yields RuntimeEvents + @abstractmethod + async def health(self) -> bool: ... + @abstractmethod + def validate_config(self, config: dict) -> list[str]: ... + async def bootstrap(self) -> None: ... # one-time env/config setup (OpenAlice bootstrap()) +``` + +Registry: same decorator pattern as `channels/registry.py`; discovery import in `__init__`. + +Split-by-concern composition (from OpenAlice `CliAdapter`): argv composition, env composition, +provider-config translation, and session-id acquisition are separate small methods on stdio +adapters — never one monolithic `spawn()`. + +Session identity ⊥ process lifetime (OpenAlice registry/pool split): a small durable +`RuntimeSessionRegistry` (session_id, runtime_type, resume hint, state) survives agent restarts; +the ephemeral process/connection pool does not. + +## 3. Per-runtime adapters + +- **`pi`** (P0, first): subprocess `pi --mode rpc`, LF-delimited JSONL over stdio. Same shape as + our existing shell-out pattern — least new plumbing. Steal from OpenAlice `adapters/pi.ts`: + `--session-id` create-or-reopen (launcher assigns ids → resume_by_id=True), provider override + via `PI_CODING_AGENT_DIR` redirect, skills injection into `/.pi/skills`, tools via + CLI-shim-on-PATH since pi speaks no MCP. +- **`langgraph`** (P1): local sidecar server (open-source `langgraph-api`, the `langgraph dev` + machinery) on a loopback port; adapter = httpx client speaking Assistants/Threads/Runs + SSE. + Best checkpoint story (Postgres/SQLite checkpointers) → the runtime for long/resumable + workflows. Sidecar lifecycle owned by our supervisor (below). +- **`voltagent`** (P2): Node sidecar with `@voltagent/server`, REST+SSE per its OpenAPI 3.1 spec. + Bespoke schema (no A2A/OpenAI-compat confirmed) → adapter does the translation, nothing else does. + +Sidecar supervisor in agent_server: spawn on first use, health-probe, restart-with-backoff, +SIGTERM watchdog + kill grace (OpenAlice headless-task pattern; plain subprocess, never PTY — +PTY mangles JSON streams). + +## 4. Wire protocol extension (center ⇄ edge) + +Today: ws reverse channel carries `collect` → single `result` (request/response, +`ws_agent_manager.resolve_response`). Agent runs are long and streaming: + +- New message types: `agent_task` (center→edge), `agent_event` (edge→center, many, + carries request_id + one RuntimeEvent), final `agent_result`. +- `ws_agent_manager` grows a per-request event callback/queue alongside the existing + single-shot future (existing collect path untouched). +- **Runtime advertisement**: register handshake gains `runtimes: ["pi", ...]` — the center + learns node capabilities the same way it learns mode/node_type today; scheduler can route + agent tasks only to nodes advertising the runtime (analog of `session_affinity`). +- Fleet auth: already covered — ws handshake carries the bearer token (8fab4fe). + +## 5. Center side + +- Thin `agent_channel` (AbstractChannel impl): declares capabilities, one fetch() = one agent + run dispatched via the reverse channel; runner keeps owning retry/rate/cursor. Agent runs + emit the same evidence (accepted/error_kind → SourceMeasurement) so the control loop + (PR-Control-*) covers agent tasks with zero new sensor machinery — agent runtimes are just + another 被控对象 class with different observability/controllability. +- MCP callback surface (P1+): expose center MCP endpoint to edge runtimes so agents report + structured results by calling back (OpenAlice `inbox_push` inversion) instead of us parsing + heterogeneous stdout. opencli-admin already ships `backend/mcp_server.py` — reuse. + +## 6. Docker image layering + +Base image = agent_server only (unchanged). Runtimes are opt-in layers/build-args: +`INSTALL_RUNTIME_PI=true` (adds node + pinned pi), `INSTALL_RUNTIME_LANGGRAPH=true` +(pip layer), `INSTALL_RUNTIME_VOLTAGENT=true` (node layer). Image advertises what it has +via the register handshake — no config drift. LLM API keys: node-local env first (P0); +center-side encrypted distribution later (provider api_key store already exists, `06684a7`). + +## 7. Phasing + +- **P0**: contract + registry + pi adapter (stdio) + ws `agent_task`/`agent_event` protocol + + runtime advertisement + tests. Proves the seam end-to-end with the lightest runtime. +- **P1**: LangGraph sidecar adapter + sidecar supervisor + center `agent_channel` + + session registry (resume) + MCP callback. +- **P2**: VoltAgent adapter + control-loop evidence integration + credential distribution + + UI (node runtime badges on the topology canvas). + +## 8. Non-goals / rejected + +- In-process Python embedding of LangGraph as the primary path (couples versions, blocks + TS runtimes from ever being first-class; kept possible later as an optimization behind the + same adapter contract). +- Normalizing framework semantics (graph vs supervisor vs tool-loop) — we normalize only the + run/stream/result protocol; workflow definitions stay runtime-native in `AgentTask.config`. +- A2A/AG-UI as the wire format now — only LangGraph speaks AG-UI today; revisit if a second + runtime adopts it. diff --git a/GOAL.md b/GOAL.md new file mode 100644 index 0000000..c6a909d --- /dev/null +++ b/GOAL.md @@ -0,0 +1,76 @@ +# GOAL — opencli-admin strangler-fig 自动环 + +> `/loop` 自驱目标文件。**每轮重读本文件** → 取下个未完 PR → 端到端做(码+测) → +> pytest 绿闸 → 自动 commit(仅列出路径) → 勾掉状态+记一行进度 → 下一轮。 +> 命中任一停止条件 → 停+报，**别瞎猜**。 + +--- + +## 北极星 +接一个正经数据源 ≈ 100 行，只写它独有的「发一次请求 + 解析成条目」。 +横切脏活(刷 token / 翻页 / 限速 / 存游标 / 写目的地)全归框架，渠道不碰。 +迁移 = **Strangler Fig**：旧路径不破坏、新路径旁路验证、逐源切主路。 +**绝不为新架构破坏旧行为。** + +## 坐标 +- repo: `D:\projects\opencli-admin` 分支: `refactor/thin-channel-thick-runner` +- 测试闸: `uv run pytest tests/unit --no-cov -q`（须全绿，当前基线 347） +- 跑测试用 PowerShell（`cd D:\projects\opencli-admin; uv run ...`）；Bash 被 RTK hook 改写易炸。 +- ⚠️ **永不 stage**: `backend/api/v1/chat.py`、`PR-DESCRIPTION.md`、`HANDOFF-strangler-fig.md`、`GOAL.md`（用户 dock WIP + 本控制文件）。 + +## 提交策略（用户 2026-07-01 授权，仅限此 goal） +每 PR 绿即**自动 commit**。granular，一 PR 一 commit。提交前**显式 `git add <精确路径>` + `git diff --cached --name-only` 自检**，绝不 `git add -A`/`add .`。**push 仍等用户**。 + +--- + +## 状态机（每轮更新） +- [x] **PR1** — LegacyDbSink 写缝（`b33416a`，行为零变） +- [x] **PR2** — 锁旧 ODP 契约 + `backend/odp/`{schemas,mapper} + odp_client 走 mapper（`532291b`） +- [x] **PR3** — ODP forward→`OdpSink` + `LegacyDbSink(forward_to_odp)` gate + `DualSink` 不双发（`74ac704`，355 passed） +- [x] **PR4** — `write_strategy` 状态机→选 sink + column/migration `o5j6k7l8m9n0`（`db10450`，365 passed） +- [x] **PR5a** — DB cursor 表+migration `p6k7l8m9n0o1` + `DBCursorStore` + RSS `fetch()` etag/304 增量 + `identity()`=item id（`9cbfb80`，374 passed；纯加性，未碰 live pipeline） +- [x] **PR5b** — collect `collect()`→`run_channel()`(incremental opt-in) + cursor 后置 commit(sink durable 后)（`09e4860`，379 passed） + +> ✅ **GOAL 完成**（2026-07-01）：strangler-fig 重构 PR2→PR5b 全落，325→**379 passed**，零回归，每刀行为零变。**未 push**（push 等用户）。范围外后续：AuthManager+加密凭据、session affinity 泛化。 + +### ✅ PR5b 分叉（已决议 2026-07-01：安全切——仅 incremental opt-in + cursor 后置 commit） +1. **路由范围**：(a) 仅 `capabilities.incremental` 渠道走 run_channel(opt-in strangler，RSS先)；(b) 全渠道切 run_channel；(c) 仅 RSS 显式特判。 +2. **cursor 前进时机**：现 `run_channel` 每页 fetch 后立即 save(channel_runner.py:82-84)。规则要「只在进可靠写入层才前进」→ 需重构：(a) run_channel 返回 (items, pending_cursor)，pipeline sink 写成功后才 commit cursor；(b) 把 sink 注入 run_channel，每页写+commit(保翻页 resumability 但耦合 runner↔sink)。 +3. **durability 判据**：odp_only/odp_primary 下「durable」= ODP 真落(Redis Stream queued 算不算？)；memory-only 假 202 不能让 cursor 前进。 + +## 每 PR 验收（DoD） +1. 全 `tests/unit` 绿（≥ 上一基线） +2. 旧路径行为零变 / 新路径有 characterization 或新测护栏 +3. commit（仅码+测路径，自检 staged 集） +4. 更新本文件状态框 + 一行进度 + +## 停止条件（任一 → 停+报） +- 全 PR 完 +- pytest 红且 2 轮内修不动 +- **真分叉**：设计有多条不等价路 / 必须破坏旧行为 / 缺凭据或外部依赖 / 要碰 WIP 文件 +- 需要 push（push 永远等用户） + +--- + +## PR 详细规格（出自 HANDOFF §4 + memory `opencli-admin-channel-runner-refactor`） + +### PR3 — OdpSink + 双发陷阱解 +- **双发陷阱**：`storer.py:34-45` 在 `ODP_INGEST_URL` 设了时已 forward 到 ODP（上游既有 shadow）。所以 `LegacyDbSink` 现含此 forward。将来 `DualSink(LegacyDbSink+OdpSink)` 会**双发** → 污染 shadow。 +- **做**： + 1. `LegacyDbSink(forward_to_odp: bool = True)` 加 gate；storer 的 forward 受此控制（默认 True = 行为零变）。 + 2. 新 `backend/pipeline/sinks/odp_sink.py` `OdpSink`：normalize → `odp_client.forward_triples`（复用 PR2 mapper），`SinkResult.records=[]`（forward-only，AI/notify no-op）。accepted=queued、duplicates、rejected 按 ODP 响应。 + 3. 新 `DualSink(legacy=LegacyDbSink(forward_to_odp=False), odp=OdpSink)`：legacy 写 DB（不 forward）+ OdpSink 发**一次**；ODP 失败不阻断 legacy。 +- **验收**：legacy 模式同 PR1；odp_shadow=legacy 写 DB + ODP 发一次（非两次）；ODP 失败 legacy 照常。 + +### PR4 — write_strategy 状态机 +- `data_sources.write_strategy` ∈ {legacy / odp_shadow / odp_dual_required / odp_primary / odp_only} → 选 sink。 +- 一旦显式策略，ODP forward 不能再藏 storer 里（PR3 已把它收进 sink）。 +- 选 sink 的工厂 + pipeline 注入点（`run_pipeline(sink=)` 已存在）。 + +### PR5 — RSS 真实竖切（含原 Phase 1b） +- `source_cursors` 表 + alembic migration + `DBCursorStore`（实现 PR1a 的 `CursorStore` Protocol）。 +- `RSSChannel.fetch()` 走 etag/If-None-Match 增量（304 = 无新条目）；`identity()` = item id。 +- **规则**：cursor **只在数据进了可靠写入层才前进**；prod 的 odp-ingest 不能 memory-only 假 202。 + +## 其后（不在本 goal 范围，到此停） +AuthManager + 加密凭据（堵 `channel_config` 明文 key）；会话亲和 `pipeline.py:45-56` 特判 → `Capabilities.session_affinity` 泛化 + 按域名并发上限。 diff --git a/GRILL-KICKOFF.md b/GRILL-KICKOFF.md new file mode 100644 index 0000000..6f58e22 --- /dev/null +++ b/GRILL-KICKOFF.md @@ -0,0 +1,49 @@ +# Grill Kickoff — opencli-admin 收口定调 + +> 用法:新 session,cwd=`D:\projects\opencli-admin`,跑 `/grill-with-docs`,把本文件当输入喂进去。 +> 目的:对齐收口目标 → 裁剪残留清单 → 出 CONTEXT.md/ADR 收口宪法。收口 = 裁剪,不是做完。 + +## 现成输入(盘问前先读) + +- `AUDIT-cybernetic-remediation.md` — 控制论审核残留账 +- `docs/CONTROL_THEORY_ARCHITECTURE.md` — 控制回路架构 +- Memory: `opencli-admin-cybernetic-audit`(Control-4 硬规格:recovery rate 阈值 per-state gate automatic mode)、`opencli-admin-channel-runner-refactor`(留白清单) + +## 已定事实(不许在盘问里重新翻案) + +- PR-Control-3(advisory 决策引擎)+ 3.5(证据台账 `control_actions` + outcome 判定 + recovery 报表)已推 fork = 4f3b2fe +- 控制回路语义已定:Advisory-Gated Automatic Execution;recovery 阈值 per-state 门禁 automatic +- PR#4(薄渠道+厚 runner)已合 main = f731897 +- 回测/画布等其他摊子与本收口无关 + +## 盘问必须逼出答案的问题 + +1. **收口线定义**:Control-4 actuator 落地算完?还是 advisory 攒证据阶段就封版?证据要攒几天(天然分界)? +2. **残留裁剪**(每项:进收口 / 弃权写 ADR): + - crawl4ai call-time SSRF + - per-source objective 存储 + - odp-store 心跳 producer(Rust) + - error_kinds histogram + - trend fallback + - 前端 3 处遗留 / CLI ctrl+c / opencli_channel 路由(channel-runner 留白) +3. **悬而未决**: + - 部署面:纯本机 vs LAN(→鉴权 P1/P2) + - gitea push 凭证 + - topology ODP 节点 +4. **每项验收标准**:测试断言级,不是"做了"。 + +## 收口后流程(同一窗,不 compact 不断窗) + +grill → `/to-prd` → `/to-issues`。多 session build(Control-4 单 PR 都嫌大)。 + +## Issue 模板硬规则(写死进每个 issue,喂 Sonnet 5 子 agent) + +- 契约 pin 死:endpoint schema、复用 `control_actions` 表(mode=automatic / executed=True)、零变异测试:原断言不许动 +- HARD RULE:禁 Agent 工具、禁 commit(主模型验收后统一 commit) +- 验收闸门写进 issue:全量 pytest + cov≥80 + alembic 单 head +- 复用现成机制,不手搓(memory: feedback-reuse-wheels) + +## 不走的岔路 + +- `/triage` 不用(issue 全自产) +- `/prototype` 不用(控制回路语义已定,没有跑起来才能答的问题) diff --git a/HANDOFF-strangler-fig.md b/HANDOFF-strangler-fig.md new file mode 100644 index 0000000..0de2b97 --- /dev/null +++ b/HANDOFF-strangler-fig.md @@ -0,0 +1,88 @@ +# HANDOFF — opencli-admin 渠道系统 Strangler Fig 重构 + +> 用法：开一个**新 session**（本窗已过 smart zone），先读这份文件 + 读 memory +> `opencli-admin-channel-runner-refactor`，然后从 **PR2** 接着干。一口气做完一刀再 commit。 + +--- + +## 0. 一句话北极星 + +接一个新数据源 ≈ 100 行，只写它独有的「发一次请求 + 解析成条目」。刷 token / 翻页 / 限速 / 存游标 / 写目的地 —— 全归框架，渠道不碰。 + +迁移打法 = **Strangler Fig**：旧路径不破坏、新路径旁路验证、逐源切主路。**绝不为新架构破坏旧行为。** + +--- + +## 1. 坐标 + +- 机器：5090，`D:\projects\opencli-admin`（FastAPI Python） +- 分支：`refactor/thin-channel-thick-runner` +- remotes：`origin`=xjh1994(上游，**零接触**)、`fork`=2233admin(我们的，已推)、`gitea`=Curry 镜像 +- 测试：`uv run --directory D:\projects\opencli-admin pytest tests/unit --no-cov -q`（现 **325 passed**） +- ⚠️ 同仓 `backend/api/v1/chat.py`(M) + `PR-DESCRIPTION.md`(??) = 用户 dock WIP，**不碰、不提交、不 stage**。 + +--- + +## 2. 已落（提交在分支上，已推 fork） + +| commit | 内容 | +|---|---| +| `b33416a` | **PR1** — LegacyDbSink 写缝（行为零变） | +| `56fa0c4` | Phase 1a — 厚 runner 地基（cursor store + 限速重试 client + 翻页） | +| `de52c25` | Phase 0 — 加厚渠道契约（Capabilities/FetchContext/FetchResult） | + +「Phase 0/1a」= runner 层（`channel_runner.py`/`cursor_store.py`/`http_client.py`），仍有效复用。 + +### PR1 装了什么（写缝） + +`backend/pipeline/sinks/`： +- `base.py` — `ItemSink` Protocol（一个方法 `write_batch(ctx, items) -> SinkResult`）+ `RunContext`(task_id/source_id/provider/ingest_mode/run_id) + `SinkResult`(accepted/duplicates/rejected/normalized/**records**/errors)。 + - `SinkResult.records` 回带 ORM 行 —— `pipeline.py` 后续 ai/notify 依赖它。**必须存在**，否则 PR1 不是行为零变。 + - accepted/duplicates/rejected 语义按**各 sink 自己的 durable 边界**写死（legacy accepted=已插入行 ≠ odp accepted=已入队）。 +- `legacy_db_sink.py` — `LegacyDbSink` 包现有 `normalizer.normalize_items` + `storer.store_records`，行为照旧。 +- `pipeline.py` — step2+3 改成 `active_sink.write_batch()`；`run_pipeline` 加 `sink=` 注入口（默认 LegacyDbSink）。 + +--- + +## 3. ⚠️ 双发陷阱（最大坑，PR3 必解） + +`backend/pipeline/storer.py:34-45` 在 `ODP_INGEST_URL` 设了时**已经 forward 到 ODP**（这是上游 fork 里既有的 shadow，不是我们加的）。 + +所以 **`LegacyDbSink` 现在不是纯 legacy** —— 它经 storer，带着这个 ODP forward。PR1 故意保留（行为零变）。 + +后果：将来 `DualSink(LegacyDbSink + OdpSink)` 会**对 ODP 双发** → 污染 shadow 对比指标。 + +解法（PR3）：`LegacyDbSink(forward_to_odp: bool = True)` 加 gate；DualSink 用 `forward_to_odp=False` + `OdpSink`。`legacy_db_sink.py` 已留 `TODO(PR3)` 在 storer 调用处。 + +--- + +## 4. 剩余路线（**顺序依赖，不是独立可抢** —— 别拆 /to-issues） + +- **PR2（下一刀）** = 锁旧契约 + mapper，**不搬 forward**： + 1. 读死 `storer.py` 当前 ODP forward 的 payload shape（经 `odp_client.forward_triples`）。 + 2. 给该 forward 加 **characterization test**：`ODP_INGEST_URL` 设了时 `store_records()` 会 forward，payload 与当前一致。锁旧行为，证明 PR3 搬迁前后等价。 + 3. 新增 `backend/odp/schemas.py`：`RecordEvent` / `OdpIngestResponse`，字段对齐 Rust `odp-rs/crates/odp-contracts`（RecordEvent v2）+ `IngestBatchResponse`(accepted/duplicates/rejected/errors)。 + 4. 新增 `RecordEventMapper`，输入**对齐 normalized record**（沿用现有 normalizer 结果），**别从 raw collector item 另起一套语义**（否则 legacy DB 字段语义 ≠ ODP payload 语义）。 + 5. **暂不搬** storer 的 forward；只把 mapper/client 备好。`backend/pipeline/odp_client.py` **已存在**（commit 97b8d93）→ 扩展，别重建。 + 6. 全 `tests/unit` 保持绿。 +- **PR3** = ODP forward 从 storer 搬进 `OdpSink` + `LegacyDbSink(forward_to_odp)` gate + DualSink 不双发。验收：legacy 模式同 PR1；odp_shadow=legacy 写 DB + ODP 发**一次**；ODP 失败不阻断 legacy。 +- **PR4** = `data_sources.write_strategy` 状态机(legacy / odp_shadow / odp_dual_required / odp_primary / odp_only)→选 sink。一旦显式策略，ODP forward 不能再藏 storer 里。 +- **PR5** = RSS 真实竖切。并入原计划「Phase 1b」：`source_cursors` 表 + alembic migration + `DBCursorStore`；`RSSChannel.fetch()` 走 etag/If-None-Match 增量(304=无新条目)；`identity()`=item id。规则：cursor **只在数据进了可靠写入层才前进**；prod 的 odp-ingest 不能 memory-only 假 202。 +- 其后：AuthManager + 加密凭据（堵 `channel_config` 明文 key）；会话亲和 `pipeline.py:45-56` 特判 → `Capabilities.session_affinity` 泛化 + 按域名并发上限。 + +--- + +## 5. 工作纪律（用户定，硬约束） + +- 回复中文、代码/路径/commit 英文；caveman 简洁。 +- **接到明确方向就端到端做完**（自己 build/test/真验证再交），中途不一步一问、不开菜单挑下一步；只真分叉才问。 +- **只在用户说 "commit" 时提交**；stage 时显式列路径，**绝不** stage `chat.py` / `PR-DESCRIPTION.md`。 +- 不向上游 PR，自己 fork 开发。「PR1/PR2」只是每刀的叫法 = 本地 commit。 + +--- + +## 6. 指针 + +- 路线 + 双发陷阱全文：memory `opencli-admin-channel-runner-refactor`（新窗自动 recall）。 +- odp-rs Rust 热路径子系统简报（给 GPT 调研）：本机 scratchpad `odp-rs-briefing.md`（2 二进制 odp-ingest:8040/odp-store + 3 crate contracts/bus/store；Redis Streams + Postgres odp_records；两层去重；缺 XAUTOCLAIM/DLQ/outbox）。 +- 关键源：`backend/pipeline/{pipeline,collector,normalizer,storer,odp_client,channel_runner,cursor_store,http_client}.py`、`backend/pipeline/sinks/`、`backend/channels/base.py`、`backend/models/{record,source}.py`。 diff --git a/backend/api/v1/__init__.py b/backend/api/v1/__init__.py index ef9655f..21a17a9 100644 --- a/backend/api/v1/__init__.py +++ b/backend/api/v1/__init__.py @@ -10,6 +10,7 @@ control, cookies, dashboard, + model_defaults, nodes, notifications, plan_ir, @@ -37,6 +38,7 @@ v1_router.include_router(chat.router) v1_router.include_router(control.router) v1_router.include_router(cookies.router) +v1_router.include_router(model_defaults.router) v1_router.include_router(nodes.router) v1_router.include_router(plan_ir.router) v1_router.include_router(plans.router) diff --git a/backend/api/v1/chat.py b/backend/api/v1/chat.py index faca62c..12b52eb 100644 --- a/backend/api/v1/chat.py +++ b/backend/api/v1/chat.py @@ -191,15 +191,51 @@ async def _pick_provider(db: AsyncSession, provider_id: Optional[str]) -> ModelP return provider -def _build_client(provider: ModelProvider): +async def _build_client(provider: ModelProvider): + """Build the agent dock's OpenAI-compatible tool-calling client. + + GOAL-6 PR-E: consolidates what used to be a private ``AsyncOpenAI(...)`` + construction here into :class:`~backend.llm.openai_compat.OpenAICompatAdapter` + via :func:`~backend.llm.factory.build_openai_compat_adapter` — the same + guarded client :class:`OpenAICompatAdapter` gives every other PR-E + consumer, so this file stops duplicating the SSRF-guard + DNS-rebind- + pinning wiring. The tool-calling loop below stays exactly as it was + (needs the *raw* client for ``tools=``/``tool_choice=``, which the + adapter's thin ``chat()`` doesn't support) — only client *construction* + moves. + + Preserved exactly: the ``OPENAI_API_KEY`` env fallback when the selected + provider has no ``api_key`` configured, and this file's pre-existing + behavior of treating ANY selected provider (regardless of + ``provider_type``) as an OpenAI-compatible endpoint — ``_pick_provider`` + never filtered by ``provider_type``, so neither does this. + + Deliberate, narrow behavior change (decision #6): the previous + ``_build_client`` had NO SSRF guard at all. Routing through + ``OpenAICompatAdapter`` now validates ``provider.base_url`` before + attaching the api_key to a client pointed at it — closing an SSRF/key- + exfil gap that already existed everywhere else (openai_processor, + skill_channel) but not here. No existing test exercises this path (see + ``tests/integration/test_chat_api.py``'s docstring: "the LLM round trip + itself is out of scope"), so this cannot regress the test suite; a + provider whose base_url fails the guard now gets a clear 502 instead of + an unguarded outbound call. + """ try: - from openai import AsyncOpenAI + from openai import AsyncOpenAI # noqa: F401 -- import-availability probe only except ImportError as exc: raise HTTPException(status_code=500, detail="openai package not installed") from exc import os + from backend.llm.base import LlmAdapterError + from backend.llm.factory import build_openai_compat_adapter + api_key = provider.api_key or os.environ.get("OPENAI_API_KEY", "") - return AsyncOpenAI(api_key=api_key, base_url=provider.base_url or None) + adapter = build_openai_compat_adapter(base_url=provider.base_url, api_key=api_key) + try: + return await adapter.get_client() + except LlmAdapterError as exc: + raise HTTPException(status_code=502, detail=f"模型调用失败: {exc}") from exc # ── 只读工具执行 ───────────────────────────────────────────────────────────── @@ -312,7 +348,7 @@ async def _build_proposal(db: AsyncSession, name: str, args: dict[str, Any]) -> @router.post("", response_model=ApiResponse[ChatReply]) async def chat(body: ChatRequest, db: AsyncSession = Depends(get_db)) -> ApiResponse: provider = await _pick_provider(db, body.provider_id) - client = _build_client(provider) + client = await _build_client(provider) model = provider.default_model or "gpt-4o-mini" system = SYSTEM_PROMPT diff --git a/backend/api/v1/model_defaults.py b/backend/api/v1/model_defaults.py new file mode 100644 index 0000000..765cb2a --- /dev/null +++ b/backend/api/v1/model_defaults.py @@ -0,0 +1,46 @@ +"""GET/PUT endpoints for ``model_defaults`` (GOAL-6 PR-C, decision #10). + +A top-level resource, not nested under ``/providers/{id}`` — a role's +candidate list can reference any provider, so it doesn't belong under one +provider's URL subtree. DB/validation logic lives in +``backend.services.provider_model_service`` (thin-endpoint convention). +""" + +from fastapi import APIRouter, Depends, HTTPException +from sqlalchemy.ext.asyncio import AsyncSession + +from backend.database import get_db +from backend.llm import VALID_ROLES +from backend.schemas.common import ApiResponse +from backend.schemas.model_default import ModelDefaultCandidatesBody, ModelDefaultRead +from backend.services import provider_model_service + +router = APIRouter(prefix="/model-defaults", tags=["model-defaults"]) + + +@router.get("", response_model=ApiResponse[list[ModelDefaultRead]]) +async def list_model_defaults(db: AsyncSession = Depends(get_db)) -> ApiResponse: + rows = await provider_model_service.get_defaults(db) + return ApiResponse.ok([ModelDefaultRead.model_validate(r) for r in rows]) + + +@router.put("/{role}", response_model=ApiResponse[ModelDefaultRead]) +async def put_model_default( + role: str, body: ModelDefaultCandidatesBody, db: AsyncSession = Depends(get_db) +) -> ApiResponse: + """Set (upsert) the ordered candidate list for ``role`` (index 0 = + primary, the rest are PR-D's failover order). + + ``role`` fails fast here (before any DB work) if it's outside the closed + set; each candidate's ``(provider_id, model_id)`` is validated against + real rows by ``provider_model_service.put_default`` — a candidate naming + a nonexistent provider or a model never registered in that provider's + catalog is rejected with a 400 naming exactly which candidate is bad. + """ + if role not in VALID_ROLES: + raise HTTPException(status_code=400, detail=f"invalid role: {role!r}") + try: + row = await provider_model_service.put_default(db, role, body.candidates) + except provider_model_service.ModelDefaultsValidationError as exc: + raise HTTPException(status_code=400, detail=str(exc)) from exc + return ApiResponse.ok(ModelDefaultRead.model_validate(row)) diff --git a/backend/api/v1/providers.py b/backend/api/v1/providers.py index 515d23b..0d45069 100644 --- a/backend/api/v1/providers.py +++ b/backend/api/v1/providers.py @@ -1,13 +1,24 @@ -"""CRUD endpoints for model providers.""" +"""CRUD endpoints for model providers, plus GOAL-6 PR-C's provider-scoped +API: test-connection, model-catalog sync, and model catalog CRUD (decision +#10). DB logic for the PR-C additions lives in +``backend.services.provider_model_service`` (thin-endpoint convention); this +module only does HTTP concerns (404 lookups, response shaping).""" from fastapi import APIRouter, Depends, HTTPException from sqlalchemy import select from sqlalchemy.ext.asyncio import AsyncSession from backend.database import get_db +from backend.llm.base import LlmAdapterError from backend.models.provider import ModelProvider from backend.schemas.common import ApiResponse from backend.schemas.provider import ModelProviderCreate, ModelProviderRead, ModelProviderUpdate +from backend.schemas.provider_model import ( + ProviderModelManualCreate, + ProviderModelRead, + ProviderModelUpdate, +) +from backend.services import provider_model_service router = APIRouter(prefix="/providers", tags=["providers"]) @@ -53,6 +64,102 @@ async def delete_provider(provider_id: str, db: AsyncSession = Depends(get_db)) provider = result.scalar_one_or_none() if not provider: raise HTTPException(status_code=404, detail="Provider not found") + # GOAL-6 PR-C (decision #3 / PR-A note): sqlite here never runs with + # PRAGMA foreign_keys=ON, so provider_models' ondelete=CASCADE never + # fires at runtime -- clean up the catalog explicitly or it orphans. + await provider_model_service.delete_provider_models(db, provider_id) await db.delete(provider) await db.commit() return ApiResponse.ok(None) + + +# --------------------------------------------------------------------------- +# GOAL-6 PR-C: test connection / model catalog sync + CRUD (decision #10) +# --------------------------------------------------------------------------- + + +@router.post("/{provider_id}/test", response_model=ApiResponse[dict]) +async def test_provider_connection( + provider_id: str, db: AsyncSession = Depends(get_db) +) -> ApiResponse: + """Probe the provider via its adapter (PR-B factory). Never echoes + ``api_key`` — ``ConnectionTestResult``/adapters guarantee that, not this + endpoint (see ``backend.llm.base.redact_secret``).""" + result = await provider_model_service.test_connection(db, provider_id) + if result is None: + raise HTTPException(status_code=404, detail="Provider not found") + return ApiResponse.ok(result) + + +@router.post("/{provider_id}/models/sync", response_model=ApiResponse[dict]) +async def sync_provider_models( + provider_id: str, db: AsyncSession = Depends(get_db) +) -> ApiResponse: + """Discover this provider's models and upsert them into its catalog + (decision #3: manual rows are never touched; stale discovered rows are + pruned — see ``provider_model_service.sync_models`` for the full policy). + A genuine discovery failure (connection error, bad key, ...) surfaces as + 502, not 500 — the adapter already sanitized the message.""" + try: + result = await provider_model_service.sync_models(db, provider_id) + except LlmAdapterError as exc: + raise HTTPException(status_code=502, detail=str(exc)) from exc + if result is None: + raise HTTPException(status_code=404, detail="Provider not found") + return ApiResponse.ok(dict(result)) + + +@router.get("/{provider_id}/models", response_model=ApiResponse[list[ProviderModelRead]]) +async def list_provider_models( + provider_id: str, db: AsyncSession = Depends(get_db) +) -> ApiResponse: + provider = await provider_model_service.get_provider(db, provider_id) + if not provider: + raise HTTPException(status_code=404, detail="Provider not found") + rows = await provider_model_service.list_models(db, provider_id) + return ApiResponse.ok([ProviderModelRead.model_validate(r) for r in rows]) + + +@router.post( + "/{provider_id}/models", response_model=ApiResponse[ProviderModelRead], status_code=201 +) +async def add_provider_model( + provider_id: str, + body: ProviderModelManualCreate, + db: AsyncSession = Depends(get_db), +) -> ApiResponse: + """Hand-register a catalog entry (``source="manual"`` — decision #3; + the request body has no ``source`` field, it's always forced manual + server-side).""" + provider = await provider_model_service.get_provider(db, provider_id) + if not provider: + raise HTTPException(status_code=404, detail="Provider not found") + row = await provider_model_service.add_manual_model(db, provider_id, body) + return ApiResponse.ok(ProviderModelRead.model_validate(row)) + + +@router.patch( + "/{provider_id}/models/{model_row_id}", response_model=ApiResponse[ProviderModelRead] +) +async def update_provider_model( + provider_id: str, + model_row_id: str, + body: ProviderModelUpdate, + db: AsyncSession = Depends(get_db), +) -> ApiResponse: + existing = await provider_model_service.get_model(db, model_row_id) + if existing is None or existing.provider_id != provider_id: + raise HTTPException(status_code=404, detail="Model not found") + row = await provider_model_service.update_model(db, model_row_id, body) + return ApiResponse.ok(ProviderModelRead.model_validate(row)) + + +@router.delete("/{provider_id}/models/{model_row_id}", response_model=ApiResponse[None]) +async def delete_provider_model( + provider_id: str, model_row_id: str, db: AsyncSession = Depends(get_db) +) -> ApiResponse: + existing = await provider_model_service.get_model(db, model_row_id) + if existing is None or existing.provider_id != provider_id: + raise HTTPException(status_code=404, detail="Model not found") + await provider_model_service.delete_model(db, model_row_id) + return ApiResponse.ok(None) diff --git a/backend/channels/crawl4ai_channel.py b/backend/channels/crawl4ai_channel.py index a10d833..457ed7c 100644 --- a/backend/channels/crawl4ai_channel.py +++ b/backend/channels/crawl4ai_channel.py @@ -35,6 +35,7 @@ FetchResult, ) from backend.channels.registry import register_channel +from backend.llm.factory import litellm_prefix_for from backend.security.url_guard import SSRFValidationError, avalidate_public_url logger = logging.getLogger(__name__) @@ -172,7 +173,25 @@ async def _build_llm_strategy(config: dict[str, Any]) -> Any: @staticmethod async def _resolve_llm_config(provider_id: str | None) -> Any: """Same autonomous-default convention as backend.pipeline.runner: an - explicit provider_id wins, otherwise the first enabled ModelProvider.""" + explicit provider_id wins, otherwise the first enabled ModelProvider. + + GOAL-6 PR-E exception (decision #8): crawl4ai's LLM calls go through + its own ``litellm``-backed ``LLMExtractionStrategy`` / + ``AsyncWebCrawler`` — an internal client this module has no clean + seam to route through ``backend.llm``'s adapters/resolver, so that + call stays exactly as it was. What DOES move here is the + provider-selection *resolution*: which litellm provider-name prefix + (``"openai"`` / ``"anthropic"``) a given ``provider.provider_type`` + maps to now comes from :func:`backend.llm.factory.litellm_prefix_for` + — the same place :func:`backend.llm.factory.get_adapter` dispatches + adapters from — instead of an independently hand-maintained dict + here that could silently drift out of sync with it. The provider + *selection* itself (explicit ``provider_id`` vs. first-enabled) is + untouched: it is not role/``model_defaults``-based like PR-D's + resolver, so wiring ``ProviderResolver`` in here would change WHICH + provider gets picked, not just how the prefix is computed — out of + scope for a client-construction consolidation. + """ from crawl4ai import LLMConfig from sqlalchemy import select @@ -222,9 +241,7 @@ async def _resolve_llm_config(provider_id: str | None) -> Any: error_type="SSRFValidationError", ) from exc - litellm_prefix = {"claude": "anthropic", "openai": "openai", "local": "openai"}.get( - provider.provider_type, "openai" - ) + litellm_prefix = litellm_prefix_for(provider.provider_type) default_model = ( "claude-haiku-4-5-20251001" if litellm_prefix == "anthropic" else "gpt-4o-mini" ) diff --git a/backend/channels/skill_channel.py b/backend/channels/skill_channel.py index 9f14b2a..e438ac3 100644 --- a/backend/channels/skill_channel.py +++ b/backend/channels/skill_channel.py @@ -47,12 +47,9 @@ from backend.channels.base import AbstractChannel, Capabilities, ChannelResult from backend.channels.registry import register_channel +from backend.llm.base import LlmAdapterError +from backend.llm.factory import build_openai_compat_adapter from backend.pipeline import events -from backend.security.url_guard import ( - PinnedAsyncHTTPTransport, - SSRFValidationError, - avalidate_public_url_and_ip, -) # risk / perception import only stdlib — safe at registry-load time. The loop is # imported lazily inside collect() because backend.skills.loop imports @@ -213,6 +210,18 @@ async def _build_model_call(provider: dict[str, Any]) -> Any: model. ``reply`` is the raw OpenAI chat object the loop already knows how to normalize (both ``tool_calls`` and the Qwen XML ```` path). + GOAL-6 PR-E: client construction is consolidated through + :class:`~backend.llm.openai_compat.OpenAICompatAdapter` (via + :func:`~backend.llm.factory.build_openai_compat_adapter`) — the same + guarded ``AsyncOpenAI`` construction ``chat.py`` and the ``openai`` + processor now also go through, instead of this file hand-rolling its own + copy of the SSRF-guard + DNS-rebind-pinning wiring. Behavior preserved + exactly: ``provider`` is a plain ``dict`` (``channel_config.provider``) + with no ``provider_type`` key, so the adapter's ``allow_private`` stays + ``False`` — the full, unmodified SSRF guard — exactly as this file's own + ``avalidate_public_url_and_ip(base_url)`` call (no ``allow_private=True``) + already enforced before PR-E. + Key-exfil guard: ``provider`` is DB/config-supplied (``channel_config. provider``), so its ``base_url`` is validated before the API key is ever attached to a client pointed at it — an unvalidated base_url would let a @@ -228,23 +237,13 @@ async def _build_model_call(provider: dict[str, Any]) -> Any: base_url configured leaves ``http_client`` unset (AsyncOpenAI's own default client), unchanged from before. """ - from openai import AsyncOpenAI - api_key = provider.get("api_key") or "" base_url = provider.get("base_url") or None - pinned_http_client = None - if base_url: - try: - base_url, ips = await avalidate_public_url_and_ip(base_url) - except SSRFValidationError as exc: - raise ValueError(f"skill channel: provider base_url rejected: {exc}") from exc - from urllib.parse import urlparse as _urlparse - - import httpx - - hostname = _urlparse(base_url).hostname or "" - pinned_http_client = httpx.AsyncClient(transport=PinnedAsyncHTTPTransport(hostname, ips)) - client = AsyncOpenAI(api_key=api_key, base_url=base_url, http_client=pinned_http_client) + adapter = build_openai_compat_adapter(base_url=base_url, api_key=api_key) + try: + client = await adapter.get_client() + except LlmAdapterError as exc: + raise ValueError(f"skill channel: {exc}") from exc async def model_call( messages: list[dict[str, Any]], *, tools: Any, model: str, xml: bool diff --git a/backend/llm/__init__.py b/backend/llm/__init__.py new file mode 100644 index 0000000..c7565c9 --- /dev/null +++ b/backend/llm/__init__.py @@ -0,0 +1,71 @@ +"""backend.llm — self-built model-provider runtime (GOAL-6 decision #1). + +No litellm: this package owns provider-agnostic building blocks for +chat/list_models/test_connection dispatch across ``model_providers`` rows. +PR-A shipped the closed-set vocabulary shared by the data layer +(``backend.models.provider_model``, ``backend.models.model_default``) and +their Pydantic schemas, plus the Anthropic model catalog +(``backend.llm.catalog``). PR-B added the runtime itself: ``ProviderAdapter`` (``backend.llm.base``), +``OpenAICompatAdapter`` (``backend.llm.openai_compat``, ``provider_type in +{"openai", "local"}``), ``AnthropicAdapter`` (``backend.llm.anthropic``, +``provider_type == "claude"``), and ``factory.get_adapter()`` +(``backend.llm.factory``) to dispatch a +:class:`~backend.models.provider.ModelProvider` row to the right one. PR-D +(this state) adds the failover resolver: ``ProviderResolver``/ +``ResolverError``/the module-level ``resolver`` singleton +(``backend.llm.resolver``) — pure logic only, not yet wired into any +consumption point (that is PR-E's job). +""" + +from typing import Any + +from backend.llm.anthropic import AnthropicAdapter +from backend.llm.base import ConnectionTestResult, LlmAdapterError, ProviderAdapter +from backend.llm.factory import get_adapter +from backend.llm.openai_compat import OpenAICompatAdapter +from backend.llm.resolver import ProviderResolver, ResolverError, resolver + +#: provider_models.model_type — v1 ships only "llm"; the column itself stays +#: a plain string so future embedding/rerank rows don't need a migration. +VALID_MODEL_TYPES = frozenset({"llm"}) + +#: model_defaults.role — the three consumption points GOAL-6 collapses onto +#: ModelProvider (decision #4): agent dock chat, skill_channel's cheap +#: executor model, pipeline enrichment fallback. +VALID_ROLES = frozenset({"chat", "executor", "enrichment"}) + +#: provider_models.source — "discovered" rows come from sync (OpenAI-compat +#: /v1/models or the Anthropic catalog), "manual" rows are hand-entered and +#: must never be overwritten/deleted by a sync (decision #3). +VALID_MODEL_SOURCES = frozenset({"discovered", "manual"}) + + +def is_valid_model_type(value: Any) -> bool: + return value in VALID_MODEL_TYPES + + +def is_valid_role(value: Any) -> bool: + return value in VALID_ROLES + + +def is_valid_model_source(value: Any) -> bool: + return value in VALID_MODEL_SOURCES + + +__all__ = [ + "VALID_MODEL_TYPES", + "VALID_ROLES", + "VALID_MODEL_SOURCES", + "is_valid_model_type", + "is_valid_role", + "is_valid_model_source", + "ProviderAdapter", + "LlmAdapterError", + "ConnectionTestResult", + "OpenAICompatAdapter", + "AnthropicAdapter", + "get_adapter", + "ProviderResolver", + "ResolverError", + "resolver", +] diff --git a/backend/llm/anthropic.py b/backend/llm/anthropic.py new file mode 100644 index 0000000..ba0f171 --- /dev/null +++ b/backend/llm/anthropic.py @@ -0,0 +1,150 @@ +"""Anthropic adapter (GOAL-6 PR-B) — ``provider_type == "claude"`` (decision +#2).""" + +from __future__ import annotations + +import time +from typing import Any +from urllib.parse import urlparse + +import httpx + +from backend.llm.base import ( + ConnectionTestResult, + LlmAdapterError, + ProviderAdapter, + classify_retryable, + redact_secret, +) +from backend.llm.catalog import anthropic_catalog +from backend.security.url_guard import ( + PinnedAsyncHTTPTransport, + SSRFValidationError, + avalidate_public_url_and_ip, +) + +#: Fallback model when neither an explicit ``model=`` kwarg nor +#: ``provider.default_model`` is set — mirrors +#: ``backend.processors.claude_processor.ClaudeProcessor``'s own default so +#: behaviour is unchanged for callers that relied on that default. +_DEFAULT_MODEL = "claude-haiku-4-5-20251001" + + +class AnthropicAdapter(ProviderAdapter): + """Adapter for ``provider_type == "claude"``. + + Uses ``anthropic.AsyncAnthropic`` (mirrors + ``backend.processors.claude_processor.ClaudeProcessor``'s SDK usage). + Unlike ``OpenAICompatAdapter``, there is no ``provider_type == "local"`` + case here — Anthropic's endpoint is effectively fixed + (``https://api.anthropic.com`` by SDK default), so this adapter never + passes ``allow_private=True`` to the guard: a ``base_url`` override (rare + — e.g. a proxy in front of the real API) is validated with the full, + unmodified SSRF guard, exactly like ``openai``-type providers. + + ``list_models()`` returns the hardcoded + :func:`backend.llm.catalog.anthropic_catalog` model ids rather than + hitting the network — decision #5: Anthropic has no ``GET /v1/models``- + style discovery endpoint. + """ + + def __init__(self, provider: Any) -> None: + super().__init__(provider) + self._client: Any = None + self._pinned_http_client: httpx.AsyncClient | None = None + + async def _get_client(self) -> Any: + if self._client is not None: + return self._client + import anthropic + + api_key = self.provider.api_key or "" + base_url = getattr(self.provider, "base_url", None) or None + client_kwargs: dict[str, Any] = {"api_key": api_key} + if base_url: + # Same SSRF-guard + DNS-rebind-pinning pattern as + # OpenAICompatAdapter/skill_channel/openai_processor — + # allow_private is always False here (see class docstring). + try: + base_url, ips = await avalidate_public_url_and_ip(base_url) + except SSRFValidationError as exc: + raise LlmAdapterError( + self._sanitize(f"provider base_url rejected: {exc}") + ) from exc + hostname = urlparse(base_url).hostname or "" + self._pinned_http_client = httpx.AsyncClient( + transport=PinnedAsyncHTTPTransport(hostname, ips) + ) + client_kwargs["base_url"] = base_url + client_kwargs["http_client"] = self._pinned_http_client + self._client = anthropic.AsyncAnthropic(**client_kwargs) + return self._client + + async def aclose(self) -> None: + """Close the pinned ``http_client`` this adapter opened, if any.""" + if self._pinned_http_client is not None: + await self._pinned_http_client.aclose() + self._pinned_http_client = None + + async def get_client(self) -> Any: + """Public accessor for the guarded ``AsyncAnthropic`` client (GOAL-6 + PR-E) — mirrors :meth:`OpenAICompatAdapter.get_client`. Used by + ``claude_processor`` to consolidate client construction while keeping + its own per-record loop + usage-token logging (which needs the raw + SDK response object, not ``chat()``'s plain-text return). + """ + return await self._get_client() + + def _resolve_model(self, model: str | None) -> str: + return model or self.provider.default_model or _DEFAULT_MODEL + + def _sanitize(self, message: str) -> str: + return redact_secret(message, self.provider.api_key) + + async def chat( + self, + messages: list[dict[str, Any]], + *, + model: str | None = None, + **kwargs: Any, + ) -> str: + client = await self._get_client() + max_tokens = kwargs.pop("max_tokens", 1024) + try: + response = await client.messages.create( + model=self._resolve_model(model), + max_tokens=max_tokens, + messages=messages, + **kwargs, + ) + except LlmAdapterError: + raise + except Exception as exc: + raise LlmAdapterError( + self._sanitize(f"chat completion failed: {exc}"), + retryable=classify_retryable(exc), + ) from exc + return response.content[0].text if response.content else "" + + async def list_models(self) -> list[str]: + return [entry["model_id"] for entry in anthropic_catalog()] + + async def test_connection(self) -> ConnectionTestResult: + started = time.monotonic() + try: + client = await self._get_client() + except LlmAdapterError as exc: + return {"ok": False, "error": str(exc)} + model = self.provider.default_model or _DEFAULT_MODEL + try: + await client.messages.create( + model=model, max_tokens=1, messages=[{"role": "user", "content": "ping"}] + ) + except Exception as exc: + return {"ok": False, "error": self._sanitize(str(exc))} + latency_ms = (time.monotonic() - started) * 1000 + return { + "ok": True, + "latency_ms": latency_ms, + "models_sample": (await self.list_models())[:10], + } diff --git a/backend/llm/base.py b/backend/llm/base.py new file mode 100644 index 0000000..b231f32 --- /dev/null +++ b/backend/llm/base.py @@ -0,0 +1,200 @@ +"""``ProviderAdapter`` ABC (GOAL-6 PR-B). + +Every concrete adapter (:class:`~backend.llm.openai_compat.OpenAICompatAdapter`, +:class:`~backend.llm.anthropic.AnthropicAdapter`) implements the same three +async methods against a :class:`~backend.models.provider.ModelProvider` row, +so :func:`backend.llm.factory.get_adapter` can hand callers (PR-C's API +routes, PR-D's resolver, PR-E's consumption points) a uniform surface +regardless of which vendor SDK sits underneath. + +Kept deliberately thin: ``chat()`` returns plain assistant text (not the raw +SDK response object) because PR-B has no consumer yet that needs anything +richer — a later PR can widen the return type without touching this ABC's +shape if a real need shows up. ``test_connection()`` returns a dict rather +than a dataclass so it serializes straight into an API response body (PR-C) +with no extra marshalling step. +""" + +from __future__ import annotations + +import asyncio +from abc import ABC, abstractmethod +from typing import Any, TypedDict + +import anthropic +import httpx +import openai + + +def redact_secret(message: str, secret: str | None) -> str: + """Strip a literal ``secret`` (the provider's plaintext ``api_key``) out + of an error ``message`` before it is ever raised/returned. + + Every concrete adapter routes SDK exception text through this before + wrapping it in :class:`LlmAdapterError` or a + :class:`~backend.llm.base.ConnectionTestResult`'s ``error`` field — SDK + exceptions can echo back request details (e.g. a vendor's HTTP client + repr'ing its own auth header) and this is the one seam that guarantees + the key never reaches a log line, an API response, or a raised message. + A missing/empty ``secret`` is a no-op (nothing to redact). + """ + if not secret: + return message + return message.replace(secret, "***REDACTED***") + + +class LlmAdapterError(Exception): + """Raised for any adapter-level failure (bad provider_type, SDK error, + connection failure, etc). + + Mirrors :class:`backend.browser_act.cli.BrowserActError`'s guarantee: the + message is built to be diagnosable but must NEVER include the provider's + ``api_key`` (or any other credential/env value) — every concrete adapter + sanitizes SDK exception text before wrapping it here (see each adapter's + ``_sanitize_error`` helper). Treat a message containing a raw key as a + bug in the adapter that raised it, not something this class can enforce + on its own. + + ``retryable`` (GOAL-6 PR-D, decision #7) tells + :class:`~backend.llm.resolver.ProviderResolver` whether this failure is + connection-level (connect error / timeout / 5xx — the provider is + unreachable or broken right now, worth failing over to the next + candidate) or a business error (4xx — bad key, malformed request; a + *configuration* problem no other candidate would fix, so failing over + would just mask it). Defaults to ``False`` — an adapter that raises this + directly (e.g. :func:`~backend.llm.factory.get_adapter`'s unknown + ``provider_type``) is a config error, not a transient one. Concrete + adapters set it explicitly via :func:`classify_retryable` when wrapping a + caught SDK exception. + """ + + def __init__(self, message: str, *, retryable: bool = False) -> None: + super().__init__(message) + self.retryable = retryable + + +def classify_retryable(exc: BaseException) -> bool: + """Classify a caught SDK/transport exception as connection-level + (``True``) vs business-level (``False``) for GOAL-6 decision #7. + + Connection-level (worth a failover): stdlib/httpx transport timeouts and + connection failures, and each SDK's own connection/timeout/5xx exception + types (``openai.APIConnectionError``/``APITimeoutError``/ + ``InternalServerError``, the ``anthropic`` equivalents). + + Business-level (never fails over — decision #7: a 4xx is a config + error, not a liveness problem): auth/permission/bad-request/not-found/ + rate-limit/conflict — checked explicitly, and, as a catch-all for any + other ``APIStatusError`` subclass this list doesn't name, any exception + carrying a ``status_code`` is classified by that code (``>= 500`` -> + retryable, else not) before falling back to ``False`` for anything + unrecognized. + """ + if isinstance( + exc, + ( + asyncio.TimeoutError, + httpx.TimeoutException, + httpx.NetworkError, + openai.APIConnectionError, + openai.APITimeoutError, + openai.InternalServerError, + anthropic.APIConnectionError, + anthropic.APITimeoutError, + anthropic.InternalServerError, + ), + ): + return True + + if isinstance( + exc, + ( + openai.AuthenticationError, + openai.PermissionDeniedError, + openai.BadRequestError, + openai.NotFoundError, + openai.UnprocessableEntityError, + openai.RateLimitError, + openai.ConflictError, + anthropic.AuthenticationError, + anthropic.PermissionDeniedError, + anthropic.BadRequestError, + anthropic.NotFoundError, + anthropic.UnprocessableEntityError, + anthropic.RateLimitError, + anthropic.ConflictError, + ), + ): + return False + + status_code = getattr(exc, "status_code", None) + if isinstance(status_code, int): + return status_code >= 500 + + return False + + +class ConnectionTestResult(TypedDict, total=False): + """Shape returned by :meth:`ProviderAdapter.test_connection`. + + ``total=False`` — a failed test omits ``latency_ms``/``models_sample`` + rather than filling them with ``None`` noise; callers should use + ``.get(...)``. + """ + + ok: bool + latency_ms: float | None + error: str | None + models_sample: list[str] | None + + +class ProviderAdapter(ABC): + """Uniform async runtime surface over one :class:`ModelProvider` row. + + Constructed directly from the ORM instance (not its individual fields) + so an adapter can read whichever columns it needs (``base_url``, + ``api_key`` — via the model's decrypting property, ``default_model``) + without the factory having to know each adapter's field list. + """ + + def __init__(self, provider: Any) -> None: + self.provider = provider + + @abstractmethod + async def chat( + self, + messages: list[dict[str, Any]], + *, + model: str | None = None, + **kwargs: Any, + ) -> str: + """Send ``messages`` (OpenAI chat-message shape: ``{"role", "content"}`` + dicts) and return the assistant's reply text. + + ``model`` defaults to ``self.provider.default_model`` when omitted. + Raises :class:`LlmAdapterError` on failure (never leaks ``api_key``). + """ + + @abstractmethod + async def list_models(self) -> list[str]: + """Return the model ids this provider exposes. + + OpenAI-compat: discovered via ``GET {base_url}/v1/models`` (decision + #5). Anthropic: returned from the hardcoded + :func:`backend.llm.catalog.anthropic_catalog` (no discovery endpoint + exists). Raises :class:`LlmAdapterError` on failure — callers that + want a non-raising "discovery failed, here's why" surface should use + :meth:`test_connection` instead (decision #5: discovery failure must + not crash the caller). + """ + + @abstractmethod + async def test_connection(self) -> ConnectionTestResult: + """Probe the provider cheaply and report ``{ok, latency_ms, error, + models_sample}``. + + Never raises for an ordinary connection/auth failure — those come + back as ``{"ok": False, "error": }`` so PR-C's + ``POST /providers/{id}/test`` endpoint can return this dict straight + through as the response body. ``error`` never contains ``api_key``. + """ diff --git a/backend/llm/catalog.py b/backend/llm/catalog.py new file mode 100644 index 0000000..19e770d --- /dev/null +++ b/backend/llm/catalog.py @@ -0,0 +1,56 @@ +"""Anthropic model catalog (GOAL-6 decision #5). + +Anthropic has no ``GET /v1/models``-style discovery endpoint the way +OpenAI-compatible providers do (PR-B's ``OpenAICompatAdapter`` hits +``{base_url}/v1/models`` directly, which works against ollama/model-hotel/ +deepseek/etc.) — so for ``provider_type="claude"``, model discovery falls +back to this hardcoded, maintained constant instead of a network call. + +This list is a maintained constant, not a derived value: update it by hand +as Anthropic ships new models. There is no other source of truth for it in +this codebase. +""" + +from typing import TypedDict + + +class AnthropicModel(TypedDict): + model_id: str + display_name: str + context_window: int + supports_tools: bool + supports_vision: bool + + +ANTHROPIC_CATALOG: list[AnthropicModel] = [ + { + "model_id": "claude-opus-4-8", + "display_name": "Claude Opus 4.8", + "context_window": 200000, + "supports_tools": True, + "supports_vision": True, + }, + { + "model_id": "claude-sonnet-5", + "display_name": "Claude Sonnet 5", + "context_window": 200000, + "supports_tools": True, + "supports_vision": True, + }, + { + "model_id": "claude-haiku-4-5-20251001", + "display_name": "Claude Haiku 4.5", + "context_window": 200000, + "supports_tools": True, + "supports_vision": False, + }, +] + + +def anthropic_catalog() -> list[AnthropicModel]: + """Return the maintained Anthropic model catalog. + + Returns a defensive shallow copy (new list of new dicts) so a caller + mutating the result can't corrupt the module-level constant. + """ + return [dict(entry) for entry in ANTHROPIC_CATALOG] diff --git a/backend/llm/factory.py b/backend/llm/factory.py new file mode 100644 index 0000000..ddc215a --- /dev/null +++ b/backend/llm/factory.py @@ -0,0 +1,169 @@ +"""Adapter factory (GOAL-6 PR-B, decision #6): dispatch a +:class:`~backend.models.provider.ModelProvider` row to its +:class:`~backend.llm.base.ProviderAdapter` by ``provider_type``. + +This is the single place PR-C's API routes / PR-D's resolver / PR-E's +consumption points should call to turn a stored provider row into something +that can ``chat``/``list_models``/``test_connection`` — the SSRF guard +(decision #6) lives inside the concrete adapters' client-building step +(``OpenAICompatAdapter._get_client`` / ``AnthropicAdapter._get_client``), not +here; this function only dispatches on ``provider_type``. +""" + +from __future__ import annotations + +from types import SimpleNamespace +from typing import Any + +from backend.llm.anthropic import AnthropicAdapter +from backend.llm.base import LlmAdapterError, ProviderAdapter +from backend.llm.openai_compat import OpenAICompatAdapter + +#: provider_type -> adapter class (decision #2: provider_type stays the +#: existing openai|claude|local enum; it now also selects the adapter +#: *family* — openai/local share OpenAICompatAdapter, claude gets its own). +_ADAPTERS: dict[str, type[ProviderAdapter]] = { + "openai": OpenAICompatAdapter, + "local": OpenAICompatAdapter, + "claude": AnthropicAdapter, +} + +#: provider_type -> litellm provider-name prefix (crawl4ai_channel's +#: LLMConfig, GOAL-6 PR-E decision #8's crawl4ai exception). Mirrors +#: _ADAPTERS' family grouping 1:1 (openai/local share the openai wire +#: protocol, claude is Anthropic's own) so the two mappings can't quietly +#: drift apart by being hand-maintained in two files. +_LITELLM_PREFIX: dict[str, str] = { + "openai": "openai", + "local": "openai", + "claude": "anthropic", +} + + +def get_adapter(provider: Any) -> ProviderAdapter: + """Return the :class:`ProviderAdapter` for ``provider.provider_type``. + + Raises :class:`LlmAdapterError` for an unrecognized ``provider_type`` + rather than silently defaulting — a typo'd/legacy provider_type should + fail loudly here instead of quietly getting the wrong adapter. + """ + provider_type = getattr(provider, "provider_type", None) + adapter_cls = _ADAPTERS.get(provider_type) + if adapter_cls is None: + raise LlmAdapterError( + f"no adapter registered for provider_type={provider_type!r} " + f"(expected one of {sorted(_ADAPTERS)})" + ) + return adapter_cls(provider) + + +def litellm_prefix_for(provider_type: str | None) -> str: + """Map a ``ModelProvider.provider_type`` to the litellm provider-name + prefix ``crawl4ai_channel``'s ``LLMConfig`` needs (GOAL-6 PR-E, decision + #8's crawl4ai exception): the litellm client/call itself stays untouched + there, but *which* prefix a given ``provider_type`` maps to is now + decided here — the same place :func:`get_adapter` dispatches from — + instead of an independently hand-maintained dict inside the channel that + could silently drift out of sync with ``_ADAPTERS``. + + Falls back to ``"openai"`` for an unrecognized ``provider_type``, + matching crawl4ai_channel's pre-PR-E behavior exactly. + """ + return _LITELLM_PREFIX.get(provider_type or "", "openai") + + +def _provider_view( + *, + provider_type: str | None, + base_url: str | None, + api_key: str | None, + default_model: str | None = None, +) -> Any: + """Build a minimal read-only stand-in for a + :class:`~backend.models.provider.ModelProvider` row from already-resolved + field values (GOAL-6 PR-E). + + ``OpenAICompatAdapter``/``AnthropicAdapter`` only ever read + ``provider.provider_type`` / ``.base_url`` / ``.api_key`` / + ``.default_model`` off whatever object they're constructed with — they + don't require a real ORM instance. This lets ``build_openai_compat_adapter``/ + ``build_anthropic_adapter`` below hand them a disposable view instead of + the caller's actual ``provider``, which matters for two reasons: + + * chat.py's ``provider`` is a live ORM instance still attached to a DB + session — writing an env-var API-key fallback onto it directly + (``provider.api_key = ...``) would mark it dirty and risk persisting + the env-derived key back into ``model_providers.api_key`` on the next + commit/autoflush. A throwaway view sidesteps that entirely. + * skill_channel's / the processors' ``provider``/``config`` is a plain + ``dict`` (``.get(...)`` access), not an object with attributes at all — + an adapter constructed directly from it would find every attribute + lookup falling through to nothing. + + Each PR-E call site keeps resolving its OWN fields first (attribute vs + dict-get, its own env-var fallback name, its own default) exactly as it + did before GOAL-6 — this only removes the duplicated *client + construction* step, not each caller's field-resolution rules. + """ + return SimpleNamespace( + provider_type=provider_type, + base_url=base_url, + api_key=api_key or "", + default_model=default_model, + ) + + +def build_openai_compat_adapter( + *, + base_url: str | None, + api_key: str | None, + default_model: str | None = None, + provider_type: str | None = None, +) -> OpenAICompatAdapter: + """Build an :class:`OpenAICompatAdapter` from already-resolved field + values (GOAL-6 PR-E) — for ``chat.py``/``skill_channel``/the ``openai`` + processor, which need the guarded ``AsyncOpenAI`` client construction + this adapter implements, but whose ``provider`` is either a live ORM row + (chat.py) or a plain config ``dict`` (skill_channel, the processors), not + something this helper should re-derive each caller's own field-resolution + rules for (see :func:`_provider_view`). + + ``provider_type`` defaults to ``None`` (→ ``allow_private=False``, the + full SSRF guard — see ``OpenAICompatAdapter``'s docstring / decision #6): + only ``ModelProvider.provider_type == "local"`` should ever get the + private-address exemption, and none of chat.py's provider selection, + skill_channel's dict config, or the openai processor's config carry that + distinction today — passing ``None`` here keeps every one of them fully + guarded, exactly as they were (chat.py: unguarded before, now fully + guarded — a deliberate SSRF-hole closure, decision #6 — see PR-E report; + skill_channel/openai processor: already fully guarded, unchanged). + """ + return OpenAICompatAdapter( + _provider_view( + provider_type=provider_type, + base_url=base_url, + api_key=api_key, + default_model=default_model, + ) + ) + + +def build_anthropic_adapter( + *, + api_key: str | None, + base_url: str | None = None, + default_model: str | None = None, +) -> AnthropicAdapter: + """Build an :class:`AnthropicAdapter` from already-resolved field values + (GOAL-6 PR-E) — for the ``claude`` processor. See + :func:`build_openai_compat_adapter` / :func:`_provider_view` for why this + takes field values rather than a real provider object. + """ + return AnthropicAdapter( + _provider_view( + provider_type="claude", + base_url=base_url, + api_key=api_key, + default_model=default_model, + ) + ) diff --git a/backend/llm/openai_compat.py b/backend/llm/openai_compat.py new file mode 100644 index 0000000..e2bf7ce --- /dev/null +++ b/backend/llm/openai_compat.py @@ -0,0 +1,171 @@ +"""OpenAI-compatible adapter (GOAL-6 PR-B) — ``provider_type in {"openai", +"local"}`` (decision #2: both are the same wire protocol, just different +trust levels for the target address). +""" + +from __future__ import annotations + +import time +from typing import Any +from urllib.parse import urlparse + +import httpx + +from backend.llm.base import ( + ConnectionTestResult, + LlmAdapterError, + ProviderAdapter, + classify_retryable, + redact_secret, +) +from backend.security.url_guard import ( + PinnedAsyncHTTPTransport, + SSRFValidationError, + avalidate_public_url_and_ip, +) + + +class OpenAICompatAdapter(ProviderAdapter): + """Adapter for ``provider_type in {"openai", "local"}``. + + Builds an ``openai.AsyncOpenAI`` client pointed at ``provider.base_url``, + reusing the exact SSRF-guard + DNS-rebind-pinning pattern + ``backend.channels.skill_channel._build_model_call`` and + ``backend.processors.openai_processor.OpenAIProcessor`` already use for + this same SDK: validate ``base_url`` with + ``avalidate_public_url_and_ip`` and hand ``AsyncOpenAI`` an + ``http_client`` whose transport is a ``PinnedAsyncHTTPTransport`` bound + to the validated IP(s) — see :mod:`backend.security.url_guard`'s module + docstring for the full DNS-rebind-closure mechanism. When ``base_url`` + is unset the SDK's own default endpoint is used, unvalidated and + unpinned, exactly as the existing call sites already do. + + **Local-address exemption (decision #6 — flag for reviewer + confirmation)**: ``backend.security.url_guard`` had *no* existing + localhost/private-IP allowlist before this PR (confirmed by reading the + whole module + its test file — every IP-space check was unconditional). + Yet ``provider_type == "local"`` exists specifically for self-hosted + providers that live at exactly the addresses the guard blocks: ollama on + loopback (``http://localhost:11434``), model-hotel on the NetBird + fleet-mesh CGNAT range (``100.64.0.0/10``, e.g. ``100.80.x.x``). Rather + than leave "local" providers permanently unreachable, this adapter adds + a narrow ``allow_private=True`` opt-in (see + ``backend.security.url_guard.is_ip_blocked``) that is threaded through + to both the initial validation call and the pinned transport's + connect-time re-check, and is used ONLY when + ``self.provider.provider_type == "local"`` — an ``openai`` provider's + ``base_url`` is always validated with ``allow_private=False`` (the full, + unmodified guard). The connection is still IP-pinned in both cases: + ``allow_private`` only changes which addresses pass the block-list + check, not whether DNS-rebind pinning applies. + """ + + def __init__(self, provider: Any) -> None: + super().__init__(provider) + self._allow_private = getattr(provider, "provider_type", None) == "local" + self._client: Any = None + self._pinned_http_client: httpx.AsyncClient | None = None + + async def _get_client(self) -> Any: + if self._client is not None: + return self._client + from openai import AsyncOpenAI + + api_key = self.provider.api_key or "" + base_url = self.provider.base_url or None + if base_url: + try: + base_url, ips = await avalidate_public_url_and_ip( + base_url, allow_private=self._allow_private + ) + except SSRFValidationError as exc: + raise LlmAdapterError( + self._sanitize(f"provider base_url rejected: {exc}") + ) from exc + hostname = urlparse(base_url).hostname or "" + self._pinned_http_client = httpx.AsyncClient( + transport=PinnedAsyncHTTPTransport( + hostname, ips, allow_private=self._allow_private + ) + ) + self._client = AsyncOpenAI( + api_key=api_key, base_url=base_url, http_client=self._pinned_http_client + ) + return self._client + + async def aclose(self) -> None: + """Close the pinned ``http_client`` this adapter opened, if any. + + ``AsyncOpenAI`` does not close an externally-supplied ``http_client`` + (mirrors ``OpenAIProcessor``'s own cleanup) — callers that create an + adapter directly (rather than through a request-scoped helper) should + call this when done with it. + """ + if self._pinned_http_client is not None: + await self._pinned_http_client.aclose() + self._pinned_http_client = None + + async def get_client(self) -> Any: + """Public accessor for the guarded ``AsyncOpenAI`` client (GOAL-6 + PR-E). + + ``chat.py``'s agent-dock tool-calling loop and ``skill_channel``'s + cheap-executor loop both need the *raw* SDK client (not + :meth:`chat`'s plain-text return) because they drive their own + multi-step ``tools=``/``tool_choice=`` loop that this adapter's thin + ``chat()`` doesn't support. This just exposes the same + SSRF-validated + DNS-rebind-pinned construction :meth:`chat`/ + :meth:`list_models` already use internally, so those two call sites + stop duplicating the ``AsyncOpenAI`` + ``PinnedAsyncHTTPTransport`` + wiring themselves. + """ + return await self._get_client() + + def _resolve_model(self, model: str | None) -> str: + return model or self.provider.default_model or "gpt-4o-mini" + + def _sanitize(self, message: str) -> str: + return redact_secret(message, self.provider.api_key) + + async def chat( + self, + messages: list[dict[str, Any]], + *, + model: str | None = None, + **kwargs: Any, + ) -> str: + client = await self._get_client() + try: + response = await client.chat.completions.create( + model=self._resolve_model(model), messages=messages, **kwargs + ) + except LlmAdapterError: + raise + except Exception as exc: + raise LlmAdapterError( + self._sanitize(f"chat completion failed: {exc}"), + retryable=classify_retryable(exc), + ) from exc + return response.choices[0].message.content or "" + + async def list_models(self) -> list[str]: + client = await self._get_client() + try: + page = await client.models.list() + except LlmAdapterError: + raise + except Exception as exc: + raise LlmAdapterError( + self._sanitize(f"model discovery failed: {exc}"), + retryable=classify_retryable(exc), + ) from exc + return [model.id for model in (page.data or [])] + + async def test_connection(self) -> ConnectionTestResult: + started = time.monotonic() + try: + models = await self.list_models() + except LlmAdapterError as exc: + return {"ok": False, "error": str(exc)} + latency_ms = (time.monotonic() - started) * 1000 + return {"ok": True, "latency_ms": latency_ms, "models_sample": models[:10]} diff --git a/backend/llm/resolver.py b/backend/llm/resolver.py new file mode 100644 index 0000000..8483812 --- /dev/null +++ b/backend/llm/resolver.py @@ -0,0 +1,191 @@ +"""Provider resolver + failover (GOAL-6 PR-D, decision #7). + +Reads a role's ``model_defaults.candidates`` (PR-A) — an ordered list of +``{"provider_id", "model_id"}`` dicts — and dispatches through +:func:`backend.llm.factory.get_adapter` (PR-B) to try providers in that +order. Only a *connection-level* failure (see +:func:`backend.llm.base.classify_retryable`: connect error, timeout, 5xx) +degrades to the next candidate; a *business* failure (4xx — bad key, +malformed request) is a configuration problem no other candidate would fix, +so it is re-raised immediately instead of being silently masked behind a +fallback model (decision #7). + +Cooldown is an in-process ``dict`` keyed by ``provider_id`` — no Redis +(decision #7): this app runs as a single process, so a plain dict plus an +injectable clock is enough, and it keeps this PR pure logic with no new +infra dependency. + +This module is pure logic + the module-level ``resolver`` singleton — it is +NOT wired into any consumption point yet (chat.py / skill_channel / +processors). That wiring is PR-E's job. +""" + +from __future__ import annotations + +import time +from dataclasses import dataclass +from typing import Any, Awaitable, Callable, TypeVar + +from sqlalchemy import select +from sqlalchemy.ext.asyncio import AsyncSession + +from backend.llm.base import LlmAdapterError, ProviderAdapter +from backend.llm.factory import get_adapter +from backend.models.model_default import ModelDefault +from backend.models.provider import ModelProvider + +T = TypeVar("T") + +#: async callable invoked per live candidate: operation(adapter, model_id) -> T +Operation = Callable[[ProviderAdapter, str], Awaitable[T]] + + +class ResolverError(Exception): + """Raised by :meth:`ProviderResolver.resolve_with_fallback` when no live + candidate exists for a role — either ``model_defaults`` has no row (or an + empty ``candidates`` list) for it, or every configured candidate is + currently unavailable (cooled down and/or just retryable-failed). + + The message only ever names the role and tried/cooled counts — never a + provider's ``api_key`` (nothing here ever touches a raw key, only ids). + """ + + +@dataclass +class ResolvedModel: + """One resolved ``(provider, model_id, adapter)`` triple — what + :meth:`ProviderResolver.resolve` returns for a role's primary candidate. + """ + + provider: ModelProvider + model_id: str + adapter: ProviderAdapter + + +class ProviderResolver: + """Resolves a consumption ``role`` (PR-A: ``chat``/``executor``/ + ``enrichment``) to a live provider, trying ``model_defaults.candidates`` + in order and skipping/cooling down ones that just failed. + + ``now`` is an injectable monotonic clock (defaults to + ``time.monotonic``) so unit tests can fast-forward a cooldown window + without a real sleep. Cooldown state (``self._cooldown_until``) is a + plain in-process ``dict`` — no Redis (decision #7) — so it is + per-process only; that matches this app's current single-process + deployment. + """ + + def __init__( + self, + *, + cooldown_seconds: float = 60.0, + now: Callable[[], float] = time.monotonic, + ) -> None: + self.cooldown_seconds = cooldown_seconds + self._now = now + self._cooldown_until: dict[str, float] = {} + + def _is_cooled(self, provider_id: str) -> bool: + until = self._cooldown_until.get(provider_id) + return until is not None and self._now() < until + + def _set_cooldown(self, provider_id: str) -> None: + # Single synchronous read-then-write, no `await` in between — safe + # under concurrent asyncio callers sharing this resolver (nothing + # can interleave inside one un-awaited statement). + self._cooldown_until[provider_id] = self._now() + self.cooldown_seconds + + async def _candidates_for(self, db: AsyncSession, role: str) -> list[dict[str, Any]]: + result = await db.execute(select(ModelDefault).where(ModelDefault.role == role)) + row = result.scalar_one_or_none() + if row is None: + return [] + return list(row.candidates or []) + + async def resolve(self, db: AsyncSession, role: str) -> ResolvedModel | None: + """Return the primary (first) candidate configured for ``role``. + + Returns ``None`` — rather than raising — when no ``model_defaults`` + row exists for ``role``, its ``candidates`` list is empty, or the + first candidate's ``provider_id`` no longer resolves to a real + provider row: this is a direct "what's configured" lookup for + callers that want the single default, not a "try until one works" + search (that's :meth:`resolve_with_fallback`) — there is nothing to + fail over to here, so a clean ``None`` is more useful to a caller + than an exception for what is often just "nothing configured yet". + """ + candidates = await self._candidates_for(db, role) + if not candidates: + return None + candidate = candidates[0] + provider = await db.get(ModelProvider, candidate["provider_id"]) + if provider is None: + return None + return ResolvedModel( + provider=provider, + model_id=candidate["model_id"], + adapter=get_adapter(provider), + ) + + async def resolve_with_fallback( + self, db: AsyncSession, role: str, operation: Operation[T] + ) -> T: + """Try ``role``'s candidates in order, calling ``await + operation(adapter, model_id)`` for the first live one. + + - A candidate whose ``provider_id`` is currently cooled down is + skipped WITHOUT building its adapter (a cooled provider is + assumed still broken; no point re-probing it every call before its + window expires). + - A candidate whose ``provider_id`` no longer resolves to a real + provider row (deleted since the default was configured) is + skipped the same way, without being put in cooldown (there is no + "it" to cool down). + - ``operation`` succeeding returns that result immediately — no + further candidates are tried. + - ``operation`` raising :class:`~backend.llm.base.LlmAdapterError` + with ``retryable=True`` (connection-level) puts that + ``provider_id`` in cooldown and moves on to the next candidate. + - ``operation`` raising ``LlmAdapterError`` with ``retryable=False`` + (business/4xx) is re-raised IMMEDIATELY: no cooldown is set, no + further candidate is tried (decision #7 — this is a config error, + not a liveness problem, and failing over would mask it). + - once every candidate has been skipped (cooled/missing) or has + retryable-failed, raises :class:`ResolverError` naming ``role`` + and how many candidates were tried vs. skipped-as-cooled. + """ + candidates = await self._candidates_for(db, role) + if not candidates: + raise ResolverError(f"no model_defaults candidates configured for role={role!r}") + + tried = 0 + cooled = 0 + for candidate in candidates: + provider_id = candidate["provider_id"] + if self._is_cooled(provider_id): + cooled += 1 + continue + provider = await db.get(ModelProvider, provider_id) + if provider is None: + # Candidate references a since-deleted provider — dead, but + # not "this provider just failed", so no cooldown to set. + continue + adapter = get_adapter(provider) + tried += 1 + try: + return await operation(adapter, candidate["model_id"]) + except LlmAdapterError as exc: + if not exc.retryable: + raise + self._set_cooldown(provider_id) + continue + + raise ResolverError( + f"role={role!r}: no live provider candidate " + f"(tried={tried}, cooled={cooled}, total={len(candidates)})" + ) + + +#: Module-level default instance — production code imports this singleton; +#: tests build their own ``ProviderResolver(now=fake_clock)`` (decision #7). +resolver = ProviderResolver() diff --git a/backend/migrations/versions/d8e9f0a1b2c3_add_provider_models_and_model_defaults.py b/backend/migrations/versions/d8e9f0a1b2c3_add_provider_models_and_model_defaults.py new file mode 100644 index 0000000..0580464 --- /dev/null +++ b/backend/migrations/versions/d8e9f0a1b2c3_add_provider_models_and_model_defaults.py @@ -0,0 +1,56 @@ +"""add provider_models and model_defaults tables + +Revision ID: d8e9f0a1b2c3 +Revises: a7v8w9x0y1z2 +Create Date: 2026-07-09 + +GOAL-6 PR-A (model-provider-mgmt, decisions #3/#4): the model catalog +(``provider_models`` — one row per model a provider exposes, sourced from +discovery sync or manual entry) and system default candidates per +consumption role (``model_defaults``). Adds ONLY these two tables — no +adapters/factory/resolver (PR-B/D), no API routes (PR-C). +""" + +import sqlalchemy as sa +from alembic import op + +revision = "d8e9f0a1b2c3" +down_revision = "a7v8w9x0y1z2" +branch_labels = None +depends_on = None + + +def upgrade() -> None: + op.create_table( + "provider_models", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("provider_id", sa.String(length=36), nullable=False), + sa.Column("model_id", sa.String(length=255), nullable=False), + sa.Column("model_type", sa.String(length=50), nullable=False), + sa.Column("capabilities", sa.JSON(), nullable=True), + sa.Column("source", sa.String(length=50), nullable=False), + sa.Column("enabled", sa.Boolean(), nullable=False), + sa.Column("created_at", sa.DateTime(timezone=True), nullable=False), + sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False), + sa.ForeignKeyConstraint(["provider_id"], ["model_providers.id"], ondelete="CASCADE"), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint("provider_id", "model_id", name="uq_provider_models_provider_model"), + ) + op.create_index("ix_provider_models_provider_id", "provider_models", ["provider_id"]) + + op.create_table( + "model_defaults", + sa.Column("id", sa.String(length=36), nullable=False), + sa.Column("role", sa.String(length=50), nullable=False), + sa.Column("candidates", sa.JSON(), nullable=False), + sa.Column("created_at", sa.DateTime(timezone=True), nullable=False), + sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False), + sa.PrimaryKeyConstraint("id"), + sa.UniqueConstraint("role", name="uq_model_defaults_role"), + ) + + +def downgrade() -> None: + op.drop_table("model_defaults") + op.drop_index("ix_provider_models_provider_id", table_name="provider_models") + op.drop_table("provider_models") diff --git a/backend/models/__init__.py b/backend/models/__init__.py index c8cd3a0..5648d8a 100644 --- a/backend/models/__init__.py +++ b/backend/models/__init__.py @@ -4,12 +4,14 @@ from backend.models.control_action import ControlActionRecord from backend.models.cookie_jar import CookieJarEntry from backend.models.edge_node import EdgeNode, EdgeNodeEvent +from backend.models.model_default import ModelDefault from backend.models.notification import NotificationLog, NotificationRule from backend.models.odp_system_measurement import OdpSystemMeasurement from backend.models.plan import Plan from backend.models.plan_health import PlanHealthRecord from backend.models.plan_source_index import PlanSourceIndex from backend.models.provider import ModelProvider +from backend.models.provider_model import ProviderModel from backend.models.record import CollectedRecord from backend.models.schedule import CronSchedule from backend.models.skill import Skill @@ -30,6 +32,8 @@ "EdgeNode", "EdgeNodeEvent", "ModelProvider", + "ProviderModel", + "ModelDefault", "Plan", "PlanHealthRecord", "PlanSourceIndex", diff --git a/backend/models/model_default.py b/backend/models/model_default.py new file mode 100644 index 0000000..1c5200f --- /dev/null +++ b/backend/models/model_default.py @@ -0,0 +1,27 @@ +from sqlalchemy import JSON, String +from sqlalchemy.orm import Mapped, mapped_column + +from backend.models.base import TimestampMixin + + +class ModelDefault(TimestampMixin): + """System default model candidates for one consumption role (GOAL-6 + decision #4). + + One row per role (``role`` is UNIQUE): ``candidates`` is an ordered list + of ``{"provider_id": ..., "model_id": ...}`` dicts — index 0 is the + primary pick, the rest are failover order tried in sequence by the + resolver (PR-D; this table only defines the shape, no resolve logic + lives here). Roles map to the three consumption points GOAL-6 collapses + onto ModelProvider (decision #4): ``chat`` (agent dock conversation), + ``executor`` (skill_channel's cheap execution model), ``enrichment`` + (pipeline processor fallback). + """ + + __tablename__ = "model_defaults" + + #: chat | executor | enrichment — closed-set validated, see + #: backend.llm.VALID_ROLES. unique=True: exactly one defaults row per role. + role: Mapped[str] = mapped_column(String(50), nullable=False, unique=True) + #: Ordered [{"provider_id": ..., "model_id": ...}, ...]; index 0 = primary. + candidates: Mapped[list] = mapped_column(JSON, nullable=False, default=list) diff --git a/backend/models/provider_model.py b/backend/models/provider_model.py new file mode 100644 index 0000000..5722356 --- /dev/null +++ b/backend/models/provider_model.py @@ -0,0 +1,46 @@ +from typing import Optional + +from sqlalchemy import JSON, Boolean, ForeignKey, String, UniqueConstraint +from sqlalchemy.orm import Mapped, mapped_column + +from backend.models.base import TimestampMixin + + +class ProviderModel(TimestampMixin): + """One model in a provider's catalog (GOAL-6 decision #3). + + Populated either by discovery sync (``source="discovered"`` — OpenAI-compat + ``GET {base_url}/v1/models``, or the Anthropic hardcoded catalog for + ``provider_type="claude"``, both PR-B/C) or entered by hand + (``source="manual"``). Sync is an upsert that must never overwrite or + delete a ``manual`` row (PR-C concern; this table only defines the shape). + + ``provider_id`` is a real FK — unlike ``AIAgent.provider_id``, which stays + a loose string column per GOAL-6 decision #9 (the migration cost isn't + worth it there) — so deleting a ``ModelProvider`` cascades its whole + catalog away instead of leaving orphan rows. + """ + + __tablename__ = "provider_models" + __table_args__ = ( + UniqueConstraint("provider_id", "model_id", name="uq_provider_models_provider_model"), + ) + + provider_id: Mapped[str] = mapped_column( + String(36), + ForeignKey("model_providers.id", ondelete="CASCADE"), + nullable=False, + index=True, + ) + #: Provider-native model name (e.g. "claude-sonnet-5", "gpt-4o"). + model_id: Mapped[str] = mapped_column(String(255), nullable=False) + #: llm | embedding | rerank — v1 only ever writes "llm"; the column stays + #: a plain string (closed-set validated at the Pydantic/backend.llm + #: layer, see backend.llm.VALID_MODEL_TYPES) so embedding/rerank rows can + #: land later without a migration. + model_type: Mapped[str] = mapped_column(String(50), nullable=False, default="llm") + #: e.g. {"tools": true, "vision": false, "context_window": 200000}. + capabilities: Mapped[Optional[dict]] = mapped_column(JSON, nullable=True) + #: discovered | manual — closed-set validated, see backend.llm.VALID_MODEL_SOURCES. + source: Mapped[str] = mapped_column(String(50), nullable=False) + enabled: Mapped[bool] = mapped_column(Boolean, default=True, nullable=False) diff --git a/backend/pipeline/ai_processor.py b/backend/pipeline/ai_processor.py index fd2bf8f..a83ac12 100644 --- a/backend/pipeline/ai_processor.py +++ b/backend/pipeline/ai_processor.py @@ -1,14 +1,83 @@ """Pipeline Step 4: Optional AI enrichment of records.""" +import logging from typing import Any from backend.models.record import CollectedRecord from backend.processors.registry import get_processor +logger = logging.getLogger(__name__) + + +async def _resolve_llm_config(ai_config: dict[str, Any], source_id: Any) -> dict[str, Any]: + """GOAL-6 PR-F (decision #9): soft dual-track convergence between + ``DataSource.ai_config``'s legacy inline ``api_key``/``base_url`` and the + governed ``ModelProvider`` catalog (``backend.models.provider``). + + * No ``provider_id`` in ``ai_config`` -> returned byte-identical to + today's behavior (same dict, same keys, same values) — the only + addition is a deprecation warning when inline ``api_key``/``base_url`` + are present, since that's the legacy channel this decision is steering + callers away from. + * ``provider_id`` set and resolves -> the provider's ``provider_type``/ + ``api_key``/``base_url``/``default_model`` win over anything supplied + inline (a warning is logged when inline creds were *also* supplied, + since they're now silently ignored in favor of the provider). + * ``provider_id`` set but does NOT resolve (deleted/bad id) -> warn and + fall back to ``ai_config`` unchanged, exactly like the no-``provider_id`` + case (inline creds if present, otherwise whatever the processor's own + env-var fallback does) — this mirrors this module's existing fail-soft + posture (unknown ``processor_type`` / missing config never raise, they + just skip/continue) rather than introducing a new crash path. + """ + provider_id = ai_config.get("provider_id") + has_inline = bool(ai_config.get("api_key") or ai_config.get("base_url")) + + if not provider_id: + if has_inline: + logger.warning( + "DataSource %s ai_config uses inline LLM credentials; " + "reference a provider_id instead (inline config is deprecated)", + source_id, + ) + return ai_config + + from backend.database import AsyncSessionLocal + from backend.services.provider_model_service import get_provider + + async with AsyncSessionLocal() as session: + provider = await get_provider(session, provider_id) + + if provider is None: + logger.warning( + "DataSource %s ai_config.provider_id=%s does not resolve to an " + "existing ModelProvider; falling back to inline config", + source_id, provider_id, + ) + return ai_config + + if has_inline: + logger.warning( + "DataSource %s ai_config supplies both provider_id=%s and inline " + "api_key/base_url; provider_id takes precedence", + source_id, provider_id, + ) + + resolved = dict(ai_config) + resolved["processor_type"] = provider.provider_type + resolved["api_key"] = provider.api_key + resolved["base_url"] = provider.base_url + if provider.default_model: + resolved["model"] = provider.default_model + return resolved + async def process_with_ai( records: list[CollectedRecord], ai_config: dict[str, Any] | None, + *, + source_id: Any = None, + resolve_provider: bool = True, ) -> None: """Enrich records with AI processing in-place. @@ -16,12 +85,36 @@ async def process_with_ai( processor_type: claude | openai | local model: model name prompt_template: Jinja2 template + provider_id: GOAL-6 PR-F — governed ModelProvider reference; wins + over inline api_key/base_url when both are present (decision #9) ...processor-specific options + + ``source_id`` (the owning DataSource's id) is only used to identify the + source in deprecation/fallback warning log lines above; it never affects + resolution logic. + + ``resolve_provider`` gates the decision #9 dual-track resolution above. + Callers pass ``False`` when ``ai_config`` is really an *agent*-level + config (``ai_agents.processor_config`` merged with its own + ``ai_agents.provider_id`` resolution in ``backend.pipeline.runner`` phase + 2) rather than a ``DataSource.ai_config`` value — that merge already + happened upstream through a separate, pre-existing, intentionally + untouched mechanism (decision #9 explicitly keeps ``ai_agents.provider_id`` + a loose string column, out of this PR's scope), and it routinely leaves + inline ``api_key``/``base_url`` in the dict with no ``provider_id`` key. + Running this function's deprecation-warning logic against that dict would + misfire on every agent-driven run, not just legacy inline + ``DataSource.ai_config`` — so agent-sourced configs skip resolution + entirely and are used exactly as before. """ if not ai_config or not records: return - processor_type = ai_config.get("processor_type", "claude") + resolved_config = ( + await _resolve_llm_config(ai_config, source_id) if resolve_provider else ai_config + ) + + processor_type = resolved_config.get("processor_type", "claude") try: processor = get_processor(processor_type) except ValueError: @@ -29,8 +122,8 @@ async def process_with_ai( result = await processor.process( records=records, - prompt_template=ai_config.get("prompt_template", ""), - config=ai_config, + prompt_template=resolved_config.get("prompt_template", ""), + config=resolved_config, ) for record, enrichment in zip(records, result.enrichments): diff --git a/backend/pipeline/pipeline.py b/backend/pipeline/pipeline.py index adff452..a44e5b4 100644 --- a/backend/pipeline/pipeline.py +++ b/backend/pipeline/pipeline.py @@ -390,7 +390,16 @@ async def run_pipeline( task_row.status = "ai_processing" await session.commit() try: - await ai_processor.process_with_ai(new_records, effective_ai_config) + await ai_processor.process_with_ai( + new_records, + effective_ai_config, + source_id=source.id, + # GOAL-6 PR-F decision #9 dual-track resolution only applies to + # DataSource.ai_config; an agent_config override already went + # through its own (untouched) ai_agents.provider_id resolution + # in backend.pipeline.runner phase 2, so it's used as-is. + resolve_provider=agent_config is None, + ) # Persist enrichments — new_records are detached after step3 session closed from backend.models.record import CollectedRecord async with AsyncSessionLocal() as session: diff --git a/backend/processors/claude_processor.py b/backend/processors/claude_processor.py index 720e767..a3d16b0 100644 --- a/backend/processors/claude_processor.py +++ b/backend/processors/claude_processor.py @@ -2,9 +2,11 @@ import json import logging +import os import re from typing import TYPE_CHECKING, Any +from backend.llm.factory import build_anthropic_adapter from backend.processors.base import AbstractProcessor, ProcessingResult from backend.processors.registry import register_processor @@ -33,46 +35,56 @@ async def process( config: dict[str, Any], ) -> ProcessingResult: try: - import anthropic + import anthropic # noqa: F401 -- import-availability probe only except ImportError: return ProcessingResult( success=False, error="anthropic package not installed" ) - api_key = config.get("api_key") or __import__("os").environ.get("ANTHROPIC_API_KEY", "") + api_key = config.get("api_key") or os.environ.get("ANTHROPIC_API_KEY", "") model = config.get("model", "claude-haiku-4-5-20251001") max_tokens = config.get("max_tokens", 1024) logger.info("claude processor | model=%s max_tokens=%d records=%d", model, max_tokens, len(records)) - client = anthropic.AsyncAnthropic(api_key=api_key) + # GOAL-6 PR-E: client construction consolidated through + # backend.llm.anthropic.AnthropicAdapter (via + # backend.llm.factory.build_anthropic_adapter). This processor never + # configured a base_url (Anthropic's endpoint is effectively fixed), + # so this is a pure passthrough construction — no new SSRF surface, + # no behavior change. + adapter = build_anthropic_adapter(api_key=api_key) + client = await adapter.get_client() enrichments: list[dict[str, Any]] = [] - for i, record in enumerate(records): - prompt = _render(prompt_template, record.normalized_data) - logger.debug("claude req [%d/%d] | prompt_preview=%s", - i + 1, len(records), prompt[:200]) - try: - response = await client.messages.create( - model=model, - max_tokens=max_tokens, - messages=[{"role": "user", "content": prompt}], - ) - text = response.content[0].text - usage = response.usage - logger.info("claude resp [%d/%d] | input_tokens=%d output_tokens=%d preview=%s", - i + 1, len(records), - usage.input_tokens, usage.output_tokens, - text[:200]) + try: + for i, record in enumerate(records): + prompt = _render(prompt_template, record.normalized_data) + logger.debug("claude req [%d/%d] | prompt_preview=%s", + i + 1, len(records), prompt[:200]) try: - enrichment = json.loads(text) - except json.JSONDecodeError: - enrichment = {"analysis": text} - enrichments.append(enrichment) - except Exception as exc: - logger.error("claude error [%d/%d] | %s", i + 1, len(records), exc) - enrichments.append({"error": str(exc)}) + response = await client.messages.create( + model=model, + max_tokens=max_tokens, + messages=[{"role": "user", "content": prompt}], + ) + text = response.content[0].text + usage = response.usage + logger.info("claude resp [%d/%d] | input_tokens=%d output_tokens=%d preview=%s", + i + 1, len(records), + usage.input_tokens, usage.output_tokens, + text[:200]) + try: + enrichment = json.loads(text) + except json.JSONDecodeError: + enrichment = {"analysis": text} + enrichments.append(enrichment) + except Exception as exc: + logger.error("claude error [%d/%d] | %s", i + 1, len(records), exc) + enrichments.append({"error": str(exc)}) + finally: + await adapter.aclose() logger.info("claude processor done | success=%d errors=%d", sum(1 for e in enrichments if "error" not in e), diff --git a/backend/processors/local_processor.py b/backend/processors/local_processor.py index cd232a4..189e1a2 100644 --- a/backend/processors/local_processor.py +++ b/backend/processors/local_processor.py @@ -1,4 +1,28 @@ -"""Local model processor via Ollama/vLLM compatible API.""" +"""Local model processor via Ollama/vLLM compatible API. + +GOAL-6 PR-E note: deliberately NOT routed through +``backend.llm.factory``/``OpenAICompatAdapter`` like the openai/claude +processors. Two real incompatibilities, not just "not bothered yet": + + * ``api_style="ollama"`` (the default) speaks Ollama's *native* + ``POST /api/generate`` protocol (``{"model", "prompt", "stream"}`` in, + ``{"response": ...}`` out) — a different wire protocol entirely from + ``OpenAICompatAdapter``'s ``AsyncOpenAI`` client, which only knows the + OpenAI ``/v1/chat/completions`` shape. There is no adapter call that + reaches ``/api/generate`` without changing what gets sent over the wire. + * even the ``api_style="openai"`` branch has a per-call configurable + ``timeout`` (``config.get("timeout", 120)``) threaded straight into the + raw ``httpx.AsyncClient`` — ``OpenAICompatAdapter`` (frozen behavior, + PR-B, 1599-test baseline) has no parameter to accept a caller-supplied + timeout, so swapping in the adapter here would silently drop that + config knob for anyone using it. + +Also has no SSRF guard today for either branch (raw ``httpx.AsyncClient``, +no ``url_guard`` call) — unlike ``openai_processor``/``skill_channel``. This +is pre-existing behavior, left as-is; closing it would need its own change +(not a client-construction consolidation) and is out of PR-E's "zero +regression" scope. +""" import json import re diff --git a/backend/processors/openai_processor.py b/backend/processors/openai_processor.py index 658732e..67ec4fa 100644 --- a/backend/processors/openai_processor.py +++ b/backend/processors/openai_processor.py @@ -2,16 +2,14 @@ import json import logging +import os import re from typing import TYPE_CHECKING, Any +from backend.llm.base import LlmAdapterError +from backend.llm.factory import build_openai_compat_adapter from backend.processors.base import AbstractProcessor, ProcessingResult from backend.processors.registry import register_processor -from backend.security.url_guard import ( - PinnedAsyncHTTPTransport, - SSRFValidationError, - avalidate_public_url_and_ip, -) if TYPE_CHECKING: from backend.models.record import CollectedRecord @@ -38,42 +36,12 @@ async def process( config: dict[str, Any], ) -> ProcessingResult: try: - from openai import AsyncOpenAI + from openai import AsyncOpenAI # noqa: F401 -- import-availability probe only except ImportError: return ProcessingResult(success=False, error="openai package not installed") - api_key = config.get("api_key") or __import__("os").environ.get("OPENAI_API_KEY", "") + api_key = config.get("api_key") or os.environ.get("OPENAI_API_KEY", "") base_url: str | None = config.get("base_url") or None - # Key-exfil guard: base_url is DB/config-supplied — if it doesn't pass - # the SSRF/public-host check, don't attach api_key to a client pointed - # at it. None (OpenAI's own default endpoint) is left unvalidated. - # - # Full DNS-rebinding closure (AUDIT B3 follow-up): AsyncOpenAI accepts - # an `http_client` (any httpx.AsyncClient), so — unlike a vendor SDK - # that hides its own connection handling — we CAN pin this one: build - # a PinnedAsyncHTTPTransport bound to the IP(s) validation just - # resolved and hand it in as http_client, same mechanism as - # backend.security.url_guard.guarded_async_client uses for plain - # httpx call sites. When base_url is None (SDK default endpoint, - # never validated — unchanged from before) there is nothing to pin, - # so http_client is left unset and AsyncOpenAI builds its own default - # client exactly as before. - pinned_http_client = None - if base_url: - try: - base_url, ips = await avalidate_public_url_and_ip(base_url) - except SSRFValidationError as exc: - return ProcessingResult( - success=False, error=f"openai processor: base_url rejected: {exc}" - ) - from urllib.parse import urlparse as _urlparse - - import httpx - - hostname = _urlparse(base_url).hostname or "" - pinned_http_client = httpx.AsyncClient( - transport=PinnedAsyncHTTPTransport(hostname, ips) - ) model = config.get("model", "gpt-4o-mini") max_tokens = config.get("max_tokens", 1024) use_json_mode = config.get("json_mode", base_url is None) @@ -81,7 +49,19 @@ async def process( logger.info("openai processor | model=%s base_url=%s max_tokens=%d records=%d", model, base_url or "(default)", max_tokens, len(records)) - client = AsyncOpenAI(api_key=api_key, base_url=base_url, http_client=pinned_http_client) + # GOAL-6 PR-E: client construction (SSRF guard + DNS-rebind pinning) + # is consolidated through backend.llm.openai_compat.OpenAICompatAdapter + # (via backend.llm.factory.build_openai_compat_adapter) — this used to + # be a verbatim duplicate of the same wiring in chat.py/skill_channel. + # Key-exfil guard behavior is unchanged: base_url is DB/config-supplied, + # so if it doesn't pass the SSRF/public-host check, api_key is never + # attached to a client pointed at it; None (OpenAI's own default + # endpoint) is left unvalidated, exactly as before. + adapter = build_openai_compat_adapter(base_url=base_url, api_key=api_key) + try: + client = await adapter.get_client() + except LlmAdapterError as exc: + return ProcessingResult(success=False, error=f"openai processor: {exc}") enrichments: list[dict[str, Any]] = [] try: @@ -117,8 +97,10 @@ async def process( # AsyncOpenAI does not close an externally-supplied http_client # (it doesn't own it) — close ours ourselves, same as the # `async with client:` scope guarded_async_client callers use. - if pinned_http_client is not None: - await pinned_http_client.aclose() + # OpenAICompatAdapter.aclose() is a no-op when base_url was never + # set (no pinned transport was ever created), matching the old + # `if pinned_http_client is not None` guard exactly. + await adapter.aclose() logger.info("openai processor done | success=%d errors=%d", sum(1 for e in enrichments if "error" not in e), diff --git a/backend/schemas/model_default.py b/backend/schemas/model_default.py new file mode 100644 index 0000000..1948013 --- /dev/null +++ b/backend/schemas/model_default.py @@ -0,0 +1,53 @@ +from datetime import datetime +from typing import Any + +from pydantic import BaseModel, Field, field_validator + +from backend.llm import is_valid_role +from backend.schemas.common import UTCModel + + +class ModelDefaultCandidate(BaseModel): + """One entry in ``ModelDefault.candidates`` (ordered: index 0 = primary, + the rest are failover order — PR-D resolver).""" + + provider_id: str + model_id: str + + +class ModelDefaultPut(BaseModel): + """Body for ``PUT /model-defaults`` (decision #10; PR-C wires the actual + endpoint — this is just the validated payload shape).""" + + role: str + candidates: list[ModelDefaultCandidate] = Field(default_factory=list) + + @field_validator("role") + @classmethod + def _validate_role(cls, v: str) -> str: + if not is_valid_role(v): + raise ValueError(f"invalid role: {v!r}") + return v + + +class ModelDefaultCandidatesBody(BaseModel): + """Body for ``PUT /model-defaults/{role}`` (decision #10). + + ``role`` comes from the URL path, not repeated in the body — the router + wraps this into a role-validated :class:`ModelDefaultPut` before handing + off to :func:`backend.services.provider_model_service.put_default`, so + the same closed-set check that schema already enforces gets reused + end-to-end instead of duplicated. + """ + + candidates: list[ModelDefaultCandidate] = Field(default_factory=list) + + +class ModelDefaultRead(UTCModel): + id: str + role: str + candidates: list[dict[str, Any]] + created_at: datetime + updated_at: datetime + + model_config = {"from_attributes": True} diff --git a/backend/schemas/provider_model.py b/backend/schemas/provider_model.py new file mode 100644 index 0000000..42230e3 --- /dev/null +++ b/backend/schemas/provider_model.py @@ -0,0 +1,93 @@ +from datetime import datetime +from typing import Any, Optional + +from pydantic import BaseModel, Field, field_validator + +from backend.llm import is_valid_model_source, is_valid_model_type +from backend.schemas.common import UTCModel + + +class ProviderModelCreate(BaseModel): + """Body for registering a provider model catalog entry. + + Minimal shape for PR-A: PR-C wires the actual sync/CRUD endpoints (decision + #10) that construct/consume this; here it's just the validated payload. + """ + + provider_id: str + model_id: str = Field(..., min_length=1, max_length=255) + model_type: str = "llm" + capabilities: Optional[dict[str, Any]] = None + source: str = "manual" + enabled: bool = True + + @field_validator("model_type") + @classmethod + def _validate_model_type(cls, v: str) -> str: + if not is_valid_model_type(v): + raise ValueError(f"invalid model_type: {v!r}") + return v + + @field_validator("source") + @classmethod + def _validate_source(cls, v: str) -> str: + if not is_valid_model_source(v): + raise ValueError(f"invalid source: {v!r}") + return v + + +class ProviderModelManualCreate(BaseModel): + """Body for ``POST /providers/{id}/models`` (decision #10) — the manual + catalog-entry endpoint. + + ``provider_id`` comes from the URL path, not repeated in the body. + ``source`` is always forced to ``"manual"`` server-side + (``backend.services.provider_model_service.add_manual_model``) — this is + the only way a ``source="manual"`` row gets created; sync (decision #3) + never writes one. + """ + + model_id: str = Field(..., min_length=1, max_length=255) + model_type: str = "llm" + capabilities: Optional[dict[str, Any]] = None + enabled: bool = True + + @field_validator("model_type") + @classmethod + def _validate_model_type(cls, v: str) -> str: + if not is_valid_model_type(v): + raise ValueError(f"invalid model_type: {v!r}") + return v + + +class ProviderModelUpdate(BaseModel): + """Body for ``PATCH /providers/{id}/models/{model_row_id}`` (decision + #10). All fields optional (only supplied ones change). ``model_id`` / + ``provider_id`` / ``source`` are immutable via this endpoint — flipping a + row between "discovered" and "manual" isn't a partial-update concern, + it's decided entirely by which endpoint created the row (decision #3). + """ + + model_type: Optional[str] = None + capabilities: Optional[dict[str, Any]] = None + enabled: Optional[bool] = None + + @field_validator("model_type") + @classmethod + def _validate_model_type(cls, v: Optional[str]) -> Optional[str]: + if v is not None and not is_valid_model_type(v): + raise ValueError(f"invalid model_type: {v!r}") + return v + + +class ProviderModelRead(UTCModel): + id: str + provider_id: str + model_id: str + model_type: str + capabilities: Optional[dict[str, Any]] + source: str + enabled: bool + created_at: datetime + + model_config = {"from_attributes": True} diff --git a/backend/security/url_guard.py b/backend/security/url_guard.py index 8fc86d5..8b57b8a 100644 --- a/backend/security/url_guard.py +++ b/backend/security/url_guard.py @@ -116,7 +116,9 @@ class SSRFValidationError(ValueError): """ -def is_ip_blocked(ip: ipaddress.IPv4Address | ipaddress.IPv6Address) -> bool: +def is_ip_blocked( + ip: ipaddress.IPv4Address | ipaddress.IPv6Address, *, allow_private: bool = False +) -> bool: """True if ``ip`` must not be reached by an outbound fetch. Blocks loopback (127.0.0.0/8, ::1), RFC1918 private ranges, link-local @@ -125,17 +127,31 @@ def is_ip_blocked(ip: ipaddress.IPv4Address | ipaddress.IPv6Address) -> bool: unspecified (0.0.0.0, ::), reserved, and multicast addresses. Anything not covered by one of those categories is treated as globally reachable and allowed. + + ``allow_private`` (default ``False`` — every existing call site keeps + today's behaviour unchanged): when ``True``, loopback / RFC1918-private / + link-local / IPv6-unique-local / the NetBird CGNAT shared space + (100.64.0.0/10) are treated as reachable. ``unspecified``/``multicast``/ + ``reserved`` are still always blocked — no legitimate provider endpoint + lives there, so there is no reason to ever allow them. + + This is a narrow, explicit opt-in (GOAL-6 PR-B, decision #6). It exists + because ``url_guard`` had **no** existing localhost/private-IP exemption + mechanism, yet self-hosted LLM providers (``ModelProvider.provider_type + == "local"`` — ollama on loopback, model-hotel on the NetBird fleet mesh) + legitimately live at exactly the addresses this guard otherwise blocks. + ``backend.llm.openai_compat.OpenAICompatAdapter`` is the only caller that + ever passes ``allow_private=True``, and only for ``provider_type == + "local"`` — ``openai``/``claude`` providers are always validated with + ``allow_private=False`` (the full, unmodified guard). """ + if ip.is_unspecified or ip.is_multicast or ip.is_reserved: + return True + if allow_private: + return False if isinstance(ip, ipaddress.IPv4Address) and ip in _CGNAT_SHARED_SPACE: return True - return ( - ip.is_loopback - or ip.is_private - or ip.is_link_local - or ip.is_unspecified - or ip.is_multicast - or ip.is_reserved - ) + return ip.is_loopback or ip.is_private or ip.is_link_local def resolve_hostname(hostname: str) -> list[str]: @@ -155,13 +171,13 @@ def resolve_hostname(hostname: str) -> list[str]: return sorted(addrs) -def _check_host_and_ips(hostname: str, ips: list[str]) -> None: +def _check_host_and_ips(hostname: str, ips: list[str], *, allow_private: bool = False) -> None: for raw_ip in ips: try: ip = ipaddress.ip_address(raw_ip) except ValueError: continue - if is_ip_blocked(ip): + if is_ip_blocked(ip, allow_private=allow_private): raise SSRFValidationError( f"URL host {hostname!r} resolves to a non-public address " f"({raw_ip}) — blocked to prevent SSRF against internal " @@ -169,7 +185,7 @@ def _check_host_and_ips(hostname: str, ips: list[str]) -> None: ) -def validate_public_url(url: str) -> str: +def validate_public_url(url: str, *, allow_private: bool = False) -> str: """Validate ``url`` is safe to fetch from the server; return it normalized. Synchronous — performs a blocking DNS lookup. Call @@ -181,16 +197,22 @@ def validate_public_url(url: str) -> str: * missing/invalid URL, or a scheme other than http/https, * missing hostname, * hostname/raw-IP resolves to a loopback/private/link-local/ - unique-local/unspecified/multicast address. + unique-local/unspecified/multicast address (unless ``allow_private`` + — see :func:`is_ip_blocked`). """ - return validate_public_url_and_ip(url)[0] + return validate_public_url_and_ip(url, allow_private=allow_private)[0] -def validate_public_url_and_ip(url: str) -> tuple[str, list[str]]: +def validate_public_url_and_ip( + url: str, *, allow_private: bool = False +) -> tuple[str, list[str]]: """Like :func:`validate_public_url`, but also returns the resolved IP(s) (sorted) so a caller can pin a redirect-safe connection to them — see the module docstring's DNS-rebinding note for why that pinning isn't wired through httpx in this pass. + + ``allow_private``: see :func:`is_ip_blocked` — default ``False`` keeps + every existing caller's behaviour unchanged. """ if not url or not isinstance(url, str): raise SSRFValidationError("URL is required") @@ -214,27 +236,31 @@ def validate_public_url_and_ip(url: str) -> tuple[str, list[str]]: literal_ip = None if literal_ip is not None: - _check_host_and_ips(hostname, [str(literal_ip)]) + _check_host_and_ips(hostname, [str(literal_ip)], allow_private=allow_private) return url, [str(literal_ip)] ips = resolve_hostname(hostname) - _check_host_and_ips(hostname, ips) + _check_host_and_ips(hostname, ips, allow_private=allow_private) return url, ips -async def avalidate_public_url(url: str) -> str: +async def avalidate_public_url(url: str, *, allow_private: bool = False) -> str: """Async-friendly wrapper: runs the (blocking DNS) validation off-thread via ``asyncio.to_thread`` so it never stalls the event loop. Prefer this from ``async def`` call sites; :func:`validate_public_url` remains for sync call sites (e.g. module-level helpers called before an event loop exists).""" - return await asyncio.to_thread(validate_public_url, url) + return await asyncio.to_thread(validate_public_url, url, allow_private=allow_private) -async def avalidate_public_url_and_ip(url: str) -> tuple[str, list[str]]: +async def avalidate_public_url_and_ip( + url: str, *, allow_private: bool = False +) -> tuple[str, list[str]]: """Async ``asyncio.to_thread`` wrapper around :func:`validate_public_url_and_ip`.""" - return await asyncio.to_thread(validate_public_url_and_ip, url) + return await asyncio.to_thread( + validate_public_url_and_ip, url, allow_private=allow_private + ) # ── DNS-rebinding-safe connection pinning ──────────────────────────────────── @@ -264,9 +290,12 @@ class _PinnedNetworkBackend: blocked address through this seam. """ - def __init__(self, hostname: str, ips: list[str]) -> None: + def __init__( + self, hostname: str, ips: list[str], *, allow_private: bool = False + ) -> None: self._hostname = hostname self._ips = ips + self._allow_private = allow_private self._next_ip_index = 0 from httpcore._backends.auto import AutoBackend @@ -294,7 +323,7 @@ async def connect_tcp( ip_obj = ipaddress.ip_address(dial_host) except ValueError: ip_obj = None - if ip_obj is not None and is_ip_blocked(ip_obj): + if ip_obj is not None and is_ip_blocked(ip_obj, allow_private=self._allow_private): raise SSRFValidationError( f"pinned connect target {dial_host!r} for host {host!r} is a " "non-public address — refused at connect time (SSRF guard " @@ -336,6 +365,7 @@ def __init__( hostname: str, ips: list[str], *, + allow_private: bool = False, verify: "bool | str" = True, http1: bool = True, http2: bool = False, @@ -365,12 +395,12 @@ def __init__( local_address=built_pool._local_address, uds=built_pool._uds, socket_options=built_pool._socket_options, - network_backend=_PinnedNetworkBackend(hostname, ips), + network_backend=_PinnedNetworkBackend(hostname, ips, allow_private=allow_private), ) async def guarded_async_client( - url: str, **client_kwargs: typing.Any + url: str, *, allow_private: bool = False, **client_kwargs: typing.Any ) -> tuple[httpx.AsyncClient, str]: """Validate ``url`` (SSRF guard) and return ``(client, validated_url)`` where ``client`` is an ``httpx.AsyncClient`` whose transport is pinned to @@ -382,6 +412,11 @@ async def guarded_async_client( :class:`PinnedAsyncHTTPTransport` for consistency and the connect-time ``is_ip_blocked`` defense-in-depth re-check, pinned to that single IP. + ``allow_private`` (default ``False``): see :func:`is_ip_blocked` — passed + through to both the initial validation and the pinned transport's + connect-time re-check. Only ``backend.llm.openai_compat`` passes ``True``, + and only for ``ModelProvider.provider_type == "local"``. + ``client_kwargs`` are forwarded to ``httpx.AsyncClient`` verbatim (timeout, headers, follow_redirects, etc) except ``transport``, which this function owns — passing it raises ``TypeError`` (same as httpx would for a @@ -390,8 +425,8 @@ async def guarded_async_client( if "transport" in client_kwargs: raise TypeError("guarded_async_client() sets 'transport' itself — do not pass one") - validated_url, ips = await avalidate_public_url_and_ip(url) + validated_url, ips = await avalidate_public_url_and_ip(url, allow_private=allow_private) hostname = urlparse(validated_url).hostname or "" - transport = PinnedAsyncHTTPTransport(hostname, ips) + transport = PinnedAsyncHTTPTransport(hostname, ips, allow_private=allow_private) client = httpx.AsyncClient(transport=transport, **client_kwargs) return client, validated_url diff --git a/backend/services/provider_model_service.py b/backend/services/provider_model_service.py new file mode 100644 index 0000000..b4bcaba --- /dev/null +++ b/backend/services/provider_model_service.py @@ -0,0 +1,302 @@ +"""Service-layer logic for GOAL-6 PR-C: provider model catalog (sync + CRUD) +and model_defaults (get/put), plus provider-delete catalog cleanup. + +Kept out of ``backend/api/v1/providers.py`` / ``backend/api/v1/model_defaults.py`` +(thin-endpoint convention — see ``backend/services/source_service.py`` and +friends) so the sync-upsert/manual-preservation logic (decision #3) and the +defaults validation logic are independently unit-testable without spinning up +the ASGI app, and so tests have one obvious seam (``get_adapter`` in this +module) to patch instead of reaching into the router. +""" + +from __future__ import annotations + +from typing import Any, Optional, TypedDict + +from sqlalchemy import delete, select +from sqlalchemy.ext.asyncio import AsyncSession + +from backend.llm import is_valid_role +from backend.llm.factory import get_adapter +from backend.models.model_default import ModelDefault +from backend.models.provider import ModelProvider +from backend.models.provider_model import ProviderModel + + +class ModelDefaultsValidationError(ValueError): + """Raised by :func:`put_default` for a bad role or a candidate that + doesn't resolve to a real provider/catalog entry. + + The message only ever names role/provider_id/model_id — safe to surface + verbatim as an HTTP 4xx ``detail`` (never touches ``api_key``). + """ + + +class SyncResult(TypedDict): + added: int + updated: int + kept_manual: int + pruned: int + + +# --------------------------------------------------------------------------- +# Providers (lookup helper shared by every endpoint below) +# --------------------------------------------------------------------------- + + +async def get_provider(db: AsyncSession, provider_id: str) -> Optional[ModelProvider]: + return await db.get(ModelProvider, provider_id) + + +# --------------------------------------------------------------------------- +# Catalog CRUD +# --------------------------------------------------------------------------- + + +async def list_models(db: AsyncSession, provider_id: str) -> list[ProviderModel]: + result = await db.execute( + select(ProviderModel) + .where(ProviderModel.provider_id == provider_id) + .order_by(ProviderModel.model_id) + ) + return list(result.scalars().all()) + + +async def get_model(db: AsyncSession, model_row_id: str) -> Optional[ProviderModel]: + return await db.get(ProviderModel, model_row_id) + + +async def add_manual_model(db: AsyncSession, provider_id: str, body: Any) -> ProviderModel: + """Insert a hand-entered catalog row. + + ``source`` is always forced to ``"manual"`` regardless of what ``body`` + carries (the request schema, ``ProviderModelManualCreate``, doesn't even + expose a ``source`` field) — decision #3's manual/discovered boundary is + enforced structurally: this is the ONLY function that ever writes + ``source="manual"``, :func:`sync_models` is the only one that writes + ``source="discovered"``. + """ + row = ProviderModel( + provider_id=provider_id, + model_id=body.model_id, + model_type=body.model_type, + capabilities=body.capabilities, + source="manual", + enabled=body.enabled, + ) + db.add(row) + await db.commit() + await db.refresh(row) + return row + + +async def update_model( + db: AsyncSession, model_row_id: str, body: Any +) -> Optional[ProviderModel]: + """Partial-update a catalog row (``enabled``/``capabilities``/``model_type``). + + Returns ``None`` if ``model_row_id`` doesn't exist — callers decide + whether that's a 404 (the router checks ownership against ``provider_id`` + *before* calling this, so this function itself never needs the parent + provider id). + """ + row = await db.get(ProviderModel, model_row_id) + if row is None: + return None + for field, value in body.model_dump(exclude_unset=True).items(): + setattr(row, field, value) + await db.commit() + await db.refresh(row) + return row + + +async def delete_model(db: AsyncSession, model_row_id: str) -> bool: + """Delete one catalog row. Returns ``False`` if it didn't exist.""" + row = await db.get(ProviderModel, model_row_id) + if row is None: + return False + await db.delete(row) + await db.commit() + return True + + +async def delete_provider_models(db: AsyncSession, provider_id: str) -> int: + """Wipe a provider's whole catalog. Returns the number of rows deleted. + + Called from the providers router's ``DELETE /providers/{id}`` BEFORE the + provider row itself is deleted (GOAL-6 PR-A note, decision #3): this + repo's runtime engine (``backend/database.py``) never issues ``PRAGMA + foreign_keys=ON``, so ``provider_models.provider_id``'s ``ondelete= + CASCADE`` clause never actually fires against the production sqlite file + — without this explicit cleanup, deleting a provider would silently + orphan its ``provider_models`` rows instead of cascading them away. + """ + result = await db.execute(delete(ProviderModel).where(ProviderModel.provider_id == provider_id)) + return result.rowcount or 0 + + +# --------------------------------------------------------------------------- +# Discovery sync (decision #3) +# --------------------------------------------------------------------------- + + +async def sync_models(db: AsyncSession, provider_id: str) -> Optional[SyncResult]: + """Discover a provider's models via its adapter and upsert them into the + catalog as ``source="discovered"`` rows. + + Returns ``None`` if ``provider_id`` doesn't exist (router 404s). Raises + whatever :class:`~backend.llm.base.LlmAdapterError` the adapter's + ``list_models()`` raises on a genuine discovery failure (connection + error, bad credentials, ...) — this mirrors ``list_models``'s own + contract (non-raising probes belong to ``test_connection``, not sync); + the router converts that into a 502 rather than a raw 500. + + Upsert rules (decision #3 + this PR's stale-row policy): + + * a discovered ``model_id`` with no existing row -> inserted + (``added``); + * a discovered ``model_id`` that already has a ``source="discovered"`` + row -> left alone, counted as ``updated`` (there is nothing richer + to write yet — ``list_models()`` only returns bare ids — but this + keeps the row from being re-inserted, which would violate the + ``(provider_id, model_id)`` unique constraint, and gives a future + richer discovery payload somewhere to land without reshaping this + function); + * a discovered ``model_id`` that already has a ``source="manual"`` + row -> left COMPLETELY untouched, counted as ``kept_manual`` + (decision #3, hard requirement: sync must never overwrite or delete + a manually-entered row, even one that happens to share a model_id + the provider also reports); + * an existing ``source="discovered"`` row whose ``model_id`` was NOT + in this sync's results (the provider no longer serves it) -> + deleted, counted as ``pruned``. This half is this PR's own design + choice (decision #3 only pins "manual survives", it doesn't mandate + stale-discovered pruning) — without it, re-running sync against a + provider that removed a model would leave a phantom catalog entry + forever, which defeats sync being idempotent/re-runnable as the + source of truth for "what does this provider serve right now". + Manual rows are NEVER subject to this prune, regardless of whether + their model_id appears in the fresh discovery results. + """ + provider = await get_provider(db, provider_id) + if provider is None: + return None + + adapter = get_adapter(provider) + discovered_ids = await adapter.list_models() + discovered_set = set(discovered_ids) + + existing = await list_models(db, provider_id) + existing_by_model_id = {row.model_id: row for row in existing} + + added = updated = kept_manual = pruned = 0 + + for model_id in discovered_ids: + row = existing_by_model_id.get(model_id) + if row is None: + db.add( + ProviderModel( + provider_id=provider_id, + model_id=model_id, + source="discovered", + ) + ) + added += 1 + elif row.source == "manual": + kept_manual += 1 + else: + updated += 1 + + for row in existing: + if row.source == "discovered" and row.model_id not in discovered_set: + await db.delete(row) + pruned += 1 + + await db.commit() + return SyncResult(added=added, updated=updated, kept_manual=kept_manual, pruned=pruned) + + +# --------------------------------------------------------------------------- +# Connection test +# --------------------------------------------------------------------------- + + +async def test_connection(db: AsyncSession, provider_id: str) -> Optional[dict]: + """Probe a provider via its adapter. Returns ``None`` if ``provider_id`` + doesn't exist (router 404s); otherwise the + :class:`~backend.llm.base.ConnectionTestResult` dict as-is — + ``test_connection()`` never raises and never leaks ``api_key`` (enforced + in the adapters themselves, see ``backend.llm.base.redact_secret``).""" + provider = await get_provider(db, provider_id) + if provider is None: + return None + adapter = get_adapter(provider) + return dict(await adapter.test_connection()) + + +# --------------------------------------------------------------------------- +# model_defaults +# --------------------------------------------------------------------------- + + +async def get_defaults(db: AsyncSession) -> list[ModelDefault]: + result = await db.execute(select(ModelDefault).order_by(ModelDefault.role)) + return list(result.scalars().all()) + + +async def put_default(db: AsyncSession, role: str, candidates: list[Any]) -> ModelDefault: + """Validate + upsert the ``model_defaults`` row for ``role``. + + Validation (decision #10): + + * ``role`` must be in the closed set (``backend.llm.is_valid_role`` — + the router's Pydantic path/body layer also checks this; this is + defense in depth for callers that hit this function directly, e.g. + tests); + * every candidate's ``(provider_id, model_id)`` pair must reference a + provider that exists AND a catalog row that exists for that exact + pair. A candidate naming a real provider but a ``model_id`` never + synced/registered into that provider's catalog is rejected outright + — otherwise the resolver (PR-D) would silently try to route to a + model nobody ever confirmed that provider actually serves. + + Raises :class:`ModelDefaultsValidationError` (subclass of ``ValueError``) + naming the first bad role/candidate found; the router maps that to a + 4xx with the message as ``detail``. + """ + if not is_valid_role(role): + raise ModelDefaultsValidationError(f"invalid role: {role!r}") + + for candidate in candidates: + provider_id = candidate.provider_id + model_id = candidate.model_id + provider = await get_provider(db, provider_id) + if provider is None: + raise ModelDefaultsValidationError( + f"candidate references nonexistent provider_id={provider_id!r}" + ) + result = await db.execute( + select(ProviderModel).where( + ProviderModel.provider_id == provider_id, + ProviderModel.model_id == model_id, + ) + ) + if result.scalar_one_or_none() is None: + raise ModelDefaultsValidationError( + f"candidate model_id={model_id!r} is not in provider " + f"{provider_id!r}'s catalog (sync or register it first)" + ) + + candidates_json = [c.model_dump() for c in candidates] + + result = await db.execute(select(ModelDefault).where(ModelDefault.role == role)) + row = result.scalar_one_or_none() + if row is None: + row = ModelDefault(role=role, candidates=candidates_json) + db.add(row) + else: + row.candidates = candidates_json + + await db.commit() + await db.refresh(row) + return row diff --git a/frontend/app/(app)/providers/page.tsx b/frontend/app/(app)/providers/page.tsx index ac7aba1..ec1fbe3 100644 --- a/frontend/app/(app)/providers/page.tsx +++ b/frontend/app/(app)/providers/page.tsx @@ -1,12 +1,19 @@ 'use client' -import { KeyRound } from 'lucide-react' +import { useState } from 'react' +import { ChevronDown, ChevronUp, KeyRound, Loader2, Pencil, Plug, Plus, Trash2 } from 'lucide-react' +import { toast } from 'sonner' -import { useProviders } from '@/lib/api/hooks' +import { useDeleteProvider, useProviders, useTestProvider } from '@/lib/api/hooks' +import type { ConnectionTestResult, ModelProvider } from '@/lib/api/types' +import { ModelDefaultsCard } from '@/components/providers/model-defaults-card' +import { ProviderCatalogPanel } from '@/components/providers/provider-catalog-panel' +import { ProviderFormDialog } from '@/components/providers/provider-form-dialog' import { BACKEND_HINT, EmptyState, ErrorState, LoadingState } from '@/components/shell/data-states' import { PageContainer } from '@/components/shell/page-container' import { StatusBadge } from '@/components/shell/status-badge' import { Badge } from '@/components/ui/badge' +import { Button } from '@/components/ui/button' import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card' const TYPE_LABEL: Record = { @@ -19,8 +26,45 @@ export default function ProvidersPage() { const { data, isLoading, isError, error } = useProviders() const providers = data?.data ?? [] + // Expand/test/delete-confirm state all live only for the current page + // session — none of this is persisted server-side (GET /providers never + // returns a last-test-result field), which matches the backend contract. + const [expandedId, setExpandedId] = useState(null) + const [confirmDeleteId, setConfirmDeleteId] = useState(null) + const [testResults, setTestResults] = useState>({}) + + const testMutation = useTestProvider() + const deleteMutation = useDeleteProvider() + + const handleTest = (p: ModelProvider) => { + testMutation.mutate(p.id, { + onSuccess: (result) => setTestResults((prev) => ({ ...prev, [p.id]: result })), + onError: (e: Error) => toast.error(e.message), + }) + } + + const handleDelete = (p: ModelProvider) => { + if (confirmDeleteId !== p.id) { + setConfirmDeleteId(p.id) + return + } + deleteMutation.mutate(p.id, { + onSuccess: () => { + toast.success('已删除供应商') + setConfirmDeleteId(null) + }, + onError: (e: Error) => toast.error(e.message), + }) + } + return ( - + } /> + } + > {isLoading ? ( ) : isError ? ( @@ -28,32 +72,99 @@ export default function ProvidersPage() { ) : providers.length === 0 ? ( ) : ( -
- {providers.map((p) => ( - - -
-
- - - - {p.name} -
- -
- - -
- {TYPE_LABEL[p.provider_type] ?? p.provider_type} - {p.default_model ? {p.default_model} : null} -
- {p.base_url ? ( -

{p.base_url}

- ) : null} - - - ))} -
+ <> +
+ {providers.map((p) => { + const expanded = expandedId === p.id + const testResult = testResults[p.id] + const testing = testMutation.isPending && testMutation.variables === p.id + + return ( + + +
+
+ + + + {p.name} +
+ +
+ + +
+ {TYPE_LABEL[p.provider_type] ?? p.provider_type} + {p.default_model ? {p.default_model} : null} + {p.has_api_key ? ( + + {p.api_key_preview ?? '已配置密钥'} + + ) : ( + 未配置密钥 + )} +
+ {p.base_url ? ( +

{p.base_url}

+ ) : null} + + {testResult ? ( +

+ {testResult.ok + ? `连接正常 · 延迟 ${testResult.latency_ms ?? '—'}ms` + : testResult.error ?? '连接失败'} +

+ ) : null} + +
+ + + } + triggerVariant="ghost" + triggerSize="xs" + /> + +
+ + {expanded ? : null} + + + ) + })} +
+ + + )} ) diff --git a/frontend/components/providers/model-defaults-card.tsx b/frontend/components/providers/model-defaults-card.tsx new file mode 100644 index 0000000..e7424f6 --- /dev/null +++ b/frontend/components/providers/model-defaults-card.tsx @@ -0,0 +1,217 @@ +'use client' + +import { useEffect, useState } from 'react' +import { ArrowDown, ArrowUp, Plus, X } from 'lucide-react' +import { toast } from 'sonner' + +import { useModelDefaults, useProviderModels, usePutModelDefault } from '@/lib/api/hooks' +import type { ModelDefaultCandidate, ModelProvider, ModelRole } from '@/lib/api/types' +import { Button } from '@/components/ui/button' +import { Card, CardContent, CardDescription, CardHeader, CardTitle } from '@/components/ui/card' +import { Input } from '@/components/ui/input' +import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from '@/components/ui/select' +import { Separator } from '@/components/ui/separator' + +const ROLE_META: Record = { + chat: { label: '对话', description: 'agent 坞对话模型' }, + executor: { label: '执行', description: 'skill_channel 执行模型（轻量/低成本）' }, + enrichment: { label: '富化兜底', description: 'pipeline 富化兜底模型' }, +} + +const ROLES: ModelRole[] = ['chat', 'executor', 'enrichment'] + +function candidatesEqual(a: ModelDefaultCandidate[], b: ModelDefaultCandidate[]) { + if (a.length !== b.length) return false + return a.every((c, i) => c.provider_id === b[i].provider_id && c.model_id === b[i].model_id) +} + +function RoleEditor({ + role, + initialCandidates, + providers, +}: { + role: ModelRole + initialCandidates: ModelDefaultCandidate[] + providers: ModelProvider[] +}) { + const [candidates, setCandidates] = useState(initialCandidates) + const [synced, setSynced] = useState(initialCandidates) + const [pickerProviderId, setPickerProviderId] = useState('') + const [pickerModelId, setPickerModelId] = useState('') + const putDefault = usePutModelDefault() + // Only fetch the picker provider's catalog once one is actually chosen — + // falls back to a free-text model_id input if the catalog hasn't loaded. + const providerModels = useProviderModels(pickerProviderId || null) + + // Re-seed the draft whenever the server-backed value for this role changes + // (e.g. after a save elsewhere invalidates ['model-defaults']). + useEffect(() => { + setCandidates(initialCandidates) + setSynced(initialCandidates) + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [JSON.stringify(initialCandidates)]) + + const providerName = (id: string) => providers.find((p) => p.id === id)?.name ?? id + + const move = (index: number, dir: -1 | 1) => { + setCandidates((list) => { + const target = index + dir + if (target < 0 || target >= list.length) return list + const next = [...list] + ;[next[index], next[target]] = [next[target], next[index]] + return next + }) + } + + const remove = (index: number) => setCandidates((list) => list.filter((_, i) => i !== index)) + + const add = () => { + if (!pickerProviderId || !pickerModelId.trim()) return + setCandidates((list) => [...list, { provider_id: pickerProviderId, model_id: pickerModelId.trim() }]) + setPickerModelId('') + } + + const dirty = !candidatesEqual(candidates, synced) + const availableModels = providerModels.data?.data ?? [] + + const save = () => { + putDefault.mutate( + { role, candidates }, + { + onSuccess: (result) => { + toast.success(`已保存「${ROLE_META[role].label}」候选列表`) + setSynced(result.candidates) + }, + onError: (e: Error) => toast.error(e.message), + }, + ) + } + + return ( +
+
+
+ {ROLE_META[role].label} + {ROLE_META[role].description} +
+ +
+ + {candidates.length === 0 ? ( +

尚未配置候选模型。

+ ) : ( +
    + {candidates.map((c, i) => ( +
  • + {i + 1}. + + {providerName(c.provider_id)} · {c.model_id} + + + + +
    • + ))} +
+ )} + +
+ + + {pickerProviderId && availableModels.length > 0 ? ( + + ) : ( + setPickerModelId(e.target.value)} + placeholder="model_id" + className="h-7 w-48 text-xs" + /> + )} + + +
+
+ ) +} + +export function ModelDefaultsCard({ providers }: { providers: ModelProvider[] }) { + const { data, isLoading, isError, error } = useModelDefaults() + const defaultsByRole = new Map((data?.data ?? []).map((d) => [d.role, d.candidates])) + + return ( + + + 角色默认模型 + 为对话 / 执行 / 富化兜底三种角色配置候选模型的故障转移顺序 + + + {isLoading ? ( +

加载中…

+ ) : isError ? ( +

{(error as Error)?.message ?? '加载失败'}

+ ) : ( + ROLES.map((role, i) => ( +
+ {i > 0 ? : null} + +
+ )) + )} + + + ) +} diff --git a/frontend/components/providers/provider-catalog-panel.tsx b/frontend/components/providers/provider-catalog-panel.tsx new file mode 100644 index 0000000..bcbcaf6 --- /dev/null +++ b/frontend/components/providers/provider-catalog-panel.tsx @@ -0,0 +1,163 @@ +'use client' + +import { useState } from 'react' +import { Loader2, Plus, RefreshCw, Trash2 } from 'lucide-react' +import { toast } from 'sonner' + +import { + useAddProviderModel, + useDeleteProviderModel, + useProviderModels, + useSyncProviderModels, + useUpdateProviderModel, +} from '@/lib/api/hooks' +import type { ModelProvider } from '@/lib/api/types' +import { Badge } from '@/components/ui/badge' +import { Button } from '@/components/ui/button' +import { Input } from '@/components/ui/input' +import { Switch } from '@/components/ui/switch' +import { Table, TableBody, TableCell, TableHead, TableHeader, TableRow } from '@/components/ui/table' + +const SOURCE_LABEL: Record = { + discovered: '自动发现', + manual: '手动添加', +} + +// Lazily-mounted panel — only rendered by the parent card while expanded, so +// useProviderModels only fires once the operator actually asks to see it. +export function ProviderCatalogPanel({ provider }: { provider: ModelProvider }) { + const [newModelId, setNewModelId] = useState('') + const { data, isLoading, isError, error } = useProviderModels(provider.id) + const sync = useSyncProviderModels() + const addModel = useAddProviderModel() + const updateModel = useUpdateProviderModel() + const deleteModel = useDeleteProviderModel() + + const models = data?.data ?? [] + + const handleSync = () => { + sync.mutate(provider.id, { + onSuccess: (result) => { + toast.success( + `新增 ${result.added} · 更新 ${result.updated} · 保留手动 ${result.kept_manual} · 清理 ${result.pruned}`, + ) + }, + onError: (e: Error) => toast.error(e.message), + }) + } + + const handleAdd = (e: React.FormEvent) => { + e.preventDefault() + const modelId = newModelId.trim() + if (!modelId) return + addModel.mutate( + { providerId: provider.id, data: { model_id: modelId } }, + { + onSuccess: () => { + toast.success('已添加模型') + setNewModelId('') + }, + onError: (e: Error) => toast.error(e.message), + }, + ) + } + + return ( +
+
+ 模型目录 + +
+ + {isLoading ? ( +

加载中…

+ ) : isError ? ( +

{(error as Error)?.message ?? '加载模型目录失败'}

+ ) : models.length === 0 ? ( +

暂无模型，点击「同步」自动发现，或在下方手动添加。

+ ) : ( + + + + 模型 ID + 类型 + 来源 + 启用 + + + + + {models.map((m) => ( + + {m.model_id} + {m.model_type} + + + {SOURCE_LABEL[m.source] ?? m.source} + + + + + updateModel.mutate( + { providerId: provider.id, modelRowId: m.id, data: { enabled: v } }, + { onError: (e: Error) => toast.error(e.message) }, + ) + } + aria-label="启用/停用模型" + /> + + + + + + ))} + +
+ )} + +
+ setNewModelId(e.target.value)} + placeholder="手动添加 model_id，例如 gpt-4o-mini" + className="h-7 text-xs" + /> + +
+
+ ) +} diff --git a/frontend/components/providers/provider-form-dialog.tsx b/frontend/components/providers/provider-form-dialog.tsx new file mode 100644 index 0000000..249d570 --- /dev/null +++ b/frontend/components/providers/provider-form-dialog.tsx @@ -0,0 +1,262 @@ +'use client' + +import { useEffect, useState } from 'react' +import { toast } from 'sonner' + +import { useCreateProvider, useUpdateProvider } from '@/lib/api/hooks' +import type { ModelProvider, ModelProviderInput } from '@/lib/api/types' +import { Button } from '@/components/ui/button' +import { + Dialog, + DialogContent, + DialogDescription, + DialogFooter, + DialogHeader, + DialogTitle, + DialogTrigger, +} from '@/components/ui/dialog' +import { Field, FieldDescription, FieldGroup, FieldLabel } from '@/components/ui/field' +import { Input } from '@/components/ui/input' +import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from '@/components/ui/select' +import { Switch } from '@/components/ui/switch' + +type ProviderType = ModelProvider['provider_type'] + +interface FormState { + name: string + provider_type: ProviderType + base_url: string + api_key: string + default_model: string + notes: string + enabled: boolean +} + +const EMPTY_FORM: FormState = { + name: '', + provider_type: 'openai', + base_url: '', + api_key: '', + default_model: '', + notes: '', + enabled: true, +} + +// model-hotel is the user's self-hosted OpenAI-compatible gateway (see +// MEMORY.md model-hotel-5080) — base_url below is a placeholder LAN address, +// the user fills in their real endpoint after applying the preset. +const PRESETS: { key: string; label: string; fill: Partial }[] = [ + { key: 'claude', label: 'Claude', fill: { provider_type: 'claude', name: 'Claude', base_url: '' } }, + { key: 'openai', label: 'OpenAI', fill: { provider_type: 'openai', name: 'OpenAI', base_url: '' } }, + { + key: 'model-hotel', + label: 'model-hotel', + fill: { provider_type: 'local', name: 'model-hotel', base_url: 'http://localhost:4000/v1' }, + }, +] + +function providerToForm(p: ModelProvider): FormState { + return { + name: p.name, + provider_type: p.provider_type, + base_url: p.base_url ?? '', + api_key: '', + default_model: p.default_model ?? '', + notes: p.notes ?? '', + enabled: p.enabled, + } +} + +export function ProviderFormDialog({ + mode, + provider, + triggerLabel, + triggerIcon, + triggerVariant = 'default', + triggerSize = 'sm', +}: { + mode: 'create' | 'edit' + /** Required for mode="edit" — the row being edited, prefills the form. */ + provider?: ModelProvider + triggerLabel: string + triggerIcon?: React.ReactNode + triggerVariant?: React.ComponentProps['variant'] + triggerSize?: React.ComponentProps['size'] +}) { + const [open, setOpen] = useState(false) + const [form, setForm] = useState(() => (provider ? providerToForm(provider) : EMPTY_FORM)) + const createMutation = useCreateProvider() + const updateMutation = useUpdateProvider() + const pending = createMutation.isPending || updateMutation.isPending + + // Reset the draft to the current server row (or blank) every time the + // dialog opens, so stale edits from a previous open don't leak in. + useEffect(() => { + if (open) { + setForm(provider ? providerToForm(provider) : EMPTY_FORM) + } + // eslint-disable-next-line react-hooks/exhaustive-deps + }, [open]) + + const applyPreset = (fill: Partial) => setForm((f) => ({ ...f, ...fill })) + + const handleSubmit = (e: React.FormEvent) => { + e.preventDefault() + if (!form.name.trim()) { + toast.error('请填写供应商名称') + return + } + + // ModelProviderInput fields are all optional — omit blanks instead of + // sending empty strings so PATCH's "unset = don't change" semantics hold + // (matters most for api_key: blank means "leave the stored key alone"). + const payload: ModelProviderInput = { + name: form.name.trim(), + provider_type: form.provider_type, + enabled: form.enabled, + } + if (form.base_url.trim()) payload.base_url = form.base_url.trim() + if (form.default_model.trim()) payload.default_model = form.default_model.trim() + if (form.notes.trim()) payload.notes = form.notes.trim() + if (form.api_key.trim()) payload.api_key = form.api_key.trim() + + const onSuccess = () => { + toast.success(mode === 'create' ? '已添加供应商' : '已更新供应商') + setOpen(false) + } + const onError = (e: Error) => toast.error(e.message) + + if (mode === 'create') { + createMutation.mutate(payload, { onSuccess, onError }) + } else if (provider) { + updateMutation.mutate({ id: provider.id, data: payload }, { onSuccess, onError }) + } + } + + return ( + + }> + {triggerIcon} + {triggerLabel} + + +
+ + {mode === 'create' ? '添加模型供应商' : '编辑模型供应商'} + + {mode === 'create' + ? 'AI 模型接入凭证与端点配置。' + : '留空 API Key 表示不修改已保存的凭证。'} + + + + {mode === 'create' ? ( +
+ 快速填充： + {PRESETS.map((preset) => ( + + ))} +
+ ) : null} + + + + 名称 + setForm((f) => ({ ...f, name: e.target.value }))} + required + /> + + + + 类型 + + + + + Base URL + setForm((f) => ({ ...f, base_url: e.target.value }))} + /> + + + + API Key + setForm((f) => ({ ...f, api_key: e.target.value }))} + autoComplete="new-password" + /> + {mode === 'edit' ? ( + + 当前：{provider?.has_api_key ? (provider.api_key_preview ?? '已配置密钥') : '未配置密钥'} + + ) : null} + + + + 默认模型 + setForm((f) => ({ ...f, default_model: e.target.value }))} + /> + + + + 备注 + setForm((f) => ({ ...f, notes: e.target.value }))} + /> + + + + 启用 + setForm((f) => ({ ...f, enabled: v }))} + /> + + + + + + +
+ + + ) +} diff --git a/frontend/lib/api/endpoints.ts b/frontend/lib/api/endpoints.ts index a56f583..ed0e795 100644 --- a/frontend/lib/api/endpoints.ts +++ b/frontend/lib/api/endpoints.ts @@ -3,7 +3,14 @@ import type { AIAgent, AdvisoryReport, ApiResponse, + ConnectionTestResult, + ModelDefaultCandidate, + ModelDefaultRead, ModelProvider, + ModelProviderInput, + ModelRole, + ProviderModelRead, + ProviderModelSyncResult, BrowserActPack, BrowserBinding, ChromeEndpoint, @@ -276,15 +283,72 @@ export const listNotificationLogs = (params?: { rule_id?: string }) => export const listProviders = () => apiClient.get>('/providers').then((r) => r.data) -export const createProvider = (data: Partial) => +export const createProvider = (data: ModelProviderInput) => apiClient.post>('/providers', data).then((r) => r.data.data) -export const updateProvider = (id: string, data: Partial) => +export const updateProvider = (id: string, data: ModelProviderInput) => apiClient.patch>(`/providers/${id}`, data).then((r) => r.data.data) export const deleteProvider = (id: string) => apiClient.delete>(`/providers/${id}`).then((r) => r.data) +// Probes the provider's live endpoint (backend/llm adapter). NEVER raises for +// an ordinary connection failure — the failure is `ok: false` in a normal 200 +// response; it can still 404 if the provider id doesn't exist. +export const testProvider = (id: string) => + apiClient.post>(`/providers/${id}/test`).then((r) => r.data.data) + +// Discovers models via the adapter and idempotently upserts the provider's +// catalog — 'source=manual' rows are never touched. Can 502 if discovery +// itself fails (LlmAdapterError), surfaced via the axios interceptor's +// normalized Error.message. +export const syncProviderModels = (id: string) => + apiClient + .post>(`/providers/${id}/models/sync`) + .then((r) => r.data.data) + +export const listProviderModels = (id: string) => + apiClient.get>(`/providers/${id}/models`).then((r) => r.data) + +// Manual catalog add — `source` is forced server-side to 'manual', never send it. +export const addProviderModel = ( + providerId: string, + data: { + model_id: string + model_type?: string + capabilities?: Record | null + enabled?: boolean + }, +) => + apiClient + .post>(`/providers/${providerId}/models`, data) + .then((r) => r.data.data) + +export const updateProviderModel = ( + providerId: string, + modelRowId: string, + data: { model_type?: string; capabilities?: Record | null; enabled?: boolean }, +) => + apiClient + .patch>(`/providers/${providerId}/models/${modelRowId}`, data) + .then((r) => r.data.data) + +export const deleteProviderModel = (providerId: string, modelRowId: string) => + apiClient.delete>(`/providers/${providerId}/models/${modelRowId}`).then((r) => r.data) + +// ── Model defaults (GOAL-6 — role-based failover candidate lists) ─────────────── +// A role can be entirely absent from the list on a fresh install — that's a +// legitimate "no candidates configured yet" state, not an error. +export const listModelDefaults = () => + apiClient.get>('/model-defaults').then((r) => r.data) + +// role goes in the URL path, not the body. Can 400 if a candidate names a +// nonexistent provider or a model not in that provider's catalog. +export const putModelDefault = (role: ModelRole, candidates: ModelDefaultCandidate[]) => + apiClient + .put>(`/model-defaults/${role}`, { candidates }) + .then((r) => r.data.data) + // ── Agents ───────────────────────────────────────────────────────────────────── export const listAgents = (params?: { enabled?: boolean }) => apiClient.get>('/agents', { params }).then((r) => r.data) diff --git a/frontend/lib/api/hooks.ts b/frontend/lib/api/hooks.ts index 56e9795..8f6a4a5 100644 --- a/frontend/lib/api/hooks.ts +++ b/frontend/lib/api/hooks.ts @@ -1,8 +1,9 @@ 'use client' -import { useQuery } from '@tanstack/react-query' +import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query' import * as api from './endpoints' +import type { ModelDefaultCandidate, ModelProviderInput, ModelRole } from './types' export function useDashboardStats() { return useQuery({ @@ -152,6 +153,113 @@ export function useProviders() { }) } +export function useProviderModels(providerId: string | null) { + return useQuery({ + queryKey: ['providers', providerId, 'models'], + queryFn: () => api.listProviderModels(providerId as string), + enabled: !!providerId, + }) +} + +export function useModelDefaults() { + return useQuery({ + queryKey: ['model-defaults'], + queryFn: () => api.listModelDefaults(), + }) +} + +export function useCreateProvider() { + const queryClient = useQueryClient() + return useMutation({ + mutationFn: (data: ModelProviderInput) => api.createProvider(data), + onSuccess: () => queryClient.invalidateQueries({ queryKey: ['providers'] }), + }) +} + +export function useUpdateProvider() { + const queryClient = useQueryClient() + return useMutation({ + mutationFn: ({ id, data }: { id: string; data: ModelProviderInput }) => api.updateProvider(id, data), + onSuccess: () => queryClient.invalidateQueries({ queryKey: ['providers'] }), + }) +} + +export function useDeleteProvider() { + const queryClient = useQueryClient() + return useMutation({ + mutationFn: (id: string) => api.deleteProvider(id), + onSuccess: () => queryClient.invalidateQueries({ queryKey: ['providers'] }), + }) +} + +// Result is intentionally NOT cached in react-query state long-term by the +// caller — GET /providers never returns the last test outcome, so callers +// keep it in local component state keyed by provider id for the page session. +export function useTestProvider() { + return useMutation({ + mutationFn: (id: string) => api.testProvider(id), + }) +} + +export function useSyncProviderModels() { + const queryClient = useQueryClient() + return useMutation({ + mutationFn: (id: string) => api.syncProviderModels(id), + onSuccess: (_result, id) => queryClient.invalidateQueries({ queryKey: ['providers', id, 'models'] }), + }) +} + +export function useAddProviderModel() { + const queryClient = useQueryClient() + return useMutation({ + mutationFn: ({ + providerId, + data, + }: { + providerId: string + data: Parameters[1] + }) => api.addProviderModel(providerId, data), + onSuccess: (_result, { providerId }) => + queryClient.invalidateQueries({ queryKey: ['providers', providerId, 'models'] }), + }) +} + +export function useUpdateProviderModel() { + const queryClient = useQueryClient() + return useMutation({ + mutationFn: ({ + providerId, + modelRowId, + data, + }: { + providerId: string + modelRowId: string + data: Parameters[2] + }) => api.updateProviderModel(providerId, modelRowId, data), + onSuccess: (_result, { providerId }) => + queryClient.invalidateQueries({ queryKey: ['providers', providerId, 'models'] }), + }) +} + +export function useDeleteProviderModel() { + const queryClient = useQueryClient() + return useMutation({ + mutationFn: ({ providerId, modelRowId }: { providerId: string; modelRowId: string }) => + api.deleteProviderModel(providerId, modelRowId), + onSuccess: (_result, { providerId }) => + queryClient.invalidateQueries({ queryKey: ['providers', providerId, 'models'] }), + }) +} + +export function usePutModelDefault() { + const queryClient = useQueryClient() + return useMutation({ + mutationFn: ({ role, candidates }: { role: ModelRole; candidates: ModelDefaultCandidate[] }) => + api.putModelDefault(role, candidates), + onSuccess: () => queryClient.invalidateQueries({ queryKey: ['model-defaults'] }), + }) +} + export function useNodes() { return useQuery({ queryKey: ['nodes'], diff --git a/frontend/lib/api/types.ts b/frontend/lib/api/types.ts index eaf8b2c..111d336 100644 --- a/frontend/lib/api/types.ts +++ b/frontend/lib/api/types.ts @@ -1,13 +1,94 @@ +// Self-hosted LLM-provider runtime (GOAL-6, backend/llm/, no litellm) — mirrors +// backend.schemas.provider.ModelProviderRead.from_model exactly. The raw +// api_key is NEVER returned by the backend (from_model explicitly masks it) — +// only has_api_key / api_key_preview (e.g. "sk-...wxyz", null if unset) ever +// reach the frontend. See ModelProviderInput for the write-only request body. export interface ModelProvider { id: string name: string provider_type: 'claude' | 'openai' | 'local' + base_url: string | null + has_api_key: boolean + api_key_preview: string | null + default_model: string | null + notes: string | null + enabled: boolean + created_at: string + updated_at: string +} + +// Write-only request body for POST /providers and PATCH /providers/{id} +// (ModelProviderCreate / ModelProviderUpdate on the backend share this same +// shape, all-optional so PATCH can omit any field). `api_key` is the only +// place a raw key is ever sent — it is never echoed back on read (see +// ModelProvider.api_key_preview for the masked view). +export interface ModelProviderInput { + name?: string + provider_type?: ModelProvider['provider_type'] base_url?: string api_key?: string default_model?: string notes?: string + enabled?: boolean +} + +// One row of a provider's model catalog — mirrors backend.schemas.provider. +// ProviderModelRead. `source` distinguishes rows discovered via +// POST /providers/{id}/models/sync from ones added manually through +// POST /providers/{id}/models — sync is idempotent and never touches +// 'manual' rows. +export interface ProviderModelRead { + id: string + provider_id: string + model_id: string + model_type: string + capabilities: Record | null + source: 'discovered' | 'manual' enabled: boolean created_at: string +} + +// POST /providers/{id}/models/sync response — mirrors backend.schemas. +// provider.SyncResult. All-int and always present (unlike +// ConnectionTestResult, which is a TypedDict with optional fields). +export interface ProviderModelSyncResult { + added: number + updated: number + kept_manual: number + pruned: number +} + +// POST /providers/{id}/test response — mirrors backend.llm's +// ConnectionTestResult TypedDict (total=False), so every field beyond `ok` +// is optional: an ordinary connection failure comes back as a 200 with +// ok:false and no other fields populated, this endpoint never throws for a +// probe failure (it can still 404 if the provider id doesn't exist). +export interface ConnectionTestResult { + ok: boolean + latency_ms?: number | null + error?: string | null + models_sample?: string[] | null +} + +// Role a model-defaults candidate list resolves for (backend.llm role +// registry): chat = agent 坞对话模型, executor = skill_channel 执行模型 +// （轻量/低成本）, enrichment = pipeline 富化兜底模型. +export type ModelRole = 'chat' | 'executor' | 'enrichment' + +export interface ModelDefaultCandidate { + provider_id: string + model_id: string +} + +// GET /model-defaults / PUT /model-defaults/{role} — mirrors backend.schemas. +// provider.ModelDefaultRead. A role can be entirely absent from the GET list +// (fresh install, no row created yet) — render that as "no candidates +// configured", never crash on the missing entry. +export interface ModelDefaultRead { + id: string + role: ModelRole + candidates: ModelDefaultCandidate[] + created_at: string updated_at: string } diff --git a/tests/integration/test_model_defaults_api.py b/tests/integration/test_model_defaults_api.py new file mode 100644 index 0000000..6844334 --- /dev/null +++ b/tests/integration/test_model_defaults_api.py @@ -0,0 +1,150 @@ +"""Integration tests for GOAL-6 PR-C's ``GET|PUT /model-defaults`` (decision +#10): role closed-set validation and candidate (provider_id, model_id) +existence validation, both enforced before a row is ever stored. +""" + +import pytest + + +@pytest.fixture +def provider_data(): + return { + "name": "Test Provider", + "provider_type": "openai", + "base_url": "https://api.example.com/v1", + "api_key": "sk-defaults-test-key", + "default_model": "gpt-4o-mini", + "enabled": True, + } + + +async def _create_provider(client, provider_data) -> str: + resp = await client.post("/api/v1/providers", json=provider_data) + assert resp.status_code == 201 + return resp.json()["data"]["id"] + + +async def _register_catalog_model(client, provider_id: str, model_id: str) -> None: + resp = await client.post( + f"/api/v1/providers/{provider_id}/models", json={"model_id": model_id} + ) + assert resp.status_code == 201 + + +@pytest.mark.asyncio +async def test_list_model_defaults_empty(client): + resp = await client.get("/api/v1/model-defaults") + assert resp.status_code == 200 + assert resp.json()["data"] == [] + + +@pytest.mark.asyncio +async def test_put_model_default_rejects_invalid_role(client): + resp = await client.put( + "/api/v1/model-defaults/summarizer", json={"candidates": []} + ) + assert resp.status_code == 400 + + +@pytest.mark.asyncio +async def test_put_model_default_rejects_candidate_with_nonexistent_provider(client): + resp = await client.put( + "/api/v1/model-defaults/chat", + json={"candidates": [{"provider_id": "nonexistent-provider", "model_id": "m1"}]}, + ) + assert resp.status_code == 400 + assert "nonexistent-provider" in resp.json()["detail"] + + +@pytest.mark.asyncio +async def test_put_model_default_rejects_candidate_model_not_in_catalog(client, provider_data): + provider_id = await _create_provider(client, provider_data) + # Provider exists but no provider_models row for "unregistered-model". + resp = await client.put( + "/api/v1/model-defaults/chat", + json={"candidates": [{"provider_id": provider_id, "model_id": "unregistered-model"}]}, + ) + assert resp.status_code == 400 + assert "unregistered-model" in resp.json()["detail"] + + +@pytest.mark.asyncio +async def test_put_model_default_valid_candidates_stored_and_ordered(client, provider_data): + provider_id = await _create_provider(client, provider_data) + await _register_catalog_model(client, provider_id, "primary-model") + await _register_catalog_model(client, provider_id, "backup-model") + + put_resp = await client.put( + "/api/v1/model-defaults/chat", + json={ + "candidates": [ + {"provider_id": provider_id, "model_id": "primary-model"}, + {"provider_id": provider_id, "model_id": "backup-model"}, + ] + }, + ) + assert put_resp.status_code == 200 + data = put_resp.json()["data"] + assert data["role"] == "chat" + assert data["candidates"] == [ + {"provider_id": provider_id, "model_id": "primary-model"}, + {"provider_id": provider_id, "model_id": "backup-model"}, + ] + + list_resp = await client.get("/api/v1/model-defaults") + rows = list_resp.json()["data"] + assert len(rows) == 1 + assert rows[0]["role"] == "chat" + assert rows[0]["candidates"][0]["model_id"] == "primary-model" + + +@pytest.mark.asyncio +async def test_put_model_default_upserts_same_role(client, provider_data): + provider_id = await _create_provider(client, provider_data) + await _register_catalog_model(client, provider_id, "m1") + await _register_catalog_model(client, provider_id, "m2") + + first = await client.put( + "/api/v1/model-defaults/executor", + json={"candidates": [{"provider_id": provider_id, "model_id": "m1"}]}, + ) + assert first.status_code == 200 + first_id = first.json()["data"]["id"] + + second = await client.put( + "/api/v1/model-defaults/executor", + json={"candidates": [{"provider_id": provider_id, "model_id": "m2"}]}, + ) + assert second.status_code == 200 + assert second.json()["data"]["id"] == first_id # same row, upserted + assert second.json()["data"]["candidates"] == [{"provider_id": provider_id, "model_id": "m2"}] + + list_resp = await client.get("/api/v1/model-defaults") + rows = list_resp.json()["data"] + assert len(rows) == 1 # not duplicated + + +@pytest.mark.asyncio +async def test_put_model_default_empty_candidates_allowed(client): + """An empty candidate list is a legitimate way to clear a role's + defaults (no candidates to validate, nothing rejected).""" + resp = await client.put("/api/v1/model-defaults/enrichment", json={"candidates": []}) + assert resp.status_code == 200 + assert resp.json()["data"]["candidates"] == [] + + +@pytest.mark.asyncio +async def test_put_model_default_all_three_roles_independent(client, provider_data): + provider_id = await _create_provider(client, provider_data) + await _register_catalog_model(client, provider_id, "m1") + + for role in ("chat", "executor", "enrichment"): + resp = await client.put( + f"/api/v1/model-defaults/{role}", + json={"candidates": [{"provider_id": provider_id, "model_id": "m1"}]}, + ) + assert resp.status_code == 200 + + list_resp = await client.get("/api/v1/model-defaults") + rows = list_resp.json()["data"] + assert {r["role"] for r in rows} == {"chat", "executor", "enrichment"} diff --git a/tests/integration/test_provider_models_api.py b/tests/integration/test_provider_models_api.py new file mode 100644 index 0000000..058d584 --- /dev/null +++ b/tests/integration/test_provider_models_api.py @@ -0,0 +1,352 @@ +"""Integration tests for GOAL-6 PR-C's provider-scoped API (decision #10): +``POST /providers/{id}/test``, ``POST /providers/{id}/models/sync``, and +``GET|POST|PATCH|DELETE /providers/{id}/models``. + +The adapter (``backend.services.provider_model_service.get_adapter``) is +mocked in every test — nothing here makes a real network call. Two +security-critical properties get dedicated coverage: the stored ``api_key`` +never appears anywhere in a response body, and sync never overwrites or +deletes a ``source="manual"`` catalog row (decision #3). +""" + +from unittest.mock import AsyncMock, patch + +import pytest +from sqlalchemy import select + +from backend.models.provider_model import ProviderModel + +SECRET_KEY = "sk-test-super-secret-do-not-leak-98765" + + +@pytest.fixture +def provider_data(): + return { + "name": "Test Provider", + "provider_type": "openai", + "base_url": "https://api.example.com/v1", + "api_key": SECRET_KEY, + "default_model": "gpt-4o-mini", + "enabled": True, + } + + +async def _create_provider(client, provider_data) -> str: + resp = await client.post("/api/v1/providers", json=provider_data) + assert resp.status_code == 201 + return resp.json()["data"]["id"] + + +def _patch_adapter(**method_results): + """Patch provider_model_service.get_adapter to return a mock adapter + whose async methods return the given results. + + Usage: ``_patch_adapter(test_connection={"ok": True, ...})`` or + ``_patch_adapter(list_models=["m1", "m2"])``. + """ + mock_adapter = AsyncMock() + for method_name, result in method_results.items(): + getattr(mock_adapter, method_name).return_value = result + return patch( + "backend.services.provider_model_service.get_adapter", + return_value=mock_adapter, + ) + + +# --------------------------------------------------------------------------- +# POST /providers/{id}/test +# --------------------------------------------------------------------------- + + +@pytest.mark.asyncio +async def test_test_connection_success(client, provider_data): + provider_id = await _create_provider(client, provider_data) + + with _patch_adapter( + test_connection={ + "ok": True, + "latency_ms": 12.5, + "error": None, + "models_sample": ["gpt-4o-mini"], + } + ): + resp = await client.post(f"/api/v1/providers/{provider_id}/test") + + assert resp.status_code == 200 + data = resp.json()["data"] + assert data["ok"] is True + assert data["latency_ms"] == 12.5 + assert data["models_sample"] == ["gpt-4o-mini"] + assert SECRET_KEY not in resp.text + + +@pytest.mark.asyncio +async def test_test_connection_failure(client, provider_data): + provider_id = await _create_provider(client, provider_data) + + with _patch_adapter( + test_connection={"ok": False, "error": "connection refused", "latency_ms": None} + ): + resp = await client.post(f"/api/v1/providers/{provider_id}/test") + + assert resp.status_code == 200 + data = resp.json()["data"] + assert data["ok"] is False + assert data["error"] == "connection refused" + assert SECRET_KEY not in resp.text + + +@pytest.mark.asyncio +async def test_test_connection_provider_not_found(client): + resp = await client.post("/api/v1/providers/nonexistent-id/test") + assert resp.status_code == 404 + + +# --------------------------------------------------------------------------- +# POST /providers/{id}/models/sync — decision #3 +# --------------------------------------------------------------------------- + + +@pytest.mark.asyncio +async def test_sync_creates_discovered_rows(client, provider_data): + provider_id = await _create_provider(client, provider_data) + + with _patch_adapter(list_models=["m1", "m2"]): + resp = await client.post(f"/api/v1/providers/{provider_id}/models/sync") + + assert resp.status_code == 200 + assert resp.json()["data"] == {"added": 2, "updated": 0, "kept_manual": 0, "pruned": 0} + + list_resp = await client.get(f"/api/v1/providers/{provider_id}/models") + rows = list_resp.json()["data"] + assert {r["model_id"] for r in rows} == {"m1", "m2"} + assert all(r["source"] == "discovered" for r in rows) + + +@pytest.mark.asyncio +async def test_sync_idempotent_and_manual_preserved(client, provider_data): + """The scenario from GOAL-6 PR-C's spec: seed provider, sync [m1, m2] -> + 2 discovered rows; add manual m3; sync again with only [m1] -> m3 + (manual) survives untouched, m2 (stale discovered) is pruned, m1 is not + duplicated (idempotent).""" + provider_id = await _create_provider(client, provider_data) + + with _patch_adapter(list_models=["m1", "m2"]): + first = await client.post(f"/api/v1/providers/{provider_id}/models/sync") + assert first.json()["data"] == {"added": 2, "updated": 0, "kept_manual": 0, "pruned": 0} + + manual_resp = await client.post( + f"/api/v1/providers/{provider_id}/models", json={"model_id": "m3"} + ) + assert manual_resp.status_code == 201 + assert manual_resp.json()["data"]["source"] == "manual" + + with _patch_adapter(list_models=["m1"]): + second = await client.post(f"/api/v1/providers/{provider_id}/models/sync") + assert second.json()["data"] == {"added": 0, "updated": 1, "kept_manual": 0, "pruned": 1} + + list_resp = await client.get(f"/api/v1/providers/{provider_id}/models") + rows = {r["model_id"]: r for r in list_resp.json()["data"]} + assert set(rows) == {"m1", "m3"} + assert rows["m1"]["source"] == "discovered" + assert rows["m3"]["source"] == "manual" + + # Idempotency: re-running the identical [m1] sync doesn't duplicate m1 + # or touch the manual m3 row again. + with _patch_adapter(list_models=["m1"]): + third = await client.post(f"/api/v1/providers/{provider_id}/models/sync") + assert third.json()["data"] == {"added": 0, "updated": 1, "kept_manual": 0, "pruned": 0} + list_resp = await client.get(f"/api/v1/providers/{provider_id}/models") + assert {r["model_id"] for r in list_resp.json()["data"]} == {"m1", "m3"} + + +@pytest.mark.asyncio +async def test_sync_discovering_same_model_id_as_manual_row_counts_kept_manual( + client, provider_data +): + """If discovery reports a model_id that already has a manual row, sync + must not touch/duplicate it — counted as kept_manual, not added.""" + provider_id = await _create_provider(client, provider_data) + + manual_resp = await client.post( + f"/api/v1/providers/{provider_id}/models", json={"model_id": "m1"} + ) + assert manual_resp.status_code == 201 + + with _patch_adapter(list_models=["m1"]): + resp = await client.post(f"/api/v1/providers/{provider_id}/models/sync") + + assert resp.json()["data"] == {"added": 0, "updated": 0, "kept_manual": 1, "pruned": 0} + list_resp = await client.get(f"/api/v1/providers/{provider_id}/models") + rows = list_resp.json()["data"] + assert len(rows) == 1 + assert rows[0]["source"] == "manual" + + +@pytest.mark.asyncio +async def test_sync_provider_not_found(client): + with _patch_adapter(list_models=["m1"]): + resp = await client.post("/api/v1/providers/nonexistent-id/models/sync") + assert resp.status_code == 404 + + +@pytest.mark.asyncio +async def test_sync_adapter_failure_returns_502_not_500(client, provider_data): + """A genuine discovery failure -- the adapter contract says list_models() + raises LlmAdapterError -- surfaces as 502 (sanitized message already, + per the adapter's own guarantee), not a raw 500.""" + from backend.llm.base import LlmAdapterError + + provider_id = await _create_provider(client, provider_data) + mock_adapter = AsyncMock() + mock_adapter.list_models.side_effect = LlmAdapterError("connection timed out") + with patch( + "backend.services.provider_model_service.get_adapter", return_value=mock_adapter + ): + resp = await client.post(f"/api/v1/providers/{provider_id}/models/sync") + assert resp.status_code == 502 + assert "connection timed out" in resp.json()["detail"] + + +# --------------------------------------------------------------------------- +# Catalog CRUD +# --------------------------------------------------------------------------- + + +@pytest.mark.asyncio +async def test_add_manual_model_then_list_shows_source_manual(client, provider_data): + provider_id = await _create_provider(client, provider_data) + + resp = await client.post( + f"/api/v1/providers/{provider_id}/models", + json={"model_id": "custom-model", "model_type": "llm", "enabled": True}, + ) + assert resp.status_code == 201 + assert resp.json()["data"]["source"] == "manual" + + list_resp = await client.get(f"/api/v1/providers/{provider_id}/models") + rows = list_resp.json()["data"] + assert len(rows) == 1 + assert rows[0]["model_id"] == "custom-model" + assert rows[0]["source"] == "manual" + + +@pytest.mark.asyncio +async def test_add_manual_model_provider_not_found(client): + resp = await client.post( + "/api/v1/providers/nonexistent-id/models", json={"model_id": "m1"} + ) + assert resp.status_code == 404 + + +@pytest.mark.asyncio +async def test_patch_model_updates_enabled(client, provider_data): + provider_id = await _create_provider(client, provider_data) + create_resp = await client.post( + f"/api/v1/providers/{provider_id}/models", json={"model_id": "m1"} + ) + model_row_id = create_resp.json()["data"]["id"] + + patch_resp = await client.patch( + f"/api/v1/providers/{provider_id}/models/{model_row_id}", + json={"enabled": False}, + ) + assert patch_resp.status_code == 200 + assert patch_resp.json()["data"]["enabled"] is False + + list_resp = await client.get(f"/api/v1/providers/{provider_id}/models") + assert list_resp.json()["data"][0]["enabled"] is False + + +@pytest.mark.asyncio +async def test_patch_model_not_found(client, provider_data): + provider_id = await _create_provider(client, provider_data) + resp = await client.patch( + f"/api/v1/providers/{provider_id}/models/nonexistent-row", + json={"enabled": False}, + ) + assert resp.status_code == 404 + + +@pytest.mark.asyncio +async def test_delete_model_removes_row(client, provider_data): + provider_id = await _create_provider(client, provider_data) + create_resp = await client.post( + f"/api/v1/providers/{provider_id}/models", json={"model_id": "m1"} + ) + model_row_id = create_resp.json()["data"]["id"] + + delete_resp = await client.delete(f"/api/v1/providers/{provider_id}/models/{model_row_id}") + assert delete_resp.status_code == 200 + + list_resp = await client.get(f"/api/v1/providers/{provider_id}/models") + assert list_resp.json()["data"] == [] + + +@pytest.mark.asyncio +async def test_delete_model_not_found(client, provider_data): + provider_id = await _create_provider(client, provider_data) + resp = await client.delete(f"/api/v1/providers/{provider_id}/models/nonexistent-row") + assert resp.status_code == 404 + + +@pytest.mark.asyncio +async def test_model_row_scoped_to_its_own_provider(client, provider_data): + """A model row created under provider A is not reachable through + provider B's URL subtree (ownership check, not just existence).""" + provider_a = await _create_provider(client, provider_data) + provider_b = await _create_provider(client, {**provider_data, "name": "Provider B"}) + + create_resp = await client.post( + f"/api/v1/providers/{provider_a}/models", json={"model_id": "m1"} + ) + model_row_id = create_resp.json()["data"]["id"] + + cross_patch = await client.patch( + f"/api/v1/providers/{provider_b}/models/{model_row_id}", json={"enabled": False} + ) + assert cross_patch.status_code == 404 + + cross_delete = await client.delete(f"/api/v1/providers/{provider_b}/models/{model_row_id}") + assert cross_delete.status_code == 404 + + # Still present, untouched, under its real owner. + list_resp = await client.get(f"/api/v1/providers/{provider_a}/models") + assert list_resp.json()["data"][0]["enabled"] is True + + +# --------------------------------------------------------------------------- +# Provider-delete cleanup (PR-A FK-cascade note: sqlite won't cascade) +# --------------------------------------------------------------------------- + + +@pytest.mark.asyncio +async def test_delete_provider_cleans_up_catalog_rows(client, db_session, provider_data): + provider_id = await _create_provider(client, provider_data) + await client.post(f"/api/v1/providers/{provider_id}/models", json={"model_id": "m1"}) + await client.post(f"/api/v1/providers/{provider_id}/models", json={"model_id": "m2"}) + + rows_before = ( + ( + await db_session.execute( + select(ProviderModel).where(ProviderModel.provider_id == provider_id) + ) + ) + .scalars() + .all() + ) + assert len(rows_before) == 2 + + delete_resp = await client.delete(f"/api/v1/providers/{provider_id}") + assert delete_resp.status_code == 200 + + rows_after = ( + ( + await db_session.execute( + select(ProviderModel).where(ProviderModel.provider_id == provider_id) + ) + ) + .scalars() + .all() + ) + assert rows_after == [] diff --git a/tests/unit/llm/__init__.py b/tests/unit/llm/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tests/unit/llm/test_adapters.py b/tests/unit/llm/test_adapters.py new file mode 100644 index 0000000..fa740c7 --- /dev/null +++ b/tests/unit/llm/test_adapters.py @@ -0,0 +1,312 @@ +"""Unit tests for backend.llm's runtime adapters (GOAL-6 PR-B). + +Every SDK client (openai.AsyncOpenAI / anthropic.AsyncAnthropic) is mocked at +the class level — these tests never make a real network call. The two +security-critical properties get dedicated coverage: + + * the url_guard rejection path: a blocked base_url must raise *before* the + key-bearing SDK client is ever constructed (proven by asserting the + mocked SDK class was never called), and + * api_key never appears in any error string these adapters raise/return. +""" + +from types import SimpleNamespace +from unittest.mock import AsyncMock, MagicMock, patch + +import pytest + +from backend.llm.anthropic import AnthropicAdapter +from backend.llm.base import LlmAdapterError +from backend.llm.catalog import anthropic_catalog +from backend.llm.factory import get_adapter +from backend.llm.openai_compat import OpenAICompatAdapter +from backend.models.provider import ModelProvider + +SECRET_KEY = "sk-test-super-secret-do-not-leak-12345" + + +def _provider(**overrides) -> ModelProvider: + defaults = dict( + name="test-provider", + provider_type="openai", + base_url="https://api.example.com/v1", + api_key=SECRET_KEY, + default_model="gpt-4o-mini", + enabled=True, + ) + defaults.update(overrides) + return ModelProvider(**defaults) + + +def _models_page(ids: list[str]) -> SimpleNamespace: + return SimpleNamespace(data=[SimpleNamespace(id=i) for i in ids]) + + +def _chat_response(text: str) -> SimpleNamespace: + return SimpleNamespace( + choices=[SimpleNamespace(message=SimpleNamespace(content=text))] + ) + + +# ── OpenAICompatAdapter: chat / list_models / test_connection ────────────── + + +@pytest.mark.asyncio +async def test_openai_compat_chat_returns_text(): + provider = _provider(base_url=None) # skip guard/DNS — not under test here + mock_client = MagicMock() + mock_client.chat.completions.create = AsyncMock(return_value=_chat_response("hi there")) + with patch("openai.AsyncOpenAI", return_value=mock_client) as mock_cls: + adapter = OpenAICompatAdapter(provider) + result = await adapter.chat([{"role": "user", "content": "hello"}]) + assert result == "hi there" + mock_cls.assert_called_once() + _, kwargs = mock_client.chat.completions.create.call_args + assert kwargs["model"] == "gpt-4o-mini" # provider.default_model + + +@pytest.mark.asyncio +async def test_openai_compat_chat_model_override(): + provider = _provider(base_url=None) + mock_client = MagicMock() + mock_client.chat.completions.create = AsyncMock(return_value=_chat_response("ok")) + with patch("openai.AsyncOpenAI", return_value=mock_client): + adapter = OpenAICompatAdapter(provider) + await adapter.chat([{"role": "user", "content": "hello"}], model="gpt-4o") + _, kwargs = mock_client.chat.completions.create.call_args + assert kwargs["model"] == "gpt-4o" + + +@pytest.mark.asyncio +async def test_openai_compat_list_models_returns_ids(): + provider = _provider(base_url=None) + mock_client = MagicMock() + mock_client.models.list = AsyncMock(return_value=_models_page(["gpt-4o-mini", "gpt-4o"])) + with patch("openai.AsyncOpenAI", return_value=mock_client): + adapter = OpenAICompatAdapter(provider) + models = await adapter.list_models() + assert models == ["gpt-4o-mini", "gpt-4o"] + + +@pytest.mark.asyncio +async def test_openai_compat_test_connection_success(): + provider = _provider(base_url=None) + mock_client = MagicMock() + mock_client.models.list = AsyncMock(return_value=_models_page(["gpt-4o-mini"])) + with patch("openai.AsyncOpenAI", return_value=mock_client): + adapter = OpenAICompatAdapter(provider) + result = await adapter.test_connection() + assert result["ok"] is True + assert isinstance(result["latency_ms"], float) + assert result["models_sample"] == ["gpt-4o-mini"] + + +@pytest.mark.asyncio +async def test_openai_compat_test_connection_failure_sanitized(): + provider = _provider(base_url=None) + mock_client = MagicMock() + mock_client.models.list = AsyncMock( + side_effect=Exception(f"401 unauthorized: bad key {SECRET_KEY}") + ) + with patch("openai.AsyncOpenAI", return_value=mock_client): + adapter = OpenAICompatAdapter(provider) + result = await adapter.test_connection() + assert result["ok"] is False + assert SECRET_KEY not in result["error"] + assert "REDACTED" in result["error"] + + +@pytest.mark.asyncio +async def test_openai_compat_chat_failure_raises_llm_adapter_error_without_key(): + provider = _provider(base_url=None) + mock_client = MagicMock() + mock_client.chat.completions.create = AsyncMock( + side_effect=Exception(f"upstream rejected key={SECRET_KEY}") + ) + with patch("openai.AsyncOpenAI", return_value=mock_client): + adapter = OpenAICompatAdapter(provider) + with pytest.raises(LlmAdapterError) as exc_info: + await adapter.chat([{"role": "user", "content": "hi"}]) + assert SECRET_KEY not in str(exc_info.value) + + +# ── url_guard rejection: guard runs BEFORE a key-bearing client is built ─── + + +@pytest.mark.asyncio +async def test_openai_compat_rejects_private_base_url_before_building_client(): + """A blocked base_url must raise before openai.AsyncOpenAI is ever + constructed — proving the api_key never gets attached to a client aimed + at a non-public host.""" + provider = _provider(provider_type="openai", base_url="http://10.0.0.5/v1") + with patch("openai.AsyncOpenAI") as mock_cls: + adapter = OpenAICompatAdapter(provider) + with pytest.raises(LlmAdapterError, match="rejected"): + await adapter.list_models() + mock_cls.assert_not_called() + + +@pytest.mark.asyncio +async def test_openai_compat_rejects_private_base_url_error_has_no_key(): + provider = _provider(provider_type="openai", base_url="http://127.0.0.1:8080/v1") + with patch("openai.AsyncOpenAI"): + adapter = OpenAICompatAdapter(provider) + with pytest.raises(LlmAdapterError) as exc_info: + await adapter.chat([{"role": "user", "content": "hi"}]) + assert SECRET_KEY not in str(exc_info.value) + + +# ── local-address exemption (decision #6) ─────────────────────────────────── + + +@pytest.mark.asyncio +async def test_openai_compat_local_type_allows_loopback_base_url(): + """provider_type == "local" is exempted from the private/loopback block + (ollama on 127.0.0.1) — the guard must not reject building the client.""" + provider = _provider(provider_type="local", base_url="http://127.0.0.1:11434/v1") + mock_client = MagicMock() + mock_client.models.list = AsyncMock(return_value=_models_page(["qwen3:4b"])) + with patch("openai.AsyncOpenAI", return_value=mock_client) as mock_cls: + adapter = OpenAICompatAdapter(provider) + models = await adapter.list_models() + mock_cls.assert_called_once() + assert models == ["qwen3:4b"] + + +@pytest.mark.asyncio +async def test_openai_compat_openai_type_rejects_same_loopback_url(): + """The exact same loopback URL that is fine for provider_type=="local" + must still be rejected for provider_type=="openai" — the exemption is + scoped strictly by type, not by address.""" + provider = _provider(provider_type="openai", base_url="http://127.0.0.1:11434/v1") + with patch("openai.AsyncOpenAI") as mock_cls: + adapter = OpenAICompatAdapter(provider) + with pytest.raises(LlmAdapterError, match="rejected"): + await adapter.list_models() + mock_cls.assert_not_called() + + +@pytest.mark.asyncio +async def test_openai_compat_local_type_still_pins_transport(): + """Even with the private-IP allowance, the client is still built with a + pinned http_client (DNS-rebind protection is unaffected by the + local-address exemption — allow_private only changes which addresses + pass the block-list check).""" + provider = _provider(provider_type="local", base_url="http://127.0.0.1:11434/v1") + with patch("openai.AsyncOpenAI") as mock_cls: + adapter = OpenAICompatAdapter(provider) + await adapter._get_client() + _, kwargs = mock_cls.call_args + assert kwargs["http_client"] is not None + await adapter.aclose() + + +# ── AnthropicAdapter: chat / list_models / test_connection ────────────────── + + +def _anthropic_provider(**overrides) -> ModelProvider: + defaults = dict( + name="claude-provider", + provider_type="claude", + base_url=None, + api_key=SECRET_KEY, + default_model=None, + enabled=True, + ) + defaults.update(overrides) + return ModelProvider(**defaults) + + +def _anthropic_response(text: str) -> SimpleNamespace: + return SimpleNamespace(content=[SimpleNamespace(text=text)]) + + +@pytest.mark.asyncio +async def test_anthropic_chat_returns_text(): + provider = _anthropic_provider() + mock_client = MagicMock() + mock_client.messages.create = AsyncMock(return_value=_anthropic_response("claude says hi")) + with patch("anthropic.AsyncAnthropic", return_value=mock_client): + adapter = AnthropicAdapter(provider) + result = await adapter.chat([{"role": "user", "content": "hello"}]) + assert result == "claude says hi" + + +@pytest.mark.asyncio +async def test_anthropic_list_models_returns_catalog(): + provider = _anthropic_provider() + adapter = AnthropicAdapter(provider) + models = await adapter.list_models() + assert models == [entry["model_id"] for entry in anthropic_catalog()] + + +@pytest.mark.asyncio +async def test_anthropic_test_connection_success(): + provider = _anthropic_provider() + mock_client = MagicMock() + mock_client.messages.create = AsyncMock(return_value=_anthropic_response("pong")) + with patch("anthropic.AsyncAnthropic", return_value=mock_client): + adapter = AnthropicAdapter(provider) + result = await adapter.test_connection() + assert result["ok"] is True + assert isinstance(result["latency_ms"], float) + assert result["models_sample"] + + +@pytest.mark.asyncio +async def test_anthropic_test_connection_failure_sanitized(): + provider = _anthropic_provider() + mock_client = MagicMock() + mock_client.messages.create = AsyncMock( + side_effect=Exception(f"invalid x-api-key {SECRET_KEY}") + ) + with patch("anthropic.AsyncAnthropic", return_value=mock_client): + adapter = AnthropicAdapter(provider) + result = await adapter.test_connection() + assert result["ok"] is False + assert SECRET_KEY not in result["error"] + assert "REDACTED" in result["error"] + + +@pytest.mark.asyncio +async def test_anthropic_chat_failure_raises_without_key(): + provider = _anthropic_provider() + mock_client = MagicMock() + mock_client.messages.create = AsyncMock( + side_effect=Exception(f"bad key {SECRET_KEY}") + ) + with patch("anthropic.AsyncAnthropic", return_value=mock_client): + adapter = AnthropicAdapter(provider) + with pytest.raises(LlmAdapterError) as exc_info: + await adapter.chat([{"role": "user", "content": "hi"}]) + assert SECRET_KEY not in str(exc_info.value) + + +@pytest.mark.asyncio +async def test_anthropic_rejects_private_base_url_before_building_client(): + provider = _anthropic_provider(base_url="http://10.0.0.5:9999") + with patch("anthropic.AsyncAnthropic") as mock_cls: + adapter = AnthropicAdapter(provider) + with pytest.raises(LlmAdapterError, match="rejected"): + await adapter.chat([{"role": "user", "content": "hi"}]) + mock_cls.assert_not_called() + + +# ── factory dispatch ───────────────────────────────────────────────────────── + + +def test_factory_dispatches_openai(): + assert isinstance(get_adapter(_provider(provider_type="openai")), OpenAICompatAdapter) + + +def test_factory_dispatches_local(): + assert isinstance(get_adapter(_provider(provider_type="local")), OpenAICompatAdapter) + + +def test_factory_dispatches_claude(): + assert isinstance(get_adapter(_anthropic_provider(provider_type="claude")), AnthropicAdapter) + + +def test_factory_unknown_provider_type_raises(): + with pytest.raises(LlmAdapterError, match="no adapter registered"): + get_adapter(_provider(provider_type="carrier-pigeon")) diff --git a/tests/unit/llm/test_catalog.py b/tests/unit/llm/test_catalog.py new file mode 100644 index 0000000..c581335 --- /dev/null +++ b/tests/unit/llm/test_catalog.py @@ -0,0 +1,79 @@ +"""Unit tests for backend.llm (GOAL-6 PR-A): closed-set vocabulary shared by +the model-provider data layer, plus the hardcoded Anthropic model catalog +(decision #5 — Anthropic has no /v1/models discovery endpoint).""" + +from backend.llm import ( + VALID_MODEL_SOURCES, + VALID_MODEL_TYPES, + VALID_ROLES, + is_valid_model_source, + is_valid_model_type, + is_valid_role, +) +from backend.llm.catalog import ANTHROPIC_CATALOG, anthropic_catalog + + +def test_valid_model_types_v1_is_llm_only(): + assert VALID_MODEL_TYPES == frozenset({"llm"}) + assert is_valid_model_type("llm") is True + assert is_valid_model_type("embedding") is False + assert is_valid_model_type("rerank") is False + assert is_valid_model_type("") is False + assert is_valid_model_type(None) is False + + +def test_valid_roles_closed_set(): + assert VALID_ROLES == frozenset({"chat", "executor", "enrichment"}) + for role in ("chat", "executor", "enrichment"): + assert is_valid_role(role) is True + assert is_valid_role("summarizer") is False + assert is_valid_role("Chat") is False # case-sensitive, no normalization + + +def test_valid_model_sources_closed_set(): + assert VALID_MODEL_SOURCES == frozenset({"discovered", "manual"}) + assert is_valid_model_source("discovered") is True + assert is_valid_model_source("manual") is True + assert is_valid_model_source("synced") is False + + +def test_anthropic_catalog_non_empty(): + assert len(ANTHROPIC_CATALOG) >= 3 + + +def test_anthropic_catalog_entries_have_required_fields(): + required = { + "model_id", + "display_name", + "context_window", + "supports_tools", + "supports_vision", + } + for entry in ANTHROPIC_CATALOG: + assert required <= set(entry.keys()) + assert isinstance(entry["model_id"], str) and entry["model_id"] + assert isinstance(entry["display_name"], str) and entry["display_name"] + assert isinstance(entry["context_window"], int) and entry["context_window"] > 0 + assert isinstance(entry["supports_tools"], bool) + assert isinstance(entry["supports_vision"], bool) + + +def test_anthropic_catalog_model_ids_unique(): + ids = [entry["model_id"] for entry in ANTHROPIC_CATALOG] + assert len(ids) == len(set(ids)) + + +def test_anthropic_catalog_seeds_current_model_ids(): + ids = {entry["model_id"] for entry in ANTHROPIC_CATALOG} + assert ids == {"claude-opus-4-8", "claude-sonnet-5", "claude-haiku-4-5-20251001"} + + +def test_anthropic_catalog_helper_returns_defensive_copy(): + result = anthropic_catalog() + assert result == ANTHROPIC_CATALOG + result.append({"model_id": "mutated", "display_name": "x", "context_window": 1, + "supports_tools": False, "supports_vision": False}) + result[0]["model_id"] = "mutated-in-place" + # The module-level constant must be untouched by mutating the returned list. + assert len(ANTHROPIC_CATALOG) == 3 + assert ANTHROPIC_CATALOG[0]["model_id"] != "mutated-in-place" diff --git a/tests/unit/llm/test_pr_e_consumers.py b/tests/unit/llm/test_pr_e_consumers.py new file mode 100644 index 0000000..a56e083 --- /dev/null +++ b/tests/unit/llm/test_pr_e_consumers.py @@ -0,0 +1,334 @@ +"""GOAL-6 PR-E: consumer client-construction consolidation tests. + +Covers chat.py / skill_channel.py / the openai+claude processors' switch from +each hand-rolling its own AsyncOpenAI/AsyncAnthropic + SSRF-guard wiring to +backend.llm.factory's build_openai_compat_adapter/build_anthropic_adapter + +OpenAICompatAdapter/AnthropicAdapter.get_client() — proving each consumer's +pre-PR-E behavior (env-var api_key fallback, SSRF guard, per-record error +handling) is unchanged now that construction is centralized instead of +duplicated four times over, and that leaving PR-D's resolver un-wired here is +safe (existing provider selection still works when model_defaults isn't +configured). + +Every SDK client is mocked at the class level (openai.AsyncOpenAI / +anthropic.AsyncAnthropic) — no real network call, matching +tests/unit/llm/test_adapters.py's convention. +""" + +from unittest.mock import AsyncMock, MagicMock, patch + +import pytest +from fastapi import HTTPException + +from backend.llm.anthropic import AnthropicAdapter +from backend.llm.base import LlmAdapterError +from backend.llm.factory import ( + build_anthropic_adapter, + build_openai_compat_adapter, + litellm_prefix_for, +) +from backend.llm.openai_compat import OpenAICompatAdapter +from backend.llm.resolver import resolver +from backend.models.provider import ModelProvider + + +# ── factory helpers (backend.llm.factory) ─────────────────────────────────── + + +def test_litellm_prefix_for_matches_pre_pr_e_mapping(): + """crawl4ai_channel's old inline dict, now centralized (decision #8) — + values must match exactly, including the "openai" fallback default.""" + assert litellm_prefix_for("claude") == "anthropic" + assert litellm_prefix_for("openai") == "openai" + assert litellm_prefix_for("local") == "openai" + assert litellm_prefix_for("some-unknown-type") == "openai" + assert litellm_prefix_for(None) == "openai" + + +@pytest.mark.asyncio +async def test_build_openai_compat_adapter_builds_client_with_resolved_fields(): + mock_client = MagicMock() + with patch("openai.AsyncOpenAI", return_value=mock_client) as mock_cls: + adapter = build_openai_compat_adapter(base_url=None, api_key="resolved-key") + client = await adapter.get_client() + assert client is mock_client + _, kwargs = mock_cls.call_args + assert kwargs["api_key"] == "resolved-key" + + +@pytest.mark.asyncio +async def test_build_openai_compat_adapter_guards_private_base_url_by_default(): + """No provider_type passed -> allow_private False (the full guard) — + matches every PR-E consumer, none of which carry a "local" distinction.""" + with patch("openai.AsyncOpenAI") as mock_cls: + adapter = build_openai_compat_adapter(base_url="http://10.0.0.5/v1", api_key="k") + with pytest.raises(LlmAdapterError, match="rejected"): + await adapter.get_client() + mock_cls.assert_not_called() + + +@pytest.mark.asyncio +async def test_build_anthropic_adapter_builds_client_with_resolved_fields(): + mock_client = MagicMock() + with patch("anthropic.AsyncAnthropic", return_value=mock_client) as mock_cls: + adapter = build_anthropic_adapter(api_key="claude-key") + client = await adapter.get_client() + assert client is mock_client + _, kwargs = mock_cls.call_args + assert kwargs["api_key"] == "claude-key" + + +@pytest.mark.asyncio +async def test_openai_compat_public_get_client_matches_private_get_client(): + provider = ModelProvider( + name="p", provider_type="openai", base_url=None, api_key="k", + default_model="gpt-4o-mini", enabled=True, + ) + mock_client = MagicMock() + with patch("openai.AsyncOpenAI", return_value=mock_client): + adapter = OpenAICompatAdapter(provider) + client = await adapter.get_client() + assert client is mock_client + + +@pytest.mark.asyncio +async def test_anthropic_public_get_client_matches_private_get_client(): + provider = ModelProvider( + name="p", provider_type="claude", base_url=None, api_key="k", + default_model=None, enabled=True, + ) + mock_client = MagicMock() + with patch("anthropic.AsyncAnthropic", return_value=mock_client): + adapter = AnthropicAdapter(provider) + client = await adapter.get_client() + assert client is mock_client + + +# ── chat.py: _build_client env fallback + guard preserved ─────────────────── + + +@pytest.mark.asyncio +async def test_chat_build_client_uses_provider_api_key_when_set(monkeypatch): + from backend.api.v1.chat import _build_client + + monkeypatch.delenv("OPENAI_API_KEY", raising=False) + provider = ModelProvider( + name="p", provider_type="openai", base_url=None, api_key="provider-key", + default_model="gpt-4o-mini", enabled=True, + ) + mock_client = MagicMock() + with patch("openai.AsyncOpenAI", return_value=mock_client) as mock_cls: + client = await _build_client(provider) + assert client is mock_client + _, kwargs = mock_cls.call_args + assert kwargs["api_key"] == "provider-key" + + +@pytest.mark.asyncio +async def test_chat_build_client_falls_back_to_openai_api_key_env(monkeypatch): + """chat.py's pre-PR-E ``_build_client`` fell back to os.environ when the + provider had no api_key configured — must still work through the + consolidated adapter-based construction.""" + from backend.api.v1.chat import _build_client + + monkeypatch.setenv("OPENAI_API_KEY", "env-fallback-key") + provider = ModelProvider( + name="p", provider_type="openai", base_url=None, api_key=None, + default_model="gpt-4o-mini", enabled=True, + ) + mock_client = MagicMock() + with patch("openai.AsyncOpenAI", return_value=mock_client) as mock_cls: + client = await _build_client(provider) + assert client is mock_client + _, kwargs = mock_cls.call_args + assert kwargs["api_key"] == "env-fallback-key" + + +@pytest.mark.asyncio +async def test_chat_build_client_rejects_private_base_url(): + """New in PR-E (decision #6): chat.py's _build_client had NO SSRF guard + at all before — routing through OpenAICompatAdapter closes that gap. No + existing test exercised the old unguarded path (see + tests/integration/test_chat_api.py's module docstring), so this is a + deliberate hardening, not a regression.""" + from backend.api.v1.chat import _build_client + + provider = ModelProvider( + name="p", provider_type="openai", base_url="http://10.0.0.5/v1", api_key="k", + default_model="gpt-4o-mini", enabled=True, + ) + with patch("openai.AsyncOpenAI") as mock_cls: + with pytest.raises(HTTPException) as exc_info: + await _build_client(provider) + assert exc_info.value.status_code == 502 + mock_cls.assert_not_called() + + +# ── skill_channel.py: _build_model_call preserves dict-config shape ───────── + + +@pytest.mark.asyncio +async def test_skill_channel_build_model_call_uses_dict_provider_fields(): + from backend.channels.skill_channel import _build_model_call + + mock_client = MagicMock() + mock_client.chat.completions.create = AsyncMock(return_value=MagicMock()) + with patch("openai.AsyncOpenAI", return_value=mock_client) as mock_cls: + model_call = await _build_model_call( + {"api_key": "sk-skill", "base_url": None, "model": "qwen3:4b"} + ) + await model_call([{"role": "user", "content": "hi"}], tools=None, model="qwen3:4b", xml=False) + _, kwargs = mock_cls.call_args + assert kwargs["api_key"] == "sk-skill" + + +@pytest.mark.asyncio +async def test_skill_channel_build_model_call_rejects_private_base_url(): + """dict provider carries no provider_type -> allow_private stays False, + matching pre-PR-E behavior exactly (skill_channel never allowed private + addresses through its own inline avalidate_public_url_and_ip call + either).""" + from backend.channels.skill_channel import _build_model_call + + with patch("openai.AsyncOpenAI") as mock_cls: + with pytest.raises(ValueError, match="rejected"): + await _build_model_call({"api_key": "k", "base_url": "http://127.0.0.1:11434/v1"}) + mock_cls.assert_not_called() + + +# ── openai/claude processors: env fallback + guard + per-record errors ───── + + +@pytest.mark.asyncio +async def test_openai_processor_env_fallback_and_success(monkeypatch): + from backend.processors.openai_processor import OpenAIProcessor + + monkeypatch.setenv("OPENAI_API_KEY", "env-openai-key") + mock_client = MagicMock() + mock_response = MagicMock() + mock_response.choices = [MagicMock(message=MagicMock(content='{"summary": "ok"}'))] + mock_response.usage = MagicMock(prompt_tokens=10, completion_tokens=5) + mock_client.chat.completions.create = AsyncMock(return_value=mock_response) + + record = MagicMock() + record.normalized_data = {"content": "hello"} + + with patch("openai.AsyncOpenAI", return_value=mock_client) as mock_cls: + proc = OpenAIProcessor() + result = await proc.process([record], "{{content}}", {}) + + assert result.success is True + assert result.enrichments == [{"summary": "ok"}] + _, kwargs = mock_cls.call_args + assert kwargs["api_key"] == "env-openai-key" + + +@pytest.mark.asyncio +async def test_openai_processor_rejects_private_base_url(): + from backend.processors.openai_processor import OpenAIProcessor + + record = MagicMock() + record.normalized_data = {} + with patch("openai.AsyncOpenAI") as mock_cls: + proc = OpenAIProcessor() + result = await proc.process([record], "{{content}}", {"base_url": "http://10.0.0.5/v1"}) + assert result.success is False + assert "rejected" in result.error + mock_cls.assert_not_called() + + +@pytest.mark.asyncio +async def test_openai_processor_per_record_error_isolation(): + """One record's LLM call failing must not fail the whole batch — the + existing per-record try/except is preserved through the consolidated + client construction (client is built once, reused for every record).""" + from backend.processors.openai_processor import OpenAIProcessor + + mock_client = MagicMock() + ok_response = MagicMock() + ok_response.choices = [MagicMock(message=MagicMock(content='{"ok": true}'))] + ok_response.usage = MagicMock(prompt_tokens=1, completion_tokens=1) + mock_client.chat.completions.create = AsyncMock(side_effect=[Exception("boom"), ok_response]) + + records = [MagicMock(normalized_data={}), MagicMock(normalized_data={})] + with patch("openai.AsyncOpenAI", return_value=mock_client): + proc = OpenAIProcessor() + result = await proc.process(records, "{{content}}", {"api_key": "k"}) + + assert result.success is True + assert "error" in result.enrichments[0] + assert result.enrichments[1] == {"ok": True} + + +@pytest.mark.asyncio +async def test_claude_processor_env_fallback_and_success(monkeypatch): + from backend.processors.claude_processor import ClaudeProcessor + + monkeypatch.setenv("ANTHROPIC_API_KEY", "env-claude-key") + mock_client = MagicMock() + mock_response = MagicMock() + mock_response.content = [MagicMock(text='{"summary": "ok"}')] + mock_response.usage = MagicMock(input_tokens=3, output_tokens=2) + mock_client.messages.create = AsyncMock(return_value=mock_response) + + record = MagicMock() + record.normalized_data = {"content": "hi"} + + with patch("anthropic.AsyncAnthropic", return_value=mock_client) as mock_cls: + proc = ClaudeProcessor() + result = await proc.process([record], "{{content}}", {}) + + assert result.success is True + assert result.enrichments == [{"summary": "ok"}] + _, kwargs = mock_cls.call_args + assert kwargs["api_key"] == "env-claude-key" + + +@pytest.mark.asyncio +async def test_claude_processor_per_record_error_isolation(): + from backend.processors.claude_processor import ClaudeProcessor + + mock_client = MagicMock() + ok_response = MagicMock() + ok_response.content = [MagicMock(text='{"ok": true}')] + ok_response.usage = MagicMock(input_tokens=1, output_tokens=1) + mock_client.messages.create = AsyncMock(side_effect=[Exception("boom"), ok_response]) + + records = [MagicMock(normalized_data={}), MagicMock(normalized_data={})] + with patch("anthropic.AsyncAnthropic", return_value=mock_client): + proc = ClaudeProcessor() + result = await proc.process(records, "{{content}}", {"api_key": "k"}) + + assert result.success is True + assert "error" in result.enrichments[0] + assert result.enrichments[1] == {"ok": True} + + +# ── resolver fallback safety (decision #8): consumers unaffected when +# model_defaults is not configured ────────────────────────────────────────── + + +@pytest.mark.asyncio +async def test_resolver_absent_model_defaults_does_not_affect_chat_provider_pick(db_session): + """PR-E deliberately does NOT wire ProviderResolver into chat.py / + skill_channel / the processors (see PR-E report — decision #8 only + mandates factory adoption for these three, not resolver adoption). This + proves that choice is safe: with zero ModelDefault rows, resolver.resolve + reports "nothing configured" (None) while chat.py's own pre-existing + _pick_provider selection keeps working, completely unaffected and never + consulting model_defaults at all.""" + from backend.api.v1.chat import _pick_provider + + provider = ModelProvider( + name="Existing Provider", provider_type="openai", base_url=None, + api_key="k", default_model="gpt-4o-mini", enabled=True, + ) + db_session.add(provider) + await db_session.commit() + await db_session.refresh(provider) + + resolved = await resolver.resolve(db_session, "chat") + assert resolved is None # no model_defaults row for role="chat" + + picked = await _pick_provider(db_session, None) + assert picked.id == provider.id # existing selection unaffected, no crash diff --git a/tests/unit/llm/test_resolver.py b/tests/unit/llm/test_resolver.py new file mode 100644 index 0000000..c3eddbf --- /dev/null +++ b/tests/unit/llm/test_resolver.py @@ -0,0 +1,408 @@ +"""Unit tests for GOAL-6 PR-D: ``backend.llm.resolver`` (failover) and +``backend.llm.base.classify_retryable`` (decision #7's connection-vs-business +error split). + +``get_adapter`` is patched at ``backend.llm.resolver.get_adapter`` so tests +can assert exactly which candidates had an adapter built (cooled candidates +must never reach it) without any real SDK/network involvement; ``operation`` +(the callable ``resolve_with_fallback`` invokes per live candidate) is always +a test double, never a real adapter method. A fake injectable clock replaces +``time.monotonic`` so cooldown-window assertions never depend on real sleeps. +""" + +from __future__ import annotations + +import asyncio +from unittest.mock import AsyncMock, patch + +import anthropic +import httpx +import openai +import pytest +from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker + +from backend.llm.base import LlmAdapterError, classify_retryable +from backend.llm.resolver import ProviderResolver, ResolverError +from backend.models.model_default import ModelDefault +from backend.models.provider import ModelProvider + +SECRET_KEY = "sk-resolver-test-secret-do-not-leak" + + +class FakeClock: + """Injectable monotonic clock (GOAL-6 PR-D): starts at 0.0, only moves + when the test calls :meth:`advance` — no real sleeps anywhere here.""" + + def __init__(self) -> None: + self.t = 0.0 + + def __call__(self) -> float: + return self.t + + def advance(self, seconds: float) -> None: + self.t += seconds + + +async def _make_provider(db: AsyncSession, name: str) -> ModelProvider: + provider = ModelProvider( + name=name, + provider_type="openai", + base_url=None, + api_key=SECRET_KEY, + default_model="gpt-4o-mini", + enabled=True, + ) + db.add(provider) + await db.commit() + await db.refresh(provider) + return provider + + +async def _make_default(db: AsyncSession, role: str, candidates: list[dict]) -> ModelDefault: + row = ModelDefault(role=role, candidates=candidates) + db.add(row) + await db.commit() + await db.refresh(row) + return row + + +def _response(status_code: int) -> httpx.Response: + request = httpx.Request("POST", "https://api.example.com/v1/chat/completions") + return httpx.Response(status_code, request=request) + + +# --------------------------------------------------------------------------- +# resolve() — primary candidate, no failover +# --------------------------------------------------------------------------- + + +@pytest.mark.asyncio +async def test_resolve_returns_first_candidate(db_session: AsyncSession): + provider_a = await _make_provider(db_session, "provider-a") + provider_b = await _make_provider(db_session, "provider-b") + await _make_default( + db_session, + "chat", + [ + {"provider_id": provider_a.id, "model_id": "model-a"}, + {"provider_id": provider_b.id, "model_id": "model-b"}, + ], + ) + + resolver = ProviderResolver(now=FakeClock()) + resolved = await resolver.resolve(db_session, "chat") + + assert resolved is not None + assert resolved.provider.id == provider_a.id + assert resolved.model_id == "model-a" + assert resolved.adapter is not None + + +@pytest.mark.asyncio +async def test_resolve_returns_none_when_no_default_row(db_session: AsyncSession): + resolver = ProviderResolver(now=FakeClock()) + assert await resolver.resolve(db_session, "chat") is None + + +@pytest.mark.asyncio +async def test_resolve_returns_none_when_candidates_empty(db_session: AsyncSession): + await _make_default(db_session, "executor", []) + resolver = ProviderResolver(now=FakeClock()) + assert await resolver.resolve(db_session, "executor") is None + + +# --------------------------------------------------------------------------- +# resolve_with_fallback() — sequential failover +# --------------------------------------------------------------------------- + + +@pytest.mark.asyncio +async def test_sequential_failover_a_retryable_fails_b_succeeds(db_session: AsyncSession): + provider_a = await _make_provider(db_session, "provider-a") + provider_b = await _make_provider(db_session, "provider-b") + await _make_default( + db_session, + "chat", + [ + {"provider_id": provider_a.id, "model_id": "model-a"}, + {"provider_id": provider_b.id, "model_id": "model-b"}, + ], + ) + + clock = FakeClock() + resolver = ProviderResolver(now=clock) + + operation = AsyncMock( + side_effect=[LlmAdapterError("connection reset", retryable=True), "b-result"] + ) + + with patch("backend.llm.resolver.get_adapter", side_effect=lambda p: object()) as mock_get_adapter: + result = await resolver.resolve_with_fallback(db_session, "chat", operation) + + assert result == "b-result" + assert operation.call_count == 2 + assert mock_get_adapter.call_count == 2 + # provider-a called with model-a, provider-b called with model-b + assert operation.call_args_list[0].args[1] == "model-a" + assert operation.call_args_list[1].args[1] == "model-b" + # provider-a is now cooled down (it retryable-failed) + assert resolver._is_cooled(provider_a.id) is True + assert resolver._is_cooled(provider_b.id) is False + + +@pytest.mark.asyncio +async def test_no_default_raises_resolver_error(db_session: AsyncSession): + resolver = ProviderResolver(now=FakeClock()) + operation = AsyncMock() + with pytest.raises(ResolverError): + await resolver.resolve_with_fallback(db_session, "chat", operation) + operation.assert_not_called() + + +# --------------------------------------------------------------------------- +# 4xx business error — NO failover (decision #7, required coverage) +# --------------------------------------------------------------------------- + + +@pytest.mark.asyncio +async def test_4xx_business_error_does_not_fail_over(db_session: AsyncSession): + provider_a = await _make_provider(db_session, "provider-a") + provider_b = await _make_provider(db_session, "provider-b") + await _make_default( + db_session, + "chat", + [ + {"provider_id": provider_a.id, "model_id": "model-a"}, + {"provider_id": provider_b.id, "model_id": "model-b"}, + ], + ) + + clock = FakeClock() + resolver = ProviderResolver(now=clock) + operation = AsyncMock(side_effect=LlmAdapterError("401 bad api key", retryable=False)) + + with patch("backend.llm.resolver.get_adapter", side_effect=lambda p: object()): + with pytest.raises(LlmAdapterError, match="401 bad api key"): + await resolver.resolve_with_fallback(db_session, "chat", operation) + + # B was NEVER tried. + assert operation.call_count == 1 + assert operation.call_args_list[0].args[1] == "model-a" + # A got NO cooldown entry — a 4xx is a config error, not a liveness one. + assert resolver._is_cooled(provider_a.id) is False + + +# --------------------------------------------------------------------------- +# cooldown skip + window expiry +# --------------------------------------------------------------------------- + + +@pytest.mark.asyncio +async def test_cooldown_skip_within_window_then_retry_after_expiry(db_session: AsyncSession): + provider_a = await _make_provider(db_session, "provider-a") + provider_b = await _make_provider(db_session, "provider-b") + await _make_default( + db_session, + "chat", + [ + {"provider_id": provider_a.id, "model_id": "model-a"}, + {"provider_id": provider_b.id, "model_id": "model-b"}, + ], + ) + + clock = FakeClock() + resolver = ProviderResolver(cooldown_seconds=60.0, now=clock) + + async def operation(adapter, model_id): + if model_id == "model-a": + raise LlmAdapterError("boom", retryable=True) + return f"result-{model_id}" + + with patch("backend.llm.resolver.get_adapter", side_effect=lambda p: object()) as mock_get_adapter: + # First call: A fails (retryable), cooldown set, B succeeds. + result1 = await resolver.resolve_with_fallback(db_session, "chat", operation) + assert result1 == "result-model-b" + assert mock_get_adapter.call_count == 2 # A (failed) + B (succeeded) + + # Second call, still inside the cooldown window: A must be skipped + # WITHOUT building its adapter at all. + mock_get_adapter.reset_mock() + result2 = await resolver.resolve_with_fallback(db_session, "chat", operation) + assert result2 == "result-model-b" + assert mock_get_adapter.call_count == 1 # only B — A skipped pre-adapter-build + built_for = [call.args[0].id for call in mock_get_adapter.call_args_list] + assert built_for == [provider_b.id] + + # Advance the clock past the cooldown window: A is tried again. + clock.advance(60.1) + mock_get_adapter.reset_mock() + result3 = await resolver.resolve_with_fallback(db_session, "chat", operation) + assert result3 == "result-model-b" # A still fails when retried... + assert mock_get_adapter.call_count == 2 # ...but it WAS retried this time + assert mock_get_adapter.call_args_list[0].args[0].id == provider_a.id + + +# --------------------------------------------------------------------------- +# all candidates exhausted -> clear ResolverError, no key leak +# --------------------------------------------------------------------------- + + +@pytest.mark.asyncio +async def test_all_candidates_exhausted_raises_resolver_error_without_key( + db_session: AsyncSession, +): + provider_a = await _make_provider(db_session, "provider-a") + provider_b = await _make_provider(db_session, "provider-b") + await _make_default( + db_session, + "chat", + [ + {"provider_id": provider_a.id, "model_id": "model-a"}, + {"provider_id": provider_b.id, "model_id": "model-b"}, + ], + ) + + resolver = ProviderResolver(now=FakeClock()) + operation = AsyncMock( + side_effect=LlmAdapterError(f"connection refused near {SECRET_KEY}", retryable=True) + ) + + with patch("backend.llm.resolver.get_adapter", side_effect=lambda p: object()): + with pytest.raises(ResolverError) as exc_info: + await resolver.resolve_with_fallback(db_session, "chat", operation) + + message = str(exc_info.value) + assert "chat" in message + assert SECRET_KEY not in message + assert operation.call_count == 2 + assert resolver._is_cooled(provider_a.id) is True + assert resolver._is_cooled(provider_b.id) is True + + +# --------------------------------------------------------------------------- +# concurrency: cooldown dict must not corrupt under concurrent callers +# --------------------------------------------------------------------------- + + +@pytest.mark.asyncio +async def test_concurrent_resolve_with_fallback_is_consistent(db_engine): + """N concurrent resolve_with_fallback calls, sharing one resolver + instance, where the first candidate always retryable-fails: no crash, + every call still lands on the working second candidate, and the shared + cooldown dict ends up in a sane state (provider-a cooled).""" + session_factory = async_sessionmaker(db_engine, class_=AsyncSession, expire_on_commit=False) + + async with session_factory() as setup_session: + provider_a = await _make_provider(setup_session, "provider-a") + provider_b = await _make_provider(setup_session, "provider-b") + await _make_default( + setup_session, + "chat", + [ + {"provider_id": provider_a.id, "model_id": "model-a"}, + {"provider_id": provider_b.id, "model_id": "model-b"}, + ], + ) + + clock = FakeClock() + resolver = ProviderResolver(now=clock) + + async def operation(adapter, model_id): + if model_id == "model-a": + raise LlmAdapterError("boom", retryable=True) + return f"result-{model_id}" + + async def run_one(): + async with session_factory() as session: + with patch("backend.llm.resolver.get_adapter", side_effect=lambda p: object()): + return await resolver.resolve_with_fallback(session, "chat", operation) + + results = await asyncio.gather(*(run_one() for _ in range(20))) + + assert results == ["result-model-b"] * 20 + assert resolver._is_cooled(provider_a.id) is True + assert resolver._is_cooled(provider_b.id) is False + + +# --------------------------------------------------------------------------- +# classify_retryable — connection-level vs business (4xx) exceptions +# --------------------------------------------------------------------------- + + +def test_classify_retryable_openai_connection_error(): + exc = openai.APIConnectionError(request=httpx.Request("POST", "https://api.openai.com")) + assert classify_retryable(exc) is True + + +def test_classify_retryable_openai_timeout_error(): + exc = openai.APITimeoutError(request=httpx.Request("POST", "https://api.openai.com")) + assert classify_retryable(exc) is True + + +def test_classify_retryable_openai_internal_server_error(): + exc = openai.InternalServerError("500 boom", response=_response(500), body=None) + assert classify_retryable(exc) is True + + +def test_classify_retryable_openai_bad_request_error(): + exc = openai.BadRequestError("400 malformed", response=_response(400), body=None) + assert classify_retryable(exc) is False + + +def test_classify_retryable_openai_authentication_error(): + exc = openai.AuthenticationError("401 bad key", response=_response(401), body=None) + assert classify_retryable(exc) is False + + +def test_classify_retryable_anthropic_connection_error(): + exc = anthropic.APIConnectionError(request=httpx.Request("POST", "https://api.anthropic.com")) + assert classify_retryable(exc) is True + + +def test_classify_retryable_anthropic_internal_server_error(): + exc = anthropic.InternalServerError("500 boom", response=_response(500), body=None) + assert classify_retryable(exc) is True + + +def test_classify_retryable_anthropic_bad_request_error(): + exc = anthropic.BadRequestError("400 malformed", response=_response(400), body=None) + assert classify_retryable(exc) is False + + +def test_classify_retryable_anthropic_authentication_error(): + exc = anthropic.AuthenticationError("401 bad key", response=_response(401), body=None) + assert classify_retryable(exc) is False + + +def test_classify_retryable_httpx_connect_error(): + assert classify_retryable(httpx.ConnectError("connection refused")) is True + + +def test_classify_retryable_httpx_timeout_exception(): + assert classify_retryable(httpx.TimeoutException("timed out")) is True + + +def test_classify_retryable_asyncio_timeout_error(): + assert classify_retryable(asyncio.TimeoutError()) is True + + +def test_classify_retryable_generic_status_code_5xx_fallback(): + class _FakeStatusError(Exception): + def __init__(self, status_code: int) -> None: + super().__init__("fake upstream error") + self.status_code = status_code + + assert classify_retryable(_FakeStatusError(503)) is True + + +def test_classify_retryable_generic_status_code_4xx_fallback(): + class _FakeStatusError(Exception): + def __init__(self, status_code: int) -> None: + super().__init__("fake upstream error") + self.status_code = status_code + + assert classify_retryable(_FakeStatusError(404)) is False + + +def test_classify_retryable_unrecognized_exception_defaults_false(): + assert classify_retryable(ValueError("something unrelated")) is False diff --git a/tests/unit/pipeline/test_ai_processor.py b/tests/unit/pipeline/test_ai_processor.py index f95d39b..c8fc36f 100644 --- a/tests/unit/pipeline/test_ai_processor.py +++ b/tests/unit/pipeline/test_ai_processor.py @@ -1,5 +1,7 @@ """Unit tests for ai_processor pipeline step.""" +import logging + import pytest from unittest.mock import AsyncMock, MagicMock, patch @@ -45,3 +47,209 @@ async def test_process_with_ai_enriches_records(): assert records[0].ai_enrichment == {"summary": "Summary 1"} assert records[1].ai_enrichment == {"summary": "Summary 2"} assert records[0].status == "ai_processed" + + +# ─── GOAL-6 PR-F (decision #9): DataSource.ai_config <-> ModelProvider ───── +# soft dual-track convergence at the ai_config -> processor-config seam. + + +def _session_cm(session): + """Wrap an already-open (test-fixture) AsyncSession in the async context + manager shape ``backend.database.AsyncSessionLocal()`` normally returns, + so code under test that does ``async with AsyncSessionLocal() as + session:`` transparently reuses the real ``db_session`` fixture instead + of hitting the module-level production engine. Same pattern as + ``tests/unit/worker/test_redbeat_sync.py``'s ``_session_cm`` helper.""" + cm = AsyncMock() + cm.__aenter__ = AsyncMock(return_value=session) + cm.__aexit__ = AsyncMock(return_value=False) + return cm + + +def _logged(caplog, substring: str) -> bool: + return any(substring in r.getMessage() for r in caplog.records) + + +def _make_record(): + record = MagicMock() + record.ai_enrichment = None + record.status = "normalized" + return record + + +def _mock_processor(enrichment: dict | None = None): + mock_result = ProcessingResult(success=True, enrichments=[enrichment or {"summary": "ok"}]) + processor = AsyncMock() + processor.process = AsyncMock(return_value=mock_result) + return processor + + +@pytest.mark.asyncio +async def test_process_with_ai_provider_id_resolves(db_session): + """ai_config.provider_id resolves to a real ModelProvider row: the + resulting processor config carries the provider's + api_key/base_url/model/provider_type, not whatever was inline.""" + from backend.models.provider import ModelProvider + + provider = ModelProvider( + name="Governed Provider", + provider_type="openai", + base_url="https://provider.example.com/v1", + api_key="sk-provider-secret", + default_model="gpt-4o-mini", + enabled=True, + ) + db_session.add(provider) + await db_session.flush() + + records = [_make_record()] + mock_processor = _mock_processor() + + ai_config = { + "provider_id": provider.id, + "processor_type": "claude", # must be overridden by provider.provider_type + "prompt_template": "Summarize: {{content}}", + } + + with ( + patch("backend.database.AsyncSessionLocal", return_value=_session_cm(db_session)), + patch("backend.pipeline.ai_processor.get_processor", return_value=mock_processor) as mock_get, + ): + await process_with_ai(records, ai_config, source_id="src-provider") + + mock_get.assert_called_once_with("openai") + passed_config = mock_processor.process.call_args.kwargs["config"] + assert passed_config["api_key"] == "sk-provider-secret" + assert passed_config["base_url"] == "https://provider.example.com/v1" + assert passed_config["model"] == "gpt-4o-mini" + assert passed_config["processor_type"] == "openai" + assert records[0].status == "ai_processed" + + +@pytest.mark.asyncio +async def test_process_with_ai_inline_only_is_byte_identical_and_warns(caplog): + """No provider_id at all: the resolved processor config is untouched + (byte-identical to pre-PR-F behavior) but a deprecation warning fires.""" + records = [_make_record()] + mock_processor = _mock_processor() + + ai_config = { + "processor_type": "openai", + "api_key": "sk-inline-secret", + "base_url": "https://inline.example.com/v1", + "model": "gpt-4o-mini", + "prompt_template": "Summarize: {{content}}", + } + original = dict(ai_config) + + with ( + patch("backend.pipeline.ai_processor.get_processor", return_value=mock_processor), + caplog.at_level(logging.WARNING), + ): + await process_with_ai(records, ai_config, source_id="src-inline") + + passed_config = mock_processor.process.call_args.kwargs["config"] + assert passed_config == original + assert passed_config is ai_config # same object — nothing copied/rebuilt + assert _logged(caplog, "deprecated") + + +@pytest.mark.asyncio +async def test_process_with_ai_both_supplied_provider_id_wins(db_session, caplog): + """provider_id AND inline api_key/base_url both present: provider_id + wins outright, the inline fields are ignored, and a warning is logged.""" + from backend.models.provider import ModelProvider + + provider = ModelProvider( + name="Winning Provider", + provider_type="claude", + base_url=None, + api_key="sk-provider-wins", + default_model="claude-sonnet-5", + enabled=True, + ) + db_session.add(provider) + await db_session.flush() + + records = [_make_record()] + mock_processor = _mock_processor() + + ai_config = { + "provider_id": provider.id, + "processor_type": "openai", + "api_key": "sk-inline-loses", + "base_url": "https://inline-loses.example.com", + "model": "gpt-4o-mini", + "prompt_template": "Summarize: {{content}}", + } + + with ( + patch("backend.database.AsyncSessionLocal", return_value=_session_cm(db_session)), + patch("backend.pipeline.ai_processor.get_processor", return_value=mock_processor), + caplog.at_level(logging.WARNING), + ): + await process_with_ai(records, ai_config, source_id="src-both") + + passed_config = mock_processor.process.call_args.kwargs["config"] + assert passed_config["api_key"] == "sk-provider-wins" + assert passed_config["base_url"] is None + assert passed_config["model"] == "claude-sonnet-5" + assert passed_config["processor_type"] == "claude" + assert _logged(caplog, "precedence") + + +@pytest.mark.asyncio +async def test_process_with_ai_provider_id_not_found_falls_back(db_session, caplog): + """provider_id set but no such ModelProvider exists (deleted/bad id): + warn, fall back to ai_config unchanged, never crash the pipeline.""" + records = [_make_record()] + mock_processor = _mock_processor() + + ai_config = { + "provider_id": "does-not-exist", + "processor_type": "claude", + "prompt_template": "Summarize: {{content}}", + } + original = dict(ai_config) + + with ( + patch("backend.database.AsyncSessionLocal", return_value=_session_cm(db_session)), + patch("backend.pipeline.ai_processor.get_processor", return_value=mock_processor) as mock_get, + caplog.at_level(logging.WARNING), + ): + await process_with_ai(records, ai_config, source_id="src-missing") + + mock_get.assert_called_once_with("claude") + passed_config = mock_processor.process.call_args.kwargs["config"] + assert passed_config == original + assert _logged(caplog, "does not resolve") + assert records[0].status == "ai_processed" + + +@pytest.mark.asyncio +async def test_process_with_ai_resolve_provider_false_skips_resolution(caplog): + """resolve_provider=False (the agent_config path from + backend.pipeline.runner's own ai_agents.provider_id merge) bypasses + decision #9 entirely: no DB lookup, no deprecation warning, even though + the dict carries inline api_key/base_url with no provider_id key — the + exact shape that merge produces.""" + records = [_make_record()] + mock_processor = _mock_processor() + + ai_config = { + "processor_type": "claude", + "api_key": "sk-agent-provider-key", + "base_url": "https://agent.example.com", + "prompt_template": "Summarize: {{content}}", + } + original = dict(ai_config) + + with ( + patch("backend.pipeline.ai_processor.get_processor", return_value=mock_processor), + caplog.at_level(logging.WARNING), + ): + await process_with_ai(records, ai_config, source_id="src-agent", resolve_provider=False) + + passed_config = mock_processor.process.call_args.kwargs["config"] + assert passed_config == original + assert not _logged(caplog, "deprecated") diff --git a/tests/unit/security/test_url_guard.py b/tests/unit/security/test_url_guard.py index dff9b62..06a3c2e 100644 --- a/tests/unit/security/test_url_guard.py +++ b/tests/unit/security/test_url_guard.py @@ -272,7 +272,7 @@ def _tracking_getaddrinfo(host, *args, **kwargs): return _fake_getaddrinfo("10.6.6.6") monkeypatch.setattr("socket.getaddrinfo", _tracking_getaddrinfo) - monkeypatch.setattr(url_guard, "is_ip_blocked", lambda ip: False) + monkeypatch.setattr(url_guard, "is_ip_blocked", lambda ip, **kwargs: False) transport = PinnedAsyncHTTPTransport("rebind-test.invalid", ["127.0.0.1"]) async with httpx.AsyncClient(transport=transport, follow_redirects=False) as client: diff --git a/tests/unit/test_model_default.py b/tests/unit/test_model_default.py new file mode 100644 index 0000000..47c36de --- /dev/null +++ b/tests/unit/test_model_default.py @@ -0,0 +1,114 @@ +"""Unit tests for ModelDefault (GOAL-6 PR-A, decision #4): the per-role +system default candidates table (role UNIQUE) and the Pydantic schemas' +closed-set validation for role. +""" + +import pytest +from pydantic import ValidationError +from sqlalchemy import select +from sqlalchemy.exc import IntegrityError +from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker + +from backend.models.model_default import ModelDefault +from backend.schemas.model_default import ModelDefaultPut, ModelDefaultRead + + +def _sessionmaker(db_engine): + return async_sessionmaker(db_engine, class_=AsyncSession, expire_on_commit=False) + + +# --------------------------------------------------------------------------- +# role uniqueness +# --------------------------------------------------------------------------- + + +@pytest.mark.asyncio +async def test_duplicate_role_raises_integrity_error(db_engine): + sm = _sessionmaker(db_engine) + async with sm() as session: + session.add(ModelDefault(role="chat", candidates=[{"provider_id": "p1", "model_id": "m1"}])) + await session.commit() + + async with sm() as session: + session.add(ModelDefault(role="chat", candidates=[{"provider_id": "p2", "model_id": "m2"}])) + with pytest.raises(IntegrityError): + await session.commit() + + +@pytest.mark.asyncio +async def test_distinct_roles_are_fine(db_engine): + sm = _sessionmaker(db_engine) + async with sm() as session: + session.add(ModelDefault(role="chat", candidates=[])) + session.add(ModelDefault(role="executor", candidates=[])) + session.add(ModelDefault(role="enrichment", candidates=[])) + await session.commit() + + async with sm() as session: + rows = (await session.execute(select(ModelDefault))).scalars().all() + assert {r.role for r in rows} == {"chat", "executor", "enrichment"} + + +@pytest.mark.asyncio +async def test_candidates_ordering_round_trips(db_engine): + """candidates[0] is the primary pick, the rest are failover order — the + JSON column must preserve list order through a round trip.""" + sm = _sessionmaker(db_engine) + ordered = [ + {"provider_id": "primary-provider", "model_id": "primary-model"}, + {"provider_id": "backup-provider", "model_id": "backup-model"}, + ] + async with sm() as session: + md = ModelDefault(role="chat", candidates=ordered) + session.add(md) + await session.commit() + md_id = md.id + + async with sm() as session: + loaded = (await session.execute( + select(ModelDefault).where(ModelDefault.id == md_id) + )).scalar_one() + assert loaded.candidates == ordered + + +# --------------------------------------------------------------------------- +# Pydantic schema closed-set validation (role) +# --------------------------------------------------------------------------- + + +def test_model_default_put_accepts_each_valid_role(): + for role in ("chat", "executor", "enrichment"): + payload = ModelDefaultPut(role=role, candidates=[]) + assert payload.role == role + + +def test_model_default_put_accepts_candidates_list(): + payload = ModelDefaultPut( + role="chat", + candidates=[ + {"provider_id": "p1", "model_id": "m1"}, + {"provider_id": "p2", "model_id": "m2"}, + ], + ) + assert len(payload.candidates) == 2 + assert payload.candidates[0].provider_id == "p1" + + +def test_model_default_put_rejects_invalid_role(): + with pytest.raises(ValidationError): + ModelDefaultPut(role="summarizer", candidates=[]) + + +def test_model_default_read_from_attributes(): + from datetime import datetime, timezone + + class _Row: + id = "md-1" + role = "chat" + candidates = [{"provider_id": "p1", "model_id": "m1"}] + created_at = datetime.now(timezone.utc) + updated_at = datetime.now(timezone.utc) + + read = ModelDefaultRead.model_validate(_Row()) + assert read.role == "chat" + assert read.candidates == [{"provider_id": "p1", "model_id": "m1"}] diff --git a/tests/unit/test_provider_model.py b/tests/unit/test_provider_model.py new file mode 100644 index 0000000..a568f62 --- /dev/null +++ b/tests/unit/test_provider_model.py @@ -0,0 +1,229 @@ +"""Unit tests for ProviderModel (GOAL-6 PR-A, decision #3): the model catalog +table, its real FK to ModelProvider (ondelete CASCADE — unlike +AIAgent.provider_id, which stays a loose string per decision #9), the +(provider_id, model_id) uniqueness constraint, and the Pydantic schemas' +closed-set validation for model_type/source. + +Follows the DB-fixture pattern from tests/unit/security/test_provider_key_* +(db_engine -> own sessionmaker -> `async with sm() as session`). +""" + +import pytest +from pydantic import ValidationError +from sqlalchemy import select, text +from sqlalchemy.exc import IntegrityError +from sqlalchemy.ext.asyncio import AsyncSession, async_sessionmaker + +from backend.models.provider import ModelProvider +from backend.models.provider_model import ProviderModel +from backend.schemas.provider_model import ProviderModelCreate, ProviderModelRead + + +def _sessionmaker(db_engine): + return async_sessionmaker(db_engine, class_=AsyncSession, expire_on_commit=False) + + +async def _make_provider(session, name="Test Provider") -> ModelProvider: + provider = ModelProvider(name=name, provider_type="openai", enabled=True) + session.add(provider) + await session.flush() + return provider + + +# --------------------------------------------------------------------------- +# Unique constraint: (provider_id, model_id) +# --------------------------------------------------------------------------- + + +@pytest.mark.asyncio +async def test_duplicate_provider_id_model_id_pair_raises_integrity_error(db_engine): + sm = _sessionmaker(db_engine) + async with sm() as session: + provider = await _make_provider(session) + session.add(ProviderModel( + provider_id=provider.id, model_id="gpt-4o", source="manual", + )) + await session.commit() + + async with sm() as session: + provider = (await session.execute( + select(ModelProvider) + )).scalars().first() + session.add(ProviderModel( + provider_id=provider.id, model_id="gpt-4o", source="manual", + )) + with pytest.raises(IntegrityError): + await session.commit() + + +@pytest.mark.asyncio +async def test_same_provider_different_model_id_is_fine(db_engine): + sm = _sessionmaker(db_engine) + async with sm() as session: + provider = await _make_provider(session) + session.add(ProviderModel( + provider_id=provider.id, model_id="gpt-4o", source="manual", + )) + session.add(ProviderModel( + provider_id=provider.id, model_id="gpt-4o-mini", source="manual", + )) + await session.commit() + provider_id = provider.id + + async with sm() as session: + rows = (await session.execute( + select(ProviderModel).where(ProviderModel.provider_id == provider_id) + )).scalars().all() + assert len(rows) == 2 + + +@pytest.mark.asyncio +async def test_same_model_id_different_provider_is_fine(db_engine): + sm = _sessionmaker(db_engine) + async with sm() as session: + p1 = await _make_provider(session, name="Provider A") + p2 = await _make_provider(session, name="Provider B") + session.add(ProviderModel(provider_id=p1.id, model_id="shared-model", source="manual")) + session.add(ProviderModel(provider_id=p2.id, model_id="shared-model", source="manual")) + await session.commit() + + async with sm() as session: + rows = (await session.execute( + select(ProviderModel).where(ProviderModel.model_id == "shared-model") + )).scalars().all() + assert len(rows) == 2 + + +# --------------------------------------------------------------------------- +# Defaults / column shape +# --------------------------------------------------------------------------- + + +@pytest.mark.asyncio +async def test_defaults_model_type_llm_source_manual_enabled_true(db_engine): + sm = _sessionmaker(db_engine) + async with sm() as session: + provider = await _make_provider(session) + pm = ProviderModel(provider_id=provider.id, model_id="gpt-4o", source="manual") + session.add(pm) + await session.commit() + pm_id = pm.id + + async with sm() as session: + loaded = (await session.execute( + select(ProviderModel).where(ProviderModel.id == pm_id) + )).scalar_one() + assert loaded.model_type == "llm" + assert loaded.enabled is True + assert loaded.capabilities is None + + +@pytest.mark.asyncio +async def test_capabilities_json_round_trips(db_engine): + sm = _sessionmaker(db_engine) + caps = {"tools": True, "vision": False, "context_window": 200000} + async with sm() as session: + provider = await _make_provider(session) + pm = ProviderModel( + provider_id=provider.id, model_id="claude-sonnet-5", source="discovered", + capabilities=caps, + ) + session.add(pm) + await session.commit() + pm_id = pm.id + + async with sm() as session: + loaded = (await session.execute( + select(ProviderModel).where(ProviderModel.id == pm_id) + )).scalar_one() + assert loaded.capabilities == caps + + +# --------------------------------------------------------------------------- +# FK cascade: deleting a ModelProvider cascade-deletes its provider_models +# rows (real FK, ondelete=CASCADE — decision #3). SQLite does not enforce +# FK constraints by default (matches this repo's production database.py, +# which never issues `PRAGMA foreign_keys=ON`), so this test explicitly +# turns enforcement on for its own session/connection to prove the +# ondelete=CASCADE clause baked into the migration/model actually works at +# the DB level when enforcement is active. Documented limitation: outside +# tests, this repo's runtime engine does not enable the pragma, so today +# nothing relies on DB-level cascade actually firing in production sqlite — +# PR-C/E service code must not assume it (delete provider_models explicitly +# if that ever matters), same caveat table.py's own FK columns are under. +# --------------------------------------------------------------------------- + + +@pytest.mark.asyncio +async def test_deleting_provider_cascade_deletes_provider_models(db_engine): + sm = _sessionmaker(db_engine) + async with sm() as session: + await session.execute(text("PRAGMA foreign_keys=ON")) + provider = await _make_provider(session) + provider_id = provider.id + session.add(ProviderModel(provider_id=provider_id, model_id="gpt-4o", source="manual")) + session.add(ProviderModel(provider_id=provider_id, model_id="gpt-4o-mini", source="manual")) + await session.commit() + + rows_before = (await session.execute( + select(ProviderModel).where(ProviderModel.provider_id == provider_id) + )).scalars().all() + assert len(rows_before) == 2 + + await session.delete(provider) + await session.commit() + + rows_after = (await session.execute( + select(ProviderModel).where(ProviderModel.provider_id == provider_id) + )).scalars().all() + assert rows_after == [] + + +# --------------------------------------------------------------------------- +# Pydantic schema closed-set validation (model_type / source) +# --------------------------------------------------------------------------- + + +def test_provider_model_create_accepts_valid_model_type_and_source(): + payload = ProviderModelCreate( + provider_id="p1", model_id="gpt-4o", model_type="llm", source="manual", + ) + assert payload.model_type == "llm" + assert payload.source == "manual" + + +def test_provider_model_create_accepts_discovered_source(): + payload = ProviderModelCreate( + provider_id="p1", model_id="gpt-4o", source="discovered", + ) + assert payload.source == "discovered" + + +def test_provider_model_create_rejects_invalid_model_type(): + with pytest.raises(ValidationError): + ProviderModelCreate(provider_id="p1", model_id="gpt-4o", model_type="embedding") + + +def test_provider_model_create_rejects_invalid_source(): + with pytest.raises(ValidationError): + ProviderModelCreate(provider_id="p1", model_id="gpt-4o", source="synced") + + +def test_provider_model_read_from_attributes(db_engine): + # ProviderModelRead just needs from_attributes wiring; exercised via a + # plain namespace rather than a DB round-trip (that's covered above). + from datetime import datetime, timezone + + class _Row: + id = "pm-1" + provider_id = "p-1" + model_id = "gpt-4o" + model_type = "llm" + capabilities = {"tools": True} + source = "manual" + enabled = True + created_at = datetime.now(timezone.utc) + + read = ProviderModelRead.model_validate(_Row()) + assert read.id == "pm-1" + assert read.model_type == "llm" diff --git a/tests/unit/test_runner.py b/tests/unit/test_runner.py index c58ed04..e3c4c11 100644 --- a/tests/unit/test_runner.py +++ b/tests/unit/test_runner.py @@ -451,6 +451,90 @@ def phase2_get(model, obj_id): assert result["success"] is True +@pytest.mark.asyncio +async def test_run_pipeline_agent_processor_config_overrides_provider_base(): + """GOAL-6 PR-E / decision #8: provider supplies BASE fields (api_key, + base_url) via provider_config; agent.processor_config is layered on top + and WINS on any overlapping key. test_run_pipeline_with_agent_id (above) + only asserts the pipeline succeeds — it never inspects the merged + agent_config dict run_pipeline actually receives, so it can't catch a + precedence regression. This captures that dict via a fake run_pipeline + and asserts the override, locking the merge order PR-E's consumer + refactor must not disturb.""" + task = _make_task() + task.agent_id = "agent-1" + run = _make_run() + + def capture_add(obj): + obj.id = "run-1" + + session1 = AsyncMock() + session1.get = AsyncMock(return_value=task) + session1.add = MagicMock(side_effect=capture_add) + session1.flush = AsyncMock() + session1.commit = AsyncMock() + + source = _make_source() + mock_agent = MagicMock() + mock_agent.enabled = True + mock_agent.provider_id = "prov-1" + mock_agent.processor_type = "claude" + mock_agent.model = "claude-3-haiku" + mock_agent.prompt_template = "Summarize: {{content}}" + # Agent overrides the provider's base_url and adds a field the provider + # never had; it does NOT set api_key, so the provider's key must survive. + mock_agent.processor_config = { + "base_url": "https://agent-override.example.com", + "max_tokens": 2048, + } + + mock_provider = MagicMock() + mock_provider.enabled = True + mock_provider.api_key = "sk-provider-key" + mock_provider.base_url = "https://provider-base.example.com" + + def phase2_get(model, obj_id): + if "DataSource" in str(model): + return source + if "AIAgent" in str(model): + return mock_agent + if "ModelProvider" in str(model): + return mock_provider + return None + + session2 = AsyncMock() + session2.get = AsyncMock(side_effect=phase2_get) + session2.expunge = MagicMock() + + session3 = AsyncMock() + session3.get = AsyncMock(return_value=run) + session3.commit = AsyncMock() + + pipeline_result = _make_pipeline_result(success=True) + captured_kwargs: dict = {} + + async def fake_run_pipeline(**kwargs): + captured_kwargs.update(kwargs) + return pipeline_result + + with patch( + "backend.pipeline.runner.AsyncSessionLocal", + side_effect=[make_session_cm(session1), make_session_cm(session2), make_session_cm(session3)], + ): + with patch("backend.pipeline.runner.run_pipeline", side_effect=fake_run_pipeline): + result = await run_collection_pipeline("task-1", {}) + + assert result["success"] is True + agent_config = captured_kwargs["agent_config"] + # Provider supplies the base field the agent didn't override... + assert agent_config["api_key"] == "sk-provider-key" + # ...but agent.processor_config wins on any overlapping/added key. + assert agent_config["base_url"] == "https://agent-override.example.com" + assert agent_config["max_tokens"] == 2048 + assert agent_config["processor_type"] == "claude" + assert agent_config["model"] == "claude-3-haiku" + + # ── GOAL-4 PR-B: retryable run_pipeline failure records then re-propagates ────── @pytest.mark.asyncio