Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ htmlcov/
*.coveragerc
coverage.xml
.tmp-smoke/
.tmp/opencli-conformance/

# Database files
*.db
Expand Down Expand Up @@ -81,3 +82,7 @@ odp-rs/target/
.gstack/
.repowise/
.understand-anything/
.codex/
.sentrux/agent-sessions/
.sentrux/baseline.json
artifacts/
100 changes: 100 additions & 0 deletions LIVE_WEBHOOK_ACCEPTANCE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# Live Webhook Acceptance

Date: 2026-07-07

## WSL Test Environment

- Repo: `/mnt/c/c/Users/Administrator/projects/opencli-admin-backend`
- WSL distro: Ubuntu
- Working Python: `3.12.13`
- WSL venv: `/root/.cache/codex/venvs/opencli-admin-backend-py312`
- Install command:

```bash
uv venv /root/.cache/codex/venvs/opencli-admin-backend-py312 \
--python /root/.local/share/uv/python/cpython-3.12-linux-x86_64-gnu/bin/python3.12

uv pip install \
--python /root/.cache/codex/venvs/opencli-admin-backend-py312/bin/python \
-e '.[dev]'
```

Notes:

- The repo-local `.venv` is a Windows venv (`Scripts/python.exe`), not usable from WSL.
- Ubuntu's system Python is `3.14.4`; it was not used for acceptance because `lxml==5.4.0` does not build cleanly on this image.
- `uv python install 3.11` did not complete in this WSL session, but uv already had Python `3.12.13`, which satisfies the project `>=3.11` requirement.

## Baseline Pytest Acceptance

Command:

```bash
cd /mnt/c/c/Users/Administrator/projects/opencli-admin-backend
/root/.cache/codex/venvs/opencli-admin-backend-py312/bin/python \
-m pytest -q -m 'not live' --maxfail=20
```

Result:

```text
1430 passed, 1 skipped, 9 deselected, 92 warnings in 235.18s
Required test coverage of 80% reached. Total coverage: 89.72%
```

One test adjustment was needed: compile API binding assertions now allow the runtime binding to include the new `contract` manifest while still asserting the original stable binding fields and matching `contract.bindingId`.

## Generic Webhook Live Acceptance

Added test:

```text
tests/integration/test_generic_webhook_live.py
```

Behavior:

- If `OPENCLI_GENERIC_WEBHOOK_LIVE_URL` is set, the test posts to that URL.
- If unset, the test creates a temporary Webhook.site token with `POST https://webhook.site/token`.
- For Webhook.site URLs, the test reads `request/latest/raw` and verifies the captured payload.
- The test exercises the real project path: `execute_workflow_webhook_delivery()` -> `WebhookNotifier` -> public HTTPS POST.

Command:

```bash
cd /mnt/c/c/Users/Administrator/projects/opencli-admin-backend
/root/.cache/codex/venvs/opencli-admin-backend-py312/bin/python \
-m pytest -q -m live tests/integration/test_generic_webhook_live.py --no-cov
```

Result:

```text
1 passed, 27 warnings in 2.89s
```

Manual smoke also passed before the pytest was added:

```text
delivery_result.delivered=true
captured_event=workflow.evidence_batch.ready
captured_title=WSL live webhook acceptance
```

## Secret Handling

No Feishu, DingTalk, WeCom, Hookdeck, or other private keys were added to the repo or written into this document.

For provider-specific live checks, inject secrets through WSL environment variables or an external secret manager:

```bash
export OPENCLI_FEISHU_WEBHOOK_URL='...'
export OPENCLI_DINGTALK_WEBHOOK_URL='...'
export OPENCLI_WECOM_WEBHOOK_URL='...'
```

Then add or run provider-specific `pytest -m live` tests that skip unless the matching env var is present.

## Next Step

After generic webhook live is green, wire Feishu/DingTalk/WeCom live smoke tests behind env-var skips. Keep their keys out of git, shell history, docs, and chat.
68 changes: 57 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# OpenCLI Admin

[![Docker](https://img.shields.io/badge/Docker%20Hub-0.3.6-blue?logo=docker)](https://hub.docker.com/u/xjh1994)
[![Docker](https://img.shields.io/badge/Docker%20Hub-0.3.6-blue?logo=docker)](https://hub.docker.com/u/2233admin)

**现代化的数据采集系统** — 可视化管理多渠道数据采集,接入 [opencli](https://github.com/jackwener/opencli) 驱动国内外主流平台,支持 AI 处理、多节点分布式调度与实时通知推送。

Expand All @@ -11,10 +11,10 @@
- 后端位于 `backend/`,前端通过 `/api/v1/*` 与 `/health` 代理对接后端。
- 扩展仍在 `chrome/extension-src/` 下独立构建。

**OpenCLI WebUI** OpenCLI 可视化界面 [opencli-webui](https://github.com/xjh1994/opencli-webui)
**OpenCLI WebUI** OpenCLI 可视化界面 [opencli-webui](https://github.com/2233admin/opencli-webui)

**仪表盘**
<img width="1600" height="900" alt="dashboard" src="https://raw.githubusercontent.com/xjh1994/opencli-admin/develop/docs/dashboard.png" />
<img width="1600" height="900" alt="dashboard" src="https://raw.githubusercontent.com/2233admin/opencli-admin/develop/docs/dashboard.png" />

**Agent 节点自动路由**
<img width="949" height="726" alt="clipboard-image-1774003758" src="https://github.com/user-attachments/assets/2838af3b-2ecb-4d3b-8d8e-21c69db174fc" />
Expand Down Expand Up @@ -101,6 +101,52 @@ docker compose up -d # 启动中心 + agent-1

---

### 舆情监控实战闭环

当前实战链路已经投到真实运行面,而不是只停留在配置说明:

1. **多账号 / 多节点采集** — 通过「节点管理」和站点绑定,把 `opencli` 采集路由到指定 WS agent;验收脚本会证明 `chrome_endpoint` 和 `node_url` 都落在绑定节点。
2. **AI 摘要与打标** — `collect → normalize → store → ai → notify` 流水线会把模型输出写入 `ai_enrichment`,监控台读取真实记录展示摘要、标签和情绪分布。
3. **飞书推送** — 飞书模板可以直接引用 `{{summary}}`、`{{tags}}`、`{{sentiment}}`,把 AI 处理后的内容推到群机器人。
4. **可视化验收** — 「监控台」的舆情监控卡片读取 `/api/v1/dashboard/opinion-monitor`,展示最近热点、AI 处理量、Feishu sent/failed 证据和来源贡献。

一键生成实战配置:

```bash
curl -X POST http://localhost:8000/api/v1/presets/opinion-monitor/apply \
-H "Content-Type: application/json" \
-d '{
"source_prefix": "实战舆情",
"feishu_webhook_url": "https://open.feishu.cn/open-apis/bot/v2/hook/xxx"
}'
```

这会创建两条默认 `aibase news` 多账号采集源、对应定时计划,以及一个飞书规则。
如果暂时不填 `feishu_webhook_url`,飞书规则会以 disabled 状态创建,不会伪造推送成功。

关键验收命令:

```powershell
scripts\acceptance\fleet-acceptance.ps1 `
-Site aibase `
-Command news `
-Limit 1 `
-CenterPort 8032 `
-AgentPort 19824 `
-FreshDb
```

如果本机已有旧 API/agent 进程占用端口,可以换一组固定端口,例如
`-CenterPort 8035 -AgentPort 19828`。

最终应输出:

```text
ACCEPTANCE: PASS
```

---

## 快速开始

### 方式零:前后端本地开发(推荐)
Expand Down Expand Up @@ -225,15 +271,15 @@ docker run -d --name opencli-agent --restart unless-stopped \
-e CENTRAL_API_URL=http://<center-ip>:8030 \
-e AGENT_REGISTER=ws -e AGENT_MODE=bridge \
-p 19823:19823 \
xjh1994/opencli-admin-agent:0.3.6
2233admin/opencli-admin-agent:0.3.6

# HTTP 模式(局域网)
docker run -d --name opencli-agent --restart unless-stopped \
--add-host=host.docker.internal:host-gateway \
-e CENTRAL_API_URL=http://<center-ip>:8030 \
-e AGENT_REGISTER=http -e AGENT_MODE=bridge \
-p 19823:19823 \
xjh1994/opencli-admin-agent:0.3.6
2233admin/opencli-admin-agent:0.3.6
```

**一键脚本安装**
Expand Down Expand Up @@ -426,20 +472,20 @@ TAG=0.3.6
docker buildx build --builder multiarch \
--platform linux/amd64,linux/arm64 \
--build-arg IMAGE_TAG=${TAG} \
-t xjh1994/opencli-admin-api:${TAG} --push .
-t 2233admin/opencli-admin-api:${TAG} --push .

# Agent 基础版(~100 MB,通过宿主机 Chrome 连接)
docker buildx build --builder multiarch \
--platform linux/amd64,linux/arm64 \
-f agent/Dockerfile \
-t xjh1994/opencli-admin-agent:${TAG} --push .
-t 2233admin/opencli-admin-agent:${TAG} --push .

# Agent 内置 Chrome 版(~450 MB,完全自包含)
docker buildx build --builder multiarch \
--platform linux/amd64,linux/arm64 \
-f agent/Dockerfile \
--build-arg INSTALL_CHROME=true \
-t xjh1994/opencli-admin-agent:${TAG}-chrome --push .
-t 2233admin/opencli-admin-agent:${TAG}-chrome --push .
```

如需并行构建所有镜像:
Expand All @@ -448,13 +494,13 @@ docker buildx build --builder multiarch \
TAG=0.3.6
docker buildx build --builder multiarch --platform linux/amd64,linux/arm64 \
--build-arg IMAGE_TAG=${TAG} \
-t xjh1994/opencli-admin-api:${TAG} --push . > /tmp/build-api.log 2>&1 &
-t 2233admin/opencli-admin-api:${TAG} --push . > /tmp/build-api.log 2>&1 &
docker buildx build --builder multiarch --platform linux/amd64,linux/arm64 \
-f agent/Dockerfile \
-t xjh1994/opencli-admin-agent:${TAG} --push . > /tmp/build-agent.log 2>&1 &
-t 2233admin/opencli-admin-agent:${TAG} --push . > /tmp/build-agent.log 2>&1 &
docker buildx build --builder multiarch --platform linux/amd64,linux/arm64 \
-f agent/Dockerfile --build-arg INSTALL_CHROME=true \
-t xjh1994/opencli-admin-agent:${TAG}-chrome --push . > /tmp/build-agent-chrome.log 2>&1 &
-t 2233admin/opencli-admin-agent:${TAG}-chrome --push . > /tmp/build-agent-chrome.log 2>&1 &
wait && echo "done"
```

Expand Down
53 changes: 53 additions & 0 deletions TODOS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# TODOs

## Workflow Runtime Conformance

### Maintain config-blocked conformance cases

- **Status:** First taxonomy-backed fixtures are implemented for missing webhook URL, missing source credential, and missing runtime resource.
- **What:** Keep `config-blocked` conformance cases current as new runtime bindings add config, credential, or resource gates.
- **Why:** The first conformance slice covers happy path, permission-blocked, and missing-binding evidence; config absence is a separate failure class that open-source users must be able to diagnose without guessing.
- **Pros:** Extends the drift gate beyond permissions and missing bindings, and makes live/preview/simulated claims harder to overstate.
- **Cons:** New bindings must register stable reason taxonomy entries before tests can assert exact failures without churn.
- **Context:** `Administrator-codex-opinion-monitor-quickstart-design-20260706-182303.md` defines runtime truth as registry declaration plus executable fixture plus observed event transcript. Config-blocked cases were intentionally deferred from the first PR-sized slice to keep the initial harness small.
- **Depends on / blocked by:** Future binding-specific config gates and their stable block reason definitions.

### Maintain SSE event stream smoke coverage

- **Status:** Canonical happy-path `/events/stream` smoke coverage now reuses the snapshot expected transcript matcher.
- **What:** Keep `/api/v1/workflows/runs/{runId}/events/stream` smoke coverage aligned with the snapshot-based conformance matcher as event shapes evolve.
- **Why:** The conformance matcher should use the deterministic `/events` snapshot as its primary evidence source, but the live UI still depends on the stream endpoint.
- **Pros:** Protects the live event-stream user path without making the main conformance gate depend on polling windows or SSE timing.
- **Cons:** SSE tests are more timing-sensitive than snapshot tests and should stay out of the primary conformance matcher.
- **Context:** The `/plan-eng-review` decision for performance was to use `/events` snapshots for conformance evidence and reserve `/events/stream` for a later smoke test only.
- **Depends on / blocked by:** Future event shape changes must update both snapshot expected transcripts and SSE parser expectations.

### Maintain ODP/Redis-stream conformance after the interim harness

- **Status:** Workflow-run event mirror conformance now publishes stable event facts through the Redis stream interface and reads them back with the shared transcript matcher.
- **What:** Keep ODP/Redis-stream conformance cases current as the event mirror moves from fixture Redis clients to deployment Redis and later ODP consumers.
- **Why:** The first conformance slice certifies the current `/events` API, but the longer-term architecture makes ODP/event streams the runtime source of truth.
- **Pros:** Prevents the interim snapshot harness from becoming the permanent definition of runtime truth, and keeps the open-source conformance story aligned with the event-stream-first architecture.
- **Cons:** Expands scope into executor/source-of-truth migration and should wait until the transcript schema is stable.
- **Context:** The current design deliberately separates interim workflow-run event evidence from long-term ODP/Redis event-stream evidence.
- **Depends on / blocked by:** Deployment Redis/ODP consumer coverage.

### Maintain real node I/O contract coverage

- **Status:** Runtime bindings now declare stable input shape, output shape, permission gate, config gate, event shape, and fixture coverage through `backend/workflow/runtime_contracts.py`.
- **What:** Keep each new runtime binding in the contract table before exposing it as runnable in compile output or Canvas capability/status surfaces.
- **Why:** A binding is not runtime-certified just because compile can produce a node or the UI can place it on the Canvas; it needs a stable I/O contract first.
- **Pros:** Prevents runnable/status drift and keeps resource internals out of user-entered fields.
- **Cons:** New runtime bindings must update both the contract table and focused fixture coverage before they can honestly appear runnable.
- **Context:** The contract is projected into both registry metadata and capability manifests; webhook delivery now has its own deterministic request-capture fixture.
- **Depends on / blocked by:** Future runtime bindings must add contract declarations and fixture evidence before being exposed as runnable.

### Maintain webhook real delivery fixtures

- **Status:** `workflow.notifier.webhook.send` now sends through the registered webhook notifier when send permission, configured URL, and upstream EvidenceBatch projection are present.
- **What:** Keep the success request-capture fixture and negative missing-permission, missing-URL, and missing-projection fixtures aligned with the delivery payload.
- **Why:** Webhook delivery is the final runtime layer; regressions here would silently turn real delivery back into a projection-only claim.
- **Pros:** Confirms actual POST construction while keeping SSRF-safe notifier plumbing and blocked preconditions visible.
- **Cons:** Payload schema changes must update both request-capture assertions and expected transcript evidence.
- **Context:** Capability/status surfaces remain blocked by default because each run still needs user configuration and upstream projection input, but the backend delivery path is now executable.
- **Depends on / blocked by:** Future webhook payload schema or notifier security changes.
Loading
Loading