[Integration] Integrating LLM-proxy (#54/#63) onto PR stack

cli/ARCHITECTURE.md

@@ -0,0 +1,200 @@
+# CanyonOS CLI — Architecture
+
+## The one idea to keep in your head
+
+**The CLI does almost nothing. The container does everything.**
+
+`canyonos` is a thin client. It never builds, compiles, or runs your workflow
+itself — it manages a **Global Controller (GC) container**, ships your project
+into it, and drives it over a small HTTP API. Everything you see in your
+terminal is the CLI *narrating* what the container is doing.
+
+If you remember only one picture, remember this:
+
+```
+        YOU                  CLI (host)                 GLOBAL CONTROLLER (container)
+         │                       │                                │
+         │  canyonos deploy      │                                │
+         ├──────────────────────▶│   pull + run container         │
+         │                       ├───────────────────────────────▶│
+         │                       │   copy project in (docker cp)  │
+         │                       ├───────────────────────────────▶│  /workspace
+         │                       │   POST /deploy                 │
+         │                       ├───────────────────────────────▶│  ventis build + launch
+         │                       │◀── log stream (docker logs) ───┤     │
+         │◀── readable progress ─┤                                │     ▼
+         │                       │                          spawns Redis + agents
+         │                       │                          (sibling containers)
+```
+
+---
+
+## How the pieces connect
+
+```
+┌───────────────────────────── your machine ─────────────────────────────┐
+│                                                                         │
+│   ┌───────────┐        HTTP :8000         ┌──────────────────────────┐  │
+│   │  canyonos │ ───── /deploy /clean ────▶│   Global Controller       │  │
+│   │    CLI    │       /status /endpoints  │   container               │  │
+│   │           │ ───── docker cp ─────────▶│   ├─ /workspace (a copy   │  │
+│   │           │ ───── docker logs -f ────▶│   │  of your project)     │  │
+│   └─────┬─────┘                           │   └─ runs `ventis`        │  │
+│         │                                 └───────────┬──────────────┘  │
+│         │ docker compose                              │ docker.sock     │
+│         ▼                                             ▼ (spawns siblings)│
+│   ┌───────────────────────┐              ┌───────────────────────────┐  │
+│   │  Dashboard stack      │◀── traces ───│  Redis + your agent /      │  │
+│   │  web · api · postgres │  (OTLP)      │  workflow containers       │  │
+│   └───────────────────────┘              └───────────────────────────┘  │
+│                                                                         │
+└─────────────────────────────────────────────────────────────────────────┘
+```
+
+Three things worth internalizing about this diagram:
+
+1. **The container talks to the host Docker daemon.** The GC mounts the host's
+   `docker.sock`, so the Redis and agent/workflow containers it launches are
+   **siblings on your machine**, not nested inside it. (This is why teardown has
+   to be explicit — see `stop` vs `quit` below.)
+2. **Your project is a *copy*, not a live mount.** Files are `docker cp`'d into
+   a named volume at `/workspace`. Editing files on the host after a deploy does
+   **not** reach the running build.
+3. **The dashboard is separate.** It's its own compose stack that just *renders*
+   the OTLP traces your workflow emits — it isn't in the deploy critical path.
+
+State connecting the CLI to its container is a single file:
+`~/.canyonos/state.json` (container id + port). Every command that needs the
+container reads it.
+
+---
+
+## The three commands that matter
+
+### `build` — get your code into CanyonOS shape
+
+```
+ you ──▶ canyonos build ──▶ pick agent (Claude/Codex) + scope
+                        └─▶ fetch the porting skill from GitHub
+                        └─▶ launch your coding agent with it
+                                      │
+                                      ▼
+                              generates  .car/   ◀── canyonos-formatted project
+                                                     (originals untouched)
+```
+
+A **host-side, agent-driven** step. The CLI installs the CanyonOS porting skill
+onto your coding agent and hands it a prompt; the agent produces a `.car/`
+folder — the canyonos-ready version of your project plus its config. **No
+container is involved yet.**
+
+### `deploy` — the main path
+
+```
+ canyonos deploy
+   │
+   ├─ 1. start fresh   → ensure Docker up, tear down any old controller,
+   │                      pull + run the GC container, save state (previous canyonos init)
+   │
+   ├─ 2. ship code     → docker cp your project into /workspace
+   │
+   ├─ 3. trigger       → POST /deploy   (container runs `ventis`:
+   │                      build stubs/images + launch the workflow)
+   │
+   └─ 4. narrate       → tail container logs, boil them down to phases,
+                         and when the workflow reports "up":
+                           • auto-start the dashboard (canyonos serve)
+                           • print where everything lives
+```
+
+Everything after step 3 happens *inside* the container. The CLI's real job in
+step 4 is turning a very noisy log stream (a full image-build transcript, etc.)
+into a short, readable progression — and, on failure, revealing the part it had
+been hiding so you can see the actual cause.
+
+When it finishes you get a summary panel: the **dashboard URL** and each
+**workflow endpoint** (`POST /main`), using the real address the container
+placed the workflow at.
+
+```
+   ┌─ Deploy is live ─────────────────────────────┐
+   │  Dashboard   http://127.0.0.1:8080           │
+   │  POST        http://127.0.0.1:8000/main      │
+   │  body        {"query": "..."}                │
+   └──────────────────────────────────────────────┘
+```
+
+### `config` — view or edit settings
+
+```
+ canyonos config ──▶ View   → pretty tables of agents / otel / general
+                 └─▶ Change → interactive editor (comments & order preserved)
+```
+
+The important mental model isn't the editor — it's **what a change costs you**:
+canyonos config only allows you to change the config file, changing the source code requires a redeploy.
+
+```
+ change type              takes effect by...
+ ───────────────────────  ───────────────────────────────────
+ config value only        reloads in place  (no rebuild)
+ workflow *code* changes   full redeploy     (container holds a copy)
+```
+
+---
+
+## Lifecycle: what stays and what goes
+
+Because the deploy spawns real sibling containers, "make it stop" has two levels of "stop":
+
+```
+                   deploy   sibling      GC         project files
+                   stops?   containers?  container? (volume)?
+  ───────────────  ───────  ───────────  ─────────  ─────────────
+  canyonos stop      ✅        ✅          keep         keep
+  canyonos quit      ✅        ✅          remove       remove
+```
+
+- **`stop`** — pause the show, keep the stage set. Redeploy without re-pulling.
+- **`quit`** — full teardown. Removes the container *and* the `/workspace`
+  volume (your copied files). Every `deploy` quietly does this to any previous
+  controller, so each deploy starts clean.
+
+And to observe without changing anything:
+
+- **`logs`** — re-attach to the same live log stream `deploy` shows. Useful
+  after you Ctrl+C out of a deploy: the deploy keeps running; you just stopped
+  *watching*. (Ctrl+C on `logs` likewise only detaches.)
+
+```
+  deploy ──▶ (Ctrl+C) ──▶ still running in the container
+     │                          ▲
+     └── logs ─────────────────┘   re-attach anytime
+```
+
+---
+
+## The whole loop, one screen
+
+```
+  cd your-project
+     │
+     ▼
+  build     port your code → .car/            (opens your coding agent)
+     │
+     ▼
+  deploy    build + launch in the container   (dashboard opens itself)
+     │
+     ├─ status   where does the workflow answer?
+     ├─ config   tweak settings (live reload); redeploy for code changes
+     ├─ logs     re-attach to the stream
+     │
+     ▼
+  stop      halt the deploy, keep container + files
+     or
+  quit      full teardown, remove everything
+```
+
+That's the entire system: a thin CLI, one container that does the heavy
+lifting, a pile of sibling containers it spawns, and a dashboard watching the
+whole thing.

cli/README.md

@@ -2,6 +2,12 @@ Lightweight CLI for CanyonOS
 
 Serves as a thin API layer, connecting to the global controller container.
 
+## Architecture
+
+For a full walkthrough of the `build`, `deploy`, and `config` flows — plus how
+`logs`, `stop`, and `quit` fit into the container lifecycle — see
+[ARCHITECTURE.md](ARCHITECTURE.md).
+
 ## Serve
 
 `canyonos serve` starts the local CanyonOS dashboard against the Postgres bundled in its compose
@@ -14,15 +20,3 @@ Need uv or pip
 Need docker and docker compose
 
 If you have a workflow running, and want to make a config change, canyonos config automatically would reload the project with your config. If you change the workflow files itself though and want the changes to take effect, you need to redeploy from scratch, running canyonos build for good measure
-
-# Use: canyonos -h
-### To Republish to PyPi
-
-```Terminal
-cd cli
-# Go into, pyproject.toml, and increment version number 
-rm -rf dist/ # Removes the old distro, causes conflicts
-
-uv build
-uv publish # Needs PyPi Auth Token, ask Saaketh
-```

examples/portfolio/agents/advisor_agent.py

@@ -2,8 +2,9 @@
 #
 # Final stage. Turns the computed portfolio metrics and risk figures into a
 # short, plain-English briefing using a small, cheap model on AWS Bedrock
-# (Converse API), called via ventis.controller.bedrock so token/cost telemetry gets
-# recorded onto this execution's future:<future_id> hash. Configure
+# (Converse API), called directly via boto3. Token/cost telemetry is recorded
+# onto this execution's future:<future_id> hash transparently by the Ventis LLM
+# proxy each agent container's boto3 calls are routed through. Configure
 # with env vars:
 #   BEDROCK_MODEL_ID  (default: meta.llama3-8b-instruct-v1:0)
 #   AWS_REGION        (default: us-east-1)
@@ -15,10 +16,7 @@
 
 import os
 
-try:
-    from ventis.controller.bedrock import call_bedrock
-except ImportError:
-    from bedrock import call_bedrock
+import boto3
 
 
 class AdvisorAgent(object):
@@ -28,16 +26,16 @@ def __init__(self):
             "BEDROCK_MODEL_ID", "meta.llama3-8b-instruct-v1:0"
         )
         self.region = os.environ.get("AWS_REGION", "us-east-1")
+        self._client = boto3.client("bedrock-runtime", region_name=self.region)
 
     def summarize(self, holdings: dict, metrics: dict, risk: dict) -> str:
         """Write a short plain-English briefing on the portfolio."""
         prompt = self._build_prompt(holdings, metrics, risk)
         try:
-            response = call_bedrock(
-                model_id=self.model_id,
+            response = self._client.converse(
+                modelId=self.model_id,
                 messages=[{"role": "user", "content": [{"text": prompt}]}],
-                inference_config={"maxTokens": 400, "temperature": 0.2},
-                region=self.region,
+                inferenceConfig={"maxTokens": 400, "temperature": 0.2},
             )
             return response["output"]["message"]["content"][0]["text"]
         except Exception as e:

examples/portfolio/agents/intent_agent.py

@@ -7,9 +7,11 @@
 #      ->  {"holdings": {"AAPL": 0.4, "MSFT": 0.35, "NVDA": 0.25},
 #           "lookback_days": 180}
 #
-# Calls AWS Bedrock (Converse API) via ventis.controller.bedrock -- same pattern as
-# AdvisorAgent -- so token/cost telemetry gets recorded onto this execution's
-# future:<future_id> hash. Configure with env vars:
+# Calls AWS Bedrock (Converse API) directly via boto3 -- same pattern as
+# AdvisorAgent. Token/cost telemetry is recorded onto this execution's
+# future:<future_id> hash transparently by the Ventis LLM proxy, which each
+# agent container's boto3 calls are routed through (AWS_ENDPOINT_URL_BEDROCK_RUNTIME).
+# Configure with env vars:
 #   BEDROCK_MODEL_ID  (default: meta.llama3-8b-instruct-v1:0)
 #   AWS_REGION        (default: us-east-1)
 #
@@ -24,10 +26,7 @@
 import re
 import json
 
-try:
-    from ventis.controller.bedrock import call_bedrock
-except ImportError:
-    from bedrock import call_bedrock
+import boto3
 
 DEFAULT_LOOKBACK_DAYS = 365
 
@@ -39,14 +38,14 @@ def __init__(self):
             "BEDROCK_MODEL_ID", "meta.llama3-8b-instruct-v1:0"
         )
         self.region = os.environ.get("AWS_REGION", "us-east-1")
+        self._client = boto3.client("bedrock-runtime", region_name=self.region)
 
     def parse(self, query: str) -> dict:
         """Parse a natural-language portfolio request into holdings + lookback."""
-        response = call_bedrock(
-            model_id=self.model_id,
+        response = self._client.converse(
+            modelId=self.model_id,
             messages=[{"role": "user", "content": [{"text": self._build_prompt(query)}]}],
-            inference_config={"maxTokens": 300, "temperature": 0.0},
-            region=self.region,
+            inferenceConfig={"maxTokens": 300, "temperature": 0.0},
         )
         text = response["output"]["message"]["content"][0]["text"]
         if not text:

examples/portfolio/config/global_controller.yaml

@@ -7,7 +7,7 @@
 
 agents:
   # Stage 0: parse the free-text request into structured holdings + lookback
-  # window (calls Bedrock directly via ventis.llm.bedrock). Cheap CPU, one
+  # window (calls Bedrock via boto3, routed through the Ventis LLM proxy). Cheap CPU, one
   # call per request, on the critical path before the fan-out.
   - name: IntentAgent
     redis_port: 6379

examples/text2sql/agents/vllm_agent.py

@@ -1,9 +1,10 @@
 # VLLM Agent
 #
 # LLM backend for SQL candidate generation, called remotely by
-# SQLGeneratorAgent. Calls AWS Bedrock (Converse API) via ventis.controller.bedrock
-# so token/cost telemetry gets recorded onto this execution's
-# future:<future_id> hash — same pattern as
+# SQLGeneratorAgent. Calls AWS Bedrock (Converse API) directly via boto3.
+# Token/cost telemetry is recorded onto this execution's future:<future_id>
+# hash transparently by the Ventis LLM proxy each agent container's boto3 calls
+# are routed through — same pattern as
 # examples/portfolio/agents/advisor_agent.py.
 # Configure with env vars:
 #   BEDROCK_MODEL_ID  (default: meta.llama3-8b-instruct-v1:0)
@@ -13,26 +14,23 @@
 
 import os
 
-try:
-    from ventis.controller.bedrock import call_bedrock
-except ImportError:
-    from bedrock import call_bedrock
+import boto3
 
 
 class VllmAgent(object):
     def __init__(self):
         self.tools = [self.generate]
         self.model_id = os.environ.get("BEDROCK_MODEL_ID", "meta.llama3-8b-instruct-v1:0")
         self.region = os.environ.get("AWS_REGION", "us-east-1")
+        self._client = boto3.client("bedrock-runtime", region_name=self.region)
 
     def generate(self, prompt: str) -> str:
         """Generates a response using an LLM model based on the given prompt."""
         try:
-            response = call_bedrock(
-                model_id=self.model_id,
+            response = self._client.converse(
+                modelId=self.model_id,
                 messages=[{"role": "user", "content": [{"text": prompt}]}],
-                inference_config={"maxTokens": 400, "temperature": 0.2},
-                region=self.region,
+                inferenceConfig={"maxTokens": 400, "temperature": 0.2},
             )
             return response["output"]["message"]["content"][0]["text"]
         except Exception as e:

pyproject.toml

@@ -12,6 +12,7 @@ dependencies = [
     "psycopg[binary]",
     "pyyaml",
     "flask",
+    "requests",
     "psutil",
     "opentelemetry-api>=1.44.0",
     "opentelemetry-sdk>=1.44.0",

requirements.txt

@@ -4,6 +4,7 @@ grpcio-tools
 redis
 pyyaml
 flask
+requests
 sqlalchemy
 psycopg[binary]
 psutil