Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/kb-gate-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7

# Twice, deliberately: the barrier is scoped by a monotonic turn counter
# rather than by wall-clock time, and a regression to timestamps would show
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/sync-blocks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v7

- name: Verify SYNC blocks exist and agree across both skills + SYNC-BLOCKS.md
run: |
Expand Down
30 changes: 30 additions & 0 deletions .github/workflows/sync-memories-test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: sync-memories-test

on:
pull_request:
paths:
- "hooks/sync-memories.sh"
- "tests/sync-memories-test.sh"
- ".github/workflows/sync-memories-test.yml"
push:
branches: [main]
paths:
- "hooks/sync-memories.sh"
- "tests/sync-memories-test.sh"

permissions:
contents: read

concurrency:
group: sync-memories-test-${{ github.ref }}
cancel-in-progress: true

jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7

# qmd is stubbed, so no model or index is needed.
- name: Run the sync-memories suite
run: bash tests/sync-memories-test.sh
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
# Thumbnails
._*

.maestri

# Custom folder icons
Icon

Expand All @@ -15,4 +17,4 @@ Icon
.TemporaryItems
.Trashes
.VolumeIcon.icns
.com.apple.timemachine.donotpresent
.com.apple.timemachine.donotpresent
Expand Down
27 changes: 22 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,10 @@ The consequence that matters most: **nothing here executes from the repo.** Edit

| Task | Command |
|---|---|
| Run the hook test suite | `bash tests/kb-gate-test.sh` |
| Run the KB gate suite | `bash tests/kb-gate-test.sh` |
| Run the indexer suite | `bash tests/sync-memories-test.sh` (stubs qmd; no model needed) |
| Verify the SYNC blocks agree | the snippet below (full version at the bottom of `SYNC-BLOCKS.md`) |
| Check the manifest parses | `python3 -c "import yaml;yaml.safe_load(open('techpack.yaml'))"` |
| Check the manifest | `mcs pack validate` — validates structure and component references, not just YAML syntax. A raw parse needs a Python that has PyYAML, which is not guaranteed to be `/usr/bin/python3` |
| Install a change locally | `mcs sync --global`, or `mcs sync` inside a project |
| Check installed health | `mcs doctor` |

Expand All @@ -38,22 +39,38 @@ Two harness details are load-bearing rather than incidental:
- It runs from a temp dir **outside any git repo**, because the hook resolves its project root with `git rev-parse --show-toplevel` first. Run from the checkout, the harness would write state into the working tree and read your real session files.
- The fixture project must contain `.claude/memories/`, or every `PreToolUse` call takes the `no_memories_dir` skip and nothing is gated. The denial count asserted at the end is what turns that into a loud failure instead of a green run that asserted nothing.

`tests/sync-memories-test.sh` shares the outside-a-git-repo rule and stubs `qmd` on `PATH`, so it needs no model. Its own trap: a first run reindexes under any version of the hook, because the config file does not exist yet. The assertions that mean anything are the **second** runs, and they only discriminate because the stub creates `$INDEX_PATH` — a staleness gate guarded on that file falls through without it.

## Invariants that span files

**Placeholders are baked at sync time, not read at runtime.** `prompts:` in `techpack.yaml` declares `KB_GATE_MODE`; `hooks/kb-gate.sh` carries `MODE="__KB_GATE_MODE__"`, substituted during install. Changing the mode means re-running `mcs sync` — there is no runtime setting. The test suite injects modes the same way (`sed s/__KB_GATE_MODE__/$m/`).

**One dispatcher, four hook events.** `hooks/kb-gate.sh` is registered four times in `techpack.yaml` (UserPromptSubmit, PostToolUse, PreToolUse, SubagentStart) and branches on `hook_event_name`. Matchers are broad on purpose; which agent types count as "discovery" is decided in exactly one place, `GATED_AGENTS`. `hooks/sync-memories.sh` is likewise registered twice, on SessionStart and UserPromptSubmit.

**That dispatcher deliberately omits `set -e` and `set -u`**, unlike `sync-memories.sh` which uses `set -uo pipefail`. Its file header explains why and lists rules that are load-bearing: fail open, never `exit 2`, never call `docs-mcp-server` (too slow for `PreToolUse`), log every evaluation. Read that header before editing it.
**That dispatcher deliberately omits `set -e` and `set -u`**, unlike `sync-memories.sh` which uses `set -uo pipefail`. Its file header explains why and lists rules that are load-bearing: fail open, never `exit 2`, never call `qmd` (it loads an embedding model; far too slow for `PreToolUse`), log every evaluation. Read that header before editing it.

**Project-root derivation must be byte-identical everywhere it appears.** Both hooks and every shell string in `techpack.yaml` — the `memory-loop` launcher and each of its doctor checks — resolve git toplevel → `CLAUDE_PROJECT_DIR` → `$PWD`. The hooks must agree on which project they are looking at; the manifest must additionally agree with `sync-memories.sh` on `.claude/.kb-index/`, because one writes the index the others read. `commandExists` runs every check in its own process with no way to share a value, so the duplication is forced; keeping the copies textually identical is the only thing that makes a future edit obviously propagatable. Do not abbreviate one of them, and do not "simplify" the ladder to `$PWD` — a shortened copy that dropped the `CLAUDE_PROJECT_DIR` rung shipped once already, and made those checks silently pass without checking anything in non-git projects.

**The indexing hook and a doctor check are coupled through a file.** `sync-memories.sh` writes `.claude/.kb-index/memory-loop.log` when a run fails or leaves documents unembedded, and deletes it on success; the "Memory indexing completed" check reports the file's existence. Move or rename it on one side and the check passes forever without testing anything.

**The index is reached by `--index`, never by a project-local `.qmd/`.** Two reasons, and the second is the dangerous one. A user may keep their own `.qmd/` at the project root for their own code, which this pack must not touch. And a project-local `.qmd/index.yml` falls under qmd's trust gate, which covers a non-default `models.embed` — for a non-interactive caller the gate does not prompt or fail, it *skips*, silently substituting a much weaker default model. Named indexes are never gated. `QMD_CONFIG_DIR` and `INDEX_PATH` are what move a named index back under the project directory.

**Reranking and query expansion are disabled by pointing their model slots at the embedding model.** The MCP `query` tool hard-defaults `rerank: true` with no server-side way to turn it off, and a *missing* model is downloaded mid-query with no progress output. An embedding model has no ranking head, so qmd fails to build a ranking context, warns, and falls back to RRF — measured at MRR 0.792 against 0.800 for an explicit `rerank: false`, and it buys back zero R@5 versus a real reranker. Those numbers come from a 20-query fixture over this project's own memories, kept outside the repo — nothing here reproduces them, so treat them as recorded measurements rather than something CI re-checks. The fallback is not a *strict* no-op: on a later fixture, a document RRF ranked 9th was dropped entirely when `rerank: true` was passed. Top ranks were unaffected, which is why this is a safe structural disable, but do not describe it as "reranking is off and nothing changes".

To measure any of this, `qmd bench <fixture.json> -c memories` is usable as shipped. Its fixture `query` field accepts the structured multi-line form (`intent:`/`lex:`/`vec:`), and a fixture written that way is passed through **unexpanded** — only a bare query string goes down the expansion path. So its `hybrid` row measures the pack's real configuration, not a degraded one. This depends on qmd's graceful-degradation path rather than a documented switch, which is why `@tobilu/qmd` is pinned to an exact version and why one doctor check issues a *default-argument* query: that check is what would catch the behaviour changing under an upgrade.

**The search call shape is stated in three places, deliberately.** "Typed `lex`+`vec` lines, `rerank: false`, `limit: 6`" appears in `templates/continuous-learning.md` (the only thing that reaches a user's `CLAUDE.md`), `skills/continuous-learning/SKILL.md`, and the `SubagentStart` briefing in `hooks/kb-gate.sh`. No single mechanism reaches all three consumers, so this is three copies rather than one source — change one and check the other two. It matters because the unguided path is measurably worse, not just slower.

**The index's `global_context` is deliberately not a fourth copy.** qmd serves that one string two ways: as the MCP server's `instructions`, once per connection, and as the `context` field of *every* search result. Guidance placed there is therefore paid for per result — at 342 characters it was 38% of a six-result response — while the only consumer it uniquely reaches is a client with no installed `CLAUDE.md` section, which cannot happen because the template is `isRequired`. So it carries identity only ("this is a project memory KB, not external documentation") and the guidance lives in the three copies that are not echoed. Resist putting the call shape back into it.

**Project-root and library derivation must match across two hooks.** `sync-memories.sh` and `resolve_paths()` in `kb-gate.sh` both resolve git toplevel → `CLAUDE_PROJECT_DIR` → `$PWD`, and derive the library name as the root directory's basename. kb-gate quotes that name back to Claude, and it has to be the one sync-memories indexed. Both files carry "keep in sync" comments.
**The same three copies carry "retrieve before relying on a result", and the reason is only recorded here.** qmd's MCP snippet is at most five lines and 300 characters (`extractSnippet` in `store.js`, called with a hardcoded `300` from `mcp/server.js`), and it is anchored by literal substring matching of the first `lex` sub-query. When those terms are not in the matched text it falls back to the top of the chunk — in practice the file's first three lines, which for a memory is its title and `Applies to:`. Measured on a 527-document corpus, 28% of results came back title-only. So a search result is a lead, and answering from it is guessing; `get`/`multi_get` is the step that makes the answer real. The instruction is phrased as an absolute in all three copies on purpose — stating the failure condition invites the reader to decide a snippet looks complete this time. Do not "simplify" it back into step 2's old wording (`Read matching memories`), which worked only because the previous backend returned a whole chunk.

**Three text blocks must stay byte-identical across three files.** `capture-rules`, `strip-the-anchors`, and `applies-to` appear in both `SKILL.md`s and in `SYNC-BLOCKS.md`, enforced by `.github/workflows/sync-blocks.yml`. Two rules when touching them:

- Blocks are verdict-neutral. Each skill adds its own verb *outside* the fence — capture says "skip", audit says "DROP". Never move an action verb inside the locked block.
- Never write a real tag name in prose. The drift check's `awk` range grabs the first matching opener, so a literal mention would shadow the canonical block and make it invisible to the verifier. `SYNC-BLOCKS.md` uses a placeholder form for exactly this reason.

**Templates are installed as marked sections inside someone's `CLAUDE.md`, not as files.** The `templates:` block in `techpack.yaml` maps `templates/continuous-learning.md` to a section fenced by `<!-- mcs:begin memory.continuous-learning -->`. On a global sync it lands in `~/.claude/CLAUDE.md`; on a project sync, in that project's `CLAUDE.local.md`. Edit the template here and re-sync, because editing inside the markers drifts and is overwritten. `__PROJECT_DIR_NAME__` is substituted at install time, like the hook's mode.
**Templates are installed as marked sections inside someone's `CLAUDE.md`, not as files.** The `templates:` block in `techpack.yaml` maps `templates/continuous-learning.md` to a section fenced by `<!-- mcs:begin memory.continuous-learning -->`. On a global sync it lands in `~/.claude/CLAUDE.md`; on a project sync, in that project's `CLAUDE.local.md`. Edit the template here and re-sync, because editing inside the markers drifts and is overwritten. The template has no placeholders of its own; only `hooks/kb-gate.sh` carries one, `KB_GATE_MODE`.

**Installed artifacts are content-hash verified.** `mcs doctor` compares hashes of installed files, so hand-editing an installed copy registers as drift and the next `mcs sync` restores the packaged version. This is why a skill can never write to its own files: anything saved that way is destroyed on the next sync.

Expand Down
40 changes: 35 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ mcs sync --global # 3. install globally (~/.claude)
mcs doctor # 4. verify everything is healthy
```

**Prerequisites:** macOS, [Claude Code](https://docs.anthropic.com/en/docs/claude-code), and [Ollama](https://ollama.com) (local embeddings runtime). `mcs` installs the rest (Node, `gh`, `jq`) automatically.
**Prerequisites:** macOS, [Claude Code](https://docs.anthropic.com/en/docs/claude-code), and Node 22 or newer. `mcs` installs the rest (Node, `gh`, `jq`, [qmd](https://github.com/tobi/qmd)) automatically, and downloads a ~610 MB embedding model once on first sync — shared by every project, with no daemon left running afterwards.

Global is the recommended scope — this pack has no per-project config, so installing once makes memory available in every project automatically. To scope it to a single repo instead, run `mcs sync` from inside that repo.

Expand All @@ -32,11 +32,11 @@ flowchart LR
B --> C[Work session]
C --> D[Capture learnings & decisions]
D --> E[(.claude/memories/)]
E --> F["Ollama embeddings<br/>semantic index"]
E --> F["local embeddings<br/>semantic index"]
F -. re-index on session start / change .-> B
```

1. **Session starts** — a hook re-indexes `.claude/memories/` into a local vector store (Ollama `nomic-embed-text`), in the background.
1. **Session starts** — a hook re-indexes `.claude/memories/` into a local vector store in the background. Embeddings are computed in-process by `qmd`, so there is no service to start and nothing left running between sessions.
2. **Before any task** — Claude is instructed to search the knowledge base first, surfacing relevant past learnings and decisions.
3. **Before delegating** — sub-agents can't see the parent's KB results, so they'd rediscover everything from scratch. A gate hook closes that gap from both ends: it requires the findings to be pasted into the sub-agent's prompt, and tells any discovery agent to search the KB itself if they weren't. Configurable per project, from a reminder up to a hard block.
4. **During work** — a prompt-submit hook reminds Claude to notice when the current interaction produces knowledge worth saving.
Expand Down Expand Up @@ -66,7 +66,7 @@ To change the answer later, re-run `mcs sync` — the mode is baked into the ins

| Component | What it does |
|---|---|
| **docs-mcp-server** (MCP) | Read-only semantic search over `.claude/memories/`, backed by local Ollama embeddings |
| **memory-loop** (MCP) | Semantic search over `.claude/memories/`, backed by a local embedding model |
| **continuous-learning** (skill) | Extracts learnings and decisions from a session into structured memory files |
| **memory-audit** (skill) | Reviews existing memories and flags stale or duplicate entries to keep the KB lean |
| **sync-memories.sh** (hook) | Indexes/re-indexes memories on session start and when they change mid-session |
Expand All @@ -81,14 +81,44 @@ Memories come in two flavors, both stored as version-controlled, human-readable

---

## Upgrading from the Ollama version

Earlier releases indexed memories through `docs-mcp-server` backed by an Ollama daemon. `mcs sync`
converges on its own: the old MCP server is deregistered for you, and the new index is built on the
next session start. Nothing below is required.

What `mcs` cannot clean up is what the old release installed through plain shell commands. If you
want the disk space back, and **only if nothing else on your machine uses these**:

```bash
npm uninstall -g @arabold/docs-mcp-server
rm -rf ~/Library/Application\ Support/docs-mcp-server # see the warning below
ollama rm nomic-embed-text
```

Three things to check before running any of them:

- **`docs-mcp-server` indexes external documentation too.** If you ever pointed it at a library's
docs, that is what you would be uninstalling.
- **Its store is shared across every library you indexed with it**, not just your memories. Deleting
the directory deletes all of them. Run `docs-mcp-server list` first and see what is in there.
- **Ollama may be serving something else.** `ollama list` shows what it holds; if
`nomic-embed-text` is the only entry and nothing else here needs the runtime, you can also remove
the app and its data — `/Applications/Ollama.app` and `~/.ollama`. Removing the app clears the
macOS Login Item it registers.

This pack no longer installs or manages any of them.

---

## Directory structure

```
memory/
├── techpack.yaml # Manifest — defines all components
├── config/settings.json # Disables built-in auto-memory
├── hooks/
│ ├── sync-memories.sh # Ollama health + memory indexing/reindexing
│ ├── sync-memories.sh # Memory indexing/reindexing
│ ├── continuous-learning-activator.sh # Knowledge extraction reminder
│ └── kb-gate.sh # Keeps KB lookups ahead of delegated discovery
├── skills/
Expand Down
2 changes: 1 addition & 1 deletion SYNC-BLOCKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Locations:

```markdown
<!-- SYNC:applies-to -->
**The `Applies to:` field.** Place `**Applies to:**` on the line immediately after the `# Title` heading of every memory; it declares which project(s) the memory targets. Use the **git repo name** — the last path segment of `git remote get-url origin`, with `.git` stripped (e.g. `git@github.com:org/repo.git` → `repo`; `https://github.com/owner/my-app.git` → `my-app`). Fall back to the working directory's basename only when the repo has no remote configured. Use the repo name — not the directory basename — because folder names vary across clones while the repo name is stable. This is also why `Applies to:` may differ from the `library:` parameter used for `search_docs`, which is folder-based and set automatically by the indexing hook.
**The `Applies to:` field.** Place `**Applies to:**` on the line immediately after the `# Title` heading of every memory; it declares which project(s) the memory targets. Use the **git repo name** — the last path segment of `git remote get-url origin`, with `.git` stripped (e.g. `git@github.com:org/repo.git` → `repo`; `https://github.com/owner/my-app.git` → `my-app`). Fall back to the working directory's basename only when the repo has no remote configured. Use the repo name — not the directory basename — because folder names vary across clones while the repo name is stable. This is also why `Applies to:` may differ from the set of memories the search index actually covers, which is folder-based and set automatically by the indexing hook.

When a memory genuinely applies to multiple projects, list them comma-separated (e.g. `**Applies to:** web-dashboard, ios-app, api-backend`); the content must stay true in every listed project. When a memory is only partially relevant to one listed project, split it into separate memories instead of mixing.
<!-- /SYNC -->
Expand Down
Loading