Temba Docs MCP

devguide.md

@@ -2,7 +2,7 @@
 
 Notes to self and contributors on how to develop and release Temba.
 
-This repo is a monorepo containing the workspaces `packages/cli`, `packages/temba`, `docs`, and `examples`.
+This repo is a monorepo containing the workspaces `packages/cli`, `packages/temba`, `packages/mcp`, `docs`, and `examples`.
 
 > [!IMPORTANT]
 > As this is a monorepo, all commands are always called from the monorepo root folder,
@@ -13,7 +13,7 @@ This repo is a monorepo containing the workspaces `packages/cli`, `packages/temb
 You can run these commands directly from the root:
 
 ```bash
-npm test          # Runs tests for the Temba library
+npm test          # Runs tests for the Temba library and MCP package
 npm run lint      # Runs linting for the Temba library
 ```
 
@@ -23,6 +23,12 @@ Or combine them in one go:
 npm run check
 ```
 
+To run only the MCP package tests:
+
+```bash
+npm test -w packages/mcp
+```
+
 ## MongoDB E2E testing
 
 To also run the integration tests against a real MongoDB, you need a local MongoDB
@@ -113,4 +119,28 @@ Write your release notes.
 
 Commit and push the remaining changes in your feature branch.
 
-Merge the PR to `main`.
+Merge the PR to `main`.
+
+## Publishing the MCP package
+
+The MCP package is versioned independently from Temba, the CLI, examples, and docs. Do not include it in the shared `./publish.sh` release flow.
+
+To publish a new MCP version from the root folder:
+
+```bash
+./publish-mcp.sh [patch|minor|major]
+```
+
+Use `--dry-run` to inspect the release steps without changing the version or publishing:
+
+```bash
+./publish-mcp.sh patch --dry-run
+```
+
+The script bumps only `packages/mcp/package.json`, updates `packages/mcp/version.js`, publishes only the `packages/mcp` workspace, then commits the MCP package version and lockfile changes.
+
+For the first npm publish of an already prepared version, publish the workspace directly instead of bumping again:
+
+```bash
+npm publish -w packages/mcp
+```

docs/docs/getting-started.md

@@ -64,6 +64,10 @@ server.start()
 ✅ Server listening on port 8362
 ```
 
+### Temba Docs MCP
+
+If you use AI tools while building with Temba, point them at the [Temba Docs MCP](/docs/mcp) so they can use the latest documentation and current best practices.
+
 ### Configuration
 
 To opt-out or customize Temba's workings, pass a `config` object to the `create` function. Check out the individual feature pages in the sidebar, or the [config settings overview](/docs/overview#config-settings-overview).

docs/docs/mcp.md

@@ -0,0 +1,35 @@
+---
+id: mcp
+title: Temba Docs MCP
+sidebar_position: 2
+---
+
+# Temba Docs MCP
+
+You can ensure your AI tools have current Temba knowledge through the Temba Docs MCP (Model Context Protocol) server. This provides real-time access to the latest documentation, helping AI tools avoid outdated recommendations and ensuring they understand current best practices.
+
+Unlike AI models trained on static data, the MCP server provides access to the latest Temba documentation. The server is free and open-source.
+
+# Installation
+
+The setup process varies depending on your AI development tool. You may see some tools refer to MCP servers as connectors, adapters, extensions, or plugins.
+
+- [ChatGPT](https://platform.openai.com/docs/mcp#test-and-connect-your-mcp-server)
+- [Claude.ai / Claude Desktop](https://support.anthropic.com/en/articles/10168395-setting-up-integrations-on-claude-ai#h_cda40ecb32)
+- [Claude Code CLI](https://docs.anthropic.com/en/docs/claude-code/mcp)
+- [Claude Code GitHub Action](https://github.com/anthropics/claude-code-action?tab=readme-ov-file#using-custom-mcp-configuration)
+- [Codex CLI](https://developers.openai.com/codex/mcp)
+- [Cursor](https://docs.cursor.com/context/mcp)
+- [Gemini CLI](https://github.com/google-gemini/gemini-cli/blob/main/docs/tools/mcp-server.md)
+- [GitHub Copilot Coding Agent](https://docs.github.com/en/copilot/how-tos/use-copilot-agents/coding-agent/extend-coding-agent-with-mcp)
+- [Google Antigravity](https://antigravity.google/)
+- [Opencode AI](https://opencode.ai/)
+- [Raycast](https://manual.raycast.com/model-context-protocol)
+- [Visual Studio Code](https://code.visualstudio.com/docs/copilot/chat/mcp-servers#_add-an-mcp-server)
+- [Warp](https://docs.warp.dev/knowledge-and-collaboration/mcp)
+- [Windsurf](https://docs.windsurf.com/windsurf/cascade/mcp#mcp-config-json)
+- [Zed](https://zed.dev/docs/ai/mcp)
+
+### Usage
+
+Once configured, you can ask your AI tool questions about Temba, and it will retrieve information directly from the latest docs. Coding agents will be able to consult the latest documentation when performing coding tasks, and chatbots will be able to accurately answer questions about Temba features, APIs, and best practices.

docs/plugins/docusaurus-plugin-ai-docs/README.md

@@ -10,6 +10,9 @@ This plugin implements the "Third Audience" pattern for AI agents and LLM crawle
 2. **Auto-Discovery**: Injects `<link>` tags in every HTML page's `<head>` pointing to the corresponding Markdown file.
    - Example: `<link rel="alternate" type="text/markdown" href="/docs/intro.md">`
 
+3. **Search Index**: Generates a static JSON index of the documentation for client-side or external search tooling.
+   - Example: `https://temba.bouwe.io/search_index.json`
+
 ## How It Works
 
 The plugin uses two key mechanisms:
@@ -19,6 +22,7 @@ During the `postBuild` lifecycle hook, the plugin:
 - Scans the `docs/` directory for all `.md` and `.mdx` files
 - Extracts document IDs from frontmatter to determine URL routes
 - Copies each Markdown file to the build output directory at the correct path
+- Generates `search_index.json` with each page's title, URL, keywords, and Markdown content
 - Injects `<link>` tags directly into the generated HTML files
 
 ### 2. URL Matching
@@ -52,6 +56,7 @@ After building, your HTML pages will include meta tags like this:
 And the corresponding Markdown file will be accessible:
 - HTML: `https://temba.bouwe.io/docs/documentation`
 - Markdown: `https://temba.bouwe.io/docs/documentation.md`
+- Search index: `https://temba.bouwe.io/search_index.json`
 
 ## Testing
 
@@ -73,6 +78,9 @@ curl http://localhost:4444/docs/api/functions/create.md
 # Test meta tag injection
 curl http://localhost:4444/docs/getting-started.html | grep "text/markdown"
 curl http://localhost:4444/docs/api/functions/create.html | grep "text/markdown"
+
+# Test search index generation
+curl http://localhost:4444/search_index.json
 ```
 
 Expected output:
@@ -88,6 +96,16 @@ title: Documentation
 # Meta tags
 <link rel="alternate" type="text/markdown" href="/docs/getting-started.md">
 <link rel="alternate" type="text/markdown" href="/docs/api/functions/create.md">
+
+# Search index
+[
+  {
+    "title": "Getting Started",
+    "url": "/docs/getting-started",
+    "keywords": [],
+    "content": "# Getting Started\n\nPrerequisites you need to have:\n..."
+  }
+]
 ```
 
 ## Benefits

docs/plugins/docusaurus-plugin-ai-docs/index.js

@@ -1,13 +1,111 @@
 const fs = require('fs-extra');
 const path = require('path');
-const glob = require('glob');
+
+async function findMarkdownFiles(directory, rootDirectory = directory) {
+  const entries = await fs.readdir(directory, { withFileTypes: true });
+  const files = [];
+
+  for (const entry of entries) {
+    const entryPath = path.join(directory, entry.name);
+
+    if (entry.isDirectory()) {
+      files.push(...await findMarkdownFiles(entryPath, rootDirectory));
+      continue;
+    }
+
+    if (entry.isFile() && /\.(md|mdx)$/.test(entry.name)) {
+      files.push(path.relative(rootDirectory, entryPath));
+    }
+  }
+
+  return files.sort();
+}
+
+function parseFrontmatter(content) {
+  const match = content.match(/^---\s*\n([\s\S]*?)\n---\s*\n?/);
+
+  if (!match) {
+    return { data: {}, body: content };
+  }
+
+  const data = {};
+  const frontmatter = match[1];
+
+  for (const line of frontmatter.split('\n')) {
+    const fieldMatch = line.match(/^([A-Za-z0-9_-]+):\s*(.*)$/);
+
+    if (fieldMatch) {
+      data[fieldMatch[1]] = fieldMatch[2].trim();
+    }
+  }
+
+  return {
+    data,
+    body: content.slice(match[0].length),
+  };
+}
+
+function stripQuotes(value) {
+  return value.replace(/^['"]|['"]$/g, '');
+}
+
+function parseKeywords(value) {
+  if (!value) {
+    return [];
+  }
+
+  const trimmedValue = value.trim();
+
+  if (trimmedValue.startsWith('[') && trimmedValue.endsWith(']')) {
+    return trimmedValue
+      .slice(1, -1)
+      .split(',')
+      .map(keyword => stripQuotes(keyword.trim()))
+      .filter(Boolean);
+  }
+
+  return [stripQuotes(trimmedValue)].filter(Boolean);
+}
+
+function deriveTitle(file, body, frontmatterTitle) {
+  if (frontmatterTitle) {
+    return stripQuotes(frontmatterTitle);
+  }
+
+  const headingMatch = body.match(/^#\s+(.+)$/m);
+
+  if (headingMatch) {
+    return headingMatch[1].trim();
+  }
+
+  const basename = path.basename(file, path.extname(file));
+
+  return basename
+    .split(/[-_]/)
+    .filter(Boolean)
+    .map(word => word.charAt(0).toUpperCase() + word.slice(1))
+    .join(' ');
+}
+
+function resolveRoute(file, docId, routeMap) {
+  const route = routeMap[docId];
+
+  if (route) {
+    return route;
+  }
+
+  const relativePath = file.replace(/\.(md|mdx)$/, '').replace(/\/index$/, '');
+
+  return `/docs/${relativePath}`;
+}
 
 /**
  * Docusaurus Plugin: AI Documentation
  * 
  * This plugin implements the "Third Audience" pattern for AI agents:
  * 1. Copies markdown files to build output directory matching URL structure
  * 2. Injects meta tags in HTML to make markdown discoverable
+ * 3. Generates a static search_index.json for the full documentation
  */
 module.exports = function (context, options) {
   return {
@@ -21,12 +119,8 @@ module.exports = function (context, options) {
 
       const docsPath = path.join(context.siteDir, 'docs');
 
-      // Find all markdown files in the docs directory
-      const markdownFiles = glob.sync('**/*.{md,mdx}', {
-        cwd: docsPath,
-        absolute: false,
-        ignore: ['api/index.md'],
-      });
+      const markdownFiles = (await findMarkdownFiles(docsPath))
+        .filter(file => file !== 'api/index.md');
 
       console.log(`[AI Docs Plugin] Found ${markdownFiles.length} markdown files`);
 
@@ -44,26 +138,18 @@ module.exports = function (context, options) {
 
       // Map to store doc routes for HTML injection
       const docRoutes = [];
+      const searchIndex = [];
 
       // Copy each markdown file to match its URL structure
       for (const file of markdownFiles) {
         const sourcePath = path.join(docsPath, file);
 
         try {
           const content = await fs.readFile(sourcePath, 'utf-8');
+          const { data: frontmatter, body } = parseFrontmatter(content);
 
-          // Extract the doc ID from frontmatter
-          const idMatch = content.match(/^---\s*\n[\s\S]*?id:\s*(.+?)\s*\n[\s\S]*?---/m);
-          const docId = idMatch ? idMatch[1].trim() : path.basename(file, path.extname(file));
-
-          // Determine the route
-          let route = routeMap[docId];
-
-          if (!route) {
-            // Fallback: construct from file path
-            const relativePath = file.replace(/\.(md|mdx)$/, '').replace(/\/index$/, '');
-            route = `/docs/${relativePath}`;
-          }
+          const docId = frontmatter.id || path.basename(file, path.extname(file));
+          const route = resolveRoute(file, docId, routeMap);
 
           // Remove leading slash and create the destination path
           const routePath = route.replace(/^\//, '');
@@ -78,11 +164,23 @@ module.exports = function (context, options) {
 
           // Store route for HTML injection
           docRoutes.push({ route, routePath });
+          searchIndex.push({
+            title: deriveTitle(file, body, frontmatter.title),
+            url: route,
+            keywords: parseKeywords(frontmatter.keywords || frontmatter.tags),
+            content: body.trim(),
+          });
         } catch (error) {
           console.error(`[AI Docs Plugin] Error processing ${file}:`, error.message);
         }
       }
 
+      searchIndex.sort((first, second) => first.url.localeCompare(second.url));
+
+      const searchIndexPath = path.join(outDir, 'search_index.json');
+      await fs.writeFile(searchIndexPath, `${JSON.stringify(searchIndex, null, 2)}\n`, 'utf-8');
+      console.log(`[AI Docs Plugin] Generated search index: search_index.json (${searchIndex.length} entries)`);
+
       // Now inject meta tags into HTML files
       console.log('[AI Docs Plugin] Injecting meta tags into HTML files...');
 

docs/sidebars.ts

@@ -6,6 +6,7 @@ const typedocSidebar = require('./docs/api/typedoc-sidebar.cjs')
 const sidebars: SidebarsConfig = {
   tembaSidebar: [
     'getting-started',
+    'mcp',
     'overview',
     'examples',
     {