Groundwire

A RAG (Retrieval-Augmented Generation) application built with TypeScript, Express, ChromaDB, and Ollama. This system enables document ingestion and intelligent query responses using vector embeddings and language models.

Features

📄 Document Ingestion - PDF, text, and HTML processing with idempotent, per-source re-ingest
🗂️ Document Management - List and delete ingested documents per tenant (right-to-be-forgotten)
🔍 Semantic Search - Vector retrieval behind a pluggable VectorStore interface (ChromaDB today, cosine space; see ADR 0001)
🎯 Cross-Encoder Reranking - Optional local reranker that reorders candidates by true relevance (measured: MRR 0.928 → 0.990, precision 56% → 88%). See docs/EVAL_AND_RERANKING.md
🤖 AI-Powered Responses - Ollama, local llama.cpp, or Gemini; answers include cited sources
⚡ Async Ingestion - Non-blocking uploads via a BullMQ + Redis job queue (202 + job id)
🛡️ Multi-Tenant + Auth - API-key auth with per-tenant document isolation
🔒 Prompt-Injection Hardening - Retrieved context is fenced as untrusted data, and the model is told never to obey instructions inside it
🧱 Resilient - Timeouts + retry/backoff on every external call
🛰️ Observability - Prometheus /metrics, a dependency readiness probe, and a correlation id on every request/log
📊 Evaluation Harness - npm run eval scores retrieval and answer quality against a 66-case golden set over a deliberately fictional corpus; gated in CI
🧩 Modular Architecture - Extensible handlers, chunkers, embedding/rerank providers
🐳 Docker Support - Compose stack (API + ChromaDB + Redis)

Architecture

src/
├── config.ts              # Environment configuration
├── index.ts               # Express server entry point
├── app.ts                 # Express app assembly (routes + middleware)
├── core/rag/
│   ├── ingestion.ts       # Document ingestion pipeline
│   ├── rag-orchestrator.ts # Query pipeline: embed → search → rerank → generate
│   ├── chunkers/          # Text chunking strategies
│   ├── embedding/         # Embedding providers (Ollama, Llama, Gemini) + shared factory
│   ├── reranking/         # Cross-encoder reranker (local GGUF via node-llama-cpp)
│   ├── file-handlers/     # File type processors (PDF, text, HTML)
│   ├── llm/               # Language model runners (Ollama, Llama) + prompt hardening
│   ├── jobs/              # Async ingest queue (BullMQ/Redis + in-memory drivers)
│   ├── eval/              # Retrieval-quality evaluation harness + metrics
│   ├── text-processors/   # Text preprocessing utilities
│   └── vector-store/      # VectorStore interface + ChromaDB implementation
├── infrastructure/
│   ├── async/             # Lazy singleton + timeout/retry (resilience) helpers
│   ├── http/              # Auth, error handling, graceful shutdown
│   ├── observability/     # Prometheus metrics, readiness check, correlation id
│   └── logging/           # Winston logging setup
├── routes/
│   ├── health.route.ts    # Liveness + readiness probes
│   ├── ingestion.route.ts # Async upload + job status endpoints
│   ├── query.route.ts     # Query endpoint
│   └── documents.route.ts # List + delete ingested documents (tenant-scoped)
└── eval.ts                # `npm run eval` entry point

docs/adr/                  # Architecture Decision Records
eval/                      # Evaluation corpus + dataset (see EVAL_AND_RERANKING.md)

👉 New to this repo? Start here: docs/SETUP_GUIDE.md walks through getting the full stack running locally step by step.

Prerequisites

Node.js 22+
Docker / Podman (for ChromaDB, Redis, and Ollama)
Redis (for the default bull async ingest queue; or set QUEUE_DRIVER=memory)
Ollama or local llama.cpp models (for inference/embeddings) — or a Gemini API key
Optional: a GGUF reranker model to enable cross-encoder reranking

Installation

Clone the repository

git clone <repository-url>
cd groundwire

Install dependencies
```
npm install
```

Set up environment variables

Create a .env file in the project root:

# Server
PORT=3000
DEBUG=false

# Chunking
CHUNK_SIZE=1000
CHUNK_OVERLAP=150

# RAG Configuration
RAG_TOP_K=3
RETRIEVAL_THRESHOLD=0.35
MAX_TOKENS=1000

# Upload limits
MAX_UPLOAD_FILE_SIZE_MB=25
MAX_UPLOAD_FILES=10

# Models
# EMBEDDING_PROVIDER applies to BOTH ingestion and querying - documents and
# queries must be embedded by the same model or retrieval returns nonsense.
EMBEDDING_PROVIDER=ollama
EMBEDDING_MODEL=nomic-embed-text
GENERATION_MODEL=llama3.2

# Ollama
OLLAMA_HOST=http://localhost:11434

# Gemini (optional)
GEMINI_API_KEY=your-api-key

# ChromaDB
CHROMADB_HOST=localhost
CHROMADB_PORT=8000
CHROMA_COLLECTION=docs

# LangSmith (optional)
LANG_SMITH_API_KEY=
LANGSMITH_TRACING=false
LANGSMITH_ENDPOINT=

Start infrastructure services

# Using Docker Compose
docker-compose up -d

# Or for development with Ollama included
docker-compose -f docker-compose.dev.yml up -d

Pull required Ollama models

ollama pull nomic-embed-text
ollama pull llama3.2

Usage

Development

# Start development server with hot reload
npm run dev

# Start with debugging enabled
npm run debug

Production

# Build the project
npm run build

# Start the server
npm start

API Endpoints

Ingest Documents

Upload documents for processing and storage in the vector database.

curl -X POST \
  -F "docs=@/path/to/document.pdf" \
  -F "docs=@/path/to/another.txt" \
  http://localhost:3000/ingest

Ingestion runs asynchronously: the request returns immediately with a job id, and the document is processed by a background worker (BullMQ + Redis).

Response (202 Accepted):

{
  "status": "accepted",
  "jobId": "42"
}

Poll the job with GET /ingest/status/:jobId:

{
  "id": "42",
  "state": "completed",
  "result": { "chunks": 12, "sources": 1 }
}

state is one of queued, active, completed, failed (with an error on failure).

Errors: 400 no file / malformed request · 413 file too large or too many files · 415 unsupported file type · 401 when auth is enabled and no valid key is given · 404 (status endpoint) unknown job id.

Query Documents

Ask questions about the ingested documents. topK and threshold are optional and override the configured defaults for that request.

curl -X POST \
  -H "Content-Type: application/json" \
  -d '{"query": "What is the main topic of the documents?", "topK": 5, "threshold": 0.45}' \
  http://localhost:3000/query

Response:

{
  "response": "Based on the documents, the main topic is...",
  "abstained": false,
  "sources": [
    {
      "id": "chunk-8f3c...",
      "source": "handbook.pdf",
      "page": 12,
      "score": 0.82,
      "excerpt": "The first 240 characters of the retrieved chunk..."
    }
  ]
}

sources lists the chunks handed to the model, ordered from the closest match, so an answer can be traced back to the documents.

abstained is true when the documents did not contain an answer — either nothing cleared the similarity threshold, or the model judged the retrieved chunks insufficient. In that case response is a fixed "could not find an answer" message and sources is empty, because an abstention cites nothing. Check this flag rather than pattern-matching the response text: it is the difference between a grounded miss and an ungrounded guess, and it is what the evaluation harness scores as abstentionAccuracy and falseAnswerRate.

Errors: 400 invalid body (missing/empty query, topK out of range, threshold outside [-1, 1], query longer than MAX_QUERY_LENGTH) · 500 internal error.

Manage Documents

List or delete the documents a tenant has ingested. Both are behind auth (when enabled) and scoped to the requesting tenant.

# List ingested documents (with per-source chunk counts)
curl http://localhost:3000/documents

# Delete every chunk of one document (right-to-be-forgotten)
curl -X DELETE http://localhost:3000/documents/handbook.pdf

List response:

{ "documents": [ { "source": "handbook.pdf", "chunks": 12 } ] }

Delete response: { "status": "ok", "source": "handbook.pdf", "deletedChunks": 12 } (404 when the tenant has no document by that name).

Health & Readiness

GET /health is a liveness probe: it answers as long as the process can serve requests and deliberately does not call ChromaDB or Ollama. GET /health/ready is a readiness probe: it pings the backing dependencies and returns 503 while any is down, so an orchestrator can stop routing traffic without restarting the container.

curl http://localhost:3000/health        # {"status":"ok","uptime":12.34}
curl http://localhost:3000/health/ready   # 200 when ready, 503 otherwise

Readiness response:

{ "ready": true, "dependencies": [ { "name": "chroma", "ok": true }, { "name": "ollama", "ok": true } ] }

Metrics

GET /metrics exposes Prometheus metrics (unauthenticated, like /health): default process metrics, an HTTP request-duration histogram (method/route/status), and a retrieval top-score histogram. Every request also carries an x-request-id correlation id (echoed from an inbound x-correlation-id/x-request-id when present) that appears in the logs.

curl http://localhost:3000/metrics

docker-compose.yml also ships a Prometheus + Grafana stack that scrapes this endpoint — see Monitoring below.

Configuration Options

Environment Variable	Description	Default
`PORT`	Server port	`3000`
`DEBUG`	Enable debug mode	`false`
`CHUNK_SIZE`	Text chunk size	`1000`
`CHUNK_OVERLAP`	Characters each chunk repeats from the previous one (~15% of `CHUNK_SIZE`)	`150`
`EMBEDDING_BATCH_SIZE`	Chunks embedded per batch during ingestion	`64`
`QUEUE_DRIVER`	Ingest queue: `bull` (BullMQ+Redis, durable) or `memory` (in-process)	`bull`
`REDIS_URL`	Redis connection URL for the `bull` driver	`redis://localhost:6379`
`UPLOAD_DIR`	Directory where uploads are staged for the worker	`uploads`
`QUEUE_CONCURRENCY`	Ingest jobs processed concurrently	`2`
`JOB_ATTEMPTS`	Retry attempts for a failed ingest job	`3`
`EXTERNAL_TIMEOUT_MS`	Timeout per external call (Ollama/Chroma)	`30000`
`EXTERNAL_RETRY_ATTEMPTS`	Retry attempts per external call	`3`
`READINESS_TIMEOUT_MS`	Timeout per dependency ping in `/health/ready`	`3000`
`RAG_TOP_K`	Number of documents to retrieve	`3`
`RAG_MAX_TOP_K`	Upper bound a request may ask for via `topK`	`50`
`MAX_QUERY_LENGTH`	Maximum query length in characters	`2000`
`RETRIEVAL_THRESHOLD`	Minimum cosine similarity a chunk must reach; applies when reranking did not run (range `[-1, 1]`)	`0.35`
`RERANK_THRESHOLD`	Minimum cross-encoder relevance; applies when reranking ran (range `[0, 1]`)	`0.1`
`RERANK_ENABLED`	Rerank vector-search candidates with a cross-encoder before top-K	`false`
`RERANK_MODEL_PATH`	Path to a GGUF reranker model (e.g. bge-reranker)	-
`RERANK_FETCH_K`	Candidates fetched from vector search before reranking to top-K	`20`
`MAX_TOKENS`	Maximum response tokens	`1000`
`MAX_UPLOAD_FILE_SIZE_MB`	Maximum size of a single uploaded file	`25`
`MAX_UPLOAD_FILES`	Maximum files per ingestion request	`10`
`EMBEDDING_PROVIDER`	Embedding provider for ingestion and query (`ollama` \| `llama` \| `gemini`)	`ollama`
`EMBEDDING_MODEL`	Embedding model name (`ollama`, `gemini`)	-
`EMBEDDING_MODEL_PATH`	Local GGUF model path (`llama` provider)	-
`GENERATION_MODEL`	Ollama generation model	-
`OLLAMA_HOST`	Ollama server URL	`http://localhost:11434`
`GEMINI_API_KEY`	Google Gemini API key	-
`CHROMADB_HOST`	ChromaDB host	`localhost`
`CHROMADB_PORT`	ChromaDB port	`8000`
`CHROMA_COLLECTION`	ChromaDB collection name	`docs`
`AUTH_ENABLED`	Require an API key on `/ingest`, `/query`, `/documents` and scope each request to its tenant	`false`
`API_KEY_HASHES`	Comma-separated `sha256Hash:tenantId` or `sha256Hash:tenantId:scope1\|scope2` pairs. Scopes are `read` (query, list, job status), `write` (ingest), `delete` (remove a document); omitting the scope segment grants all three. Raw keys are never configured — hash one with `node -e "console.log(require('crypto').createHash('sha256').update('<key>').digest('hex'))"` and store the hash here; hand the raw key to the tenant once	-
`DEFAULT_TENANT`	Tenant assigned to every request when auth is disabled	`default`
`CORS_ORIGINS`	Comma-separated allowed origins (`*` for any, empty disables CORS)	-
`RATE_LIMIT_WINDOW_MS`	Rate-limit window in milliseconds	`60000`
`RATE_LIMIT_MAX`	Max requests per window — applied per IP on every route, and additionally per tenant (shared across `/ingest`, `/query`, `/documents`) once auth resolves a tenant	`100`
`TRUST_PROXY`	Proxy hops to trust for client IP (behind nginx/LB)	`0`

Testing

# Run tests
npm test

# Run tests with UI
npm run test:ui

# Run tests with coverage
npm run test:coverage

Evaluation & Reranking

The project ships a retrieval-quality evaluation harness (npm run eval) and an optional cross-encoder reranker. Together they let you measure retrieval quality and prove that a change (like enabling reranking) actually improves it.

# Score retrieval quality (needs ChromaDB + an embedding provider)
npm run eval

# Same, but with reranking enabled — compare the MRR against the plain run
RERANK_ENABLED=true RERANK_MODEL_PATH=./models/<reranker>.gguf npm run eval

Reranking is recommended, at its own threshold. It defaults to off only because it needs a GGUF model on disk. Measured on the shipped 66-case eval set (bge-small embeddings, k=3), each mode at its own configured threshold:

Metric	Reranker off (cosine ≥ 0.45)	Reranker on (relevance ≥ 0.1)
precision@k	56.2%	88.2%
recall@k	98.0%	99.0%
MRR	0.928	0.990
hit rate	100.0%	100.0%
snippet coverage	98.0%	97.1%
false retrieval rate	100.0%	33.3%
retrieval latency	17 ms	1449 ms

The cross-encoder improves ranking outright and cuts false retrieval by two thirds, for ~1.4 s of latency per query (still under half of generation). It reads each (question, chunk) pair together, which cosine similarity never does.

The two thresholds are separate on purpose. RETRIEVAL_THRESHOLD grades cosine similarity; RERANK_THRESHOLD grades cross-encoder relevance. They are different scales, and applying one number to both is a units error: measured at cosine's 0.45, the reranker appeared to lose recall (hit rate 92.2%), while at its own 0.1 it loses none. If you tune one, do not assume the other transfers.

The eval also runs in CI: ci.yml fails a pull request when the deterministic retrieval metrics fall below eval/gates.json, while the full run including generation is nightly and non-gating (generation is non-deterministic, so gating it would fail builds on noise).

👉 Full guide with a step-by-step walkthrough: docs/EVAL_AND_RERANKING.md

Docker

Production Setup

docker-compose up -d

This starts ChromaDB with persistent storage.

Development Setup

docker-compose -f docker-compose.dev.yml up -d

This starts ChromaDB, Ollama, and the API server with hot reloading.

Monitoring

The production compose file includes Prometheus and Grafana, wired to scrape the API's /metrics endpoint automatically:

docker-compose up -d prometheus grafana

Prometheus — http://localhost:9090 (scrape config: monitoring/prometheus/prometheus.yml, target api:3000/metrics, 15s interval)
Grafana — http://localhost:3001 (default login admin/admin, change it on first login). The Prometheus datasource and a "RAG Overview" dashboard (retrieval top-score p50/p90 + distribution, per-route request rate, p95 latency, 5xx rate) are provisioned automatically from monitoring/grafana/provisioning/ — no manual setup needed.

This tracks live production query traffic, not npm run eval runs — the eval harness is a short-lived batch script with nothing for Prometheus to scrape continuously. See Evaluation & Reranking for that side.

Supported File Types

Text files (.txt, text/plain)
HTML files (.html, text/html) — markup stripped to readable text (scripts/styles removed)
PDF files (.pdf, application/pdf)
- Standard PDF processing
- Page-by-page PDF processing (with metadata)

Project Scripts

Command	Description
`npm run dev`	Start development server
`npm run debug`	Start with Node.js inspector
`npm run build`	Compile TypeScript
`npm start`	Run production build
`npm test`	Run tests
`npm run test:ui`	Run tests with Vitest UI
`npm run test:coverage`	Run tests with coverage
`npm run compose:up`	Start Docker services
`npm run compose:down`	Stop Docker services

Tech Stack

Runtime: Node.js 22+
Language: TypeScript
Framework: Express 5
Vector Database: ChromaDB
LLM Runtime: Ollama
Embeddings: Ollama / Google Gemini
File Processing: pdf-parse, node-html-parser
Async Queue: BullMQ + Redis (in-memory fallback)
Observability: prom-client (Prometheus)
Logging: Winston
Testing: Vitest
Validation: Zod

License

MIT License - see LICENSE for details.

Author

Ahmet Atar

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
bruno/RAG API		bruno/RAG API
docs		docs
eval		eval
monitoring		monitoring
src		src
test-data		test-data
.dockerignore		.dockerignore
.editorconfig		.editorconfig
.gitignore		.gitignore
Dockerfile		Dockerfile
LICENSE		LICENSE
README.md		README.md
docker-compose.dev.yml		docker-compose.dev.yml
docker-compose.yml		docker-compose.yml
download.sh		download.sh
package-lock.json		package-lock.json
package.json		package.json
tsconfig.build.json		tsconfig.build.json
tsconfig.json		tsconfig.json
vitest.config.ts		vitest.config.ts

Navigation Menu

Repository files navigation

Groundwire

Features

Architecture

Prerequisites

Installation

Usage

Development

Production

API Endpoints

Ingest Documents

Query Documents

Manage Documents

Health & Readiness

Metrics

Configuration Options

Testing

Evaluation & Reranking

Docker

Production Setup

Development Setup

Monitoring

Supported File Types

Project Scripts

Tech Stack

License

Author

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

Groundwire

Features

Architecture

Prerequisites

Installation

Usage

Development

Production

API Endpoints

Ingest Documents

Query Documents

Manage Documents

Health & Readiness

Metrics

Configuration Options

Testing

Evaluation & Reranking

Docker

Production Setup

Development Setup

Monitoring

Supported File Types

Project Scripts

Tech Stack

License

Author

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages