Dockerized RAG System

Still in development, Dockerization not implemented yet.

This repository contains a Retrieval-Augmented Generation (RAG) system packaged to run with Docker and docker-compose. The system indexes documents, builds embeddings with SentenceTransformers, stores vectors in a FAISS index, and uses a local generative LLM to answer user queries using only retrieved context.

Table of contents

• Requirements
• Quick start (installation & run)
• Configuration and environment variables
• Models and cache
• Running the application (interactive)
• Logs
• Troubleshooting

Requirements

• Docker and docker-compose installed on the host.
• Internet access for the first run to download models and dependencies (unless models are pre-cached).
• Sufficient disk space (several GB) for model caches.

Quick start

1. Clone the repository or copy the project to your machine.
2. Build the containers:

docker-compose build

3. Start the services:

docker-compose up

The first run will download required models and install dependencies inside the containers. This may take several minutes depending on network speed and machine resources.

Configuration and environment variables

Most runtime settings are controlled through environment variables defined in the project's configuration code or docker-compose. Key variables used by the code:

• PROJECT_ROOT: auto-detected project root in the container.
• CACHE_ROOT: base folder for model caches (set by config module).
• NUM_CHUNKS: number of top chunks to retrieve per query (default: 3).
• EMBEDDING_MODEL: HF model ID used for embeddings (default: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2).
• REPO_ID / GENERATIVE_MODEL: identifiers/filenames used to load the generative LLM from cache or Hugging Face Hub.

Check src/functions/config.py for current defaults and adjust via docker-compose environment entries if needed.

Models and cache

Downloaded models and related cache files are stored under the cache directory inside the project (CACHE_ROOT). Docker volumes defined in docker-compose map those folders to host paths so models persist between runs and avoid re-downloading.

Typical behavior:

• When a model is already present in the cache folder, the service will load it from disk.
• On first execution (or after removing cache), models and dependencies are downloaded into the cache folder.

To clear cached models, stop the containers and remove the mapped cache directory on the host (for example ./cache). On next start the system will re-download required models.

Running the application (interactive)

Inside the running container the project provides an interactive CLI (src/main.py). It prompts for a question, retrieves relevant document chunks from the FAISS index, and queries the local LLM with the retrieved context.

Typical flow:

1. Ensure you have text documents to index in the documents/ folder (container path: PROJECT_ROOT/documents). The project reads *.txt files from that folder.

2. Start the container (docker-compose up). The index and embeddings are created at startup.

3. Attach to the container or use the service's console. The main process prompts:

   Ask something to be found in the documents (or type exit):

4. Type a query and press Enter. Type exit to quit.

Note: The system enforces that the LLM uses ONLY the provided context. If the answer is not present in the retrieved context, the model is instructed to reply exactly: "I don't have enough information to answer this question.".

Logs

Logs are written to the logs/ folder inside the project. The logger creates daily files with debug and short formats. Check those files for detailed startup, embedding, and model-loading information.

Troubleshooting

• If documents are not found: create a documents/ directory in the project root and add plain .txt files.
• If model loading fails: check network access and the mapped cache folder permissions. Review logs/log_date.log for detailed errors.
• If FAISS or embeddings fail: ensure NUM_CHUNKS and memory constraints are appropriate for your environment.

If you need to run in an air-gapped environment, pre-download and place the model files under the configured cache directory before starting the containers.

---

For more details on the internals, inspect the source files under src/functions (config, embeddings, generative_model, documents_processing).