LLM Lab

A private, local lab of Ollama tools that run on your machine. This repository hosts several tools behind one Gradio app — Lingo (rewrite + translate) and Chat, with more to come.

Run local LLM tools with Ollama — Lingo, Chat, and more.

Features

• Shared local Gradio UI (no data sent to cloud APIs)
• Tool nav: Chat and Lingo
• Lingo: bidirectional language pairs (e.g. en-es, de-fr), native-style rewrite, then translation
• Chat: talk to a selected Ollama model with system/user prompts and a scrollable history
• Copy buttons on corrected text and translation results
• Prompt controls live on the Lingo page: view the real system prompt (read-only) and add an extra user instruction
• Benchmark (green button next to the title): last-run timing and eval rate for lab tools, plus a history of the last 10 calls (saved in the browser)

Requirements

• macOS / Linux / Windows
• About 8 GB RAM for the default model (gemma3:4b). Larger models need more (e.g. gemma3:12b around 16 GB)
• Ollama installed and running
• uv (recommended; it can install Python 3.11+ for you)
• Python 3.11+ (only if you skip uv and use pip)

1. Install and start Ollama

1. Download Ollama from https://ollama.com/download.
2. Open the Ollama app (or run ollama serve in a terminal).
3. Confirm it is running:

curl http://localhost:11434

You should see a response like Ollama is running.

Useful Ollama commands

# List installed models
ollama list

# See which models are currently loaded in memory
ollama ps

# Pull (download) a model
ollama pull gemma3:4b

# Run a model interactively (optional test)
ollama run gemma3:4b

# Unload a model from memory
ollama stop gemma3:4b

# Remove a model from disk
ollama rm gemma3:4b

Suggested models

Model	Approx. size	RAM (typical)	Notes
gemma3:4b	~3 GB	~8 GB	Default; faster and lighter
gemma3:12b	~8 GB	~16 GB	Higher quality rewrite + translate
qwen2.5:7b	~4–5 GB	~10 GB	Strong multilingual alternative
translategemma:12b	~8 GB	~16 GB	Strong for pure translation; weaker for rewrite prompts

Default used by the app: gemma3:4b.

ollama pull gemma3:4b

First request after pulling can take longer while the model loads into memory.

2. Project setup

Install uv if you do not have it yet, then:

git clone git@github.com:arcadas/local-llm-lab.git
cd local-llm-lab
uv sync

uv sync creates .venv and installs locked dependencies. It will also fetch a compatible Python (3.11+) if needed.

pip (alternative)

git clone git@github.com:arcadas/local-llm-lab.git
cd local-llm-lab
python3 -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate
python -m pip install -e .

On macOS and many Linux systems the interpreter is python3, not python. If python3 is missing, install Python 3.11+ from python.org or Homebrew (brew install python). Windows: py -3 -m venv .venv. After source .venv/bin/activate, python points at this project’s environment.

3. Start the web app

uv run local-llm-lab

Or equivalently: uv run python -m local_lingo.

Or with auto-reload while developing:

uv run gradio app.py --watch-dirs src/local_lingo

Then open the URL shown in the terminal (usually http://127.0.0.1:7860).

If you installed with pip instead, activate the venv and run local-llm-lab (or python -m local_lingo).

4. How to use the app

The header shows LLM Lab. Tool pages are Chat and Lingo. Benchmark is the green button on the right of the title (shared across tools).

Chat

Two-column layout like Lingo: pick a Model (default gemma3:4b; last choice is saved in the browser), optional System prompt, write a message, and Send. Each send includes the full visible thread (up to 10 messages) plus the new message. The right panel shows a scrollable history with per-message timestamps (to the second), model name, and reply time. History is saved in browser localStorage and restored on reload. Summarize (next to Clear) collapses the thread into one recap message so later sends stay smaller. Clear wipes the thread. Replies longer than 10 lines show a preview; Show N more lines expands only the remaining lines. While a reply is coming, a loader appears in the next message box. “Completed in …” timing sits under the thread. Each send also updates the shared Benchmark history (Tool = Chat).

Lingo

1. Choose the two Languages (type to filter; full names like “English”).
2. Choose the Model (installed Ollama models; e.g. gemma3:4b vs gemma3:12b). The last choice is saved in the browser.
3. Paste your text (either language of the pair).
4. Click Correct & Translate.
5. Read:
   • Detected language
   • Corrected (native rewrite)
   • Target language
   • Translation
6. Use the copy buttons on the corrected and translation fields to copy results.

Prompt controls are on the same page, in a separate box below the form:

• System instructions (read-only) — the guidelines from src/local_lingo/prompts.py. Placeholders {name_a} and {name_b} become the selected language names at run time.
• Extra user instruction — optional add-on (tone, names to keep, and so on). Saved in the browser.
• Required result fields — the locked three-line output format the app always appends so it can fill Detected / Corrected / Translation.

Thinking is turned off on every Ollama call (think: false), so reasoning models such as Qwen 3 do not dump a hidden chain-of-thought into the reply.

Benchmark

After a run from Chat or Lingo, open Benchmark to compare models (and tools):

• Wall time, eval rate, prompt eval rate, and token counts for the last call
• Current Ollama settings (temperature, context, keep-alive, timeout)
• A history of the last 10 calls (newest first), with a Tool column (Chat / Lingo), stored in browser localStorage
• Model-level aggregates (wall, rates, tokens, load) from those runs; rewrite / translation star ratings still come from Lingo only
• Reset (below the history) clears that history

Language pair, source text, extra prompt, and benchmark history persist in the browser across reloads.

Language selection

• Pick from the searchable dropdowns (full language names)
• The pair is bidirectional: the model detects which side the text is in, corrects it, then translates to the other
• Both languages must be different
• Defaults come from DEFAULT_LANGUAGE_PAIR in src/local_lingo/config.py (e.g. en-es → English / Spanish)

Model selection

• The dropdown is filled from Ollama (GET /api/tags) when you open the app
• Embedding models are omitted
• MODEL in config.py is selected by default if that model is installed; otherwise the first installed model is used

5. Configuration

Edit src/local_lingo/config.py:

Setting	Default	Meaning
MODEL	"gemma3:4b"	Default Ollama model in the UI dropdown, if installed
DEFAULT_LANGUAGE_PAIR	"en-es"	Default pair shown in Lingo
OLLAMA_BASE_URL	"http://localhost:11434/v1"	Ollama OpenAI-compatible API
OLLAMA_API_KEY	"ollama"	Dummy key (required by the OpenAI client; Ollama ignores it)
REQUEST_TIMEOUT_SECONDS	180.0	Max wait for a model response
NUM_CTX	8192	Ollama context window; smaller is faster for typical snippets
KEEP_ALIVE	"30m"	How long to keep the model loaded in VRAM between requests
TEMPERATURE	0.0	Lower = more deterministic output

Example: prefer a larger model in the dropdown:

MODEL = "gemma3:12b"
DEFAULT_LANGUAGE_PAIR = "en-de"

Then restart the app (or let gradio reload if you are in watch mode).

Lingo prompts live in src/local_lingo/prompts.py. Parsing and Ollama calls live in src/local_lingo/service.py. Validation lives in src/local_lingo/validation.py. UI lives in src/local_lingo/ui.py.

6. Project layout

.
  README.md
  docs/screenshot.png
  docs/screenshot-benchmark.png
  docs/local-llm-lab-icon.png
  pyproject.toml
  app.py                     # Gradio watch-friendly entry
  src/local_lingo/           # Installable package (`import local_lingo`)
    config.py                # App name, model, and Ollama settings
    ui_shared.py             # Shared Gradio helpers (loader, timing, model picker)
    prompts.py               # Lingo system / user prompts
    validation.py            # Input validation
    service.py               # Shared Ollama client + Lingo business logic
    ui.py                    # App shell (nav, Lingo page, Benchmark)
    tools/
      chat/                  # Chat tool (service + Gradio page)
    highlight.py             # Word-level correction highlighting
    app.py                   # Web entry (`uv run local-llm-lab`)
    languages.py             # Language catalog
    assets/                  # Brand icons and favicons
  tests/

7. Tests

uv run python -m unittest discover -s tests -v

These cover validation, language catalog, response parsing, mocked Ollama calls, and UI build.

8. Troubleshooting

Problem	What to try
command not found: python	Prefer uv run …. With pip: use python3 to create the venv, then source .venv/bin/activate
App cannot reach the model	Ensure Ollama is running (curl http://localhost:11434)
model not found	ollama pull the name shown in the Model dropdown
Model dropdown is empty / only the default	Ensure Ollama is running, then reload the page. Pull a model with ollama pull gemma3:4b
First request is very slow	Normal cold start; the model is loading into RAM. Later requests stay warm for KEEP_ALIVE
Later requests still slow	Mostly the model itself. Try a smaller model, or lower NUM_CTX
Out of memory / machine feels slow	Stay on gemma3:4b, or ollama stop unused models
Invalid languages	Choose two different languages from the dropdowns
Empty text error	Enter text before clicking the button
Port 7860 in use	Stop the other Gradio process, or change port in demo.launch(server_port=...)

9. Privacy

All local tools run through Ollama on your machine. Nothing is sent to OpenAI or other cloud LLM APIs unless you change the client configuration yourself.