whisper-api

An OpenAI-compatible Whisper speech-to-text REST API, running locally on an AMD GPU (ROCm) with Django + Django REST Framework.

Implements the official OpenAI audio endpoints, so any OpenAI client/SDK works by just changing base_url:

Endpoint	Description
POST /v1/audio/transcriptions	Transcribe audio (multipart upload)
POST /v1/audio/translations	Translate audio to English
GET /v1/models	List available models

Supported parameters: file, model (whisper-1 or any whisper model name: tiny, base, small, medium, large, turbo, ...), language, prompt, response_format (json, text, srt, verbose_json, vtt), temperature, timestamp_granularities[] (segment, word).

Requirements

• uv
• ffmpeg (system package, used by whisper to decode audio)
• AMD GPU with ROCm-supported kernel driver (developed on RX 7900 GRE / gfx1100). Falls back to CPU automatically if no GPU is available.

Setup & run

uv sync                                     # installs torch 2.13+rocm7.2, whisper, Django
uv run python manage.py migrate             # one-time, for admin/session tables
uv run python manage.py runserver 0.0.0.0:8000

The default model (turbo) is downloaded and loaded onto the GPU at startup.

Configuration (Dynaconf)

All configuration lives in settings.toml, layered by environment and overridable via environment variables:

[default]
whisper_model = "turbo"      # model served for "whisper-1"
whisper_preload = true       # load model at startup
device = "cuda"              # cuda (GPU) | cpu | auto
max_upload_mb = 100
log_level = "INFO"

[development]
debug = true
log_level = "DEBUG"

[production]
debug = false

Resolution order (highest wins):

1. Environment variables: WHISPER_API_<KEY> — e.g. WHISPER_API_WHISPER_MODEL=small, WHISPER_API_LOG_LEVEL=DEBUG
2. .secrets.toml (git-ignored, for local overrides like secret_key)
3. Active environment section — switch with WHISPER_API_ENV=production
4. [default] section

Example:

WHISPER_API_ENV=production WHISPER_API_WHISPER_MODEL=large-v3 \
  uv run python manage.py runserver

Logging

Colorized, detailed logs via colorlog (configured in config/settings.py):

02:19:27.717 INFO     api.whisper_engine:get_model:55 Loading whisper model 'base' on cuda (AMD Radeon RX 7900 GRE) ...
02:19:28.430 INFO     api.whisper_engine:get_model:65 Model 'base' ready in 0.71s (params: 71M, VRAM allocated: 0.27 GiB)
02:19:28.430 INFO     api.views:post:86 Transcribing 'jfk.flac' (1.10 MiB) | task=transcribe model=base language=auto format=json word_timestamps=False
02:19:29.480 INFO     api.views:post:98 Done in 1.05s | audio 10.6s (10.1x realtime) | detected language=en | 2 segment(s), 107 chars
02:19:29.530 WARNING  api.views:post:44 Bad request: missing 'file' field

• Level colors: DEBUG cyan, INFO green, WARNING yellow, ERROR red
• DEBUG level additionally logs full transcripts and model cache hits
• Level is controlled by log_level in settings.toml / WHISPER_API_LOG_LEVEL

Docker

The image is GPU-ready: PyTorch ROCm wheels bundle all user-space ROCm libraries, so the slim Debian base is enough — only the host kernel driver is used, passed through via /dev/kfd + /dev/dri (no special toolkit needed, unlike NVIDIA).

Get the image (built by GitHub Actions)

Images are built and pushed to GHCR by .github/workflows/docker.yml on every push to main / version tag — no local build needed:

docker pull ghcr.io/openprojectx/whisper-api:main
docker compose up -d          # uses the GHCR image

The Dockerfile is multi-stage with build-time caching designed for CI:

Stage	Contents	Rebuilds when
deps	uv venv (torch ROCm etc.) — the huge layer	uv.lock changes
models	whisper weights downloaded at build time	WHISPER_MODELS build arg changes
runtime	slim base + ffmpeg, copies of the two layers above	app code changes

All layers are cached in the GitHub Actions cache (cache-to: type=gha,mode=max), so unchanged stages are nearly free.

Two optional build args (set them in the workflow for forks):

• ROCM_ARCH=gfx1100 — prune the bundled ROCm kernel libraries (rocblas, hipblaslt, MIOpen DBs, aotriton) to a single GPU architecture. Cuts ~5-6 GB off the image; the container then only runs on that arch.
• WHISPER_MODELS="small turbo" — space-separated list of models to bake in (default: small, matching whisper_model in settings.toml).

Run

docker run -d -p 8000:8000 \
  --device=/dev/kfd --device=/dev/dri \
  --group-add 992 --group-add 44 \          # render/video GIDs: getent group render video
  -v whisper-cache:/cache \                 # persist runtime model downloads
  ghcr.io/openprojectx/whisper-api:main

The default model is baked into the image (symlinked into /cache/whisper), so first startup needs no download. Models requested per-request that aren't baked in are downloaded once into the /cache volume.

Choosing a Whisper model

Available models (GET /v1/models lists them at runtime):

Model	Params	Disk	~VRAM	English-only variant	Notes
tiny	39M	75M	~1 GB	tiny.en	fastest, lowest accuracy
base	74M	139M	~1 GB	base.en	good for quick tests
small	244M	461M	~2 GB	small.en	decent quality/speed balance
medium	769M	1.5G	~5 GB	medium.en	high accuracy
large-v1 / large-v2 / large-v3	1550M	2.9G	~10 GB	—	best accuracy, multilingual
turbo (= large-v3-turbo)	809M	1.6G	~4 GB	—	near-large accuracy, ~8x faster than large

The .en variants only understand English but tend to perform slightly better on English audio. turbo is the recommended default for the RX 7900 GRE (16 GB VRAM).

Selecting the model

1. Server-side default — used when clients send model=whisper-1 (or omit the field). In settings.toml:

[default]
whisper_model = "turbo"

or via environment variable:

WHISPER_API_WHISPER_MODEL=small uv run python manage.py runserver

2. Per-request — any loaded model name can be requested directly, overriding the server default:

curl http://localhost:8000/v1/audio/transcriptions \
  -F file=@speech.mp3 -F model=large-v3

client.audio.transcriptions.create(model="medium", file=f)

Requested models are downloaded on first use and cached in memory, so subsequent requests with the same model skip loading. Pre-download with:

uv run python manage.py download_models turbo medium

Pre-downloading models

Model weights are cached in $XDG_CACHE_HOME/whisper (default ~/.cache/whisper/). To download them in advance instead of at server startup or first request:

uv run python manage.py download_models turbo        # one model
uv run python manage.py download_models tiny small   # several models
uv run python manage.py download_models all          # every model

To use a different cache location, set XDG_CACHE_HOME (both when downloading and when running the server):

XDG_CACHE_HOME=/data/cache uv run python manage.py download_models turbo
XDG_CACHE_HOME=/data/cache uv run python manage.py runserver

Approximate sizes: tiny 75M, base 139M, small 461M, medium 1.5G, large/large-v3 2.9G, turbo 1.6G.

Usage

curl

curl http://localhost:8000/v1/audio/transcriptions \
  -F file=@speech.mp3 \
  -F model=whisper-1 \
  -F response_format=verbose_json \
  -F "timestamp_granularities[]=word"

OpenAI Python SDK (drop-in compatible)

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")

with open("speech.mp3", "rb") as f:
    transcript = client.audio.transcriptions.create(model="whisper-1", file=f)
print(transcript.text)

Notes

• GPU monitoring & verification: see docs/GPU_MONITORING.md.

• PyTorch ROCm wheels bundle their own ROCm runtime — the system ROCm installation does not need to match the wheel version.

• openai-whisper normally depends on CUDA-only triton; the dependency metadata override in pyproject.toml strips it (torch's ROCm wheel ships triton-rocm instead).

• The Django dev server is single-threaded; for production run under gunicorn (uv add gunicorn) — whisper GPU inference serializes on the model lock anyway.