Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
147 changes: 128 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,36 +1,145 @@
# Segmentation Annotation Studio

Manual image segmentation tool for producing COCO datasets for SAM3 fine-tuning.
A local, browser-based tool for drawing segmentation masks on scientific images and
exporting training-ready datasets. Load data from a [Tiled](https://blueskyproject.io/tiled/)
server or a local folder, annotate with a full set of drawing tools (including an
in-browser AI "Magic" wand), and export to **COCO** (for SAM3 fine-tuning) or
**DINOv3 / Lightly** semantic-segmentation format.

Everything runs on `127.0.0.1` — no data leaves your machine.

## Quick start

You need **Node.js 18+** (`npm`) and **`curl`** on your PATH. That's it — everything
else is bootstrapped for you.

### Node.js

You must have `node.js` installed on your machine. Follow the official instructions here for your operating system:
https://nodejs.org/en/download

### Mac/Linux
```bash
chmod +x start_all.sh
./start_all.sh
```

Opens:
- Frontend: http://127.0.0.1:5173
- Backend API: http://127.0.0.1:8002
- Tiled: http://127.0.0.1:8010
On first run this will automatically:

- install [`uv`](https://docs.astral.sh/uv/) if it's missing,
- create a `.venv` with Python 3.12 and install the backend dependencies,
- generate a strong Tiled API key into `backend/.env` (gitignored, never sent to the browser),
- vendor the SlimSAM model in the background so the AI Magic tool works offline,
- start Tiled, the backend API, the frontend dev server, and the docs site.

When it's ready, open the **Frontend** URL it prints. Press **Ctrl+C** to stop everything.

| Service | Default URL | Notes |
| --------- | ----------------------- | -------------------------------------------- |
| Frontend | http://127.0.0.1:5173 | The app (Vite dev server) |
| Backend | http://127.0.0.1:8002 | FastAPI — `/api/*` |
| Tiled | http://127.0.0.1:8010 | Data server (anonymous access is read-only) |
| Docs | http://127.0.0.1:8000 | MkDocs (best-effort; the in-app Docs button) |

Ports are just defaults — if one is busy, `start_all.sh` automatically picks the next
free port and wires the services together. You can also override them, e.g.
`BACKEND_PORT=9002 ./start_all.sh`.

### Windows

On Windows, use the native PowerShell launcher instead (same behavior, no WSL/Git Bash needed).
Double-click `windows\start_all.cmd`, or from a terminal:

```powershell
powershell -ExecutionPolicy Bypass -File .\windows\start_all.ps1
```

`$env:PROD = "1"` before running does the production build; `$env:BACKEND_PORT` etc. override
ports. See [windows/README.md](windows/README.md) for details. You need **Node.js 18+** and
**PowerShell 5.1+** (built into Windows 10/11); everything else is bootstrapped for you.

### Production build (single origin)

To serve the optimized SPA directly from the backend (one origin, gzip, no Vite dev
server) instead of the dev setup:

```bash
PROD=1 ./start_all.sh
```

The frontend is built to `backend/static/` and served by FastAPI. The whole app is then
available at the **Backend** URL (http://127.0.0.1:8002).

## Tabs
1. **Connect** — pick a Tiled dataset or local folder
2. **Annotate** — draw shapes (polygon, rectangle, ellipse, brush, eraser), manage classes, navigate slices
3. **Export** — write COCO dataset for SAM3 fine-tuning
## Workflow

## Output format
COCO JSON adapted for SAM3: `segmentation` is always compressed RLE, `categories[].name` is the SAM3 concept phrase.
The app is organized into four tabs:

1. **Connect** — choose a Tiled server + dataset, or point at a local folder of images
(`.tif/.tiff`, `.npy`, `.png/.jpg`).
2. **Browse** — explore and filter Tiled datasets by metadata, and open a sample into Annotate.
3. **Reference** — author a per-dataset annotation guide: for each class a label, color,
a written description, and example crops. Guide classes surface as one-click
suggestions in Annotate, keeping annotators consistent.
4. **Annotate** — draw and edit masks, manage classes, navigate slices, and export.

### Annotation tools

Polygon, magnetic lasso (live-wire), rectangle, ellipse, brush, fill, and a **Magic**
wand backed by an in-browser SAM model (with a classic intensity wand as fallback). An
eraser and a select/transform tool round out editing, with boolean clip/merge so new
strokes don't overlap existing classes. A CLAHE adaptive-contrast display filter plus
brightness/contrast/gamma controls help with low-contrast scientific data — display-only,
never affecting exported pixels.

## Export formats

Export runs from within the Annotate tab (the download action). Two targets:

- **COCO (SAM3)** — COCO JSON where `segmentation` is compressed RLE and
`categories[].name` is the SAM3 concept phrase, alongside the rendered images. Default.
- **DINOv3 / Lightly** — a semantic-segmentation layout: `images/` + `masks/` with matching
filename stems (each mask a single-channel integer PNG, pixel = class id, 0 = background)
plus a `classes.json` index. For training non-SAM3 models (e.g. via LightlyTrain).

## Development

Backend (FastAPI, Python 3.11+):

```bash
cd backend
pip install -e ".[dev,test]" # or: uv pip install -e ".[dev,test]"
flake8 . --max-line-length=120 --extend-ignore=E501,W503 # lint (isort enforced)
pytest # tests
```

Frontend (React + TypeScript + Vite):

```bash
# Backend
cd backend && pip install -e ".[dev,test]"
pytest

# Frontend
cd frontend && npm install
npm test
npm run dev
cd frontend
npm install
npm run typecheck # tsc -b
npm run test # vitest
npm run build # production bundle
npm run dev # dev server (start_all.sh runs this for you)
```

These are the same checks CI runs (see `.github/workflows/ci.yml`).

## Security & data

- All services bind to `127.0.0.1` only. Do not change the host to `0.0.0.0` without
reconsidering the auth posture.
- Tiled anonymous access is **read-only**; writes (e.g. ingest) require the API key that
`start_all.sh` generates into `backend/.env`. The key is resolved server-side and is
**never** exposed to the frontend.
- Annotation coordinates stay in image pixels throughout.

## Project layout

```
backend/ FastAPI API, COCO/Lightly export, Tiled client, local-folder access
frontend/ React SPA (Konva canvas, Zustand stores, in-browser SAM)
tiled/ Local Tiled server config
docs/ MkDocs Material documentation site
start_all.sh One-command launcher for the full stack
```
59 changes: 59 additions & 0 deletions windows/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Windows launcher

Native Windows equivalent of the repo's `start_all.sh`. Starts Tiled, the backend API,
the frontend, and the docs site together — bootstrapping everything on first run — on
stock Windows 10/11 with **no WSL and no Git Bash**.

## Prerequisites

- **Node.js 18+** (`npm`) — install from <https://nodejs.org> or `winget install OpenJS.NodeJS`.
- **PowerShell 5.1+** — ships with Windows 10/11 (PowerShell 7 also works).

Everything else (`uv`, a Python 3.12 `.venv`, backend dependencies, the SlimSAM model,
the Tiled API key) is installed/generated automatically on first run.

## Run it

Easiest — **double-click** `windows\start_all.cmd`.

Or from a terminal, in the repo root:

```powershell
powershell -ExecutionPolicy Bypass -File .\windows\start_all.ps1
```

When it's ready it prints the URLs (Frontend, Backend, Tiled, Docs). Open the **Frontend**
URL. Press **Ctrl+C** in the window to stop all services.

### Production build (single origin)

Build the optimized SPA and have the backend serve it (no Vite dev server); the whole app
is then at the **Backend** URL:

```powershell
$env:PROD = "1"; .\windows\start_all.ps1
```

### Overriding ports

Ports auto-fall back to the next free one if a default is busy. To force specific ports:

```powershell
$env:BACKEND_PORT = "9002"; $env:FRONTEND_PORT = "5273"; .\windows\start_all.ps1
```

(`TILED_PORT`, `BACKEND_PORT`, `FRONTEND_PORT`, `DOCS_PORT` are all honored.)

## Notes & troubleshooting

- **Execution policy**: `start_all.cmd` and the `-ExecutionPolicy Bypass` invocation both
bypass the policy for that one process only — no system-wide change.
- **`uv` just installed but "not found"**: open a **new** PowerShell window (so `%USERPROFILE%\.local\bin`
is on `PATH`) and re-run.
- **`pycocotools` build error**: recent versions ship Windows wheels; if the install fails,
install the **"Desktop development with C++"** workload from the Visual Studio Build Tools
and re-run, or the launcher will retry the dependency install.
- **Behavior parity**: this is a 1:1 port of `../start_all.sh` — same ports, same auto
port-fallback, same Tiled key handling (anonymous access is read-only; the generated key
authenticates writes and is never sent to the browser), same `PROD=1` prod-serving mode.
All services bind to `127.0.0.1` only.
5 changes: 5 additions & 0 deletions windows/start_all.cmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
@echo off
REM Double-click launcher for Segmentation Annotation Studio on Windows.
REM Runs start_all.ps1 with the execution policy bypassed for this process only
REM (no global policy change). Any args are forwarded, e.g. set PROD=1 first.
powershell -NoProfile -ExecutionPolicy Bypass -File "%~dp0start_all.ps1" %*
Loading
Loading