Skip to content

Segmentation Annotation Studio — annotation tooling, AI-assisted selection, exports, hardening & docs - #3

Merged
davramov merged 58 commits into
mainfrom
dabramov-fixes
Jul 17, 2026
Merged

Segmentation Annotation Studio — annotation tooling, AI-assisted selection, exports, hardening & docs#3
davramov merged 58 commits into
mainfrom
dabramov-fixes

Conversation

@davramov

Copy link
Copy Markdown
Collaborator

Overview

This PR turns the tool into a full local segmentation-annotation studio for scientific imagery. It adds a complete drawing/editing toolset, AI-assisted selection (in-browser SAM), display enhancement, a Browse/Connect data workflow over Tiled and local folders, versioned saves, COCO and DINOv3/Lightly exports, install/startup + security hardening, a Lighthouse-driven performance/accessibility pass, and MkDocs documentation. Rebranded from "SAM3 Annotation Studio" to Segmentation Annotation Studio (it's now model-agnostic).

Annotation tools & editing

  • Tools: polygon, magnetic lasso (livewire), rectangle, ellipse, brush, fill (paint-bucket), eraser, select, pan.
  • Boolean-geometry editing (via polygon-clipping): clip-to-other-classes, merge-overlapping-same-class, and erase all preserve existing vertices at full resolution — regions tile flush with no gaps and don't erode on repeated edits.
  • Eraser works on any shape kind, is radius-aware (carves on edge contact), rebuilds polygon vertices to match the cut, and can split shapes / open holes.
  • Brush builds one shape from connected strokes; disconnected dabs become separate, individually-selectable shapes.
    Polygon/lasso node-undo: Cmd/Ctrl+Z removes the last node while drafting and reopens a just-closed shape to edit mode.
  • Vertex add/delete (double-click), select-all with class/all scope (Cmd/Ctrl+A), region ops (merge/grow/shrink/remove-islands), copy/paste, brush cursor stays visible while space-panning.

AI-assisted selection

  • Magic wand with two engines: Smart (SAM / SlimSAM) running in-browser via transformers.js (WebGPU→WASM fallback, offline-vendored weights), and a classic intensity wand.
  • SAM prompts (click/box, shift-add, alt-exclude), other-class regions as auto "not" hints, and a "connected regions only" option to drop speckle.

Display, classes & workflow

  • Display (view-only, export-safe): brightness/contrast, levels histogram, colormaps, gamma, CLAHE (adaptive contrast), sharpen; sliders debounced with an optional logarithmic scale.
  • Classes: add/edit/recolor/hide, hotkeys 1–9, colorblind-safe palette, duplicate a class + all its shapes, quick-add from the annotation guide.
  • Versioned saves + crash-recovery autosave; Insights QA dashboard; a Reference guide page for expert annotation instructions.
  • Feedback button → Google Form with auto-prefilled app context; Docs & Feedback open as in-app iframes.

Browse, Connect & data

  • Connect to a Tiled server or local folder; drag-and-drop ingest; typing a comma-separated class list pre-populates a dataset's classes.
  • Browse with metadata facets, annotation-status, and star-rating filters.
  • Exports: COCO (RLE + images + semantic/per-class mask PNGs, for SAM3) and DINOv3 / Lightly (images/ + masks/ label PNGs + classes.json); optional push-masks-to-Tiled.

Install/startup & security hardening

  • start_all.sh: auto free-port finding for all services, uv auto-install, offline SAM vendoring; Tiled API keys auto-generated into gitignored .env and never exposed to the browser (all catalog/array access proxied server-side).
  • New PROD=1 ./start_all.sh mode builds the optimized SPA and serves it from FastAPI (single origin, gzip).

Performance, accessibility & SEO

  • Route-level code-splitting: the /connect entry dropped from a single ~980 kB chunk to a ~20 kB shell, with konva/AnnotatePage/polygon-clipping deferred; vendor manualChunks for caching; backend GZipMiddleware.
  • Downscaled the logo (146 KiB → 11 KiB) + explicit dimensions (fixes CLS); WCAG-AA contrast fixes; form-control labels; role="option" on class rows; meta description + robots.txt.

Docs & testing

  • MkDocs site (install, quick-start, per-tab guides, keyboard shortcuts, troubleshooting, and an expanded architecture page with design decisions and per-tool internals).
  • Frontend unit tests (Vitest) and backend tests (pytest) covering geometry, clip/merge, mask↔polygon round-trips, CLAHE, export writers, and class duplication.

Not in this PR (planned next)

  • The ipred AI/ML integration (feature compositions, CatBoost training, conformal prediction, encoder embeddings, prediction→editable-shapes) is planned and scoped but not included here.

davramov added 30 commits June 24, 2026 09:11
…ectly to Annotate page when a user uploads a dataset
…make it easier to not paint over an already annotated region with another class
…a new class, you can optionally prevent overlaps and automatically cut out holes to avoid two annotations for the same region
…opup. Adding a Reference page for expert annotations
davramov added 28 commits July 14, 2026 10:44
…ed on startup and kept in .env), adding CLAHE and sharpen adjustments
… typed before uploading data, which automatically populates the Annotate page for the dataset with those classes
…ush cursor visibility persists during panning. CLAHE integration.
…on; log checkboxes for sliders; Docs/feedback button opens in iframe instead of a new tab
…adability. Adding copyright/license documents
@davramov
davramov merged commit 614e435 into main Jul 17, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant