Skip to content

feat: HDR — single-capture local tone mapping (multi-scale exposure fusion) - #6

Open
sayed3li97 wants to merge 2 commits into
mainfrom
feat/hdr-fusion
Open

feat: HDR — single-capture local tone mapping (multi-scale exposure fusion)#6
sayed3li97 wants to merge 2 commits into
mainfrom
feat/hdr-fusion

Conversation

@sayed3li97

Copy link
Copy Markdown
Owner

What

Adds HDR exposure fusionCameraProController.captureHdr({stops}) captures an EV bracket and merges the frames into one tone-mapped image using single-scale Mertens exposure fusion. This completes the pro-capture pillar: burst → bracket → HDR. It's pure engineering (no hardware gate) and works on every platform that has the digital pipeline (macOS/iOS + web).

How

  • C core (src/core/image_processor.c): camera_pro_exposure_fusion — per pixel, weight each exposure by well-exposedness (a Gaussian around mid-grey) × saturation, normalise across the bracket, blend. Math in double; new clampd_u8.
  • Parity: byte-for-byte pure-Dart port in native_core_web.dart; the FFI/browser tests cross-check the two within 1 LSB.
  • API: captureHdr() drives the EV loop, grabs frames, and delegates fuse+encode to the backend's new fuseExposures (Apple → PNG on disk, web → RGBA bytes, stub → typed error). capabilities.supportsHdr flipped true; an HDR button added to both example apps.

Live verification

Run on the real FaceTime HD camera in a dark room:

mean luma shadow<16
Single mid-EV frame 9.2 77.0% crushed black
HDR fused (−2/0/+2) 94.1 0.0%

The subject — invisible in the single exposure — is fully recovered in the fusion (shadows pulled from the +2 EV frame).

Verification summary

  • C harness test_exposure_fusion (shadow lift, highlight recovery, n=1 identity, param validation) → 70 checks, arm64 + x86_64/Rosetta.
  • 89 VM + 70 browser tests pass, including C-vs-Dart 1-LSB cross-check, colored-bracket channel-order pin, and size-mismatch rejection.
  • dart analyze clean; dart pub publish --dry-run clean (384 KB).

Hardening from an adversarial-review pass

A multi-agent review of this diff surfaced (and this PR fixes) three real edge cases in the new code:

  1. Geometry guardcaptureHdr now rejects a bracket whose frames changed resolution mid-capture (e.g. a mobile-web orientation flip) with a typed CameraCaptureError instead of a RangeError/corrupt image.
  2. Guarded exposure-restore — the finally restore is now best-effort (try/catch) in both captureHdr and captureExposureBracket, so a restore failure can't mask the real capture error or wedge the state machine in capturing.
  3. Length validation in exposureFusion (native + web).

Note on release

Version stays at 0.0.2 (change logged under ## [Unreleased]), so merging does not auto-publish — release.yml skips because tag v0.0.2 already exists. Bump pubspec.yaml to 0.0.3 in a follow-up to cut the release.

🤖 Generated with Claude Code

SayedAliAlkamel and others added 2 commits July 11, 2026 01:13
Add single-scale Mertens exposure fusion so a captured EV bracket can be
merged into one tone-mapped image — completing the burst → bracket → HDR
pro-capture story. No hardware gate; pure engineering.

Core:
- camera_pro_exposure_fusion in image_processor.c: per-pixel weight =
  well-exposedness (Gaussian around mid-grey) × saturation, normalised and
  blended across the bracket. Math in double; new clampd_u8. Header decl.
- NativeCore.exposureFusion FFI wrapper (non-leaf) + @Native binding.
- Byte-for-byte pure-Dart port in native_core_web.dart for web parity.

API:
- CameraProController.captureHdr({stops}) drives the EV loop, grabs frames,
  and delegates fusion+encode to the backend; restores exposure in finally.
- CameraBackend.fuseExposures: Apple encodes a PNG, web returns RGBA bytes,
  stub throws. supportsHdr flipped true on both backends. HDR button in both
  example apps.

Robustness (from an adversarial-review pass):
- captureHdr guards that every bracket frame shares geometry, so a live
  resolution change mid-bracket surfaces a typed CameraCaptureError instead
  of a RangeError or a corrupt image.
- the exposure-restore in finally is now best-effort (try/catch) in both
  captureHdr and captureExposureBracket, so it can't mask the real capture
  error or wedge the state machine in `capturing`.
- exposureFusion (native + web) validates each frame's byte length.

Verification:
- C harness test_exposure_fusion (shadow lift + highlight recovery + n=1
  identity + param validation): 70 checks, arm64 + x86_64/Rosetta.
- FFI + browser tests cross-check the C core vs the pure-Dart port within
  1 LSB, pin channel order on a colored bracket, and reject size mismatch.
- 89 VM + 70 browser tests pass.
- Verified LIVE on the FaceTime HD camera: a mid frame 77% crushed-black
  fused to mean-luma 94, 0% crushed shadows.

Also: .pubignore was overriding .gitignore, leaking a local build/ cache into
the archive (29 MB → 384 KB); exclude build/, example/build/, *.dng.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The first cut was unusable: a temporal 3-frame EV bracket (~360 ms apart)
ghosted on any motion, and the naive single-scale weighted average looked
hazy and haloed. Reworked into proper single-capture local tone mapping.

What changed:
- Capture ONE frame and synthesize an exposure stack from it (gain = 2^ev in
  linear light, default stops [-3,-1.5,0,1.5,3]) — no temporal bracket, so the
  result is pixel-sharp and ghost-free (single-image exposure fusion, a la
  Wronski / Hessel WACV'20).
- Replace the naive fuser with real MULTI-SCALE Mertens: per-pixel weight =
  contrast(|Laplacian|) x saturation x well-exposedness, blended through a
  Laplacian pyramid (binomial reduce/expand) so local contrast is preserved
  with no seams or halos. New camera_pro_local_tonemap; camera_pro_exposure_fusion
  rewritten multi-scale. All in float.
- Pure-Dart port of the whole pyramid path for web (native_core_web.dart).
  C (float, -ffast-math) vs Dart (double) now agree to a few LSB (was 1).
- API: backend fuseExposures -> renderHdr(frame, {stops}); controller.captureHdr
  simplified to a single grab (no EV loop, no exposure restore, no bracket
  geometry guard — all moot with one frame). Example HDR button uses defaults.

Verification:
- Live on the FaceTime HD camera: tone-mapped still is razor-sharp (ghosting
  gone) and natural — shadows opened, highlights held, local contrast intact.
- C harness gains test_local_tonemap + multi-scale test_exposure_fusion (78
  checks, arm64 + x86_64/Rosetta); clean under AddressSanitizer + UBSan.
- 90 VM + 71 browser tests; C-vs-Dart fusion/tonemap cross-check.

Docs (README, ROADMAP, CHANGELOG, diagram, doc comments) updated to describe
single-capture local tone mapping. Removed a stray cp_test.dng harness artifact.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@sayed3li97 sayed3li97 changed the title feat: HDR exposure fusion (captureHdr) feat: HDR — single-capture local tone mapping (multi-scale exposure fusion) Jul 15, 2026
@sayed3li97

Copy link
Copy Markdown
Owner Author

Reworked: HDR is now sharp and usable

The first cut produced ghosted, washed-out results. Root causes and fixes:

Problem Cause Fix
Ghosting / blur 3-frame temporal EV bracket ~360 ms apart → motion smears the average Capture one frame and synthesize the exposure stack from it (gain = 2^ev in linear light) — ghost-free
Washed-out / hazy naive single-scale weighted average Proper multi-scale Mertens: contrast × saturation × well-exposedness weights blended through a Laplacian pyramid
Lost detail no contrast weight added contrast = |Laplacian(luma)|

This is single-image exposure fusion = local tone mapping (Wronski, Hessel WACV'20).

Verified live on the FaceTime HD camera: the tone-mapped still is pixel-sharp (ghosting gone) and natural — shadows opened, highlights held, local contrast preserved.

Verification: 78 C-harness checks (arm64 + x86_64/Rosetta), clean under AddressSanitizer + UBSan (the pyramid code does manual malloc), 90 VM + 71 browser tests, C-vs-Dart cross-check. All 4 CI jobs green. Also fixed an OOM-path pyramid leak found in a manual review pass.

API: backend fuseExposuresrenderHdr(frame, {stops}); controller.captureHdr is now a single grab.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants