feat: HDR — single-capture local tone mapping (multi-scale exposure fusion) - #6
Open
sayed3li97 wants to merge 2 commits into
Open
feat: HDR — single-capture local tone mapping (multi-scale exposure fusion)#6sayed3li97 wants to merge 2 commits into
sayed3li97 wants to merge 2 commits into
Conversation
Add single-scale Mertens exposure fusion so a captured EV bracket can be merged into one tone-mapped image — completing the burst → bracket → HDR pro-capture story. No hardware gate; pure engineering. Core: - camera_pro_exposure_fusion in image_processor.c: per-pixel weight = well-exposedness (Gaussian around mid-grey) × saturation, normalised and blended across the bracket. Math in double; new clampd_u8. Header decl. - NativeCore.exposureFusion FFI wrapper (non-leaf) + @Native binding. - Byte-for-byte pure-Dart port in native_core_web.dart for web parity. API: - CameraProController.captureHdr({stops}) drives the EV loop, grabs frames, and delegates fusion+encode to the backend; restores exposure in finally. - CameraBackend.fuseExposures: Apple encodes a PNG, web returns RGBA bytes, stub throws. supportsHdr flipped true on both backends. HDR button in both example apps. Robustness (from an adversarial-review pass): - captureHdr guards that every bracket frame shares geometry, so a live resolution change mid-bracket surfaces a typed CameraCaptureError instead of a RangeError or a corrupt image. - the exposure-restore in finally is now best-effort (try/catch) in both captureHdr and captureExposureBracket, so it can't mask the real capture error or wedge the state machine in `capturing`. - exposureFusion (native + web) validates each frame's byte length. Verification: - C harness test_exposure_fusion (shadow lift + highlight recovery + n=1 identity + param validation): 70 checks, arm64 + x86_64/Rosetta. - FFI + browser tests cross-check the C core vs the pure-Dart port within 1 LSB, pin channel order on a colored bracket, and reject size mismatch. - 89 VM + 70 browser tests pass. - Verified LIVE on the FaceTime HD camera: a mid frame 77% crushed-black fused to mean-luma 94, 0% crushed shadows. Also: .pubignore was overriding .gitignore, leaking a local build/ cache into the archive (29 MB → 384 KB); exclude build/, example/build/, *.dng. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The first cut was unusable: a temporal 3-frame EV bracket (~360 ms apart)
ghosted on any motion, and the naive single-scale weighted average looked
hazy and haloed. Reworked into proper single-capture local tone mapping.
What changed:
- Capture ONE frame and synthesize an exposure stack from it (gain = 2^ev in
linear light, default stops [-3,-1.5,0,1.5,3]) — no temporal bracket, so the
result is pixel-sharp and ghost-free (single-image exposure fusion, a la
Wronski / Hessel WACV'20).
- Replace the naive fuser with real MULTI-SCALE Mertens: per-pixel weight =
contrast(|Laplacian|) x saturation x well-exposedness, blended through a
Laplacian pyramid (binomial reduce/expand) so local contrast is preserved
with no seams or halos. New camera_pro_local_tonemap; camera_pro_exposure_fusion
rewritten multi-scale. All in float.
- Pure-Dart port of the whole pyramid path for web (native_core_web.dart).
C (float, -ffast-math) vs Dart (double) now agree to a few LSB (was 1).
- API: backend fuseExposures -> renderHdr(frame, {stops}); controller.captureHdr
simplified to a single grab (no EV loop, no exposure restore, no bracket
geometry guard — all moot with one frame). Example HDR button uses defaults.
Verification:
- Live on the FaceTime HD camera: tone-mapped still is razor-sharp (ghosting
gone) and natural — shadows opened, highlights held, local contrast intact.
- C harness gains test_local_tonemap + multi-scale test_exposure_fusion (78
checks, arm64 + x86_64/Rosetta); clean under AddressSanitizer + UBSan.
- 90 VM + 71 browser tests; C-vs-Dart fusion/tonemap cross-check.
Docs (README, ROADMAP, CHANGELOG, diagram, doc comments) updated to describe
single-capture local tone mapping. Removed a stray cp_test.dng harness artifact.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Owner
Author
Reworked: HDR is now sharp and usableThe first cut produced ghosted, washed-out results. Root causes and fixes:
This is single-image exposure fusion = local tone mapping (Wronski, Hessel WACV'20). Verified live on the FaceTime HD camera: the tone-mapped still is pixel-sharp (ghosting gone) and natural — shadows opened, highlights held, local contrast preserved. Verification: 78 C-harness checks (arm64 + x86_64/Rosetta), clean under AddressSanitizer + UBSan (the pyramid code does manual malloc), 90 VM + 71 browser tests, C-vs-Dart cross-check. All 4 CI jobs green. Also fixed an OOM-path pyramid leak found in a manual review pass. API: backend |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds HDR exposure fusion —
CameraProController.captureHdr({stops})captures an EV bracket and merges the frames into one tone-mapped image using single-scale Mertens exposure fusion. This completes the pro-capture pillar: burst → bracket → HDR. It's pure engineering (no hardware gate) and works on every platform that has the digital pipeline (macOS/iOS + web).How
src/core/image_processor.c):camera_pro_exposure_fusion— per pixel, weight each exposure by well-exposedness (a Gaussian around mid-grey) × saturation, normalise across the bracket, blend. Math indouble; newclampd_u8.native_core_web.dart; the FFI/browser tests cross-check the two within 1 LSB.captureHdr()drives the EV loop, grabs frames, and delegates fuse+encode to the backend's newfuseExposures(Apple → PNG on disk, web → RGBA bytes, stub → typed error).capabilities.supportsHdrflippedtrue; an HDR button added to both example apps.Live verification
Run on the real FaceTime HD camera in a dark room:
The subject — invisible in the single exposure — is fully recovered in the fusion (shadows pulled from the +2 EV frame).
Verification summary
test_exposure_fusion(shadow lift, highlight recovery, n=1 identity, param validation) → 70 checks, arm64 + x86_64/Rosetta.dart analyzeclean;dart pub publish --dry-runclean (384 KB).Hardening from an adversarial-review pass
A multi-agent review of this diff surfaced (and this PR fixes) three real edge cases in the new code:
captureHdrnow rejects a bracket whose frames changed resolution mid-capture (e.g. a mobile-web orientation flip) with a typedCameraCaptureErrorinstead of aRangeError/corrupt image.finallyrestore is now best-effort (try/catch) in bothcaptureHdrandcaptureExposureBracket, so a restore failure can't mask the real capture error or wedge the state machine incapturing.exposureFusion(native + web).Note on release
Version stays at 0.0.2 (change logged under
## [Unreleased]), so merging does not auto-publish —release.ymlskips because tagv0.0.2already exists. Bumppubspec.yamlto0.0.3in a follow-up to cut the release.🤖 Generated with Claude Code