Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -33,3 +33,4 @@ coverage/
# Example build output
example/build/
example/.dart_tool/
cp_test.dng
3 changes: 3 additions & 0 deletions .pubignore
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
doc/
build/
example/build/
*.dng
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Added

- **HDR / single-capture local tone mapping** — `CameraProController.captureHdr({stops})`
renders one tone-mapped HDR still. It captures a **single** frame (so there is
no motion ghosting), synthesizes an exposure stack from it by scaling in linear
light at each EV in `stops` (default `[-3, -1.5, 0, 1.5, 3]`), and fuses the
stack with **multi-scale Mertens exposure fusion** — contrast × saturation ×
well-exposedness weights blended through a Laplacian pyramid, so local contrast
is preserved with no halos. Implemented in the C core
(`camera_pro_local_tonemap` + a rewritten multi-scale `camera_pro_exposure_fusion`)
with a pure-Dart port for web (cross-checked to a few LSB). Exposed through the
backend contract as `renderHdr`, advertised via `capabilities.supportsHdr`, and
wired into both example apps (an HDR button). Verified live on the FaceTime HD
camera: the result is pixel-sharp and balanced (shadows opened, highlights held,
local contrast intact). The C harness gains fusion + tone-mapping tests, at 78
checks (arm64 + x86_64/Rosetta).

## [0.0.2] - 2026-07-07

### Changed
Expand Down
24 changes: 22 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ A Flutter camera package built on a shared C/C++ core with a crash-proof Dart AP

> **Project status: working camera engine (v0.0.2, pre-release)**
>
> On macOS the example app opens the real camera and does live preview, all six manual controls, five live visual-aid overlays (histogram, focus peaking, zebra, false color, waveform — GPU-accelerated via Metal where available), PNG + RAW/DNG capture with EXIF, burst, EV bracketing, and H.264 video recording — every one of those verified live against real hardware. The same AVFoundation backend compiles for iOS with sensor-level manual controls. **Web** runs in the browser too: a getUserMedia backend with live preview, capture, and the visual aids reimplemented in pure Dart — verified in Chrome with screenshots ([see below](#web)). Linux (V4L2) and Windows (Media Foundation) backends implement the full HAL contract and pass CI on real ubuntu/windows runners (camera-hardware runtime pending machines with cameras). Android is not started — see [ROADMAP.md](ROADMAP.md) for the honest gate on every remaining item.
> On macOS the example app opens the real camera and does live preview, all six manual controls, five live visual-aid overlays (histogram, focus peaking, zebra, false color, waveform — GPU-accelerated via Metal where available), PNG + RAW/DNG capture with EXIF, burst, EV bracketing, single-capture HDR/local tone mapping, and H.264 video recording — every one of those verified live against real hardware. The same AVFoundation backend compiles for iOS with sensor-level manual controls. **Web** runs in the browser too: a getUserMedia backend with live preview, capture, and the visual aids reimplemented in pure Dart — verified in Chrome with screenshots ([see below](#web)). Linux (V4L2) and Windows (Media Foundation) backends implement the full HAL contract and pass CI on real ubuntu/windows runners (camera-hardware runtime pending machines with cameras). Android is not started — see [ROADMAP.md](ROADMAP.md) for the honest gate on every remaining item.

---

Expand Down Expand Up @@ -129,6 +129,25 @@ Burst and exposure bracketing run through the same capture path:

![burst and EV bracket](doc/diagrams/burst-bracket.svg)

`captureHdr()` renders one tone-mapped HDR still. A temporal bracket on a
hand-held camera ghosts (the frames are ~⅓ s apart), so instead it captures a
**single** frame and synthesizes an exposure stack from it — scaling it in
linear light at a range of EV offsets — then fuses that stack with **multi-scale
[Mertens exposure fusion](https://en.wikipedia.org/wiki/Exposure_fusion)**: each
synthetic exposure is weighted per pixel by contrast (|Laplacian|), saturation,
and well-exposedness, and blended through a Laplacian pyramid so local contrast
is preserved with no seams or halos. Because every exposure comes from one
instant, the result is **sharp and ghost-free** — genuine single-capture local
tone mapping. The C core and the pure-Dart web port share the algorithm
(cross-checked to a few LSB).

![HDR exposure fusion](doc/diagrams/hdr-fusion.svg)

Verified live on the FaceTime HD camera: the tone-mapped still is pixel-sharp
(no ghosting) and balances the frame — shadows opened, highlights held, local
contrast preserved. On a dark scene it lifts a mid exposure from ~9 to ~90 mean
luma; on a bright scene it gently compresses the range.

| Feature | Status | Notes |
|---|---|---|
| `capturePhoto()` API surface | ✅ | Method exists, capability-guarded, typed error on failure |
Expand All @@ -137,7 +156,8 @@ Burst and exposure bracketing run through the same capture path:
| RAW/DNG capture | ✅ | Dependency-free linear-DNG writer with EXIF; ffmpeg-verified from the real camera |
| EXIF embedding | ✅ | ISO, exposure time, timestamps in the DNG's EXIF IFD |
| libjpeg-turbo integration | — | Skipped by design (PNG via dart:ui + DNG cover stills) |
| Burst / EV bracket | ✅ | Verified: 5-shot burst ~1.2s; bracket YAVG 25.8/96.9/183.4. HDR fusion ❌ |
| Burst / EV bracket | ✅ | Verified: 5-shot burst ~1.2s; bracket YAVG 25.8/96.9/183.4 |
| HDR / local tone mapping | ✅ | `captureHdr()` — single-frame synthesis + multi-scale Mertens fusion. Verified live: sharp, ghost-free, balanced |

### Video

Expand Down
4 changes: 2 additions & 2 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ unverifiable device code).
| Live histogram / focus peaking / zebra / false color / waveform | ✅ all five live overlays |
| RAW/DNG + EXIF (ISO, exposure, timestamps) | ✅ no libtiff/libexif needed |
| Burst / EV bracketing | ✅ |
| HDR fusion (merge brackets into one image) | ❌ (brackets are captured; fusion algorithm not written) |
| HDR / local tone mapping | ✅ `captureHdr()` — single-frame exposure synthesis + multi-scale Mertens fusion (C core + pure-Dart web port); sharp, ghost-free, verified live |
| libjpeg-turbo | skipped by design — PNG via dart:ui + DNG cover stills today |

## Phase 5 — GPU Visual Aids ✅ Metal · ⛔ others
Expand Down Expand Up @@ -111,4 +111,4 @@ unverifiable device code).
| Linux/Windows camera runtime validation | machines with cameras (CI validates compile + lifecycle) |
| Streaming transport | RTMP/SRT client implementation + an endpoint to verify against |
| Web WebGPU compute path | pure engineering — CPU pure-Dart kernels ship today; WebGPU is an optimization |
| HDR fusion, HEVC/ProRes selection, texture-based preview, ffigen swap | pure engineering time — no hardware gate |
| HEVC/ProRes selection, texture-based preview, ffigen swap | pure engineering time — no hardware gate (HDR fusion ✅ shipped) |
10 changes: 10 additions & 0 deletions doc/diagrams/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,16 @@ takes three at −2 / 0 / +2 EV, with measured mean luminance.

![Burst and EV bracket](burst-bracket.svg)

## HDR / single-capture local tone mapping

`captureHdr()` captures one frame, synthesizes an exposure stack from it (scaling
in linear light at several EV offsets), and fuses the stack with multi-scale
Mertens exposure fusion — contrast × saturation × well-exposedness weights
blended through a Laplacian pyramid. One instant in, so the tone-mapped result is
sharp and ghost-free.

![HDR / local tone mapping](hdr-fusion.svg)

## CI matrix

`native.yml` runs on every push across macOS, Ubuntu, Windows, and web — every
Expand Down
73 changes: 73 additions & 0 deletions doc/diagrams/hdr-fusion.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
19 changes: 19 additions & 0 deletions example/lib/main.dart
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,18 @@ class _CapabilityPageState extends State<CapabilityPage> {
}
}

Future<void> _hdr() async {
final controller = _controller;
if (controller == null) return;
try {
final photo = await controller.captureHdr();
setState(() => _savedPath = photo.path);
_showSnack('HDR: tone-mapped → ${photo.width}x${photo.height}');
} on Object catch (e) {
setState(() => _error = '$e');
}
}

Future<void> _toggleRecording() async {
final controller = _controller;
if (controller == null) return;
Expand Down Expand Up @@ -300,6 +312,13 @@ class _CapabilityPageState extends State<CapabilityPage> {
child: const Icon(Icons.exposure),
),
const SizedBox(width: 12),
FloatingActionButton.small(
heroTag: 'hdr',
tooltip: 'HDR fusion (-2/0/+2)',
onPressed: _hdr,
child: const Icon(Icons.hdr_on),
),
const SizedBox(width: 12),
FloatingActionButton.extended(
heroTag: 'capture',
onPressed: _attemptCapture,
Expand Down
42 changes: 32 additions & 10 deletions example/lib/web_main.dart
Original file line number Diff line number Diff line change
Expand Up @@ -222,21 +222,36 @@ class _WebCameraPageState extends State<WebCameraPage> {
if (controller == null) return;
try {
final photo = await controller.capturePhoto();
final bytes = photo.bytes;
if (bytes == null) return;
final c = Completer<ui.Image>();
ui.decodeImageFromPixels(
bytes, photo.width, photo.height, ui.PixelFormat.rgba8888, c.complete);
final img = await c.future;
setState(() {
_captured?.dispose();
_captured = img;
});
await _showCaptured(photo);
} on Object catch (e) {
setState(() => _error = '$e');
}
}

Future<void> _hdr() async {
final controller = _controller;
if (controller == null) return;
try {
final photo = await controller.captureHdr();
await _showCaptured(photo);
} on Object catch (e) {
setState(() => _error = '$e');
}
}

Future<void> _showCaptured(CapturedPhoto photo) async {
final bytes = photo.bytes;
if (bytes == null) return;
final c = Completer<ui.Image>();
ui.decodeImageFromPixels(
bytes, photo.width, photo.height, ui.PixelFormat.rgba8888, c.complete);
final img = await c.future;
setState(() {
_captured?.dispose();
_captured = img;
});
}

@override
void dispose() {
_timer?.cancel();
Expand Down Expand Up @@ -330,6 +345,13 @@ class _WebCameraPageState extends State<WebCameraPage> {
label: Text(_recording ? 'Stop' : 'Record'),
),
const SizedBox(width: 12),
FloatingActionButton.small(
heroTag: 'hdr',
tooltip: 'HDR fusion (-2/0/+2 EV)',
onPressed: _hdr,
child: const Icon(Icons.hdr_on),
),
const SizedBox(width: 12),
FloatingActionButton.extended(
heroTag: 'cap',
onPressed: _capture,
Expand Down
23 changes: 23 additions & 0 deletions lib/src/controller/camera_backend.dart
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,19 @@ abstract interface class CameraBackend {

// ── Capture ──
Future<CapturedPhoto> capturePhoto({ImageFormat? format});

/// Tone-maps a single captured [frame] into an HDR still: synthesizes an
/// exposure stack from it at [stops] and runs multi-scale exposure fusion,
/// then encodes with this backend's still encoder. Single-frame, so the
/// result is sharp and ghost-free. Used by the controller's HDR capture path.
Future<CapturedPhoto> renderHdr(
Uint8List frame, {
required int width,
required int height,
required List<double> stops,
bool isBgra = true,
});

Future<void> startVideoRecording(String path);
Future<VideoResult> stopVideoRecording();

Expand Down Expand Up @@ -161,6 +174,16 @@ class StubCameraBackend implements CameraBackend {
Future<CapturedPhoto> capturePhoto({ImageFormat? format}) async =>
_unsupported('capturePhoto');

@override
Future<CapturedPhoto> renderHdr(
Uint8List frame, {
required int width,
required int height,
required List<double> stops,
bool isBgra = true,
}) async =>
_unsupported('renderHdr');

@override
Future<void> startVideoRecording(String path) async => _unsupported('recording');

Expand Down
52 changes: 51 additions & 1 deletion lib/src/controller/camera_pro_controller.dart
Original file line number Diff line number Diff line change
Expand Up @@ -390,11 +390,61 @@ class CameraProController {
photos.add(await capturePhoto(format: format));
}
} finally {
await setExposureCompensation(previous);
// Best-effort restore; a failure here must not mask a capture error.
try {
await setExposureCompensation(previous);
} on Object {
// ignore
}
}
return photos;
}

/// Captures a single frame and renders one HDR still from it with local tone
/// mapping: an exposure stack is synthesized from the frame (gain = 2^ev for
/// each ev in [stops], in linear light) and fused with multi-scale exposure
/// fusion, lifting shadows and taming highlights while preserving local
/// contrast. Because it uses one instant, the result is sharp and ghost-free.
///
/// This is the right model for cameras without sensor-level exposure
/// bracketing (all current backends): a temporal bracket on a hand-held or
/// moving subject would ghost. Throws [CameraFeatureNotSupportedError] when
/// the backend can't render HDR.
Future<CapturedPhoto> captureHdr({
List<double> stops = const <double>[-3.0, -1.5, 0.0, 1.5, 3.0],
}) async {
if (!_capabilities.supportsHdr) {
throw CameraFeatureNotSupportedError(
feature: 'HDR fusion',
platformReason: 'Backend does not support HDR capture',
);
}
if (stops.isEmpty) {
throw CameraInvalidParameterError(message: 'HDR needs >= 1 EV stop');
}
if (!state.canCapture) {
throw CameraStateException('Cannot capture in state ${state.name}');
}
_stateMachine.transition(CameraState.capturing);
try {
final frame = _backend.latestFrame();
if (frame == null) {
throw CameraCaptureError(reason: CaptureFailureReason.noFrame);
}
return await _backend.renderHdr(
frame.bytes,
width: frame.width,
height: frame.height,
isBgra: frame.isBgra,
stops: stops,
);
} finally {
if (_stateMachine.canTransitionTo(CameraState.previewing)) {
_stateMachine.transition(CameraState.previewing);
}
}
}

/// Starts a live stream. The API is modelled; the native RTMP/SRT client is
/// roadmap, so this currently throws a typed error rather than pretending.
Future<void> startStreaming(StreamConfig config) async {
Expand Down
Loading
Loading