Minimal active-Bayesian stereo demo

A thin vertical slice through the six-layer "active Bayesian 3D vision" framework, sized for a static synthetic scene. It demonstrates the two core closures (vergence-scaling and active foveated accumulation) end-to-end.

Files

• active_stereo_demo.py — the pipeline. Runs today on a numpy synthetic scene; also loads a Blender render directory. --matcher {block,sgbm} selects the Layer-2 back end.
• generate_scene_blender.py — builds a textured scene from scratch and renders it in one command (no .blend needed).
• render_stereo_blender.py — stereo + depth rendering helpers (used by the generator; also runnable on your own .blend).

Run now (no Blender)

python3 active_stereo_demo.py --out out_block --steps 18 --matcher block
python3 active_stereo_demo.py --out out_sgbm  --steps 18 --matcher sgbm   # needs opencv

--matcher sgbm needs OpenCV: pip install opencv-python-headless. Writes to the out dir: fig_scene.png, fig_result.png (posterior depth + scanpath, uncertainty, error, RMSE-vs-fixation), fig_vergence_scaling.png.

Run the whole thing from scratch (one command each)

# 1) generate + render a textured scene with depth ground truth
blender -b -P generate_scene_blender.py -- --out ./data --baseline 0.065 --res 320 240 --samples 48
# 2) run the active inference on it
python3 active_stereo_demo.py --data ./data --out out_blender --matcher sgbm --steps 18

Loading Blender output needs imageio with EXR support: pip install imageio imageio-freeimage (or OpenEXR).

Run on your own .blend

blender -b your_scene.blend -P render_stereo_blender.py -- --out ./data --baseline 0.065
python3 active_stereo_demo.py --data ./data --out out_blender --matcher sgbm --steps 18

Data contract (what the Blender script must produce)

file	meaning
left.png, right.png	rectified stereo pair (parallel cameras, offset = baseline along local X)
depth_left.exr	linear camera-space Z in metres, left camera (Blender Z pass)
params.json	{ "f_px", "baseline", "H", "W" }
Ground-truth disparity is then d_gt = f_px * baseline / Z.

How the code maps to the framework layers

• L0 geometry — rectified rig; "vergence" is the scalar fixation distance D = f*I/d_fix (ActiveStereo.vergence).
• L1 encoding — cost_volume (box-aggregated SSD) = disparity likelihood.
• L2 correspondence — pluggable front ends returning (d_sub, var_d, valid): front_end_block (WTA + parabola subpixel + ratio/LR-consistency) or front_end_sgbm (OpenCV StereoSGBM, semi-global). Swap with --matcher.
• L3 scaling/fusion — scale_to_depth (Z = D - (D^2/I)*eta_rad) + scalar precision-weighted update in ActiveStereo.step. Measurement variance adds the linearization (Taylor-remainder) error ~eta^2, so each fixation's confident estimate is confined to its foveal neighbourhood.
• L4 control — scalar disparity-vergence loop (ActiveStereo.vergence).
• L5 active loop — per-pixel Gaussian posterior, foveated precision, and a greedy argmax-variance gaze policy (ActiveStereo.step / .run).

What it deliberately drops (and where to grow)

• process dynamics F + trans-saccadic remapping (scene is static)
• accommodation loop and AC/A coupling (L4)
• vertical-disparity distance cue and true MLE multi-cue fusion (L3)
• non-myopic (expected-free-energy) gaze policy — currently greedy info-max
• SGBM's variance is a roughness heuristic; a proper confidence (e.g. PKRN or a cost-curve from cv2.ximgproc) would sharpen the L5 uncertainty map

Notes on the demo scene

Depths are kept moderate so occlusion bands stay thin relative to card size; the left-right consistency check flags the remaining occlusions. With very large disparities, small objects fall inside their own occlusion band and block matching fails there — that is the correspondence/occlusion problem, not a bug.