OptiGeo: Efficient Monocular Geometry for Embodied Perception in Optically Challenging Scenes

Muxin Liu^1,2,*, Tianbo Liu^1,*, Jing Xia^1,*, Xiaoyang Lyu¹, Xiaoshan Wu¹, Bo Wang¹, Peng Dai¹,
Zhongrui Wang³, Shaoshuai Shi^2,✉, Xiaojuan Qi^1,✉

¹The University of Hong Kong    ²Voyager Research, Didi Chuxing   
³Southern University of Science and Technology
^*Equal Contribution   ^✉Corresponding Author

This work presents OptiGeo, which redefines transparent and reflective depth estimation as localized bias correction within monocular geometry training.

· 🧭 Rehabilitates biased real-depth supervision with a clean-geometry teacher and residual-trimmed alignment.
· ⚡ Delivers a 30M model for accurate and efficient embodied perception in optically challenging scenes.

Release Status

• Release paper and project page
• Release OptiGeo model weights
• Release inference, evaluation, and training code
• Release training and evaluation configuration files
• Release OptiGeo dataset and rendering pipeline
• Release edge computing variants and navigation system setup pipeline

⚙️ Installation

git clone https://github.com/mx-liu6/OptiGeo.git
cd OptiGeo
conda create -n optigeo python=3.10 -y
conda activate optigeo
pip install -r requirements.txt

For editable local development, install the package as well:

pip install -e .

If you install PyTorch manually, choose the build that matches your CUDA version from the official PyTorch instructions before running pip install -r requirements.txt.

📦 Model Weights

The released OptiGeo model weights are hosted on Hugging Face:

Weights	Description	Parameters
mxliu-hku/OptiGeo	Efficient monocular geometry model for optically challenging scenes	30M

The weights are downloaded automatically when --pretrained is omitted or set to mxliu-hku/OptiGeo.

🚀 Quick Start

# Run inference on one image or a folder of images
python optigeo/scripts/infer.py \
  --input path/to/image_or_folder \
  --output output/optigeo \
  --pretrained mxliu-hku/OptiGeo \
  --device cuda \
  --maps \
  --ply \
  --glb

Outputs are saved under output/optigeo and can include:

• image.jpg: resized input image used by inference
• depth.exr and depth_vis.png: metric depth and visualization
• points.exr: metric point map
• mask.png: valid prediction mask
• fov.json: estimated camera field of view
• pointcloud.ply and mesh.glb: 3D exports

Useful options:

# Faster inference with half precision
python optigeo/scripts/infer.py -i path/to/images -o output/fast --fp16 --maps

# Control inference resolution; higher is sharper but slower
python optigeo/scripts/infer.py -i path/to/images -o output/high --resolution_level 9 --maps

# Use a known horizontal camera field of view in degrees
python optigeo/scripts/infer.py -i path/to/image.jpg -o output/fov --fov_x 70 --maps

📏 Evaluation

Evaluation instructions are available in docs/eval.md. The evaluation pipeline wraps baseline models, runs configured benchmarks, and writes metrics to JSON.

🏋️ Training

We provide training code and configuration files:

• OptiGeo-S: configs/train/OptiGeo.json
• OptiGeo-H+: configs/train/OptiGeo_Hplus_w_refine.json
• OptiGeo-L: configs/train/OptiGeo_Large_w_refine.json
• Multi-GPU launch script: scripts/train.sh

Data Preparation

Training datasets are expected under data/train. Each dataset should contain an index file and per-sample folders:

data/train/somedataset
├── index.txt
├── sample_000001
│   ├── image.jpg
│   ├── depth.png
│   └── meta.json
└── ...

index.txt stores one sample folder per line. meta.json should include normalized camera intrinsics:

{
  "intrinsics": [[fx, 0.0, cx], [0.0, fy, cy], [0.0, 0.0, 1.0]]
}

Depth maps can be read and written with the helpers in optigeo/utils/io.py. You can inspect prepared samples with:

python optigeo/scripts/vis_data.py data/train/somedataset/sample_000001 --ply

Run Training

For a single-machine launch, call accelerate directly and adjust --num_processes, batch size, workspace, and checkpoint path as needed:

accelerate launch --multi_gpu --num_processes 8 \
  optigeo/scripts/train.py \
  --config configs/train/OptiGeo.json \
  --workspace workspace/OptiGeo \
  --gradient_accumulation_steps 1 \
  --batch_size_forward 16 \
  --checkpoint latest \
  --enable_gradient_checkpointing False \
  --enable_mlflow True

The provided launch script is designed for multi-GPU or multi-node training environments. It reads distributed settings from environment variables such as RESOURCE_NUM_GPU, DISTRIBUTED_NODE_COUNT, DISTRIBUTED_NODE_RANK, and DISTRIBUTED_MASTER_HOSTS:

bash scripts/train.sh

More details are available in docs/train.md.

🏗️ Pipeline

Citation

If you find our work useful, please consider citing:

@misc{liu2026optigeo,
      title={OptiGeo: Efficient Monocular Geometry for Embodied Perception in Optically Challenging Scenes}, 
      author={Muxin Liu and Tianbo Liu and Jing Xia and Xiaoyang Lyu and Xiaoshan Wu and Bo Wang and Peng Dai and Zhongrui Wang and Shaoshuai Shi and Xiaojuan Qi},
      year={2026},
      eprint={2608.29881},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2608.29881}, 
}

Please also consider citing our monocular foundation geometry model, FoundationGeo:

@misc{liu2026foundationgeo,
      title={FoundationGeo: Learning Spatial Pixel-Wise Fields for Monocular Metric Geometry}, 
      author={Muxin Liu and Xiaoyang Lyu and Tianhe Ren and Peng Dai and Xiaoshan Wu and Zhiyue Zhang and Jiaqi Zhang and Jiehong Lin and Shaoshuai Shi and Xiaojuan Qi},
      year={2026},
      eprint={2607.11588},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2607.11588}, 
}

Links

• Paper
• Project Page
• Code
• Hugging Face

📄 License

OptiGeo original code and documentation are released under the MIT License. Third-party components retain their original license terms; see the source-file headers for details.

🙏 Acknowledgments

We thank the MoGe series of works and DINOv3.