A single-stage radar-camera depth estimator for real-time autonomous systems
Constant Latency · Single-Scan LiDAR Supervision · No Auxiliary Annotations
JustDepth is a real-time, single-stage radar-camera depth estimation framework for autonomous systems.
The network takes an RGB image and an automotive radar scan as input and directly predicts a dense metric depth map. It does not require an intermediate sparse or quasi-dense depth product, a pretrained monocular depth model, multi-frame LiDAR accumulation, or auxiliary annotations such as semantic masks and bounding boxes.
JustDepth compresses all radar returns in a frame into a fixed-width 1D representation. This design keeps the computation of the radar branch independent of the number of raw radar points. Image and radar features are fused through a Height Fusion Block, and a lightweight Graph Neural Network propagates depth cues across the scene.
A training-only Confidence Decoder provides direct supervision to the fusion module by learning radar-supported pixels. This decoder is discarded during inference and therefore adds no test-time latency.
To reduce stripe artifacts caused by LiDAR Distribution Leakage, JustDepth uses point upsampling and synchronized rotation augmentation with reflection padding. The paper also introduces the Vertical-Horizontal Gradient Ratio, or VHGR, to quantify scanline artifacts across the predicted depth map.
| ⚡ Single Stage | Direct dense depth prediction without intermediate depth products |
| 🏎️ Real Time | 14.8 ms per frame with the 8-layer GNN model on an NVIDIA RTX 4070 Ti |
| 📡 Radar-Camera Fusion | Dense metric depth estimation from an RGB image and automotive radar |
| 📏 Constant Latency | Fixed-width 1D radar encoding with computation independent of radar point count |
| 📍 Single-Scan LiDAR Supervision | Training with one LiDAR sweep instead of accumulated multi-frame LiDAR |
| 🚫 No Auxiliary Annotations | No semantic masks, panoptic masks, or bounding boxes |
| 🧠 Training-Only Confidence Decoder | Improves radar-supported feature learning with zero test-time overhead |
| 🌐 Global Depth Propagation | Lightweight GNN propagation over image feature tokens |
| 🧹 LDL Mitigation | Point upsampling and synchronized rotation reduce scanline artifacts |
| 📊 VHGR Metric | Gradient-based evaluation of LiDAR Distribution Leakage artifacts |
JustDepth consists of six main components:
-
Image Encoder
A ResNet-style backbone extracts hierarchical image features from the RGB input. -
Fixed-Width Radar Encoder
Calibrated radar points are projected onto the image plane and compressed into a fixed-width 1D scan. The minimum range is retained for each image column. -
Height Fusion Block
Radar and image features are fused column by column through height-wise self-attention. -
Graph-Based Global Propagation
A lightweight GNN builds a feature-space K-NN graph and propagates depth information between related image locations. -
Depth Decoder
A U-Net-style decoder combines the globally propagated features with intermediate image features to produce a dense depth map. -
Training-Only Confidence Decoder
An auxiliary decoder predicts radar-supported pixels during training and is removed during inference.
All radar returns are encoded into a fixed-width 1D representation. The radar encoder therefore has effectively constant computation with respect to the number of radar returns.
The encoded radar features are fused with image features through height-wise self-attention. A lightweight GNN then propagates depth cues globally before the final dense depth prediction.
The Confidence Decoder directly supervises the fusion module to identify pixels supported by radar measurements.
Unlike methods that first generate an explicit intermediate radar depth map, the confidence branch is used only as an auxiliary training signal. It is completely discarded during inference and introduces zero test-time overhead.
Single-scan LiDAR supervision can cause the model to reproduce the horizontal scanline structure of the LiDAR sensor.
JustDepth mitigates these artifacts using:
- Point upsampling between compatible LiDAR samples
- Synchronized rotation of the RGB image, radar, and LiDAR
- Reflection padding for rotated RGB images
- Edge-aware smoothness regularization
The paper also introduces VHGR to measure the imbalance between vertical and horizontal depth gradients caused by stripe artifacts.
JustDepth produces complete dense depth maps across diverse daytime and nighttime nuScenes scenes while reducing the horizontal stripe artifacts commonly associated with sparse single-scan LiDAR supervision.
The 8-layer GNN configuration processes one frame in approximately 14.8 ms on an NVIDIA RTX 4070 Ti.
| Property | JustDepth |
|---|---|
| Input | RGB image and automotive radar |
| Output | Dense metric depth map |
| LiDAR supervision | Single scan |
| Auxiliary annotations | None |
| Intermediate depth product | None |
| Pretrained monocular depth module | None |
| Inference device | NVIDIA RTX 4070 Ti |
| Inference time | 14.8 ms |
| GNN layers | 8 |
| Radar sweeps | 1 |
| Image frames | 1 |
| nuScenes | ✅ Supported |
| ZJU-4DRadarCam | ✅ Supported |
Place the datasets under data/, or edit the corresponding paths in configs/*.txt.
JustDepth/
├── data/
│ ├── nuscenes/
│ │ └── samples/
│ ├── nuscenes_radar_5sweeps_infos_train.pkl
│ ├── nuscenes_radar_5sweeps_infos_test.pkl
│ └── zju/
│ ├── train.txt
│ ├── test.txt
│ ├── image/
│ ├── gt/
│ └── radar/
├── configs/
├── train.py
├── eval.py
└── save_confidence_map.py
- Python 3.11.13
- CUDA-compatible GPU
- PyTorch
# Create a clean environment
conda create -n justdepth python=3.11.13 -y
# Activate the environment
conda activate justdepth
# Install dependencies
pip install -r requirements.txtTraining uses binary confidence maps as targets for the training-only Confidence Decoder.
The maps can be precomputed before training to avoid generating them repeatedly inside the data loader.
python save_confidence_map.py \
--dataset nuscenes \
--nuscenes-path data/nuscenes_radar_5sweeps_infos_train.pkl \
--nuscenes-root data/nuscenes/samples \
--rule column \
--rid-outliers \
--link-lidar \
--output-dir confidence_map/nuscenes_train \
--workers 8For nuScenes, use the column confidence rule with LiDAR linking and outlier removal enabled.
python save_confidence_map.py \
--dataset zju \
--zju-path data/zju/train.txt \
--zju-root data/zju \
--rule dot \
--output-dir confidence_map/zju_train \
--workers 8CUDA_VISIBLE_DEVICES=<GPU_IDS> torchrun \
--nproc_per_node=<NUM_GPUS> \
train.py \
--config configs/nuscenes_train.txtExample:
CUDA_VISIBLE_DEVICES=0,1 torchrun \
--nproc_per_node=2 \
train.py \
--config configs/nuscenes_train.txtCUDA_VISIBLE_DEVICES=<GPU_ID> python train.py \
--config configs/nuscenes_train.txt \
--localExample:
CUDA_VISIBLE_DEVICES=0 python train.py \
--config configs/nuscenes_train.txt \
--localCUDA_VISIBLE_DEVICES=<GPU_IDS> torchrun \
--nproc_per_node=<NUM_GPUS> \
train.py \
--config configs/zju_train.txtExample:
CUDA_VISIBLE_DEVICES=0,1 torchrun \
--nproc_per_node=2 \
train.py \
--config configs/zju_train.txtCUDA_VISIBLE_DEVICES=<GPU_ID> python train.py \
--config configs/zju_train.txt \
--localExample:
CUDA_VISIBLE_DEVICES=0 python train.py \
--config configs/zju_train.txt \
--localpython eval.py \
--config configs/nuscenes_eval.txt \
--checkpoint <PATH_TO_CKPT>Example:
python eval.py \
--config configs/nuscenes_eval.txt \
--checkpoint train_log/models/latest.ckptpython eval.py \
--config configs/zju_eval.txt \
--checkpoint <PATH_TO_CKPT>Example:
python eval.py \
--config configs/zju_eval.txt \
--checkpoint train_log/models/latest.ckptIf you find this work useful, please cite:
@article{yun2026justdepth,
title={JustDepth: Real-Time Radar-Camera Depth Estimation With Single-Scan LiDAR Supervision},
author={Yun, Wooyung and Kim, Dongwook and Lee, Soomok},
journal={IEEE Robotics and Automation Letters},
year={2026},
volume={11},
number={3},
pages={2770--2777},
doi={10.1109/LRA.2026.3655274}
}