SFMambaNet

Spectral-Frequency Enhanced Selective State Space Model for Correspondence Pruning

Spectral-frequency enhanced selective state-space modeling for robust two-view correspondence pruning.

Paper · Architecture · Results · Reproduction · Citation

Overview

Putative feature matches frequently contain many geometrically inconsistent correspondences. SFMambaNet processes an input set of normalized two-view correspondences and predicts an inlier probability for each match through iterative pruning. Its design studies frequency-domain cues at two complementary levels:

• Local spectral geometry. LSGA enriches relative neighborhood coordinates with spectral positional encoding and combines point-level and cluster-level context.
• Frequency-selective global refinement. SIGM applies learnable spectral gating to features propagated by forward and backward state-space scans.
• A compact accuracy-efficiency profile. The current manuscript reports 2.01M parameters, 5.01G FLOPs, and 73.83% mAP@5° on the unknown-scene YFCC100M/SIFT protocol without RANSAC.

This is the public project repository for the SFMambaNet manuscript.

Architecture

SFMambaNet maps N × 4 normalized correspondences to N × 1 inlier probabilities, then uses the retained evidence for parametric model estimation.

The manuscript's pruning pipeline couples a Local Spectral-Geometric Feature Extractor with a Spectral-Global Context Aggregator. A subsequent verification stage returns full-size correspondence predictions for geometric estimation.

Local Spectral-Geometric Attention (LSGA)

LSGA performs spectral expansion over local relative coordinates before graph interaction. Spectral-Geometric Encoding and Attention enhances fine-grained geometric cues, while the Spectral-Spatial Cluster Mamba combines point-level context with regional cluster consensus. Multi-scale interaction then returns the refined local representation to the original correspondence resolution.

Spectral-Integrated Global Mamba (SIGM)

SIGM operates on the cluster-ordered propagated feature sequence. It applies learnable frequency gates to the outputs of forward and backward state-space scans, providing frequency-selective refinement of long-range context without relying on quadratic self-attention. The gate is learned rather than used as a fixed handcrafted frequency filter.

Frequency-domain diagnostic from the manuscript

On the manuscript's unknown-scene YFCC100M diagnostic, the learned gate attenuates unstable high-frequency components while largely preserving the dominant low-frequency consensus observed for inliers. This is an empirical analysis under that protocol, not an assumption that every inlier or outlier must occupy a fixed frequency band.

Benchmark Highlights

Task	Dataset / descriptor	Protocol and metric	MatchMamba	SFMambaNet
Relative camera pose	YFCC100M / SIFT	Unknown scenes, mAP@5°, no RANSAC	67.60	73.83
Relative camera pose	YFCC100M / SuperPoint	Unknown scenes, mAP@5°	52.19	58.82
Outlier removal	YFCC100M	F-score (%) under the paper protocol	82.36	83.16
Homography estimation	HPatches / SIFT	ACC@3 px (%)	71.55	73.00

Efficiency

On the paper's YFCC100M unknown-scene SIFT setting, SFMambaNet reports 2.01M parameters, 5.01G FLOPs, and an average inference time of 4.80 ms. Runtime was measured in the manuscript's Ubuntu 22.04 / single NVIDIA RTX 3090 environment.

Qualitative Results

View outlier-removal examples

Examples compare the input matches with BCLNet, MatchMamba, and SFMambaNet. Green lines denote inliers and red lines denote outliers. The first four rows come from the unknown split of outdoor YFCC100M; the remaining rows come from the unknown split of indoor SUN3D.

Reproduction

Resource availability

Resource	Status	Notes
Public code snapshot	Available	Data preparation, training, and evaluation infrastructure are included, but the model is not fully synchronized with the revised manuscript.
Manuscript-aligned implementation	Not released	Exact reproduction requires a code release matching the revised LSGA, SIGM, and training schedule.
Legacy raw-data helper	Upstream unavailable	download_data.sh still points to the retired Altizure host; use the current OANet Google Drive instead.
Processed HDF5 data	Not hosted	Generate locally with the scripts under dump_match/.

1. Clone and create an environment

git clone https://github.com/Kirito14IT/SFMambaNet.git
cd SFMambaNet

conda create -n sfmambanet python=3.10 -y
conda activate sfmambanet

The reference versions documented by this project are PyTorch 2.1.1 with CUDA 11.8, OpenCV 4.7.0.72, and mamba-ssm==1.2.0.post1. The SIFT extraction code calls cv2.xfeatures2d, so use the OpenCV contrib build:

pip install torch==2.1.1 --index-url https://download.pytorch.org/whl/cu118
pip install opencv-contrib-python==4.7.0.72 mamba-ssm==1.2.0.post1
pip install h5py matplotlib numpy six tensorboard thop tqdm

Note

The repository does not currently include a dependency lock file. Install a CUDA-compatible mamba-ssm build for your PyTorch, CUDA toolkit, compiler, and operating system.

2. Prepare YFCC100M correspondences

The included download_data.sh uses an Altizure endpoint that is no longer available. Since April 2023, OANet has hosted its data and models in the official Google Drive folder. Download and extract the required YFCC100M archive manually at the repository root, then confirm this minimum layout:

raw_data/
├── yfcc100m/
└── pairs/

Generate SIFT correspondences from the extracted data:

mkdir -p data_dump
cd dump_match
python extract_feature.py --input_path ../raw_data/yfcc100m
python yfcc.py --raw_data_path ../raw_data/ --dump_dir ../data_dump/
cd ..

The expected outputs include:

data_dump/
├── yfcc-sift-2000-train.hdf5
├── yfcc-sift-2000-val.hdf5
└── yfcc-sift-2000-test.hdf5

Optional SUN3D preparation

Download and extract the SUN3D testing archive from the same OANet Google Drive folder, ensuring that raw_data/sun3d_test/ and raw_data/pairs/ exist. Then run:

cd dump_match
python extract_feature.py --input_path ../raw_data/sun3d_test
python sun3d.py --raw_data_path ../raw_data/ --dump_dir ../data_dump/
cd ..

The training section in dump_match/sun3d.py is disabled by default. Download the larger SUN3D training archive and enable that section only if you intend to train on SUN3D.

3. Train

Run the entry point from core/ so its local imports resolve correctly. Always override the machine-specific path defaults in core/config.py:

cd core
python main.py \
  --run_mode=train \
  --data_tr=../data_dump/yfcc-sift-2000-train.hdf5 \
  --data_va=../data_dump/yfcc-sift-2000-val.hdf5 \
  --log_base=../outputs/yfcc_sift \
  --gpu_id=0
cd ..

The manuscript reports training with 2,000 correspondences per pair, batch size 32, and 500,000 iterations. However, the current snapshot's hard-coded scheduler in core/train.py differs from the revised manuscript schedule. Treat this command as a snapshot workflow check rather than an exact paper-reproduction recipe.

4. Evaluate

No pretrained model is currently included. Supply a checkpoint directory containing model_best.pth:

cd core
python main.py \
  --run_mode=test \
  --data_te=../data_dump/yfcc-sift-2000-test.hdf5 \
  --model_path=/path/to/checkpoint_directory \
  --res_path=../outputs/yfcc_sift/test \
  --log_base=../outputs/yfcc_sift \
  --use_ransac=False \
  --gpu_id=0
cd ..

Useful command-line overrides

Option	Purpose
--data_tr, --data_va, --data_te	Training, validation, and test HDF5 paths.
--log_base	Training logs and checkpoint output directory.
--model_path	Directory containing model_best.pth for evaluation.
--res_path	Evaluation result directory.
--gpu_id	Value assigned to CUDA_VISIBLE_DEVICES.
--use_ransac	Enable or disable RANSAC during evaluation.
--train_iter, --train_batch_size	Training duration and batch size.

Citation

If SFMambaNet is useful in your research, please cite the manuscript:

@misc{wang2026sfmambanet,
  title         = {SFMambaNet: Spectral-Frequency Enhanced Selective State Space Model for Correspondence Pruning},
  author        = {Wang, Zhihua and Li, Yanping and Liu, Yizhang},
  year          = {2026},
  eprint        = {2606.04493},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2606.04493}
}

Acknowledgements

This implementation builds on code and evaluation conventions from OANet, CLNet, and MatchMamba. Please cite the corresponding works when using the inherited data-generation, testing, or evaluation components.

This repository does not currently declare a project-wide license. Inherited components remain subject to their respective upstream license and attribution terms; review those terms before redistribution.

---

_{SFMambaNet · Spectral-frequency modeling for two-view correspondence pruning.}