Add scButterfly method (Multiome GEX↔ATAC)

[benjaminfreyuu]

Summary

Adds scButterfly (Cao et al., Nat. Commun. 2024, doi:10.1038/s41467-024-47418-x) — a dual-VAE + adversarial translator for single-cell cross-modality prediction — as a train/predict method.

• Scope: Multiome only (GEX↔ATAC) via scButterfly's high-level Butterfly class. Direction is detected from uns["modality"], covering both swap (ATAC→GEX) and normal (GEX→ATAC) variants. CITE-seq/ADT is out of scope and raises a clear ValueError.

What's included

• src/methods/scbutterfly/ — wrapper + scbutterfly_train + scbutterfly_predict, plus shared helpers (butterfly_common.py, chrom_utils.py).
• Registered in src/workflows/run_benchmark/{main.nf,config.vsh.yaml}.
• scripts/create_datasets/test_resources.sh — Multiome-only fixture-generation step (mirrors novel/simple_mlp).

Design notes

• Custom base image. scButterfly 0.0.9 hard-pins a 2022 stack (scipy==1.9.3, scvi-tools==0.19.0, episcanpy==0.3.2) that won't build on the shared openproblems/base_* Python. The engine uses python:3.9 + torch cu117 wheels (CUDA bundled, so the gpu label still works), scButterfly --no-deps + the pinned stack, and openproblems from openproblems-bio/core (as the base images do).
• Real serialized model. Train writes scButterfly's state_dicts (<model>/model/*.pt) + metadata.pkl; predict reconstructs the architecture and loads them, so it generalises to unseen test_mod1.
• Predict needs the training data. scButterfly has no transform-only preprocessing API (data_preprocessing refits HVG/peak-filter/TF-IDF on its input), so predict rebuilds the paired object from the train data and re-runs the deterministic preprocessing before loading weights. input_train_mod1 is therefore wired into predict.
• Runtime shims (applied before importing scButterfly): translate the removed size_average loss kwarg; clamp BCE input to [0,1] (float32 sigmoid can drift >1.0); make .cuda() a no-op when no GPU is present.

Verification

Verified locally:

• viash ns test -s src/methods/scbutterfly — passes both components.
• Manual train→predict end-to-end on bmmc_multiome/swap and .../normal: output matches test_mod2 in shape and var order, no NaNs, correct method_id/dataset_id.
• Predict loads real saved weights and infers (no retraining).

Needs CI / infra validation:

• GPU execution (local runs used a CPU .cuda() shim; the native GPU path was not exercised).
• The nextflow run_wf wrapper and full run_benchmark run (blocked locally by nextflow/JDK version constraints + component build size). The wrapper follows the standard train/predict pattern.
• The method fixture must be synced to the S3 test-resource bucket (rerun test_resources.sh, which regenerates and syncs it).

🤖 Generated with Claude Code