Building Reproducible Molecular Datasets at Speed and Scale

Code, configurations, and per-run results for the publication.

MolForge

---

This repository accompanies the MolForge publication and extends its Materials and Methods with the exact configurations, code, and per-run results reported there. All reported statistics, experimental results, and figures are reproducible from the artifacts provided here. Datasets are constructed with MolForge release v1.0.0 from ChEMBL 36.

A single serializable configuration defines the ordered pipeline of actors, from ChEMBL retrieval through bioactivity and structure curation to conformer generation, and produces an analysis-ready dataset that regenerates exactly from the recorded configuration. The kinase study uses the six core actors followed by two plug-in actors for scaffold generation and distance-based splitting.

Configurations

Each experiment is defined by a MolForge ForgeParams object. Every configuration is provided as a YAML file under results/configs/, content-addressed by a hash of its parameters, so a run directory named CHEMBL####_ID<hash> is fully identified by its configuration.

experiment	endpoint / axis	configuration	hash
Benchmark harvest	IC50	benchmark_IC50.yaml	8FAF1A95
	Ki	benchmark_Ki.yaml	91C64F75
	EC50	benchmark_EC50.yaml	F3FBCB4F
	Kd	benchmark_Kd.yaml	A7E3C1EE
Curation ablation	canonical baseline	ablation_canonical.yaml	F8E7C608
	assay format — cell-based	ablation_assay_cell.yaml	DDA4CB81
	assay format — unspecified	ablation_assay_general.yaml	66456A2C
	stereochemistry — assign	ablation_stereo_assign.yaml	97E1E758
	stereochemistry — remove	ablation_stereo_remove.yaml	677FC924
	replicate aggregation — ≤ 1.0 log	ablation_aggregation_10.yaml	CB604491

The configurations are generated by scripts/harvest_kinomescan.py and scripts/ablation_harvest.py (build_params), and the hashes correspond to the configuration table in the Supporting Information. Filesystem paths in the YAML files were made relative for publication.

Kinome data landscape

Applying one configuration across the 381-target KINOMEscan panel over four activity endpoints yields the available-data landscape below: heavy-tailed dataset-size distributions, per-endpoint activity ranges, harvest runtimes, and pairwise Bemis–Murcko scaffold overlap. This large-scale, uniform harvest is the primary demonstration of MolForge.

Scaffold-isolation splitting

To evaluate under structural extrapolation, molecules are grouped into Bemis–Murcko scaffold clusters, and each cluster is scored by its size-weighted mean Tanimoto distance to all others. The most isolated clusters are assigned to the held-out sets.

$$\Large I_i = \frac{\sum_{j \neq i} n_j , d(\mathbf{V}_i, \mathbf{V}_j)}{N - n_i}$$ For cluster $i$, $n_i$ is its size, $\mathbf{V}_i$ its scaffold fingerprint, $d$ the Tanimoto distance, and $N$ the total number of molecules. Clusters are assigned in decreasing $I_i$ to test, then validation, then train.

Repository layout

path	contents
scripts/	pipeline and analysis entry points
models/	evaluation metrics and the three model wrappers
notebooks/figures.ipynb	figure generation
notebooks/support/	data loading, statistics, and figure code
results/analysis/	statistics manifest and derived tables
results/	per-run configurations, metrics, split reports, and predictions
data/	KINOMEscan panel and Manning kinase classification
jobs/	local and Slurm launch wrappers

Reproduction

The repository supports four levels of reproduction, from a single validation command to a full rebuild from ChEMBL.

1. Statistics manifest. results/analysis/stats_manifest.json holds every reported statistic, keyed by identifier. Rebuild it from the per-run results and validate against the pinned values:

conda env create -f <molforge>/environment.yaml   # MolForge base environment (github.com/molML/molforge)
conda activate molforge
pip install -r requirements.txt                    # analysis dependencies
python scripts/build_stats_manifest.py --check

2. Figures. notebooks/figures.ipynb reads the same results and writes each manuscript figure to figures/manuscript/.

3. Individual analyses. Each analysis is a standalone entry point, for example:

python scripts/aggregate_ablation.py --model-type {rf,xgb,cnn}   # curation-ablation Δ-metrics
python scripts/stereo_occlusion_analysis.py                      # stereochemistry sensitivity
python scripts/build_trainval_metrics.py --check                 # reference-set metrics

4. Full pipeline. The launch wrappers under jobs/ drive harvesting, splitting, and per-target training end to end:

bash jobs/run_local_prep.sh      # harvest and split on a local workstation
bash jobs/run_benchmark.sh       # per-target model training
bash jobs/run_ablation.sh        # curation-ablation variants

Datasets are defined entirely by their configurations (*_config.json, identified by a content hash of the parameters) and regenerate exactly against ChEMBL 36. Each completed training run writes a DONE marker, so runs resume without repeating finished work. Slurm accounts and paths in jobs/ are placeholders to be set for the target system.

Reference set

Each model is refit on train and validation combined before test evaluation; the training set therefore denotes that union throughout, for dataset descriptors, the Q² reference, and in-sample performance alike. The per-run artifacts record both conventions, and setting MOLFORGE_REFERENCE_SET=train selects the training-split variant.

Data availability

Per-compound predictions are provided as predictions.pkl.gz (one per run), and curated dataset tables carry the columns the analyses consume: structures, scaffolds, activities, and split assignment. The ChEMBL 36 source database, 3D conformers, and trained model weights are regenerable from the configurations above and are provided separately. The stereochemistry occlusion analysis requires the trained weights; every other result rebuilds from the artifacts in this repository.

Citation

@article{rossen2026molforge,
  title   = {MolForge: Building Reproducible Molecular Datasets at Speed and Scale},
  author  = {Rossen, Luke and Grisoni, Francesca},
  journal = {ChemRxiv},
  year    = {2026},
  doi     = {10.26434/chemrxiv.15006508/v1},
  url     = {https://doi.org/10.26434/chemrxiv.15006508/v1}
}

License

Released under the MIT License. See LICENSE.