ReactionFormer

ReactionFormer: A Controlled Benchmark for Infrared Spectral Reaction Monitoring

ReactionFormer is a benchmark and codebase for researchers interested in machine learning for chemical reaction monitoring, infrared spectroscopy, reaction-state prediction, and reaction-spectral representation learning.

The benchmark addresses the following question:

Can a model estimate how far a chemical reaction has progressed from an infrared spectral snapshot and the corresponding reaction context?

Infrared spectroscopy is widely used to characterize molecules and mixtures, and it is a natural candidate for reaction monitoring. As reactants are converted into products, the infrared spectrum of the reaction mixture changes accordingly. In practice, however, learning reaction progress from spectra is difficult: real reaction-monitoring data are scarce, heterogeneous, and affected by differences in instruments, protocols, solvents, concentrations, and reaction families.

ReactionFormer studies this problem in a controlled and reproducible setting. Given a reaction with two reactants and one product, the workflow generates simulated infrared mixture spectra at known product fractions and asks whether Transformer/BERT-style models can recover the reaction state from the spectral snapshot, either alone or together with molecular reaction context.

Importantly, ReactionFormer does not use experimentally measured reaction-monitoring trajectories. The pure-component spectra were simulated using Chemprop-IR, and the reaction snapshots were generated from pure-component infrared spectra through a transparent mixture-generation protocol. The terms computed-source and experiment-derived refer to the Chemprop-IR spectral resources used for the pure-component inputs; they do not refer to experimentally measured reaction trajectories.

What this repository contains

This repository contains the code used to build serialized reaction-spectral examples from reaction tables and pure-component infrared spectral databases, and to train Transformer/BERT-style models for:

• reaction-progress regression;
• practical-completion classification;
• masked-language-model pretraining;
• spectrum-only ablations;
• cross-source transfer;
• low-data adaptation.

The repository also includes utilities for generating optional noise-augmented variants for further experimentation.

ReactionFormer representation

Each reaction snapshot is serialized as a reaction-spectral sequence:

[CLS] [REAGENT1] R1 [SEP] [REAGENT2] R2 [SEP] [PRODUCT] P [SEP] [SPECTRUM] S [SEP]

where:

• R1 is the tokenized SMILES representation of the first reactant;
• R2 is the tokenized SMILES representation of the second reactant;
• P is the tokenized SMILES representation of the product;
• S is the discretized infrared mixture spectrum.

The spectral component is represented as a fixed-length sequence of discretized intensity tokens. In the benchmark setting described in the manuscript, spectra are interpolated onto a 400-point grid and discretized into integer-valued tokens.

Learning tasks

ReactionFormer supports three main learning objectives.

Reaction-progress regression

The model predicts the continuous product fraction:

y in [0, 1]

where y = 0 corresponds to a reactant-dominated mixture and y = 1 corresponds to full conversion to the product in the simulated mixture protocol.

Practical-completion classification

The model predicts whether a snapshot corresponds to practical completion:

label = 1 if y >= 0.90

label = 0 otherwise

The threshold is a pragmatic near-completion criterion used for the benchmark and can be changed through the data-preparation arguments.

Masked-language-model pretraining

BERT-style encoders can be pretrained on unlabeled reaction-spectral sequences using a masked-language-model objective. Masked molecular or spectral tokens are reconstructed from their surrounding reaction-spectral context.

Repository layout

reactionformer/
  src/reactionformer/
    prepare_data.py
    reaction_pred_training_pipeline.py
  scripts/
  docs/
  data/toy/
  environment.yml
  requirements.txt
  pyproject.toml
  CITATION.cff
  LICENSE
  README.md

Main components:

• src/reactionformer/prepare_data.py
  Dataset generation from reaction tables and infrared spectral databases.
• src/reactionformer/reaction_pred_training_pipeline.py
  Training and testing pipeline for the baseline Transformer, BERT-style MLM pretraining, supervised fine-tuning, and evaluation.
• scripts/
  Example commands with portable paths.
• docs/
  Command notes and documentation.
• data/toy/
  Small committed toy inputs for testing the pipeline.

Data

Large datasets, checkpoints, logs, and generated outputs are intentionally excluded from version control.

The only input data committed to git are the small toy files under:

data/toy/

Private or large inputs should be placed locally under:

data/

but outside:

data/toy/

Generated files may be written locally under:

outputs/
models/
logs/

These directories are not intended to be tracked by git.

Installation

Using conda:

conda env create -f environment.yml
conda activate reactionformer

Using pip:

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Required input files

The data-preparation script expects two main input files.

1. Reaction table

A CSV file containing reactions with two reactants and one product per accepted reaction.

The toy example is:

data/toy/reactions_table_toy.csv

2. Infrared spectral database

A CSV file containing one SMILES column and numeric spectral columns.

The toy example is:

data/toy/ir_spectra_toy.csv

The default SMILES column name is:

smiles

The code canonicalizes SMILES with RDKit, filters molecules by heavy-atom count, interpolates spectra onto a fixed grid, normalizes and discretizes spectra, and writes task-specific folders for:

binary/
regression/
mlm/

Dataset generation

Example using the committed toy input files:

python src/reactionformer/prepare_data.py \
  --data_path data/toy/reactions_table_toy.csv \
  --ir_db_path data/toy/ir_spectra_toy.csv \
  --output_path outputs/toy \
  --small_output_path outputs/toy_small_train \
  --ir_smiles_col smiles \
  --classification_threshold 0.90 \
  --max_heavy_atoms 19 \
  --n_tokens 400 \
  --seed 42 \
  --split_seed 3543 \
  --train_frac 0.8 \
  --val_frac 0.1 \
  --small_train_fraction 0.5 \
  --small_val_fraction 1.0 \
  --small_train_seed 123 \
  --small_val_seed 123

This command generates toy datasets for regression, binary classification, and MLM pretraining.

Noise variants

To generate noise variants:

python src/reactionformer/prepare_data.py \
  --data_path data/toy/reactions_table_toy.csv \
  --ir_db_path data/toy/ir_spectra_toy.csv \
  --output_path outputs/toy \
  --small_output_path outputs/toy_small_train \
  --noise_output_path outputs/toy_noise \
  --ir_smiles_col smiles \
  --classification_threshold 0.90 \
  --max_heavy_atoms 19 \
  --n_tokens 400 \
  --generate_noise_datasets \
  --noise_types noise_vertical,offset,multiplication

Supported noise variants depend on the implementation in prepare_data.py.

Train models

Baseline Transformer regression

python src/reactionformer/reaction_pred_training_pipeline.py baseline_supervised \
  --train_path outputs/toy/regression/regression-train.txt \
  --valid_path outputs/toy/regression/regression-val.txt \
  --tokenizer_dir models/toy/baseline/regression/tokenizer \
  --output_dir models/toy/baseline/regression/checkpoints \
  --task_type regression

BERT-style MLM pretraining

python src/reactionformer/reaction_pred_training_pipeline.py bert_mlm \
  --train_path outputs/toy/mlm/mlm-train.txt \
  --valid_path outputs/toy/mlm/mlm-val.txt \
  --tokenizer_dir models/toy/bert/mlm/tokenizer \
  --output_dir models/toy/bert/mlm/checkpoints

BERT-style supervised fine-tuning from pretrained MLM

python src/reactionformer/reaction_pred_training_pipeline.py bert_supervised \
  --train_path outputs/toy/regression/regression-train.txt \
  --valid_path outputs/toy/regression/regression-val.txt \
  --tokenizer_dir models/toy/bert/mlm/tokenizer \
  --output_dir models/toy/bert/regression_pretrained/checkpoints \
  --task_type regression \
  --init_mode pretrained \
  --pretrained_model_dir models/toy/bert/mlm/checkpoints

BERT-style supervised training from scratch

python src/reactionformer/reaction_pred_training_pipeline.py bert_supervised \
  --train_path outputs/toy/binary/binary-train.txt \
  --valid_path outputs/toy/binary/binary-val.txt \
  --tokenizer_dir models/toy/bert/binary_random/tokenizer \
  --output_dir models/toy/bert/binary_random/checkpoints \
  --task_type binary \
  --init_mode random \
  --build_tokenizer_from_train

Evaluate models

Baseline Transformer

python src/reactionformer/reaction_pred_training_pipeline.py baseline_test \
  --checkpoint_path models/toy/baseline/regression/checkpoints/best_model.pt \
  --test_path outputs/toy/regression/regression-test.txt \
  --output_json outputs/test_metrics_baseline_regression.json

BERT-style model

python src/reactionformer/reaction_pred_training_pipeline.py bert_test \
  --model_dir models/toy/bert/regression_pretrained/checkpoints \
  --test_path outputs/toy/regression/regression-test.txt \
  --task_type regression \
  --output_json outputs/test_metrics_bert_regression.json \
  --predictions_output outputs/predictions_bert_regression.csv \
  --embeddings_output outputs/embeddings_bert_regression.npz

GPU execution

For a single GPU:

export CUDA_VISIBLE_DEVICES=0

For long runs:

mkdir -p logs

nohup python -u src/reactionformer/reaction_pred_training_pipeline.py baseline_supervised \
  --train_path outputs/toy/regression/regression-train.txt \
  --valid_path outputs/toy/regression/regression-val.txt \
  --tokenizer_dir models/toy/baseline_regression/tokenizer \
  --output_dir models/toy/baseline_regression/checkpoints \
  --task_type regression \
  > logs/baseline_regression_train.out 2>&1 &

Output files

Depending on the selected command, the pipeline can generate:

• serialized training, validation, and test examples;
• tokenizer files;
• model checkpoints;
• evaluation metrics in JSON format;
• prediction files in CSV format;
• embedding files in NPZ format;
• logs for long-running experiments.

Scope and limitations

ReactionFormer is a controlled benchmark for reaction-spectral representation learning. It is not intended to reproduce all physical and instrumental effects present in real reaction-monitoring experiments.

In the current benchmark, reaction snapshots are simulated from pure-component spectra. This design makes reaction-progress labels known by construction and supports controlled evaluation of representation learning, cross-source transfer, and low-data adaptation. However, it does not explicitly model several effects that can occur in real process spectra, including:

• solvent contributions;
• baseline drift;
• peak shifts;
• scattering;
• detector saturation;
• intermediates;
• side products;
• impurities;
• nonlinear spectral interactions;
• temperature effects;
• instrument-specific acquisition artifacts.

The benchmark should therefore be interpreted as a reproducible test bed for studying reaction-spectral learning under transparent assumptions, not as a direct substitute for experimentally collected reaction-monitoring campaigns.

Reproducibility notes

The toy files under data/toy/ are intended to test that the pipeline runs end-to-end. They are not intended to reproduce the full benchmark results reported in the associated manuscript.

External spectral resources required for full-scale experiments are not distributed with this repository and must be obtained separately according to their respective licenses or access terms.

Citation

If you use this code, please cite the associated manuscript.

Citation information will be updated after the manuscript is public.

The repository also includes a CITATION.cff file that can be used by GitHub and citation managers.

License

This repository is released under the Apache-2.0 license. See the LICENSE file for details.

Contact

For questions, please contact:

Danilo Croce
Department of Enterprise Engineering
University of Rome Tor Vergata
Email: croce@info.uniroma2.it