Skip to content

Give process_dataset the seed it was already using - #27

Merged
rcannood merged 2 commits into
mainfrom
fix/process-dataset-seed
Jul 29, 2026
Merged

Give process_dataset the seed it was already using#27
rcannood merged 2 commits into
mainfrom
fix/process-dataset-seed

Conversation

@rcannood

Copy link
Copy Markdown
Member

Describe your changes

process_dataset/script.R does this:

cat("Using seed ", par$seed, "\n", sep = "")
set.seed(par$seed)

but --seed is not among the component's arguments -- the Parameters group only declares --dataset_id and --swap. So par$seed is NULL, the log line prints Using seed, and set.seed(NULL) re-seeds from the clock. Both stochastic steps -- the cell-type-balanced downsample to 1000 test cells, and the downsample to 10000 ATAC peaks -- were therefore not reproducible, and there was no way to pin them.

comp_process_datasets.yaml does declare --seed with default: 1, and it's documented in the README, so the knob was advertised but wired to nothing. The workflow's fromState didn't forward it either.

Mirroring the sampling code from script.R:105-110:

seed = NULL (what the component did):
  run A: 5 8 28 41 45
  run B: 1 7 8 22 25
seed = 1 (what it does now):
  run A: 4 5 6 11 37
  run B: 4 5 6 11 37

Verified end to end by running the component twice on bmmc_multiome -- it now logs Using seed 1.

One note on why this hid for so long: neither random branch fires on the test resources. bmmc_multiome has 427 test cells (< 1000) and 1500 peaks (< 10000), so the outputs are byte-identical regardless of seed there. It only matters on the full datasets.

Part of a series of PRs coming out of a pre-run review of the benchmark.

Checklist before requesting a review

  • I have performed a self-review of my code

  • Check the correct box. Does this PR contain:

    • Breaking changes
    • New functionality
    • Major changes
    • Minor changes
    • Bug fixes
  • Proposed changes are described in the CHANGELOG.md

  • CI Tests succeed and look good!

rcannood added 2 commits July 28, 2026 16:01
* Declare --seed on the component, matching comp_process_datasets.yaml
* Pass seed through from the process_datasets workflow

par$seed was NULL, and set.seed(NULL) re-seeds from the clock, so the
test-cell and ATAC-peak subsampling could not be reproduced.
@rcannood
rcannood merged commit f6e5f9c into main Jul 29, 2026
1 check passed
@rcannood
rcannood deleted the fix/process-dataset-seed branch July 29, 2026 10:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant