Give process_dataset the seed it was already using - #27
Merged
Conversation
* Declare --seed on the component, matching comp_process_datasets.yaml * Pass seed through from the process_datasets workflow par$seed was NULL, and set.seed(NULL) re-seeds from the clock, so the test-cell and ATAC-peak subsampling could not be reproduced.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe your changes
process_dataset/script.Rdoes this:but
--seedis not among the component's arguments -- theParametersgroup only declares--dataset_idand--swap. Sopar$seedisNULL, the log line printsUsing seed, andset.seed(NULL)re-seeds from the clock. Both stochastic steps -- the cell-type-balanced downsample to 1000 test cells, and the downsample to 10000 ATAC peaks -- were therefore not reproducible, and there was no way to pin them.comp_process_datasets.yamldoes declare--seedwithdefault: 1, and it's documented in the README, so the knob was advertised but wired to nothing. The workflow'sfromStatedidn't forward it either.Mirroring the sampling code from
script.R:105-110:Verified end to end by running the component twice on
bmmc_multiome-- it now logsUsing seed 1.One note on why this hid for so long: neither random branch fires on the test resources.
bmmc_multiomehas 427 test cells (< 1000) and 1500 peaks (< 10000), so the outputs are byte-identical regardless of seed there. It only matters on the full datasets.Part of a series of PRs coming out of a pre-run review of the benchmark.
Checklist before requesting a review
I have performed a self-review of my code
Check the correct box. Does this PR contain:
Proposed changes are described in the CHANGELOG.md
CI Tests succeed and look good!