Two related observations from running a real analysis on 0.5.0rc2:
The rebuilt engine does not consume the resources: block — every task gets a full-node dask slot regardless of declared cpus. The pre-rebuild Snakemake layer mapped threads/mem_mb to per-task dask constraints, and docs/cli/run.md still describes that behavior, so we spent a while confused about where parallelism is actually specified. In the end the only working home for a thread count is the recipe command string itself.
Which leads to the second observation: since the command string is hashed into definition_version, retuning parallelism in the recipe (--jobs 16 → 32) marks a scientifically identical artifact stale — in our case multi-day sweep outputs. It seems analogous to the env_version split in identity.py (recorded/reported but outside identity), though you will have a better sense of where the line should sit.
This was written by Claude on behalf of Cail.
Two related observations from running a real analysis on 0.5.0rc2:
The rebuilt engine does not consume the
resources:block — every task gets a full-node dask slot regardless of declared cpus. The pre-rebuild Snakemake layer mappedthreads/mem_mbto per-task dask constraints, anddocs/cli/run.mdstill describes that behavior, so we spent a while confused about where parallelism is actually specified. In the end the only working home for a thread count is the recipe command string itself.Which leads to the second observation: since the command string is hashed into
definition_version, retuning parallelism in the recipe (--jobs 16→32) marks a scientifically identical artifact stale — in our case multi-day sweep outputs. It seems analogous to theenv_versionsplit inidentity.py(recorded/reported but outside identity), though you will have a better sense of where the line should sit.This was written by Claude on behalf of Cail.