Skip to content

Stop reserving 30 CPUs for a single-threaded lm - #42

Merged
rcannood merged 2 commits into
mainfrom
chore/lm-cpu-label
Jul 29, 2026
Merged

Stop reserving 30 CPUs for a single-threaded lm#42
rcannood merged 2 commits into
mainfrom
chore/lm-cpu-label

Conversation

@rcannood

Copy link
Copy Markdown
Member

Describe your changes

lm asks for highcpu (30 cores) but runs on one. The per-gene loop is

preds <- pbapply::pblapply(
  seq_len(ncol(X_mod2)),
  function(i) { ... }
)

with no cl argument, so it's serial. And the one thing that looks like it was meant to enable parallelism:

n_cores <- parallel::detectCores(all.tests = FALSE, logical = TRUE)

is computed at the top of the script and never referenced again.

Dropped the dead line and moved the label to lowcpu. Kept highmem/hightime -- the memory and runtime asks are real, it's only the CPU reservation that was fictional.

Happy to go the other way and pass cl = n_cores instead if we'd rather have the speedup; that's a bigger change though, since pblapply with a fork cluster would copy dr_train/X_mod2 per worker.

Part of a series of PRs coming out of a pre-run review of the benchmark.

Checklist before requesting a review

  • I have performed a self-review of my code

  • Check the correct box. Does this PR contain:

    • Breaking changes
    • New functionality
    • Major changes
    • Minor changes
    • Bug fixes
  • Proposed changes are described in the CHANGELOG.md

  • CI Tests succeed and look good!

rcannood added 2 commits July 29, 2026 12:05
n_cores was computed and never used, and pblapply() runs serially
without a cl argument, so highcpu was reserving 30 cores to use one.
@rcannood
rcannood merged commit f37fbcc into main Jul 29, 2026
@rcannood
rcannood deleted the chore/lm-cpu-label branch July 29, 2026 13:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant