Skip to content

Dedup dataset metadata by dataset_id - #48

Merged
rcannood merged 2 commits into
mainfrom
fix/meta-ch-dedup-by-dataset
Jul 29, 2026
Merged

Dedup dataset metadata by dataset_id#48
rcannood merged 2 commits into
mainfrom
fix/meta-ch-dedup-by-dataset

Conversation

@rcannood

Copy link
Copy Markdown
Member

Describe your changes

The metadata channel wants one entry per dataset, but expresses that as "keep the log_cp10k ones":

  meta_ch = dataset_ch
    // only keep one of the normalization methods
    | filter{ id, state ->
      state.rna_norm == "log_cp10k"
    }

That holds only while log_cp10k is among the normalizations we materialise. The day a dataset arrives under a different one -- or log_cp10k gets renamed -- the filter passes nothing, joinStates never fires, and output_dataset_info / output_method_configs / output_metric_configs / output_task_info are simply absent from the final state. Silently, since an empty channel isn't an error.

De-duplicating on dataset_id instead says the intended thing directly and doesn't care how many normalizations there are. Checked the Groovy under nextflow:

kept 2 of 3: [[dataset_id:a, x:1], [dataset_id:b, x:2]]

(three states, two datasets, normalization_id stripped as before.)

No behavioural change today, since log_cp10k is the only normalization in play.

Part of a series of PRs coming out of a pre-run review of the benchmark.

Checklist before requesting a review

  • I have performed a self-review of my code

  • Check the correct box. Does this PR contain:

    • Breaking changes
    • New functionality
    • Major changes
    • Minor changes
    • Bug fixes
  • Proposed changes are described in the CHANGELOG.md

  • CI Tests succeed and look good!

rcannood added 2 commits July 29, 2026 14:15
The metadata channel kept one entry per dataset by filtering on
rna_norm == "log_cp10k". That says "one per dataset" only as long as
log_cp10k is one of the normalizations we run; otherwise the channel is
empty and the output join has nothing to merge. Dedup on dataset_id,
which is what was meant.
@rcannood
rcannood merged commit e19b140 into main Jul 29, 2026
@rcannood
rcannood deleted the fix/meta-ch-dedup-by-dataset branch July 29, 2026 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant