Skip to content

Stop losing downstream scores to NA - #24

Merged
rcannood merged 3 commits into
mainfrom
fix/downstream-na-scores
Jul 28, 2026
Merged

Stop losing downstream scores to NA#24
rcannood merged 3 commits into
mainfrom
fix/downstream-na-scores

Conversation

@rcannood

Copy link
Copy Markdown
Member

Two ways the downstream metrics were reporting NA where they should have reported a number.

generate_cosine() filtered the wrong thing. real[!is.na(real) & !is.na(sim)] operates on the list returned by BivariateMoransI(), not on the Moran's I values inside it -- is.na() on a list of matrices is FALSE for every element, so it never dropped anything. One NaN anywhere then took the whole metric with it: crosscor_cosine was NA for 32 of 99 runs, while generate_mantel() passes na.rm = TRUE on the same inputs and lost none.

identical inputs  -> 1        (want 1)
one NaN in sim    -> 1        (was NaN before)
unrelated inputs  -> 0.8618
all NaN           -> NA       (want NA)

A simulation with no spatially variable genes now scores 0, not NA. calculate_precision() returned NA when tp + fp == 0, which is exactly what a simulator producing nothing spatially variable does -- the worst possible outcome, recorded as a missing result. Both negative controls were NA on all 10 datasets, so QC reported 8 of 30 expected control scores and nothing anchored the bottom of the scale.

calculate_recall() deliberately keeps NA in the same situation, and I added a comment saying why: there the empty denominator is a property of the real dataset (brain and hindlimbmuscle have no significant SVGs at all), so there is nothing for any simulator to recover, and scoring 0 would penalise every method for a property of the data.

Checklist before requesting a review

  • I have performed a self-review of my code

  • Check the correct box. Does this PR contain:

    • Breaking changes
    • New functionality
    • Major changes
    • Minor changes
    • Bug fixes
  • Proposed changes are described in the CHANGELOG.md

  • CI Tests succeed and look good!

rcannood added 3 commits July 28, 2026 14:19
* generate_cosine(): filter the Moran's I values rather than the objects
  holding them. `real[!is.na(real) & !is.na(sim)]` operates on a list, so it
  never dropped anything and one NaN took the whole metric with it -- 32 of 99
  runs came out NA, while generate_mantel() passes na.rm and lost none.
* calculate_precision(): report 0 when the simulation has no spatially
  variable genes at all. That is a bad simulation rather than a missing
  result, and reporting NA left both negative controls without a score on any
  dataset -- 8 of 30 expected control scores -- so nothing anchored the bottom
  of the scale.
* calculate_recall(): left as NA on an empty denominator, and said why. There
  it is the real dataset that has no spatially variable genes, so there is
  nothing for any simulator to recover.
@rcannood
rcannood merged commit e95f41a into main Jul 28, 2026
1 check passed
@rcannood
rcannood deleted the fix/downstream-na-scores branch July 28, 2026 13:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant