Stop losing downstream scores to NA - #24
Merged
Merged
Conversation
* generate_cosine(): filter the Moran's I values rather than the objects holding them. `real[!is.na(real) & !is.na(sim)]` operates on a list, so it never dropped anything and one NaN took the whole metric with it -- 32 of 99 runs came out NA, while generate_mantel() passes na.rm and lost none. * calculate_precision(): report 0 when the simulation has no spatially variable genes at all. That is a bad simulation rather than a missing result, and reporting NA left both negative controls without a score on any dataset -- 8 of 30 expected control scores -- so nothing anchored the bottom of the scale. * calculate_recall(): left as NA on an empty denominator, and said why. There it is the real dataset that has no spatially variable genes, so there is nothing for any simulator to recover.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two ways the
downstreammetrics were reporting NA where they should have reported a number.generate_cosine()filtered the wrong thing.real[!is.na(real) & !is.na(sim)]operates on the list returned byBivariateMoransI(), not on the Moran's I values inside it --is.na()on a list of matrices isFALSEfor every element, so it never dropped anything. One NaN anywhere then took the whole metric with it:crosscor_cosinewas NA for 32 of 99 runs, whilegenerate_mantel()passesna.rm = TRUEon the same inputs and lost none.A simulation with no spatially variable genes now scores 0, not NA.
calculate_precision()returned NA whentp + fp == 0, which is exactly what a simulator producing nothing spatially variable does -- the worst possible outcome, recorded as a missing result. Both negative controls were NA on all 10 datasets, so QC reported 8 of 30 expected control scores and nothing anchored the bottom of the scale.calculate_recall()deliberately keeps NA in the same situation, and I added a comment saying why: there the empty denominator is a property of the real dataset (brain and hindlimbmuscle have no significant SVGs at all), so there is nothing for any simulator to recover, and scoring 0 would penalise every method for a property of the data.Checklist before requesting a review
I have performed a self-review of my code
Check the correct box. Does this PR contain:
Proposed changes are described in the CHANGELOG.md
CI Tests succeed and look good!