Reuse the precomputed spatial clustering - #30
Open
rcannood wants to merge 2 commits into
Open
Conversation
`downstream` threw away the `spatial_cluster` that process_datasets/generate_sim_spatialcluster had just computed and ran BayesSpace again. BayesSpace is stochastic and unseeded, so the two runs disagree: on the positive control clustering_ari averages 0.55 with a per-dataset range of 0.14-0.92, while the best simulator averages 0.22. The metric was measuring its own noise more than the simulators. `reclassify_simsce()` was called but its result never used -- ARI and NMI do not care about label permutations -- so that call goes as well.
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
downstreamthrew away thespatial_clusterthatprocess_datasets/generate_sim_spatialclusterhad just computed and ranBayesSpace::spatialCluster()again. It is stochastic and unseeded, so the two runs disagree.That shows up on the positive control, whose "simulated" dataset is the real dataset and which should therefore score 1:
clustering_ariaverages 0.55, with a per-dataset spread of 0.14 to 0.92, while the best simulator averages 0.22. The metric was measuring its own noise more than it was measuring the simulators.It now reads the column that was already computed, and errors out with a clear message if it is missing. That also saves a BayesSpace MCMC run per method per dataset.
reclassify_simsce()was being called there too, but its result was never used -- ARI and NMI do not care about label permutations -- so that call goes as well. The function itself stays;generate_sim_spatialclusterstill uses it.Changes published numbers for
clustering_ariandclustering_nmi.Checklist before requesting a review
I have performed a self-review of my code
Check the correct box. Does this PR contain:
Proposed changes are described in the CHANGELOG.md
CI Tests succeed and look good!