Skip to content

Normalise the simulated data the same way as the real data - #18

Merged
rcannood merged 3 commits into
mainfrom
fix/metric-normalisation
Jul 28, 2026
Merged

Normalise the simulated data the same way as the real data#18
rcannood merged 3 commits into
mainfrom
fix/metric-normalisation

Conversation

@rcannood

@rcannood rcannood commented Jul 28, 2026

Copy link
Copy Markdown
Member

Describe your changes

downstream built the simulated logcounts with log1p(counts), but read the real dataset's stored logcounts layer for the other side of the comparison. generate_moransI() then compared the two, so the metric partly measured the difference in transform rather than the difference in the data.

The stored layer turns out to be exactly log2(counts / sizeFactor + 1), and sizeFactor is exactly the library size factor -- which is precisely what scater::logNormCounts() computes. Checked against the shipped test dataset:

stored logcounts vs candidate formulas (max abs diff):
  log1p(counts)                       2.5304
  log2(counts + 1)                    1.3949
  log2(counts/sizeFactor + 1)         0.0
  log2(counts/librarySizeFactor + 1)  0.0

So both sides now go through a shared compute_logcounts(). It reproduces the stored layer to 9e-16, meaning the real side does not move at all -- only the simulated side changes, from log1p to the same normalisation the real data already had.

On the positive control, whose "simulated" dataset is the real dataset and which should therefore score exactly 1:

positive control crosscor_cosine: 1   (ideal 1)
positive control crosscor_mantel: 1   (ideal 1)
before this PR, same inputs     : 0.9544862

Changes published numbers for crosscor_cosine and crosscor_mantel.

A correction to what I said earlier

An earlier version of this PR also routed ks_statistic_sc_features through the helper, and claimed the same problem applied there. It does not. That component was already computing scater::logNormCounts() for its simulated side, which is bit-identical to compute_logcounts():

old sc_features sim path vs compute_logcounts: max abs diff = 0

So I have dropped that hunk -- it was a no-op.

That leaves its positive-control deviations (ks_statistic_L_stats -1.246, _nn_correlation -1.463, _morans_I 0.345, all of which should be 0) unexplained by normalisation. The likelier cause is that it takes the real cell type proportions from the precomputed obsm[["celltype_proportions"]] but recomputes the simulated ones with CARD_processing(), so the two sides come from different code paths. I'd rather look into that properly than fold a guess into this PR.

Checklist before requesting a review

  • I have performed a self-review of my code

  • Check the correct box. Does this PR contain:

    • Breaking changes
    • New functionality
    • Major changes
    • Minor changes
    • Bug fixes
  • Proposed changes are described in the CHANGELOG.md

  • CI Tests succeed and look good!

@rcannood
rcannood force-pushed the fix/metric-normalisation branch from 76a5826 to 7471a5a Compare July 28, 2026 12:24
@rcannood rcannood changed the title Normalise real and simulated data the same way, and stop re-clustering Normalise real and simulated data the same way Jul 28, 2026
@rcannood
rcannood force-pushed the fix/metric-normalisation branch from 7471a5a to 8d12522 Compare July 28, 2026 12:31
rcannood added 2 commits July 28, 2026 14:31
`downstream` built the simulated `logcounts` with `log1p(counts)`, but read the
real dataset's stored `logcounts` layer for the other side of the comparison.
That layer is `log2(counts / sizeFactor + 1)` -- verified exactly against the
shipped dataset -- so the two sides differed by a whole transform and Moran's I
partly measured that rather than the data.

Both sides now go through a shared `compute_logcounts()`, which reproduces the
stored layer to 9e-16, so the real side is unchanged and only the simulated
side moves. On the positive control, where the simulated data is the real data,
crosscor_cosine and crosscor_mantel go from 0.95 to exactly 1.
@rcannood
rcannood force-pushed the fix/metric-normalisation branch from 8d12522 to db076bb Compare July 28, 2026 12:32
@rcannood rcannood changed the title Normalise real and simulated data the same way Normalise the simulated data the same way as the real data Jul 28, 2026
@rcannood
rcannood merged commit 5d5c9cb into main Jul 28, 2026
1 check passed
@rcannood
rcannood deleted the fix/metric-normalisation branch July 28, 2026 13:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant