Report an uncomputed annotation score as null in both text exports

[htsugawa]

Problem

.mdpeak and .mdalign disagreed on how to represent an annotation score that was never computed. On the FastLC demo, a precursor-only suggestion (named no MS2: because MS2RawSpectrumID < 0) came out as:

.mdalign  Simple/Weighted/Reverse dot product = null
.mdpeak   Simple/Weighted/Reverse dot product = 0.000

Both wrote Matched peaks count = -1.00 and Matched peaks percentage = -1.00 on the same row, which is an explicit not-applicable marker. A consumer reading 0.000 as a value concludes that a spectral comparison was performed and returned zero similarity, which is materially stronger than "no comparison was possible".

Why they differed

Neither value was a decision. BaseAnalysisMetadataAccessor and BaseMetadataAccessor each declare their own ValueOrNull, and the two call sites differ by one null-conditional operator, so overload resolution picks a different helper:

	.mdpeak	.mdalign
Call	ValueOrNull(matchResult?.SimpleDotProduct, "F3")	ValueOrNull(matchResult.SimpleDotProduct, "F3")
Argument	float?	float
Overload	ValueOrNull(double?, string)	ValueOrNull(float, string)
Body	value?.ToString(f) ?? "null"	Math.Abs(v) > 1e-10 ? v.ToString(f) : "null"

Underneath, MsScanMatching.GetSimpleDotProduct and its siblings return -1 when there is nothing to compare — the same condition that makes GetMatchedPeaksScores return {-1, -1}. Before #589 that -1 reached both exports and agreed with the matched-peak columns. #589 made the dot products derived properties clamped with Math.Max(squared, 0f), which turned the sentinel into 0. From 5.5.250625 on, .mdpeak reported a score for a comparison that never happened.

.mdalign's eps rule is lossy the other way. Demo alignment ID 264, low score: NAGly 11:0, MS/MS assigned = True: Simple 0.007, Weighted 0.230, Reverse null, Matched peaks count null. A real comparison produced 0 and .mdalign discarded it. So "null means never computed" described neither export.

Change

Both exports now take the five shared score columns from one decision in MsdialCore/Export/AnnotationScoreFormat.cs: null when and only when the score was never computed; the value otherwise, including an exact 0.

MsScanMatchResult.IsSpectrumComparisonPerformed holds the result-side test — reads the sentinel from the raw Squared* fields where it survives, and excludes an Unknown result and a TextDB result (a text database holds no reference spectrum, so its annotator never scores one). DimsMspAnnotator scores through Ms2MatchCalculator, which collapses the sentinel to Ms2MatchResult.Empty before it reaches the match result, so the formatter additionally treats an entirely unset score block as uncomputed when the peak or spot carries no product-ion spectrum. That second test cannot discard a computed score.

The selector reads the score as double deliberately: .NET Framework formats a Single through a 7-significant-digit intermediate, so 0.59349996 snaps to 0.5935 and "F3" rounds it up to 0.594 instead of down to 0.593. Widening matches what .mdpeak already did and is the accurate rounding of the stored value.

docs/adr/0003-uncomputed-annotation-score-representation.md records the decision, the history, the divergences deliberately left in place, and the measured diff.

Release note

.mdpeak and .mdalign change content. A score that was never computed is now null in both files.

Outcome	.mdpeak dot products	.mdpeak matched peaks	.mdalign dot products	.mdalign matched peaks
reference match	value	value	value	value
low score, non-zero	value	value	value	value
low score, computed 0	0.000	0.00	null → 0.000	null → 0.00
precursor only	0.000 → null	-1.00 → null	null	-1.00 → null
text database	0.000 → null	0.00 → null	null	null
unknown	null	null	null	null

A parser that read -1 as the not-applicable marker must now also accept null. MS-DIAL Interactive is updated in systemsomicslab/msdial-interactive-app#11 to read either notation.

Validation

Console built Release/net48 from this branch at tests/MSDIAL5/MsdialCoreTestApp/bin/Release/net48/MSDIALCUI.exe, the path Interactive uses for a local source build. A baseline Console was built the same way from origin/master in a separate worktree, so every differing cell is attributable to this change. MsdialGuiApp also builds Release/net48 with -p:SkipLibraryDownload=true; without that switch MSBuild fails on MSB3923 because Zenodo answers 403 for the bundled .lbm2, which is not a compilation failure.

Determinism control. The baseline Console ran the demo twice and produced byte-identical .mdpeak and .mdalign; only the timestamp-derived MTD mzTab-ID line differed. The checked-in demo output used as the reference is itself byte-identical to that baseline, so the comparison has no confounds.

Run A — the demo's own method.txt, LBM annotation, 7 SCIEX WIFF files, 13,193 .mdpeak rows and 2,512 .mdalign rows:

Artefact	Changed cells
.mdpeak ×7	3,419 precursor-only rows × 5 score columns; dot products 0.000 → null, matched peaks -1.00 → null
.mdalign	1,021 precursor-only rows, matched peaks -1.00 → null; 68 Simple, 74 Weighted, 74 Reverse, 74 Matched peaks count, 71 Matched peaks percentage cells of low-score rows recovered as 0.000/0.00; one reference-matched row recovered a Weighted dot product
.mdmsp (per-file and alignment), .qa.tsv	byte-identical
.mzTab	only the timestamp-derived MTD mzTab-ID line

No cell outside the five score columns changed in any artefact.

The recovered reference-matched row is the clearest case for the decision:

alignment ID 803  RIKEN N-VS1 ID-2003 from Mouse_Plasma_ApoEKO_N_F1EPA
  MS/MS assigned True   MS/MS matched True   tag 430   235 spectrum peaks
  Simple 0.021   Reverse 0.866   Matched peaks count 1.00
  Weighted dot product   before null   after 0.000

Run B — same method plus lib\MSMS-Public_all-neg-VS19.msp and lib\20200121_MsdialTxtDB_Neg_EquiSPLASH_rapid.txt, to reach the text-database case with real data. Same shape, plus 2–6 text-database rows per .mdpeak (LPC 18:1(d7), SM 18:1;2O/18:1(d9), annotation tag 530) moving from 0.000/0.00 to null. Those rows do carry a product-ion spectrum, so only the TextDB test catches them.

Downstream ingest. Both Run A outputs were ingested into a scratch msdial_spectrum_catalog database:

annotation_kind	rows	before	after
precursor_only	4,440	dot products and matched peaks NULL	unchanged
low_score	824	68 NULL simple dot products, 74 NULL matched-peak counts	0 NULL; those become 0.0
msms_matched	2,246	unchanged	unchanged

The catalog already normalized precursor_only exact-0.0 dot products and negative matched-peak values to NULL, so that normalization becomes a no-op. The low_score figures are the measurements the eps rule was discarding.

Tests. New tests/MSDIAL5/MsdialCoreTests/Export/AnnotationScoreRepresentationTests.cs asserts the representation of all four annotation outcomes plus a text-database annotation in both formats, that the two formats agree cell for cell on every outcome, and that the sentinel is visible only in the squared fields.

Suites run, all passing: MsdialCoreTests 302, CommonStandardTests 833, MsdialLcMsApiTests 66, MsdialDimsCoreTests 33, MsdialImmsCoreTests 56, MsdialLcImMsApiTests 52, MsdialGcMsApiTests 6, MsdialCoreTestAppTests 9.

Deliberately out of scope

Recorded in the ADR, not changed here:

• GC-MS .mdscan writes the string -1 through its own NegativeIfNull helper — a third convention.
• GC-MS .mdalign overrides Fragment presence % as MatchedPeaksPercentage * 100, so the sentinel renders as -100.0. Its dot-product and Matched peaks count columns do come from BaseMetadataAccessor and so do follow this ADR.
• mzTab-M writes matchResult.SimpleDotProduct with no not-applicable handling, so id_confidence_measure reads 0 for a precursor-only row.
• ResultExport.cs formats the same properties as {0:0.00} — a fifth convention.
• A DI-MS row whose reference spectrum was empty while the peak did carry a product-ion spectrum still reports 0.000, because Ms2MatchCalculator discarded the sentinel. Fixing that changes DI-MS scoring input and needs its own validation.

Separate finding, filed as #786

MsReferenceScorer.CalculateScore guards the MS/MS term of the total score with result.WeightedDotProduct >= 0 && result.SimpleDotProduct >= 0 && result.ReverseDotProduct >= 0. Since #589 clamped those getters to 0 the guard can no longer fail, so the MS/MS term is now always added — including for a precursor-only candidate whose MatchedPeaksPercentage is -1. That is why Total score reads -0.143 for a no MS2: row instead of being computed from mass and RT similarity alone. This changes candidate ranking, not just formatting, so it is a scoring question and is not touched here. IsSpectrumComparisonPerformed gives that guard a correct expression when it is addressed.

🤖 Generated with Claude Code

[htsugawa]

CI note: the test job fails at build with

src/MSFINDER/MsfinderCommonStandard/MsfinderCommonStandard.csproj(19,4): error MSB3923:
Failed to download file "http://prime.psc.riken.jp/compms/code/InchikeyClassyfireDB-VS5.icd".
An existing connection was forcibly closed by the remote host.

That is the only hard error in the log; error CS count is 0 and the test step never ran. It is pre-existing and unrelated to this branch: the same MSB3923 on the same URL fails the Test and publish workflow on master itself (run 33589194115, 2026-09-02) and on feature/tiered-annotation-pipeline. The last green run of this workflow was 2026-08-29, before that RIKEN host became unreachable.

Local evidence for this branch instead, all passing:

• MsdialCoreTests 302 (including the 6 new representation tests), CommonStandardTests 833, MsdialLcMsApiTests 66, MsdialDimsCoreTests 33, MsdialImmsCoreTests 56, MsdialLcImMsApiTests 52, MsdialGcMsApiTests 6, MsdialCoreTestAppTests 9
• tests/MSDIAL5/MsdialCoreTestApp builds Release/net48
• src/MSDIAL5/MsdialGuiApp builds Release/net48 with -p:SkipLibraryDownload=true; without it the same class of Zenodo/RIKEN download failure appears locally too

The FastLC demo comparison in the description was produced with that Release/net48 Console.

[Copilot]

🟡 Changes recommended

The new unit tests assume dot-decimal formatting without forcing invariant culture, which can make the suite non-deterministic under non-English locales.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR standardizes how uncomputed annotation score columns are represented across the LC-MS text exports, ensuring .mdpeak and .mdalign both emit null only when a spectral comparison was never performed (and preserve a computed exact 0).

Changes:

• Centralized shared score-column formatting in AnnotationScoreFormat.Score() and updated both analysis and alignment exporters to use it.
• Added MsScanMatchResult.IsSpectrumComparisonPerformed to distinguish “never computed” from “computed as zero”, including a TextDB exclusion.
• Added unit tests and an ADR documenting the decision and historical context.

File summaries

File	Description
tests/MSDIAL5/MsdialCoreTests/Export/AnnotationScoreRepresentationTests.cs	Adds coverage asserting consistent null/0 behavior across .mdpeak and .mdalign.
src/MSDIAL5/MsdialCore/Export/IMetadataAccessor.cs	Switches .mdalign shared score columns to the centralized formatter.
src/MSDIAL5/MsdialCore/Export/IAnalysisMetadataAccessor.cs	Switches .mdpeak score columns (including enhanced/entropy) to the centralized formatter.
src/MSDIAL5/MsdialCore/Export/AnnotationScoreFormat.cs	Introduces the single formatting decision point for “not computed” vs “computed value”.
src/Common/CommonStandard/DataObj/Result/MsScanMatchResult.cs	Adds IsSpectrumComparisonPerformed predicate used by exporters/formatter.
docs/adr/0003-uncomputed-annotation-score-representation.md	Documents the rationale, decision, and known divergences left out of scope.

Review details

• Files reviewed: 6/6 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.