Skip to content

feat: check ID uniqueness and ID references - #455

Merged
webern merged 1 commit into
mainfrom
m/mxdev-iduniq
Sep 18, 2026
Merged

webern merged 1 commit into
mainfrom
m/mxdev-iduniq

Conversation

@webern

@webern webern commented Sep 18, 2026 •

Copy link
Copy Markdown
Owner

Human Summary

Checking the global uniqueness of IDs is pretty awkward for mx, so we do it during parses and writes, but not during dom interactions.

In keeping with the design philosophy, we coerce duplicate IDs into unique ones if encountered, but at least now we have a mechanism for raising a diagnostic warning when we do.

Summary

MusicXML types an element's id two ways. An xs:ID has to be unique in the whole document; an
xs:IDREF has to match an xs:ID somewhere in the same document. mx::core models both as
Token, which keeps a value a legal name but cannot see the rest of the document, so neither rule
was checked anywhere.

Three things followed from that.

Token repair can manufacture a collision. It strips illegal characters, so two ids that differ in
the file can come out identical. The import diagnostic already warned that "the repaired ID may
duplicate another ID in the document" without anything ever confirming or denying it.

A dangling reference was imported in silence. ScoreConversions.cpp checks that a <part> resolves
to a <score-part>, but only on the timewise path, so a plain score-partwise read never ran it.
data/lysuite/ly41h_TooManyParts.xml declares one <score-part id="P1"> and then parts P1, P3
and P4. We imported it without a word.

getCoreDocument() hands out a mutable reference to the core DOM, and the header carried a standing
TODO about the ID loophole that opens. A check living in mx::impl would miss that caller entirely.

What this adds

One pass over the raw pugixml tree, in mx::core, run at both document seams. It is in core because
that is where the schema is modelled, so it is available to code using mx::core directly and it
covers the getCoreDocument() escape hatch, with nothing duplicated in mx::api.

Values are compared after Token repair, since that is what the parsed document holds. The two
directions differ on purpose:

  • Reading reports and changes nothing. Renaming on import would break the parse/serialize round
    trip, and the caller is owed fidelity to the file they handed us.
  • Writing renames a duplicate so we do not emit an invalid document. The first element to claim an
    id keeps it, later ones are renamed, and references are left pointing at the first, because
    nothing in the document says which was meant.

Both report at warning, since the result stays usable. Two new diagnostic codes, duplicateId and
danglingIdReference, because a caller may care about one and not the other. writeToFile and
writeToStream gain a Diagnostics & overload, which every other public entry point already had.

Classifying an attribute needs a table of which element and attribute pairs are xs:ID and which
are xs:IDREF. That table is hand-written in core, and IdSchemaDriftTest re-derives it from
docs/musicxml-4.0-ed15c23.xsd and fails if the two disagree. So there is drift protection without
touching gen/.

Checking that a reference names the right kind of element, a <part> naming a <score-part> rather
than any id at all, is stronger than XSD can express. The schema declares no xs:key or xs:keyref
and states the targets only in its documentation, so those twelve are supplied by hand, with a
comment for whoever extends them.

Not in scope

The xlink:href fragment on <link> that names a <bookmark>, which the schema types as
xs:anyURI. And the synthesized <score-instrument> id in PartWriter.cpp, which writes a
reference that does not match the definition it creates. That one is #445, and this pass will start
reporting it once #445 lands.

Testing

make fmt-check and make test-all on this branch:

Suite Result
core round-trip 2 assertions in 841 test cases
core unit 603 assertions in 69 test cases
api 5879 assertions in 657 test cases
api round-trip 414 passed, 0 failed of 414 pinned

src/private/mxtest/api/roundtrip-baseline.txt is unchanged, which is the point: reading reports
and does not touch the document, and nothing in data/ has a duplicate id for the write side to
rename.

New tests, 17 in all:

  • IdIntegrityTest, 12 cases over the core pass: a clean document, a duplicate, a duplicate across
    two different element kinds, a collision that only Token repair creates, a dangling reference, a
    reference naming the wrong kind of element, a reference naming the right one, that reading changes
    nothing, that writing renames and skips a name already used later in the document, that writing
    reports a dangling reference without changing it, and that writing still renames when no handler
    is listening.
  • IdSchemaDriftTest, 2 cases, re-deriving the table from docs/musicxml-4.0-ed15c23.xsd: 65
    xs:ID element names and 12 xs:IDREF element names, each element named once, and every
    reference carrying a target.
  • DiagnosticsTest, 3 cases at the api seam: a duplicate id and a dangling reference reported from
    fromStream, and a duplicate renamed by writeToStream.

Scanning all 1698 files under data/ for what the new pass would say: 0 duplicate ids, 0 references
to the wrong kind of element, and 14 dangling references in 13 files. Two of those are the
ly41h_TooManyParts.xml parts described above, one is musuite/testPartsSpecialCases.xml, and the
other eleven are single-element fixtures that carry a reference with no document around it. All
warn, none fail.

References

MusicXML types an element id as xs:ID, which has to be unique in the
document, or xs:IDREF, which has to match one. mx::core models both as
Token, which keeps a value a legal name but cannot see the rest of the
document, so neither rule was checked.

Adds one pass over the pugixml tree in mx::core, run when a document is
read and when it is written. Reading reports and changes nothing, so the
parse/serialize round trip still holds. Writing renames a duplicate,
leaving the id on the first element to claim it, so we do not emit an
invalid document. Both report at warning through the diagnostics channel,
with two new codes for a duplicate and for a reference that does not
resolve.

writeToFile and writeToStream gain a Diagnostics overload, which the
other public entry points already had. This also resolves the standing
TODO about the ID loophole that getCoreDocument opens.

The element and attribute table is hand written. IdSchemaDriftTest
re-derives it from the schema and fails if the two disagree, so gen/ is
untouched.

Closes #397
@webern webern added feature new feature request non-breaking fixes or implementation that do not require breaking changes api Affects the mx::api layer core Affects the mx::core layer ai Issues opened by, or through, a coding agent. labels Sep 18, 2026
@webern
webern merged commit 806222a into main Sep 18, 2026
8 checks passed
@webern
webern deleted the m/mxdev-iduniq branch September 18, 2026 21:33
webern added a commit that referenced this pull request Sep 19, 2026
## Human Summary

This appears to be a bug with uniqueness of synthetically created IDs
that was discovered thanks to the new diagnostic mechanism plus the scan
for duplicate IDs recently added.

## Summary

A part carrying MIDI playback data but no instrument id of its own was
written with two unrelated
ids: `score-instrument/@id` came from a counter that lived in the
process (`ID1000000`,
`ID1000001`, ...), while `midi-instrument/@id` was taken from the empty
api field and repaired to
`X`. The reference did not resolve, and because the counter outlived the
call, writing the same
score twice produced different files.

The writer now computes one id per part and uses it everywhere that
instrument is named. It is the
caller's `InstrumentData::uniqueId` when one was set, which is what
keeps parsed files unchanged,
and otherwise an id derived from the part's position in the score
(`ID1000000`, `ID1000001`, ...).
A caller who sets only `midiData` no longer has to invent an id, two
parts in one score cannot
collide, and the same `ScoreData` always writes the same file. The
counter is gone, and with it the
empty-id case that produced `X`.

This is the writer half of #445. #455 landed the identity machinery that
reports a duplicate or a
dangling reference, and it reports a dangling reference rather than
repairing it; with the writer
choosing the right id here, there is nothing left to report for an
instrument.

One consequence of the old behavior is worth stating, because it was
silent rather than merely
invalid: with two parts that both left the instrument id empty, both
`score-instrument`s and both
`midi-instrument`s were `X`, and #455's repair renamed only the second
`score-instrument`, leaving
the second `midi-instrument` on `X`. That part's playback attached to
the first part's instrument.
The fix removes the case.

## Testing

- [x] The new tests fail before the change and pass after:
`partInstrumentId_ScoreWriter`,
`partInstrumentIdIsTheSameOnEveryWrite_ScoreWriter`,
`midiInstrumentId_isTheSameOnEveryWrite`,
`midiInstrumentId_isReferencedByTheMidiInstrument`,
`midiInstrumentId_survivesWriteAndRead`
and `authoredMidiPartWritesNoIdDiagnostics_Diagnostics`. A seventh,
`aCallersInstrumentIdIsKept_ScoreWriter`,
      passes both ways and pins the unchanged caller-supplied path.
- [x] `make api-test` - all tests passed (5893 assertions in 663 test
cases)
- [x] `make test-all` - core round trip, core unit, api tests and api
round trip all pass
- [x] `make api-roundtrip` - 414 passed, 0 failed (of 414 pinned); no
pinned file changes
- [x] `make api-roundtrip-discover` - 414 PASS, 426 FAIL; no file newly
round-trips, so nothing was pinned
- [x] `make fmt-check` passed

## References

- Closes #445
- Related to #397 and #455
webern added a commit that referenced this pull request Sep 19, 2026
…ty (#460)

## Human Summary

The ability to specify the line level for pedal lines was missing. Added
here.

## Summary

`mx::api::PedalLineData` now carries a `SpannerNumber number`, and both
translation directions
honour MusicXML's `<pedal number="...">` attribute. The field is the
same identity type wedges,
curves and wavy lines already use: leave it unspecified for a lone pedal
line, give an explicit
level to write that level verbatim, or give the events of one line a
shared identity label and the
writer assigns a level from serialization order.

The reader takes the number off the element in
`DirectionReader::parsePedal`. The writer emits it
from `DirectionWriter::emitPedal` through `SpannerResolver`, which now
gives pedal lines their own
pool of numbers 1..16 like every other spanner family. A pedal line
opens with `start`, `sostenuto`
or `resume` and closes with `stop` or `discontinue`; `change` and
`continueLine` happen while the
line stays open.

Nothing changes for an ordinary score: a single pedal line with no
number writes no number
attribute, and every other spanner family keeps its own pool and
assignments.

Deliberately left out, as separate gaps: `pedal/@sign` and
`pedal/@abbreviated` (the sign form is
modelled through `MarkType::pedal` / `MarkType::damp`), the font and
color half of
`print-style-align` on `<pedal>`, and numbers on the pedal sign form.

## Testing

- [x] New reader test: `<pedal type="start" line="yes" number="2">`
reads as `PedalLineKind::start` with explicit level 2; a pedal with no
number reads as unspecified
- [x] New resolver/writer tests: two overlapping identity pedal lines
get distinct numbers and both survive a round trip; an explicit pedal
level is written verbatim and round-trips; a pedal number and a wedge
number come from separate pools
- [x] New round-trip tests: an explicit pedal number comes back; a pedal
line without a number writes no `number` attribute
- [x] Reverting just the reader/writer/resolver sites fails 5 of the new
test cases (10 assertions), so they are real regressions
- [x] `make api-test`: all pass (5909 assertions in 664 test cases)
- [x] `make api-roundtrip`: 414 passed, 0 failed (of 414 pinned)
- [x] `make api-roundtrip-discover`: 414 PASS, 426 FAIL, 0
LOADFAIL/GETDATAFAIL/CREATEFAIL -- no file unlocked, so no baseline pin.
`synthetic/pedal.3.1.xml` still fails because its pedal also carries
`abbreviated`, font and color attributes that `mx::api` does not model
- [x] `make test-all`, `make fmt-check`

## References

- Closes #411
- Related to #455
- Related to #393

---------

Signed-off-by: Matthew James Briggs <matthew.james.briggs@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai Issues opened by, or through, a coding agent. api Affects the mx::api layer core Affects the mx::core layer feature new feature request non-breaking fixes or implementation that do not require breaking changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ID uniqueness is not constrained by mx::core.

1 participant