Summary
A single track with an invalid UTF-8 byte (NUL, \x00) in its artist tag aborts the entire library sync, not just that one track — even though _resolve_canonical_artist_id explicitly tries to catch and swallow per-track failures.
Root cause
backend/app/services/scanner.py:638-661:
async def _resolve_canonical_artist_id(
self, artist_tag: str | None, mbid: str | None
) -> Any | None:
"""Resolve a track's artist tag to ``Artist.id`` (Pass 1 dual-write).
Failures don't abort the scan — the track is inserted with a NULL
``canonical_artist_id`` and the next backfill run reconciles it.
"""
if not artist_tag or not artist_tag.strip():
return None
try:
artist = await resolve_canonical_artist(
self.db,
artist_tag,
musicbrainz_artist_id=mbid,
do_mb_lookup=settings.scanner_mb_artist_lookup,
create_if_missing=True,
)
except Exception as e:
logger.warning(
f"canonical artist resolve failed for {artist_tag!r}: {e}"
)
return None
return artist.id if artist else None
The try/except Exception catches the Python exception, but that's not enough with asyncpg: once Postgres rejects a statement server-side (here, CharacterNotInRepertoireError: invalid byte sequence for encoding "UTF8": 0x00 from an artist name literally containing \x00), the whole session-level transaction is marked aborted on the server. Catching the exception in Python doesn't clear that — it just suppresses the traceback for that call. Every subsequent statement on the same self.db session (i.e. every other track in the batch) then fails with:
asyncpg.exceptions.InFailedSQLTransactionError: current transaction is aborted, commands ignored until end of transaction block
which propagates up as Scan worker failed: ...InFailedSQLTransactionError... and the sync as a whole reports failure, even though only one file was actually bad.
Reproduction
- Tag any track's artist field with an embedded NUL byte (e.g.
Richie Havens\x00 — this is what a FLAC ripped/tagged by a buggy tool produced in our case).
- Add it to the scanned library and trigger a sync.
- Observe:
canonical artist resolve failed for 'Richie Havens\x00': ...CharacterNotInRepertoireError... followed immediately by a cascade of InFailedSQLTransactionError for unrelated tracks, and the overall sync fails.
Suggested fix
Wrap the resolve_canonical_artist call in a SAVEPOINT (SQLAlchemy async with self.db.begin_nested():) so a failure there only rolls back to the savepoint, leaving the outer transaction/session usable for the rest of the batch:
try:
async with self.db.begin_nested():
artist = await resolve_canonical_artist(...)
except Exception as e:
logger.warning(f"canonical artist resolve failed for {artist_tag!r}: {e}")
return None
Alternatively (or additionally), it'd be nice if the scanner sanitized/rejected NUL bytes in tag values before they ever reach a query, with a clear per-file warning surfaced in Settings → Library, rather than the whole sync silently dying partway through.
Environment
- Deployed via the
seethroughlab/familiar Docker image, Postgres backend (pgvector), library synced from a local FLAC/MP3 collection.
Summary
A single track with an invalid UTF-8 byte (NUL,
\x00) in its artist tag aborts the entire library sync, not just that one track — even though_resolve_canonical_artist_idexplicitly tries to catch and swallow per-track failures.Root cause
backend/app/services/scanner.py:638-661:The
try/except Exceptioncatches the Python exception, but that's not enough with asyncpg: once Postgres rejects a statement server-side (here,CharacterNotInRepertoireError: invalid byte sequence for encoding "UTF8": 0x00from an artist name literally containing\x00), the whole session-level transaction is marked aborted on the server. Catching the exception in Python doesn't clear that — it just suppresses the traceback for that call. Every subsequent statement on the sameself.dbsession (i.e. every other track in the batch) then fails with:which propagates up as
Scan worker failed: ...InFailedSQLTransactionError...and the sync as a whole reports failure, even though only one file was actually bad.Reproduction
Richie Havens\x00— this is what a FLAC ripped/tagged by a buggy tool produced in our case).canonical artist resolve failed for 'Richie Havens\x00': ...CharacterNotInRepertoireError...followed immediately by a cascade ofInFailedSQLTransactionErrorfor unrelated tracks, and the overall sync fails.Suggested fix
Wrap the
resolve_canonical_artistcall in aSAVEPOINT(SQLAlchemyasync with self.db.begin_nested():) so a failure there only rolls back to the savepoint, leaving the outer transaction/session usable for the rest of the batch:Alternatively (or additionally), it'd be nice if the scanner sanitized/rejected NUL bytes in tag values before they ever reach a query, with a clear per-file warning surfaced in Settings → Library, rather than the whole sync silently dying partway through.
Environment
seethroughlab/familiarDocker image, Postgres backend (pgvector), library synced from a local FLAC/MP3 collection.