Skip to content

fix(wintermute): suppress duplicate_table NOTICE flood from bulk staging tables - #239

Open
afbase wants to merge 1 commit into
mainfrom
fix/wintermute-suppress-notice-flood
Open

fix(wintermute): suppress duplicate_table NOTICE flood from bulk staging tables#239
afbase wants to merge 1 commit into
mainfrom
fix/wintermute-suppress-notice-flood

Conversation

@afbase

@afbase afbase commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

The bulk indexer creates its _bulk_* staging tables with CREATE TEMP TABLE IF NOT EXISTS on every batch. Pools recycle with RecyclingMethod::Fast, which never discards
temp tables, so they persist for the life of a connection and every batch after the first
raises a duplicate_table NOTICE that tokio-postgres logs at INFO. That stream dominates
the log output and shortens how far back the journal reaches, which makes incident
forensics harder.

This adds config::pg_connect_options() and sets it on every pool, so the server is told
client_min_messages=warning and never generates or transmits the notice. A client-side
log filter would still pay for generating and sending it.

The redundant DDL itself is left in place here and addressed separately, because removing
it requires every pool that reaches the bulk path to pre-create the staging tables.

Test plan

  • cargo clippy -p rsky-wintermute --all-targets --no-deps -- -D warnings
  • cargo test -p rsky-wintermute
  • cargo build --release -p rsky-wintermute
  • Indexer logs show no tokio_postgres NOTICE lines after restart
  • Indexing throughput and error rates unchanged
  • direct_index and car_loader still index a repo end to end

…ing tables

The bulk indexer creates its _bulk_* staging tables with CREATE TEMP TABLE IF
NOT EXISTS on every batch. Pools recycle with RecyclingMethod::Fast, which never
discards temp tables, so they survive for the life of a connection and every
batch after the first raises a duplicate_table NOTICE that tokio-postgres logs
at INFO. That stream dominates the log output and shortens how far back the
journal reaches.

Set client_min_messages=warning on every pool through a shared
config::pg_connect_options(), so the server never generates or transmits the
notice. A client-side log filter would still pay to generate and send it.
@afbase
afbase requested a review from rudyfraser August 24, 2026 04:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant