Classification - run condition text automatch as an async task, batched per import #1780

[davmlaw]

🤖 Written by Claude

Addresses #1780 — during a Shariant upload, condition text automatching ran synchronously in the publish signal and could call the external Monarch search API per record (60s timeout + retry), crawling the whole upload when Monarch was slow or down.

Design

Publishing no longer automatches at all — it just sets a flag, and a periodic sweep does the external work. There is no task dispatch, no signal-driven queuing, and no explicit locking:

• ConditionText.pending_automatch (new field + migration): sync_condition_text_classification sets it when a new root/gene level appears (the case that previously triggered the inline attempt_automatch) and does the quick count update. The flag is written in the same transaction as the publish, so it can't be lost, and it's the crash recovery — anything a dead worker leaves behind is picked up by the next sweep.
• condition_text_automatch_task — a celery beat sweep every 5 minutes (same shape as reclassification_events_update). While a ClassificationImportRun is ongoing it stands aside, so a whole Shariant sync accumulates flags and the first sweep after completion drains them as one batch, deduped to one automatch per distinct condition text. Each text is claimed with a single-statement UPDATE ... WHERE pending_automatch before its automatch, so overlapping sweeps can't process the same text twice, and the Monarch call itself runs outside any transaction with no rows locked.
• attempt_automatch itself is unchanged (it still does the full count recompute), so ConditionTextMatch.sync_all() from the management resync path behaves exactly as before.

The Shariant sync uploader passes import_id, so it gets the batched path: the API accepts and returns quickly, and Monarch is hit at most once per distinct condition text, after the import completes, on a db worker.

Testing

• New classification/tests/models/test_condition_text_automatch.py covers the sweep's claim-once-and-clear behaviour and that it defers while an import run is ongoing then drains afterwards.
• Full classification suite passes (141 tests).

🤖 Generated with Claude Code