Skip to content

Fix batch assembly timeout, progress tracking, and error reporting - #19

Closed
jjroelofs wants to merge 1 commit into
feature/centralized-batch-processingfrom
jur/feature/centralized-batch-processing/#18-fix-batch-processing
Closed

Fix batch assembly timeout, progress tracking, and error reporting#19
jjroelofs wants to merge 1 commit into
feature/centralized-batch-processingfrom
jur/feature/centralized-batch-processing/#18-fix-batch-processing

Conversation

@jjroelofs

Copy link
Copy Markdown
Contributor

Summary

Fixes #18. Three bugs in the centralized batch processing infrastructure:

  • Assembly timeout: getFullyAnalyzedEntityIds() rendered every entity in the bundle via hasResults() just to build an exclusion list, causing 504s even with limit=1. Replaced with getAnalyzedEntityIds() DB queries that never load or render entities.
  • Infinite loop: $context['finished'] was calculated as a fraction of total entities across all operations, but each chunk is a separate batch operation. Drupal re-invoked the same operation endlessly. Now sets finished = 1 per chunk.
  • Silent failures: batchFinished() only reported processed count. Now reports failed count and individual error messages.

Changes

  • Add getAnalyzedEntityIds() to BatchableAnalyzerInterface with default [] in AnalyzePluginBase
  • Rewrite getFullyAnalyzedEntityIds() to intersect per-analyzer DB results
  • Fix $context['finished'] = 1 at end of each chunk operation
  • Report failures and errors prominently in batchFinished()

Test plan

  • Run batch with limit < total entities, verify batch completes and advances through all chunks
  • Run batch on a bundle with 1000+ entities, verify no 504 during form submission
  • Trigger an entity rendering error, verify it appears as a named error in the batch results
  • Run batch with force refresh off on already-analyzed content, verify entities are correctly skipped

Replace entity-loading approach in getFullyAnalyzedEntityIds() with
getAnalyzedEntityIds() DB queries to avoid rendering every entity
during batch assembly.

Fix $context['finished'] calculation that caused infinite loops when
entities exceeded chunk size -- each chunk is a complete batch
operation so finished must be set to 1.

Report failed entity count and individual errors in batchFinished()
instead of only showing the success count.

Closes #18
@jjroelofs

Copy link
Copy Markdown
Contributor Author

Splitting into separate PRs, one per fix.

@jjroelofs jjroelofs closed this May 11, 2026
@jjroelofs
jjroelofs deleted the jur/feature/centralized-batch-processing/#18-fix-batch-processing branch May 11, 2026 11:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant