feat(functions): implement cron scheduling with BullMQ by kkopanidis · Pull Request #1507 · ConduitPlatform/Conduit

kkopanidis · 2026-06-21T16:32:52Z

Problem

functionType: 'cron' was accepted by the admin API and UI but never ran on a schedule. The runtime fell through to the event-bus path and subscribed to a channel named after the cron expression, so scheduled functions silently never executed.

Summary

Add BullMQ functions-cron-queue with idempotent syncCronJobs() on every refreshRoutes()
Stable repeatable job IDs (cron-{functionId}); orphan cleanup on delete/type change
Canonical inputs.cronPattern with legacy inputs.event alias; admin validation on upload/patch
Cron invocations reuse the VM sandbox, FunctionExecutions, and existing metrics
Migration copies legacy inputs.event → inputs.cronPattern for existing cron docs

What kind of change does this PR introduce?

Does this PR introduce a breaking change?

Yes
No

The PR fulfills these requirements:

It's submitted to the main branch

If adding a new feature, the PR's description includes:

A convincing reason for adding this feature

Test plan

Create a cron function via Admin API (*/5 * * * *); confirm FunctionExecutions records appear within ~5 minutes
Update the cron pattern; confirm cadence changes without duplicate Redis repeatable keys
Delete the function; confirm executions stop after refresh
Verify functionType: 'event' still subscribes to the bus (no regression)
Set functions module active: false; confirm queue drains and ticks stop
Rolling deploy with two replicas: single execution per tick

Cron functions now register repeatable BullMQ jobs instead of erroneously subscribing to the event bus, with pattern validation and execution logging.

…sed ticks - Check if job with same pattern and timezone already exists before recreating - Track unchanged jobs separately from registered/updated - Only remove and recreate jobs when pattern or timezone changes - Prevents duplicate ticks from delete-recreate on every refresh

- Wrap each function processing in try-catch to prevent one bad function from aborting entire refresh - Log errors for individual functions and continue processing the rest - Ensures cron sync and route registration continue even if some functions fail

- Drop else-if branch that validated inputs.event as cron pattern - Only normalize cron inputs when functionType is actually 'cron' - Prevents PATCH to event functions with event: 'order.created' from throwing invalid cron error

- Wrap removeRepeatableByKey and queue.add operations in try-catch per job - Log errors and continue processing remaining jobs - Track errors counter separately - Prevents Redis failure on one cron job from aborting sync of all other jobs

Fixes @typescript-eslint/no-unused-vars CodeFactor warning

- Update package.bundle-lock.json to include bullmq and cron-parser dependencies - Required for bundle build verification to pass

- Remove getCronPatternFromInputs (unused after dropping else-if) - Remove validateCronPattern (unused after dropping else-if) - Add void statement for _grpcSdk in migrations per repo eslint pattern - Fixes CodeFactor @typescript-eslint/no-unused-vars warnings

- Check if service-bundle dist/cli.js exists before running pnpm build - Prevents pnpm install triggering prepare hooks that fail in workspace - In CI, dependencies are pre-built by turbo before build:bundle runs - Fixes 'Verify functions bundle' CI failure

- Create service-bundle.config.json to include bullmq and cron-parser in bundle manifest - Change cron-parser import from named to default import for CommonJS/ESM interop - Fixes 'Verify functions bundle' CI failure: packages are now in runtime deps and import works

…ifest/lockfile - Revert conditional prebuild:bundle check (commit 8c8403f) - Regenerate package.bundle.json with generate-manifest (deps reordered, uuid 14.0.2) - Regenerate package.bundle-lock.json with generate-lockfile to match - Bundle verification passes locally

Rethrow from executeBackgroundFunction so cron ticks fail the BullMQ job and event handlers do not log success after a sandbox error. Validate cron pattern before scheduling, lock multi-replica sync, and stop dual-writing the expression into inputs.event. Co-authored-by: Christina Papadogianni <ChrisPdgn@users.noreply.github.com>

cron-parser accepts unknown tz names; validate via Intl before parse so inputs.timezone is enforced rather than silently ignored.

Re-query schedulable cron functions inside the sync lock, size the worker lock past function timeouts, and drain Redis repeatables when the module is turned off. Keep inputs.event for operators and add GET /list/cron-jobs. Also await generateSecret so typecheck passes, and set pnpm allowBuilds for parcel watcher and scarf.

Flip a local disable flag before closing the worker so an in-flight refresh cannot add Redis jobs or restart the worker after the module is turned off.

…tivate - On active:false → active:true, call refreshRoutes to sync cron jobs immediately rather than waiting for an unrelated refresh - Add FunctionController.dispose() to unsubscribe from event bus - Call dispose on deactivate to prevent duplicate bus subscriptions - Reuse controller instance instead of creating new one on router reconnect Fixes two mark-ready blockers on PR #1507 Co-authored-by: Christina Papadogianni <ChrisPdgn@users.noreply.github.com>

ChrisPdgn · 2026-09-10T08:21:19Z

Review notes

Items from review that we are not changing, and why:

Job-id matching — Withdrawn. repeatable.id === cron-{functionId} is correct in this queue; matching is unchanged.
Remove-then-add — Residual only: pattern/tz change and queue.add fails after remove. Next successful sync reschedules. Not a merge blocker.
Await admin refresh — Intentional fire-and-forget. Operators inspect Redis via GET /list/cron-jobs.
Controller tests — Out of scope for this pass. Existing cron.utils tests stay (updated so event is kept).
event vs cronPattern — cronPattern is the schedule. We do not strip or dual-write event; runtime prefers cronPattern.
GitOps import — Pre-existing import path; not part of the cron scheduler fix.

Flatten refreshRoutes, drop redundant cron prep/active checks, and move unit tests into src/__tests__. Revert unrelated 2FA and pnpm allowBuilds drive-bys from the cron PR. Co-authored-by: Christina Papadogianni <ChrisPdgn@users.noreply.github.com>

TypeScript 6 infers Array<number | undefined>.reduce as possibly undefined; use an explicit loop so tsc --noEmit stays clean. Co-authored-by: Christina Papadogianni <ChrisPdgn@users.noreply.github.com>

Co-authored-by: Christina Papadogianni <ChrisPdgn@users.noreply.github.com>

CodeFactor flags @typescript-eslint/no-unused-vars. This ESLint config does not ignore underscore-prefixed args, so remove the unused parameter.

BullMQ 5 no longer surfaces repeat.jobId as repeatable id, so sync treated every job as an orphan (remove-all+re-add) and list/cron-jobs returned null ids. Use repeat.key = cron-{functionId} as the stable identity.

cursor Bot force-pushed the feat/functions-cron-scheduler branch from a5e106d to 04d1173 Compare September 1, 2026 12:47

github-code-quality Bot found potential problems Sep 1, 2026

View reviewed changes

Comment thread modules/functions/src/admin/index.ts Fixed

ChrisPdgn marked this pull request as draft September 2, 2026 09:29

kkopanidis and others added 13 commits September 2, 2026 09:50

feat(functions): implement cron scheduling with BullMQ

af227b3

Cron functions now register repeatable BullMQ jobs instead of erroneously subscribing to the event bus, with pattern validation and execution logging.

fix(functions): add bullmq and cron-parser to bundle dependencies

626b421

fix(functions): prefix unused grpcSdk parameter in migrations

eeeb2c3

Fixes @typescript-eslint/no-unused-vars CodeFactor warning

fix(functions): regenerate bundle lockfile with bullmq and cron-parser

ac274b8

- Update package.bundle-lock.json to include bullmq and cron-parser dependencies - Required for bundle build verification to pass

cursor Bot force-pushed the feat/functions-cron-scheduler branch from 8baf439 to 7cc4ec5 Compare September 2, 2026 09:52

ChrisPdgn and others added 3 commits September 2, 2026 09:55

fix(functions): reject invalid IANA timezones on cron validate

9fe0248

cron-parser accepts unknown tz names; validate via Intl before parse so inputs.timezone is enforced rather than silently ignored.

fix(functions): stop cron sync from rescheduling after drain

a7c2b5b

Flip a local disable flag before closing the worker so an in-flight refresh cannot add Redis jobs or restart the worker after the module is turned off.

ChrisPdgn force-pushed the feat/functions-cron-scheduler branch from 000bc08 to a7c2b5b Compare September 10, 2026 08:08

ChrisPdgn mentioned this pull request Sep 10, 2026

refactor(functions): deslop cron scheduler from #1507 #1599

Merged

11 tasks

cursoragent and others added 4 commits September 10, 2026 12:24

fix(functions): type lock-duration helper without reduce inference

6ec8220

TypeScript 6 infers Array<number | undefined>.reduce as possibly undefined; use an explicit loop so tsc --noEmit stays clean. Co-authored-by: Christina Papadogianni <ChrisPdgn@users.noreply.github.com>

style(functions): format cronQueue controller with prettier

c70ed1d

Co-authored-by: Christina Papadogianni <ChrisPdgn@users.noreply.github.com>

fix(functions): drop unused runMigrations parameter

345efbe

CodeFactor flags @typescript-eslint/no-unused-vars. This ESLint config does not ignore underscore-prefixed args, so remove the unused parameter.

fix(functions): identify BullMQ cron jobs via custom repeat.key

b81ed1d

BullMQ 5 no longer surfaces repeat.jobId as repeatable id, so sync treated every job as an orphan (remove-all+re-add) and list/cron-jobs returned null ids. Use repeat.key = cron-{functionId} as the stable identity.

github-code-quality Bot found potential problems Sep 11, 2026

View reviewed changes

Comment thread modules/functions/src/__tests__/cron.utils.test.ts Fixed

fix(functions): drop unused getRepeatableCronJobId test import

0415c30

ChrisPdgn mentioned this pull request Sep 11, 2026

feat(functions): cron timezone + stop dual-write event ConduitPlatform/Conduit-UI#330

Open

5 tasks

ChrisPdgn marked this pull request as ready for review September 11, 2026 15:10

ChrisPdgn self-requested a review September 11, 2026 15:11

ChrisPdgn approved these changes Sep 11, 2026

View reviewed changes

Merge branch 'main' into feat/functions-cron-scheduler

8f1be75

kkopanidis merged commit d9ed82e into main Sep 11, 2026
17 checks passed

kkopanidis deleted the feat/functions-cron-scheduler branch September 11, 2026 17:40

Navigation Menu

Uh oh!

feat(functions): implement cron scheduling with BullMQ - #1507

feat(functions): implement cron scheduling with BullMQ#1507
kkopanidis merged 23 commits into
mainfrom
feat/functions-cron-scheduler

kkopanidis commented Jun 21, 2026

Uh oh!

Uh oh!

ChrisPdgn commented Sep 10, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

kkopanidis commented Jun 21, 2026

Problem

Summary

Test plan

Uh oh!

Uh oh!

ChrisPdgn commented Sep 10, 2026

Review notes

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants