Skip to content

safe-bash awk: string concat is O(n²) with no per-byte step charge — 1-line accumulator loop rides the 30 s CPU limit #650

Description

@kamilio

Where: packages/safe-bash/src/commands/text-programs/awk-runtime.ts:296 — string concatenation is string(this.budget.check(this.asText(left) + this.asText(right))): a full string copy per concat, with budget.check bounding only the result size (≤ 32 MB maxBufferBytes) and no budget.step() proportional to bytes copied. The enclosing for loop charges 1 step per iteration regardless of how large the accumulated string is.

PoC (Node 22, default limits):

echo | awk '{ s=""; for(i=0;i<100000;i++) s = s "x"; print length(s) }'

Measured: 100k iters → 2.3–4.9 s; 200k → > 20 s; 1M → maxCpuMs at 30 s. Classic O(n²): iteration i copies i bytes ⇒ Σ = n²/2 byte-copies. The step budget (5M) would permit 5M iterations ≈ 12.5 TB of byte copying (hours) if the CPU clock didn't intervene; the only real bound is the 30 s wall/CPU deadline, and each concat of a multi-MB string is a synchronous stretch between checkpoints.

Impact: (d) — a one-line awk program pins a core for the full wall-clock window per exec, repeatable. Same under-priced-step class as v2 M4 (awk sprintf), #634 (ERE), and #646 (join -o), but on the single most common awk operation — string accumulation — which real scripts do constantly (building output in a variable). The in-repo model for the fix already exists: the yq encoder and the arrayJoin/rebuild paths charge budget.step(bytes).

Fix: charge budget.step(Math.max(1, Math.ceil(resultBytes / 64))) (or similar) in the concat branch so byte-work is priced; optionally intern/rope the accumulator to make repeated concat O(1) amortized. Either way the loop should hit a catchable ProgramError/step limit in well under a second, not ride the wall clock.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething is not working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions