Skip to content

fix(runtime): force lazy seqs at every eager consumer - #26

Open
BuddhiLW wants to merge 3 commits into
mainfrom
fix/lazy-seq-forcing
Open

fix(runtime): force lazy seqs at every eager consumer#26
BuddhiLW wants to merge 3 commits into
mainfrom
fix/lazy-seq-forcing

Conversation

@BuddhiLW

@BuddhiLW BuddhiLW commented Sep 5, 2026

Copy link
Copy Markdown
Owner

Closes the CLJEL-LAZY-FORCE card (20260905155515-51248d63).

The defect

clel-map and clel-filter return a lazy sequence, which is correct. Almost
nothing forced one, and Elisp primitives cannot: length measures the
four-element clel-lazy-seq struct, apply signals wrong-type-argument.

Measured against the shipped 0.7.1 runtime:

expression 0.7.1 correct
(reduce + (map inc c)) wrong-type-argument 9
(apply + (map inc c)) wrong-type-argument 9
(count (map inc c)) 4 3
(last (map inc c)) nil 4
(into [] (map inc c)) the struct (2 3 4)

The two silent answers are the dangerous half.

Why no test caught it

All 603 tests assert on emitted strings, and the emitted string was correct
the whole time. The defect lives one layer down, in what that string does when
Emacs runs it. There was no tier that executed the runtime, and the one ERT
suite that existed was behind a make target CI never invoked.

The fix

One coercion in the runtime's Promote layer, clel-realize, and a Boundary
stratum: every fn that hands a sequence to a raw Elisp primitive crosses
through it first. count, apply, second, butlast, reverse, flatten
and remove gained clel- wrappers instead of mapping onto Elisp directly.

Rather than fixing N mapping rows by hand, the rule is stated once as data:
mappings/lazy-seq-consuming-fns declares which fns read a possibly-lazy
sequence, and validate-tables! rejects any of them pointed at a raw
primitive. A future mapping edit cannot quietly reintroduce this.

Three more defects the execution tier surfaced

  • clel-rest returned an unforced thunk as its tail, which is truthy, so every
    while walking with clel-first/clel-rest ran one iteration past the end.
    That is why frequencies reported a phantom (nil . 1), every? answered
    nil for a sequence that satisfies its predicate, and reduce threw.
  • distinct, keep, dedupe and interpose dispatch on &optional coll,
    which cannot tell "no collection given" from "collection is empty", so
    recursing onto an empty tail returned a transducer mid-sequence.
  • distinct allocated a fresh seen table per recursion, so
    (distinct [1 2 1]) returned (1 2 1).

Evidence

  • 604 Clojure tests, 3047 assertions, 0 failures
  • 25 ERT tests across two suites, 0 unexpected
  • The new suite is not vacuous: 12 of its 13 tests fail against the 0.7.1
    runtime
    , and pass against this one
  • End-to-end through the real compiler: .cljel source using reduce/map,
    count/filter, join/map, last/sort-by, apply, into/distinct,
    remove and reverse compiled and executed in batch Emacs, all correct
  • Runtime byte-compiles with no new warnings

Compatibility

Emitted output for count, apply, second, butlast, reverse, flatten
and remove differs from 0.7.1. Behaviour only becomes more correct, and
remove is now lazy as in Clojure, but recompile rather than mixing 0.7.1
output against the 0.7.2 runtime.

clel-map and clel-filter return a lazy seq, but almost nothing forced
one. Elisp primitives cannot: `length` measures the four-element struct
and `apply` signals wrong-type-argument. So (reduce + (map inc c)) threw,
and (count (map inc c)) answered 4 for three elements.

The runtime now has one coercion, clel-realize, and every fn that hands a
sequence to a raw primitive crosses through it. count, apply, second,
butlast, reverse, flatten and remove gained clel- wrappers instead of
mapping onto Elisp directly; mappings/lazy-seq-consuming-fns declares the
rule and validate-tables! enforces it.

Also fixes three defects the execution-rung tests surfaced:
- clel-rest returned an unforced thunk as its tail, so every walking loop
  ran one iteration past the end (frequencies reported a phantom nil,
  every? answered nil, reduce threw)
- distinct/keep/dedupe/interpose dispatch on `&optional coll`, which
  cannot tell "no collection" from "empty collection", so recursing onto
  an empty tail returned a transducer mid-sequence
- distinct allocated a fresh seen table per step, keeping distant
  duplicates

None of this was visible to the 603 existing tests: they assert on
emitted strings, and the emitted string was right the whole time. Adds
test/elisp/clojure-elisp-runtime-test.el, which loads the runtime into
Emacs and calls it, and runs the elisp suites in CI. 12 of its 13 tests
fail against the 0.7.1 runtime.
The header is stamped from the VERSION resource at regen time, so a
regen that runs before the bump leaves the shipped runtime naming the
previous version.
sync-version writes the VERSION resource but cannot restamp the .el
header, which is only written by a regen. Bumping VERSION after a regen
therefore ships a runtime naming the previous release, silently. Part of
kanban 20260905161109-0e698b23.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant