Skip to content

Only mark a closure dynamic-extent once every reader has been checked - #34

Open
dg1sbg wants to merge 1 commit into
s-expressionists:mainfrom
dg1sbg:fix/dx-closure-extent-premature-mark
Open

Only mark a closure dynamic-extent once every reader has been checked#34
dg1sbg wants to merge 1 commit into
s-expressionists:mainfrom
dg1sbg:fix/dx-closure-extent-premature-mark

Conversation

@dg1sbg

@dg1sbg dg1sbg commented Aug 3, 2026

Copy link
Copy Markdown

DETERMINE-CLOSURE-EXTENT sets the enclose's extent to :dynamic from inside the loop over the variable's readers, so the mutation happens as soon as the first reader turns out to be a DX call. A later reader that escapes bails out with RETURN-FROM, which does not undo it, and the closure stays marked :dynamic.

The backend then builds it on the stack — under Clasp, cc_stack_enclose over an alloca in the enclosing frame — and it escapes anyway. Using it after that frame is gone is a use-after-return.

Why it is intermittent

Which reader is visited first decides whether this fires, and that order is unspecified. Set/set.lisp selects the hashset representation, because the guard on :listset is #+(or) and therefore always false, so DOSET walks an EQ hash table and the order follows object addresses.

Compiling

(lambda (x)
  (let ((g (lambda (y) (+ x y))))
    (mapcar g '(1 2 3))
    g))

300 times with Clasp's native compiler stack-allocated the returned closure 154 times. MAPCAR carries the :dx-call attribute, and the RETURNI is the escape. Calling one of the resulting closures gives a hard fault — SEGMENTATION-VIOLATION on one GC variant, EXT:BUS-ERROR on the other.

The fix

Move the assignment after the loop, where the conclusion it records has actually been established.

A variable with no readers now qualifies as well, which is correct — a value that is never read cannot escape. Previously the loop body simply never ran, so the case was missed rather than decided.

Verification

Through Clasp, since that is the client I have:

  • escaping closure: 154/300 stack-allocated before, 0/300 after
  • non-escaping closure: 300/300 still stack-allocated, so the optimisation is preserved
  • 100 escaped closures called after their frames were overwritten and a collection forced: all correct
  • full regression suite on both GC variants and the ANSI conformance suite: no unexpected failures

Origin

Introduced in 3884e93, which fixed the related but distinct problem in #12 — that a closure reachable through an escaping sibling could be stack-allocated. That fix added the reader walk; this one corrects when the walk commits its result.

DETERMINE-CLOSURE-EXTENT set the ENCLOSE's extent to :dynamic from inside
the loop over the variable's readers, so the mutation happened as soon as
the first reader turned out to be a DX call. A later reader that escapes
bails out with RETURN-FROM, which does not undo it, and the closure stays
marked :dynamic. The backend then builds it with cc_stack_enclose over an
alloca in the enclosing frame -- and it escapes anyway. Using it once that
frame is gone is a use-after-return; under Clasp it surfaces as
EXT:BUS-ERROR.

Which reader is seen first decides whether this fires, and that order is
unspecified: Set/set.lisp picks the hashset representation, because the
#+(or) guard on :listset is always false, so DOSET walks an EQ hash table
and the order follows object addresses. Compiling

  (lambda (x) (let ((g (lambda (y) (+ x y)))) (mapcar g '(1 2 3)) g))

300 times with Clasp's native compiler stack-allocated the returned
closure 154 times. MAPCAR carries the :dx-call attribute, and the RETURNI
is the escape.

Move the assignment after the loop, where the conclusion it records has
actually been established. A variable with no readers now qualifies too,
which is correct -- a value that is never read cannot escape; previously
the loop body simply never ran.

Introduced in 3884e93.
@dg1sbg

dg1sbg commented Aug 3, 2026

Copy link
Copy Markdown
Author

Standalone and independently valuable; #35 builds on it but this needs no part of that. Clasp-side consumer, for context: clasp-developers/clasp#1821.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant