Skip to content

Unit 6b: include, computed defconstant, and defun-inline - #56

Merged
EvanWinget merged 7 commits into
mainfrom
unit-6b-includes
Aug 17, 2026
Merged

Unit 6b: include, computed defconstant, and defun-inline#56
EvanWinget merged 7 commits into
mainfrom
unit-6b-includes

Conversation

@EvanWinget

Copy link
Copy Markdown
Owner

What changed

The three unit 6b language surfaces, tooling only, nothing in spec/ or python/bitlisp/ touched:

  • include: (include "file") splices one declaration file, resolved through the repeatable -I search path of bitlisp-compile and the REPL, first match winning, no implicit current directory. One parenthesized declaration list per file, nested includes resolved, repeat includes skipped by resolved file, cycles a compile error naming the chain.
  • Computed defconstant: the value is now an expression, compiled against the declarations above it and run on the reference VM at declaration time under the default budget. A deliberate, pinned break with unit 4's verbatim-data semantics.
  • defun-inline: call-by-name inline functions. Arguments compile once and splice at each parameter reference, so an argument used twice evaluates twice and an unused argument never evaluates. Nothing enters the function tree or symbol table.

Authorizing decisions: docs/execution-plan.md unit 6b (scope expanded and resequenced 2026-08-16 on the two-corpus census of tibetswap and chia-gaming) and the approved unit plan of the same day. This unit is the recorded decision against the self-containment rule, now amended in docs/lang/language.md: compiling uses nothing outside the form and the include files it names, resolved through the same search path everywhere.

Every divergence from classic Chialisp was established by executing the frozen clvm_tools stage 2 reference, and each is stated in the language doc's deviations section: classic reads only an include file's first form, dies on double include (tibetswap's tibet_utils.clib hand-copies code around exactly that), recurses unboundedly on cycles, quotes defconstant values verbatim, and its defun-inline drops extra arguments, substitutes into quoted data and operator position, shadows built-ins, and hangs on self-recursion.

Read the commits in this order

  1. docs: state include, computed defconstant, and defun-inline — the language surface and the amended self-containment rule
  2. compiler+tools: the include declaration
  3. compiler: defconstant computes its value at compile time
  4. compiler+tools: defun-inline, call-by-name inline functions
  5. vectors: pin the unit 6b compiled representatives
  6. docs: flip unit 6b to landed, ignore generated symbol files

Verify independently

.venv/bin/pytest python/tests                      # 750 tests
.venv/bin/python tools/run_vectors.py              # 1006 cases
.venv/bin/python tools/diff_clvm.py --count 10000 --seed 20260816   # VM untouched
PATH=.venv/bin:$PATH ci/lint/lint.sh

# Paste identity: the include build and the pasted build agree byte for byte
mkdir -p /tmp/blib && printf '((defconstant FEE 400) (defun pay (AMT) (- AMT FEE)))' > /tmp/blib/pay.blib
.venv/bin/bitlisp-compile '(program (X) (include "pay.blib") (pay X))' -I /tmp/blib
.venv/bin/bitlisp-compile '(program (X) (defconstant FEE 400) (defun pay (AMT) (- AMT FEE)) (pay X))'

# The classic behaviors each divergence responds to, reproduced from the frozen reference
# (references/clvm_tools, tools/fetch-references.sh): double include dies, defconstant
# quotes verbatim, inline drops extra arguments. See the commit messages for the probes.

The three unit 6b surfaces enter the language reference in the
form-first style, the REPL doc gains the -I search-path flag and
the two new declaration rows, and the glossary gains rows for
include and defun-inline. The self-containment rule is amended as
the unit's recorded decision: compiling uses nothing outside the
form and the include files it names, resolved through the same
search path everywhere.

Each deliberate divergence from classic Chialisp is stated in the
deviations section and was verified by executing the frozen
clvm_tools stage 2 compiler: classic reads only an include file's
first form, errors on double include (the production workaround in
tibetswap's tibet_utils.clib is hand-copied code), recurses
unboundedly on cycles, quotes defconstant values verbatim, and its
defun-inline drops extra arguments, substitutes into quoted data
and operator position, shadows built-ins, and hangs on
self-recursion. clvm_tools stage 2 informed the design throughout.
Decisions by Evan, 2026-08-16, via the approved unit plan.
(include "file") splices one declaration file in place: exactly
one parenthesized list of declarations per file, nested includes
resolved depth first, a repeat include of the same resolved file
skipped, an include cycle a compile error naming the chain, and a
name collision across files the ordinary redefinition error of the
one namespace. Resolution walks the explicit -I search path only,
first match winning, with no implicit current directory, so where
a command runs never changes what a program compiles to.
bitlisp-compile and the REPL take the same repeatable -I flag, and
an (include ...) line at the prompt splices into the session
definitions through the same machinery.

clvm_tools stage 2 informed the design: the file shape (one list
of declarations) and explicit-path resolution match its include,
verified by executing the frozen reference. The recorded
divergences, stated in the language doc's deviations section:
trailing content after the list is rejected where classic ignores
everything past the first form, a repeat include is skipped where
classic dies on the resulting redefinition (tibetswap's
tibet_utils.clib hand-copies code to work around exactly that),
and a cycle errors where classic recurses without bound.

include joins the reserved words, a compatibility break in both
directions pinned by the loader test and the reserved-set pin.
Errors from inside an include file name the file, the same rule
that names a function around its body's errors.
The value is now an expression, compiled against the declarations
above it and run on the reference VM at the declaration, under
CONSTANT_COST_BUDGET, the runner's default spend budget. The
computed node stores and inlines as a quoted literal exactly as
before, so the emitted shape for atom constants is unchanged.
Evaluation failures, budget exhaustion included, are compile
errors naming the constant, the same wrapping a function body's
errors get. Declaration order matters for constants alone, which
keeps file and REPL behavior identical since the session is
already declaration ordered.

This deliberately breaks the language core's verbatim-data
semantics, the deviation recorded in the language doc:
(defconstant K (+ 1 2)) now binds 3 where it bound the tree, and
structured data needs an explicit quote. The pinned verbatim test
flips to pin the computed semantics, authorized by the unit 6b
scope decision (docs/execution-plan.md unit 6b, decision by Evan,
2026-08-16). Classic clvm_tools quotes the value verbatim, so this
matches the modern dialect's defconst under the classic keyword.

The REPL lists a computed pair constant quoted, so a defs line
re-reads as the same constant instead of re-evaluating.
An inline call compiles by splicing: each argument compiles once,
and the body compiles at the call site with parameter references
bound to the arguments' compiled expressions instead of paths.
Destructured names reach their components through f and r steps
over the argument expression, and a tail name binds the remaining
arguments as a built list, so the one parameter grammar covers
both declaration kinds. Bindings become node valued throughout
(_bind_params stores path atoms), which keeps _reference to one
rule. Nothing enters the function tree or the symbol table, an
inline call pays no apply and no path lookup, and reachability
walks inline bodies so a defun reached only through a splice still
compiles.

The call-by-name contract is Chialisp's inline semantics, kept
deliberately: an argument used twice evaluates twice, an unused
argument never evaluates, both pinned. The classic sharp edges are
closed as recorded divergences, each verified by executing the
frozen clvm_tools stage 2 reference: arity is checked where
classic silently drops extra arguments, quoted content never
rewrites where classic substitutes into (q . X) data, a parameter
in operator position stays an error where classic substitutes
there too, shadowing is impossible under the one-namespace rule
where a classic inline silently wins over a built-in operator, and
nested expansion is capped at INLINE_DEPTH_LIMIT where a classic
self-recursive inline hangs the compiler. The depth error wraps
once at the outermost frame rather than stacking prefixes.

defun-inline joins the reserved words and DECLARATION_KEYWORDS,
so the REPL declares, lists, and undefs inlines through the
existing seams. The compatibility break is pinned by the loader
test and the reserved-set pin.
Four cases join compiled-programs.json, replaying on the consensus
VM what the unit's tests claim: an include-built program matching
its pasted equivalent, a defconstant holding a sha256tree digest
computed at compile time, the inline laziness contract (the unused
argument's raise vanishes from the program), and its converse (an
argument used twice arms its raise). The corpus-replay sweeps pick
the cases up unchanged. Spec citation: VM.md sections 3 and 4, the
file's existing scope, since compiled output is ordinary program
trees and the language layer changes nothing the VM sees.
The unit bullet records the landed surfaces and the ratified
decisions: the amended self-containment rule, the include dedup
and cycle behavior, the computed defconstant break, and the
defun-inline narrowings. Symbol files are generated output
(bitlisp-compile --symbols, and the frozen clvm_tools reference
writes main.sym as a side effect of any probe), so *.sym joins
.gitignore.
Three Fable finder rounds ran over the branch diff, every finding
probe-verified before fixing. The fixes, by finding:

- Include names escaping the search path (major, two finders): an
  absolute name made os.path.join discard the -I directory, and a
  leading .. climbed out of it, both contradicting the documented
  resolves-only-through-the-path rule. Names must now be relative
  and stay inside their include directory, subdirectory components
  stay legal, and the language doc states all three rules.
- REPL include lines shared no load-once scope and applied
  partially on failure (major, two finders): the diamond the
  language doc blesses failed across two prompt lines, and a
  collision left a half-spliced session. The session now owns the
  load-once set, threaded into the splice, and a failed include
  line restores the session whole, so a fixed collision retries
  cleanly. Stated in repl.md and pinned by session tests plus a
  piped -I main() test.
- Inline expansion size was unbounded (major): a chain of doubling
  inlines squares its emitted tree per declaration while nesting
  only linearly, so a 227-character program hung serialization and
  even compile_program via the symbol-table hash, with the depth
  cap never firing. Expansion is now capped at INLINE_SIZE_LIMIT
  emitted nodes, measured by a DAG-memoized size walk, and the
  five-level chain compiles while the six-level chain is a compile
  error.
- Nested inline errors named the outermost frame while the offset
  indexed the innermost body (minor): the wrap now happens at the
  innermost frame, carried by a marked CompileError so outer
  frames pass it through, and the depth and size errors arrive
  pre-named.
- Include dedup keyed on realpath, which aliases on this
  platform's case-insensitive filesystem and misses hard links
  (minor): identity is now the file's stat pair, pinned by a
  hard-link test.
- Smaller fixes from the same rounds: a byte-order mark in an
  include file is rejected by name, an empty search path gets its
  own message, a dotted declaration list gets the file-shaped
  message, CONSTANT_COST_BUDGET is pinned equal to the runner
  default and its exhaustion path is tested, the defconstant doc
  states that the declared-above restriction extends to everything
  the value reaches, the REPL help table is re-aligned, and the
  defs reload test now actually reloads its printed lines.
@EvanWinget

Copy link
Copy Markdown
Owner Author

Adversarial review ran before requesting merge: three Fable finder agents over the branch diff (include machinery, compile-time semantics, surfaces/docs/tests/vectors), every claim probe-verified by execution. Ten findings survived verification, three major:

  1. Include names could escape the search path (absolute names and leading ..), contradicting the documented resolution rule.
  2. REPL include lines had no session load-once scope, so the diamond pattern the docs bless failed at the prompt, and a failed include left a half-spliced session.
  3. Inline expansion size was unbounded: a 227-character chain of doubling inlines squared its tree per declaration and hung the compiler with the depth cap never firing.

All ten are fixed in the final commit (review fold-ins: confine includes, atomic REPL splices, inline size cap), each with a pinning test. The finders also cleared a long list of suspicions, notably: every example in the language and REPL docs replays byte for byte, the four new vector cases rederive from their sources, the debugger rename path survives the node-valued bindings change, and the depth cap boundary is exact at 100.

After fold-ins: 764 tests, 1006 vector cases, lint clean, diff harness clean at seed 20260816 (10000 compared, 0 failures).

@EvanWinget
EvanWinget merged commit 42120c3 into main Aug 17, 2026
2 checks passed
@EvanWinget
EvanWinget deleted the unit-6b-includes branch August 17, 2026 02:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant