diff --git a/docs/README.md b/docs/README.md index a743ffe..c0e9613 100644 --- a/docs/README.md +++ b/docs/README.md @@ -25,9 +25,6 @@ - [lang/language.md](lang/language.md): the v0 authoring language, the program form, its declarations and special forms, the compiled environment shape, and the symbol table format. -- [lang/macros.md](lang/macros.md): the macro system, defmacro with - qq and unquote, compile-time expansion, its limits, and its sharp - edges. - [lang/repl.md](lang/repl.md): the bitlisp REPL, its debugger, and the one-shot converter and compiler commands beside it. - [lang/curry.md](lang/curry.md): currying and program identity, diff --git a/docs/execution-plan.md b/docs/execution-plan.md index 94e2cdd..85903e4 100644 --- a/docs/execution-plan.md +++ b/docs/execution-plan.md @@ -79,13 +79,14 @@ descriptions, and commit messages* - [x] **Unit 3: REPL with stepper.** Landed 2026-08-15 (PR 49) as `bitlisp`, with `bitlisp-asm` and `bitlisp-disasm` beside it, surface stated in `docs/lang/repl.md`. Decisions by Evan, 2026-08-14: a standalone DebugMachine in the tools package, the consensus loop untouched, pinned differentially to `machine.run`; the command names; `def` is constants-only at the reader's unknown-symbol seam. The rendering seam and step-over predicate landed for unit 4 to consume. - [x] **Unit 4: language core.** Landed 2026-08-15 (PR 50) as the compiler, `bitlisp-compile`, and the REPL language surface, the language stated in `docs/lang/language.md`. Decisions by Evan, 2026-08-15: the module form is named `program`; `if` and `list` ship as compiler special forms; the symbol table keys on compiled-body tree hashes (refining this bullet's earlier "bytecode positions" wording); the REPL accepts source forms at the prompt under the raw-first rule, raw VM text never changing meaning. Constants inline with no optimizer, the function tree orders by declaration, one namespace covers declarations, bindings, condition constants, and reserved words. Compiled representatives pinned in `vectors/vm/compiled-programs.json`. - [x] **Unit 5: currying and program identity.** Added 2026-08-15 (decision by Evan, skeleton change under ground rule 6) and landed 2026-08-16 (PR 51): `curry` and `uncurry` as library, one-shot (`bitlisp-curry`, `bitlisp-uncurry`), and REPL surfaces, plus the tree-hash flag, the surfaces stated in `docs/lang/curry.md`. Decisions by Evan, 2026-08-15 and 2026-08-16: the standalone commands ship beside the REPL commands, the flag covers all three converters including `bitlisp-compile`, the identity story gets its own doc, and the surfaces are named `treehash` and `-T` rather than the `opc -H` spelling. The curried shape matches Chialisp tooling byte for byte, and uncurry is deliberately stricter than Chia's deployed uncurry, a divergence stated in the doc and pinned by a test. Discrete from unit 4 (no call-semantics contact) and from macros (operates on programs, not source). The unit 7 puzzles consume all three. The Phase 4 commitment-hash deferral stands. -- [x] **Unit 6: macros.** Landed 2026-08-16 (PR 52) as `defmacro` with `qq` and `unquote`, classic Chialisp semantics stated in `docs/lang/macros.md`. Decisions by Evan, 2026-08-16: macros only, includes split to their own unit below (skeleton change under ground rule 6). `if` and `list` stay compiler forms, so macros cannot shadow built-ins. Expansion is depth-capped and cost-budgeted, a recorded deviation from Chialisp's unbounded expansion. No `function` or `com` reflection form, the laziness expressiveness gap recorded in the language doc. No refactor rode along, the shared emission-primitives cleanup stays a candidate for a standalone PR. Expansion runs as a source pre-pass before reachability, macro bodies compile at declaration against earlier macros only, and read-back diverges from clvm_tools by one hop only (decision by Evan, 2026-08-16, steelmanned both ways, then narrowed twice as four review rounds showed every wider evidence scheme unsound on post-reader bytes): names the caller writes in a call's own arguments error when unresolved, REPL def spellings are barred resolution-side, and capture plus stale template spellings stay as Chialisp has them, documented sharp edges. The three new reserved words are a deliberate source and symbol-file compatibility break for earlier programs that used them as names, pinned by a loader test. Compiled representatives pinned in `vectors/vm/macro-programs.json`. Kept after the 2026-08-16 public-record review (decision by Evan): the fixed special forms and capped expansion avoid the macro-built-language failure mode on that record, and re-evaluation stays open at unit 7. -- [ ] **Unit 6b: includes.** Split from unit 6 (decision by Evan, 2026-08-16): an include mechanism needs its own recorded decision against the self-containment rule, a program compiling identically pasted into the REPL. -- [ ] **Unit 7: the four benchmark puzzles, written in the language, as the acceptance artifact:** payment pool, async offer, vault, singleton wrapper. This phase makes them exist and run, Phase 4 measures them. One PR per puzzle where size warrants. Style constraint (decision by Evan, 2026-08-16): the puzzles prefer functions over macros, macros only where they clearly improve legibility for a hostile reader, and the authoring experience feeds the typed v1 gate's macro ledger note. +- [x] **Unit 6: macros.** Landed 2026-08-16 (PR 52) as `defmacro` with `qq` and `unquote`, classic Chialisp semantics stated in `docs/lang/macros.md`. Decisions by Evan, 2026-08-16: macros only, includes split to their own unit below (skeleton change under ground rule 6). `if` and `list` stay compiler forms, so macros cannot shadow built-ins. Expansion is depth-capped and cost-budgeted, a recorded deviation from Chialisp's unbounded expansion. No `function` or `com` reflection form, the laziness expressiveness gap recorded in the language doc. No refactor rode along, the shared emission-primitives cleanup stays a candidate for a standalone PR. Expansion runs as a source pre-pass before reachability, macro bodies compile at declaration against earlier macros only, and read-back diverges from clvm_tools by one hop only (decision by Evan, 2026-08-16, steelmanned both ways, then narrowed twice as four review rounds showed every wider evidence scheme unsound on post-reader bytes): names the caller writes in a call's own arguments error when unresolved, REPL def spellings are barred resolution-side, and capture plus stale template spellings stay as Chialisp has them, documented sharp edges. The three new reserved words are a deliberate source and symbol-file compatibility break for earlier programs that used them as names, pinned by a loader test. Compiled representatives pinned in `vectors/vm/macro-programs.json`. Kept after the 2026-08-16 public-record review (decision by Evan): the fixed special forms and capped expansion avoid the macro-built-language failure mode on that record, and re-evaluation stays open at unit 7. Reversed by unit 6c the same day. +- [ ] **Unit 6b: includes.** Split from unit 6 (decision by Evan, 2026-08-16): an include mechanism needs its own recorded decision against the self-containment rule, a program compiling identically pasted into the REPL. Deferred behind unit 7 evidence (decision by Evan, 2026-08-16): the usage surveys behind unit 6c showed real include demand but reframed the design target, since chia-gaming abandoned textual include for importing a compiled sibling program plus its tree hash as a compile-time constant, the mechanism that keeps a multi-puzzle system's interlocking hashes consistent. `bitlisp-compile -T` covers that need manually in v0, and the puzzles decide whether duplication or hash plumbing justifies the mechanism. +- [x] **Unit 6c: the macro reversal.** Landed 2026-08-16 as the removal of `defmacro`, `qq`, and `unquote` and the addition of `assert`, `and`, and `or` as fixed compiler forms with classic utility_macros semantics, reversing unit 6 (decision by Evan, 2026-08-16, both sides steelmanned under ground rule 3). The evidence: three usage surveys (Chia's canonical 91-puzzle corpus, the corpora vendored in references/, and chia-gaming) found no novel macro in any deployed puzzle, short-circuit assert, and, and or the entire production vocabulary, and chia-gaming's production referee choosing built-in destructuring over the structural macro sitting unbuilt beside it. The reserved-word set change breaks compatibility in both directions, pinned by the loader test. Compiled representatives joined `vectors/vm/compiled-programs.json`, and `macro-programs.json` left with the feature. +- [ ] **Unit 7: the four benchmark puzzles, written in the language, as the acceptance artifact:** payment pool, async offer, vault, singleton wrapper. This phase makes them exist and run, Phase 4 measures them. One PR per puzzle where size warrants. Style constraint (decision by Evan, 2026-08-16, simplified by unit 6c): the puzzles are plain functions and the fixed forms, there being no macro system, and the authoring experience feeds the typed v1 gate's ledger note on whether unit 6c's cut ever binds. **Standing constraints:** -- **Hard scope boundary.** v0 is a small s-expression language with Chialisp-class ergonomics, BitLisp-native, no BLS vocabulary. Out of scope: static types, a Rue-class frontend, editor tooling, optimization beyond the obvious. The typed v1 question is a separate gate after Phase 5 review (ledger notes: 2026-08-11, types would let the REPL print by known type. 2026-08-16, whether macros carry into v1, decided on the unit 7 authoring experience). +- **Hard scope boundary.** v0 is a small s-expression language with Chialisp-class ergonomics, BitLisp-native, no BLS vocabulary. Out of scope: static types, a Rue-class frontend, editor tooling, optimization beyond the obvious. The typed v1 question is a separate gate after Phase 5 review (ledger notes: 2026-08-11, types would let the REPL print by known type. 2026-08-16, whether the unit 6c macro cut holds for v1, decided on the unit 7 authoring experience). - **Compiler correctness story.** The compiler sits outside spec-before-code, but compiler bugs become fund-loss bugs for users: every construct gets compile-and-run tests against the reference VM, and every compiled benchmark puzzle is pinned as a vector the same day. - **Fallback, pre-registered:** if v0 slips, Phase 4 proceeds with the stock Chialisp toolchain off-tree, constrained to the CLVM intersection. The stopgap never enters the tree. The fallback covers the language units only; the front end does not slip out of the phase. diff --git a/docs/glossary.md b/docs/glossary.md index eddf4c4..925f03d 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -56,7 +56,7 @@ A term enters this table in the same PR that introduces it. | REPL stepper | compare btcdeb, the Bitcoin script stepping debugger | the bllsh debugger, cldb (clvm_tools_rs) | the bitlisp command's stepping session over a pausable debug machine pinned differentially to the consensus evaluator, landed 2026-08-15 | | program form | nearest relative is a policy in miniscript, source that compiles to the committed script | mod (Chialisp) | the v0 authoring language's module form, (program params declarations body), compiling to one program tree (docs/lang/language.md), landed 2026-08-15 | | defun, defconstant | no script equivalent, script has no authoring language | defun, defconstant (Chialisp) | the two declaration forms, names adopted unchanged as generic Lisp vocabulary, defconstant literal-only (docs/lang/language.md), landed 2026-08-15 | -| defmacro, qq, unquote | no script equivalent, script has no authoring language | defmacro, qq, unquote (clvm_tools stage 2) | the macro declaration and its template forms, names adopted unchanged, expansion depth-capped and cost-budgeted where Chialisp is unbounded (docs/lang/macros.md), landed 2026-08-16 | +| assert, and, or | no script equivalent, IF/NOTIF are the nearest control flow | the assert, and, or macros (utility_macros.clib) | short-circuit fixed compiler forms where Chialisp uses library macros, the complete macro vocabulary observed in production, adopted 2026-08-16 with the macro system's removal (docs/lang/language.md deviations) | | symbol file (bitlisp-sym-v0) | compare debug symbols beside a stripped binary | main.sym (clvm_tools), .sym (clvm_tools_rs) | the compiler's JSON table mapping each compiled function body's tree hash to its source name and parameter names, consumed by the REPL stepper (docs/lang/language.md), landed 2026-08-15 | | tree hash | nearest relatives are the tagged leaf and branch hashes of BIP341's script tree | sha256tree, puzzle hash | the sha256tree digest over a program tree, one rule per atom and one per pair, the program's identity for commitment and for symbol-file keys (spec/VM.md, docs/lang/curry.md), glossed 2026-08-16 with the currying unit | | curry, uncurry | compare filling the placeholder keys of a script template before deriving its address | curry, uncurry (clvm_tools, chia_rs) | fixing values into a program to make a new committed program and reading them back out, the shape and the strict uncurry contract in docs/lang/curry.md, landed 2026-08-16 | diff --git a/docs/lang/language.md b/docs/lang/language.md index 47b29ac..89b53b3 100644 --- a/docs/lang/language.md +++ b/docs/lang/language.md @@ -11,7 +11,7 @@ name is exactly a token the raw reader rejects as an unknown symbol, so strings, hex, operator names, and decimals keep their raw spelling rules, and the language occupies only text that previously errored. The reserved words are `program`, `defun`, `defconstant`, -`defmacro`, `if`, `list`, `qq`, and `unquote`. +`if`, `list`, `assert`, `and`, and `or`. ## The program form @@ -21,9 +21,8 @@ errored. The reserved words are `program`, `defun`, `defconstant`, A source program is one self-contained form: a parameter tree, any number of declarations in any order, and exactly one body -expression. The declarations are `defun`, `defconstant`, and -`defmacro`, and nothing else may appear in declaration position. -Compiling a +expression. The declarations are `defun` and `defconstant`, and +nothing else may appear in declaration position. Compiling a program uses nothing outside the form, so a program that compiles in a file compiles identically pasted into the REPL. @@ -72,7 +71,7 @@ compiles to its quoted value at the use site. `(defconstant K (+ 1 2))` therefore binds the three-element tree whose head is the byte 0x10, not 3, exactly as `(q . (+ 1 2))` would read it. -A name is defined once: functions, constants, macros, condition +A name is defined once: functions, constants, condition constants, and reserved words share one namespace, and redefinition is an error. @@ -115,16 +114,53 @@ Builds a proper list of its evaluated arguments, folding into `c` calls: `(list A B)` compiles as `(c A (c B ()))`, and `(list)` is nil. Condition output is written with it. -## defmacro, qq, and unquote +## assert ``` -(defmacro ) +(assert * ) ``` -Declares a macro, a compile-time program whose calls expand into -source before anything else compiles, with `qq` and `unquote` as -its template forms. The system, its visibility rules, its limits, -and its sharp edges are `macros.md`. +Evaluates each condition in order and yields the final operand once +every condition holds. A falsy condition raises, failing the spend, +and nothing after it evaluates. Each level compiles to the same +apply-a-quoted-branch idiom as `if`, with the raise operator as the +untaken branch, so `(assert C V)` compiles as + +``` +(a (i C (q . V) (q 8)) 1) +``` + +With a single operand there is nothing to check and the value +compiles bare. `(assert)` is rejected at compile time: "assert +takes conditions and a final value". + +## and + +``` +(and *) +``` + +Yields 1 when every operand is truthy and nil at the first falsy +operand, whose successors never evaluate. The result is boolean, 1 +or nil, never an operand's value, exactly as Chialisp's `and` macro +behaves. `(and)` is 1. Each level compiles through the same idiom +as `assert`, the remaining chain as the taken branch and nil as +the fallback, so `(and A B)` compiles as + +``` +(a (i A (q 2 (i B (q 1 . 1) (q)) 1) (q)) 1) +``` + +## or + +``` +(or *) +``` + +Yields 1 at the first truthy operand, whose successors never +evaluate, and nil when every operand is falsy. `(or)` is nil. +`(and X)` and `(or X)` compile to the same tree, both meaning X as +a boolean. ## Expressions and quoting @@ -182,11 +218,8 @@ it, not at the declaration. A body error names its function, because in the REPL the offset indexes the declaring line's text, not the line that triggered the compile. -Macro expansion is the compiler's one source rewrite, finished -before reachability is computed and anything emits, which is why a -macro's expansion may call a function its source never names. -Emission itself is direct: what the rules above produce is what -serializes. +The compiler performs no source rewrites. Emission is direct: what +the rules above produce is what serializes. The worked example, `bitlisp-compile` output disassembled: @@ -239,32 +272,27 @@ nothing forces a change. The deliberate differences: environment tree, so no optimizer is needed to collapse them and the symbol table holds only function bodies. - The function tree is ordered by declaration, not alphabetically. -- Call arity is checked at compile time, macro calls included. +- Call arity is checked at compile time. - `defconstant` is literal-only. There is no compile-time-evaluated constant form. - Unknown bare operator atoms in call position are rejected at compile time rather than at run time. -- `if` and `list` are compiler forms, not macros, with the same - emitted semantics. Macros therefore cannot shadow them, or - anything else: where Chialisp's newest macro silently wins, the - one-namespace rule makes redefinition an error. -- Macro expansion is bounded, by the depth cap, the total - execution cap, and the execution budget `macros.md` states, - where Chialisp expands until the interpreter dies. A macro name - used as a value is an error in ordinary code, as a function - name is. -- Macro read-back catches caller-side typos. Chialisp reads an - unresolving output atom back as data, so a misspelled macro - argument silently compiles. BitLisp additionally lifts names - the caller wrote in the call's own arguments, so those fail as - unknown names, a deliberately one-hop guarantee: the call's - argument source is the only place provenance survives the VM - boundary. Capture and stale template spellings are kept as - Chialisp has them, documented sharp edges in `macros.md`. -- There is no `function` or `com` reflection form. Macro output - cannot carry names into quoted data, so a user macro cannot - introduce laziness, and lazy branching exists only through the - built-in `if`. +- `if`, `list`, `assert`, `and`, and `or` are compiler forms, not + macros, with the semantics Chialisp's stage-2 and utility_macros + macros give the same spellings. Nothing can shadow them: where + Chialisp's newest macro silently wins, the one-namespace rule + makes redefinition an error. +- There is no macro system. Chialisp's `defmacro`, `qq`, and + `unquote` are omitted, a removal decided on production evidence: + across Chia's deployed puzzle corpus and its largest application + codebases, the only macros in production use are short-circuit + `assert`, `and`, and `or`, which ship here as fixed forms. The + cut removes the expansion machinery's audit surface, and it + means user code cannot rewrite source: what is written is what + compiles. +- There is no `function` or `com` reflection form, and lazy + evaluation exists only through the built-in lazy forms, `if`, + `assert`, `and`, and `or`. - There is no include mechanism and no inline functions in v0. Currying is not a language form either: it operates on compiled programs, through the surfaces defined in `curry.md`. diff --git a/docs/lang/macros.md b/docs/lang/macros.md deleted file mode 100644 index a2d06aa..0000000 --- a/docs/lang/macros.md +++ /dev/null @@ -1,211 +0,0 @@ -# Macros - -The v0 language's macro system: `defmacro` declares a small program -that runs at compile time and writes source, and `qq` with -`unquote` is how such a body writes a code template around the -pieces it was handed. The semantics follow classic Chialisp, with -the deliberate differences recorded in `language.md` and the two -limits stated at the end of this page. - -## defmacro - -``` -(defmacro ) -``` - -A macro call looks exactly like a function call and is gone before -anything runs: the compiler replaces the call with whatever the -macro's body computes, and only that replacement compiles into the -program. - -The difference from a function is when the body runs and what it -receives. A function body runs at run time, on the values its -arguments evaluated to. A macro body runs at compile time, on the -argument source itself, unevaluated: a name arrives as its -spelling, an atom as itself, a form as the tree the reader built. -`(add 50 60)` under `(defmacro add (n1 n2) (+ n1 n2))` therefore -compiles to the constant `(q . 110)`. The addition happened inside -the compiler, and nothing of it remains at run time. - -Parameters bind positionally, exactly as function parameters do, -shapes and arity checks included. A bare-name tail binds the whole -remaining argument list, which is what a variadic macro walks. - -## What a macro body can see - -A macro body compiles at its declaration, as its own self-contained -program. It sees its parameters, the macros declared before it, the -operators, the condition constants, and the expression forms `if`, -`list`, and `qq`. It does not see the program's functions or -constants: those exist at run time, and a macro runs before run -time exists. A macro used inside another macro's body must -therefore be declared earlier. A call in a function body or the -main body may name a macro declared anywhere in the program, -because those bodies expand only when the whole program compiles. - -## qq and unquote - -``` -(qq