Make let rec build cyclic data, and compare it by bisimulation

LANGUAGE.md

@@ -211,6 +211,71 @@ that is the message you get rather than a crash.
 
 → `examples/recursion.hb`, `examples/recursion-anonymous.hb` (§9/§10)
 
+## Cyclic data
+
+`let rec` builds data that reaches itself, not just functions that call
+themselves. A `Table` literal bound with `rec` is created and named *before* its
+entries run, so an entry can mention the Table it is part of:
+
+```hashedbuild
+let rec p { .n = 1, .self = p };  p.self.self.self.n     // => 1
+let rec ones {1, ones};           ones[2][2][2][1]       // => 1
+```
+
+Friendship is mutual, so a social graph has to close — and mutual references
+between entries need nothing extra, because entries are evaluated **on demand
+rather than in source order**. Reaching `people.bob` while `.alice` is still
+being built simply evaluates `.bob` there and then:
+
+```hashedbuild
+let rec people {
+  .alice = { .name = "Alice", .friends = { people.bob, people.carol } },
+  .bob   = { .name = "Bob",   .friends = { people.alice } },
+  .carol = { .name = "Carol", .friends = { people.alice, people.bob } },
+};
+people.alice.friends[1].friends[1].name        // => "Alice", back where we started
+```
+
+That demand ordering also settles plain forward references, cycle or no cycle —
+`let rec p { .a = p.b + 1, .b = 2 }; p.a` is `3`. Entries still *print* in the
+order they were written, whatever order they ran in.
+
+**A cycle prints with a label**, so the back-edge is visible and the output is
+finite. The label marks the node the cycle returns to:
+
+```
+#1{n: 1, self: #1}
+#1{name: "Alice", friends: {{name: "Bob", friends: {#1}}}}
+```
+
+**Equality follows the cycle** rather than comparing pointers, so two rings of
+the same shape built by separate bindings are the same value:
+
+```hashedbuild
+let rec ring  { .x = { .tag = "x", .next = ring.y },  .y = { .tag = "y", .next = ring.x } };
+let rec other { .x = { .tag = "x", .next = other.y }, .y = { .tag = "y", .next = other.x } };
+ring.x == other.x        // => true
+```
+
+Two things this deliberately does **not** do. It builds cyclic structures —
+finite graphs with back-edges — and not unbounded ones: `let rec from (let n;
+{n, from (n + 1)})` has nothing to close a loop with and recurses to the depth
+limit, exactly as it did before. And an entry that needs another entry's value
+*while that one is still being computed* has no answer to give, so it fails
+rather than hanging:
+
+```hashedbuild
+let rec p { .a = p.b + 1, .b = p.a + 1 }; p.a
+// error: circular definition: p.a is needed before it has a value
+```
+
+Storing a reference to an entry still under construction is fine — that is what
+makes the back-edge. Looking *through* one is what fails. Only a `Table` literal
+written directly as the bound value gets any of this; `let rec x x + 1; x` still
+reports `undefined name`, as §10 says it should.
+
+→ `examples/cyclic-data.hb` (§6/§10)
+
 ## Branching and pattern matching
 
 Branching is built from ordinary composable operators rather than dedicated
@@ -411,7 +476,14 @@ file's executable bit, and two of the three targets cannot report one: WASI's
 So there is no way to compute the specified digest everywhere the interpreter
 runs. Building it means first deciding what a directory hashes as somewhere
 that cannot see an exec bit. `Function` is unbuilt because §15 needs it for `cached` but never
-says how a closure is encoded.
+says how a closure is encoded. A **cyclic value** (above) is the third case, and
+unbuilt for a related reason: the digest is a Merkle fold, a composite's hash
+built from its children's, and a cycle has no bottom to start from. `SPEC.md`
+§6 describes what the answer looks like — components hashed canonically, so the
+digest does not depend on where the walk entered the cycle — but §3 pins what a
+digest encodes, so it is a spec decision first. `sha256` of one fails cleanly
+meanwhile. Equality over cyclic values *is* built, and does not depend on any of
+this.
 
 Also absent: `true`/`false` literals, loops of any kind (recursion is the only
 repetition there is — see above), a `Bytes`-returning counterpart to

examples/cyclic-data.hb

@@ -0,0 +1,35 @@
+// Cyclic data (SPEC.md §10). Friendship is mutual, so a social graph has to
+// close: Alice's friends hold Bob, and Bob's hold Alice. `let rec` builds the
+// Table before its entries run, so an entry can name the Table it is part of;
+// entries are then evaluated on demand rather than in source order, so
+// reaching `people.bob` while `.alice` is still being built simply evaluates
+// `.bob` there and then. Only the reference that closes the loop - `.bob`
+// reaching back into `.alice`, which is still in progress - is left to be
+// filled in when `.alice` finishes.
+//
+// `.reordered` shows the same machinery with no cycle in it at all: `.a` is
+// written first but needs `.b`, and demand alone sorts that out.
+//
+// `.same_shape` compares two rings built by separate bindings that share no
+// entry between them. Equality is about content (§6), so walking one against
+// the other says they are the same value - which needs a walk that assumes a
+// pair equal on first meeting it, since a plain structural comparison would
+// follow the loop forever.
+//
+// Evaluates to { round_trip: "Alice", mutual: true, second_hop: "Carol",
+// reordered: 3, same_shape: true }.
+let rec people {
+  .alice = { .name = "Alice", .friends = { people.bob, people.carol } },
+  .bob   = { .name = "Bob",   .friends = { people.alice } },
+  .carol = { .name = "Carol", .friends = { people.alice, people.bob } },
+};
+let rec reordered { .a = reordered.b + 1, .b = 2 };
+let rec ring { .x = { .tag = "x", .next = ring.y }, .y = { .tag = "y", .next = ring.x } };
+let rec other { .x = { .tag = "x", .next = other.y }, .y = { .tag = "y", .next = other.x } };
+{
+  .round_trip = people.alice.friends[1].friends[1].name,
+  .mutual = people.alice.friends[1].friends[1] == people.alice,
+  .second_hop = people.alice.friends[2].name,
+  .reordered = reordered.a,
+  .same_shape = ring.x == other.x,
+}

src/eval.odin

@@ -102,6 +102,11 @@ Interpreter :: struct {
   // has_base_dir == false.
   base_dir_path: string,
 
+  // The `let rec` Table literals currently being built, innermost last
+  // (rec_build.odin). Empty for every program that doesn't write a cyclic
+  // `let rec`, which is what keeps table_access's check to a length test.
+  rec_builds: [dynamic]^Rec_Build,
+
   // Every `async` task this run has started (eval_async.odin). Shared with
   // each task's own Interpreter so that one run's tasks are all drained
   // together before it ends, however it ends.
@@ -403,7 +408,7 @@ eval :: proc(interp: ^Interpreter, node: Node_Idx, env: ^Env) -> (ret_val: Value
     if n.children_count > 0 {
       msg_val, ok := eval_slot(interp, interp.ast.extra_children[n.children_start], env)
       if ok {
-        msg_val, ok = await_value(interp, msg_val)
+        msg_val, ok = concrete_value(interp, msg_val)
       }
       if ok {
         if s, is_str := msg_val.(string); is_str do return fail(interp, s)
@@ -437,7 +442,7 @@ eval_unary :: proc(interp: ^Interpreter, node: Node_Idx, env: ^Env) -> (Value, b
   op := interp.ast.nodes[interp.ast.extra_children[n.children_start]].kind
   val, ok := eval(interp, interp.ast.extra_children[n.children_start + 1], env)
   if !ok do return nil, false
-  val, ok = await_value(interp, val)
+  val, ok = concrete_value(interp, val)
   if !ok do return nil, false
   #partial switch op {
   case .Op_Minus:
@@ -514,7 +519,7 @@ eval_binary :: proc(interp: ^Interpreter, node: Node_Idx, env: ^Env) -> (Value,
     defer pop(&interp.arg_stack)
     right_val, rok := eval(interp, right_idx, env)
     if !rok do return nil, false
-    right_val, rok = await_value(interp, right_val) // need its concrete type to know whether to call it
+    right_val, rok = concrete_value(interp, right_val) // need its concrete type to know whether to call it
     if !rok do return nil, false
     if fn, is_fn := right_val.(^Function_Value); is_fn {
       return call_function(interp, fn, left_val)
@@ -524,7 +529,7 @@ eval_binary :: proc(interp: ^Interpreter, node: Node_Idx, env: ^Env) -> (Value,
   case .Op_Call:
     fn_val, fok := eval_slot(interp, left_idx, env)
     if !fok do return nil, false
-    fn_val, fok = await_value(interp, fn_val) // need its concrete type before the is_fn check below
+    fn_val, fok = concrete_value(interp, fn_val) // need its concrete type before the is_fn check below
     if !fok do return nil, false
     arg_val, aok := eval_slot(interp, right_idx, env)
     if !aok do return nil, false
@@ -535,7 +540,7 @@ eval_binary :: proc(interp: ^Interpreter, node: Node_Idx, env: ^Env) -> (Value,
   case .Op_Dot:
     base, bok := eval(interp, left_idx, env)
     if !bok do return nil, false
-    base, bok = await_value(interp, base)
+    base, bok = concrete_value(interp, base)
     if !bok do return nil, false
     right_node := interp.ast.nodes[right_idx]
     key: Value = node_text(interp, right_idx)
@@ -547,9 +552,9 @@ eval_binary :: proc(interp: ^Interpreter, node: Node_Idx, env: ^Env) -> (Value,
     if !bok do return nil, false
     key, kok := eval(interp, right_idx, env)
     if !kok do return nil, false
-    base, bok = await_value(interp, base)
+    base, bok = concrete_value(interp, base)
     if !bok do return nil, false
-    key, kok = await_value(interp, key)
+    key, kok = concrete_value(interp, key)
     if !kok do return nil, false
     return table_access(interp, base, key)
 
@@ -564,9 +569,9 @@ eval_binary :: proc(interp: ^Interpreter, node: Node_Idx, env: ^Env) -> (Value,
       if !kok do return nil, false
       key = k
     }
-    base, bok = await_value(interp, base)
+    base, bok = concrete_value(interp, base)
     if !bok do return nil, false
-    awaited_key, kok := await_value(interp, key)
+    awaited_key, kok := concrete_value(interp, key)
     if !kok do return nil, false
     key = awaited_key
     t, is_table := base.(^Table_Value)
@@ -580,9 +585,9 @@ eval_binary :: proc(interp: ^Interpreter, node: Node_Idx, env: ^Env) -> (Value,
     if !lok do return nil, false
     r, rok := eval(interp, right_idx, env)
     if !rok do return nil, false
-    l, lok = await_value(interp, l)
+    l, lok = concrete_value(interp, l)
     if !lok do return nil, false
-    r, rok = await_value(interp, r)
+    r, rok = concrete_value(interp, r)
     if !rok do return nil, false
     if ls, lis := l.(string); lis {
       rs, ris := r.(string)
@@ -601,9 +606,9 @@ eval_binary :: proc(interp: ^Interpreter, node: Node_Idx, env: ^Env) -> (Value,
     if !lok do return nil, false
     r, rok := eval(interp, right_idx, env)
     if !rok do return nil, false
-    l, lok = await_value(interp, l)
+    l, lok = concrete_value(interp, l)
     if !lok do return nil, false
-    r, rok = await_value(interp, r)
+    r, rok = concrete_value(interp, r)
     if !rok do return nil, false
     if op == .Op_EqEq do return values_equal(l, r), true
     return compare_ordered(interp, op, l, r)
@@ -613,9 +618,9 @@ eval_binary :: proc(interp: ^Interpreter, node: Node_Idx, env: ^Env) -> (Value,
     if !lok do return nil, false
     r, rok := eval(interp, right_idx, env)
     if !rok do return nil, false
-    l, lok = await_value(interp, l)
+    l, lok = concrete_value(interp, l)
     if !lok do return nil, false
-    r, rok = await_value(interp, r)
+    r, rok = concrete_value(interp, r)
     if !rok do return nil, false
     return arithmetic(interp, op, l, r)
   }
@@ -713,6 +718,14 @@ table_concat :: proc(a: ^Table_Value, b: ^Table_Value) -> ^Table_Value {
 table_access :: proc(interp: ^Interpreter, base: Value, key: Value) -> (Value, bool) {
   t, is_table := base.(^Table_Value)
   if !is_table do return fail(interp, "cannot index a non-Table value")
+  // A Table still being built by a `let rec` (§10) answers through
+  // rec_build.odin instead: the entry asked for may not have run yet, and
+  // reading it is exactly the demand that makes it run. Ordinary Tables -
+  // every Table in a program that has no cyclic `let rec` - skip this on a
+  // length check.
+  if len(interp.rec_builds) > 0 {
+    if rb := rec_build_for(interp, t); rb != nil do return rec_access(interp, rb, key)
+  }
   val, found := table_find(t, key)
   if !found do return fail(interp, "no such key in Table")
   return val, true
@@ -818,6 +831,18 @@ eval_let_bind :: proc(interp: ^Interpreter, node: Node_Idx, env: ^Env) -> (Value
   child_env := env_make_child(env)
   bound_env := env if .Is_Rec not_in n.flags else child_env
 
+  // `let rec <name> <table literal>;` is the one shape whose entries can be
+  // reordered, so it gets §10's demand-driven construction (rec_build.odin) -
+  // which is what lets one entry reach another, and lets the Table reach
+  // itself. Every other shape, `let rec` over a function included, keeps the
+  // ordinary path below unchanged.
+  if .Is_Rec in n.flags && interp.ast.nodes[bound_idx].kind == .Table_Construct {
+    if _, rok := eval_rec_table(interp, bound_idx, node_text(interp, name_idx), child_env); !rok {
+      return nil, false
+    }
+    return eval_slot(interp, body_idx, child_env)
+  }
+
   bound_val: Value
   bok: bool
   if interp.ast.nodes[bound_idx].kind == .Hole {
@@ -850,7 +875,7 @@ eval_with_ctx :: proc(interp: ^Interpreter, node: Node_Idx, env: ^Env) -> (Value
 
   new_ctx_val, cok := eval_slot(interp, new_ctx_idx, env)
   if !cok do return nil, false
-  new_ctx_val, cok = await_value(interp, new_ctx_val) // ctx.permissions reads need a concrete Table
+  new_ctx_val, cok = concrete_value(interp, new_ctx_val) // ctx.permissions reads need a concrete Table
   if !cok do return nil, false
 
   old_ctx := interp.current_ctx
@@ -874,14 +899,14 @@ eval_chctx :: proc(interp: ^Interpreter, node: Node_Idx, env: ^Env) -> (Value, b
 
   fn_val, fok := eval_slot(interp, fn_idx, env)
   if !fok do return nil, false
-  fn_val, fok = await_value(interp, fn_val)
+  fn_val, fok = concrete_value(interp, fn_val)
   if !fok do return nil, false
   fn, is_fn := fn_val.(^Function_Value)
   if !is_fn do return fail(interp, "chctx's right side must be a function")
 
   new_ctx_val, cok := call_function(interp, fn, interp.current_ctx)
   if !cok do return nil, false
-  new_ctx_val, cok = await_value(interp, new_ctx_val)
+  new_ctx_val, cok = concrete_value(interp, new_ctx_val)
   if !cok do return nil, false
 
   old_ctx := interp.current_ctx
@@ -975,7 +1000,7 @@ eval_guard_chain :: proc(interp: ^Interpreter, node: Node_Idx, env: ^Env) -> (re
     case .Op_Is:
       subject_val, sok := eval(interp, left_idx, env)
       if !sok do return false, env, false
-      subject_val, sok = await_value(interp, subject_val)
+      subject_val, sok = concrete_value(interp, subject_val)
       if !sok do return false, env, false
       matched, new_env, mok := match_pattern(interp, right_idx, subject_val, env)
       if !mok do return false, env, false
@@ -998,7 +1023,7 @@ eval_guard_chain :: proc(interp: ^Interpreter, node: Node_Idx, env: ^Env) -> (re
   // current #arg per §8's "implicit function application in a guard position").
   val, ok1 := eval_slot(interp, node, env)
   if !ok1 do return false, env, false
-  val, ok1 = await_value(interp, val)
+  val, ok1 = concrete_value(interp, val)
   if !ok1 do return false, env, false
   if fn, is_fn := val.(^Function_Value); is_fn {
     if len(interp.arg_stack) == 0 {
@@ -1007,7 +1032,7 @@ eval_guard_chain :: proc(interp: ^Interpreter, node: Node_Idx, env: ^Env) -> (re
     }
     applied, aok := call_function(interp, fn, interp.arg_stack[len(interp.arg_stack) - 1])
     if !aok do return false, env, false
-    applied, aok = await_value(interp, applied)
+    applied, aok = concrete_value(interp, applied)
     if !aok do return false, env, false
     val = applied
   }
@@ -1058,7 +1083,7 @@ eval_then_or_else :: proc(interp: ^Interpreter, node: Node_Idx, env: ^Env) -> (V
     discarded_val, dok := eval_slot(interp, discarded_idx, guard_env)
     interp.discard_depth -= 1
     if !dok do return nil, false
-    _, aok := await_value(interp, discarded_val)
+    _, aok := concrete_value(interp, discarded_val)
     if !aok do return nil, false
   }
 
@@ -1079,7 +1104,7 @@ eval_sha256 :: proc(interp: ^Interpreter, node: Node_Idx, env: ^Env) -> (Value,
   // An operand that is still an async handle gets awaited first, exactly as
   // every other operator does - `sha256 async <expr>` hashes the result, not
   // the handle.
-  val, ok = await_value(interp, val)
+  val, ok = concrete_value(interp, val)
   if !ok do return nil, false
 
   encoded, herr := value_digest_base64(val)
@@ -1097,7 +1122,7 @@ eval_check :: proc(interp: ^Interpreter, node: Node_Idx, env: ^Env) -> (Value, b
   cond_idx := interp.ast.extra_children[n.children_start]
   cond_val, cok := eval_slot(interp, cond_idx, env)
   if !cok do return nil, false
-  cond_val, cok = await_value(interp, cond_val)
+  cond_val, cok = concrete_value(interp, cond_val)
   if !cok do return nil, false
   cond_bool, is_bool := cond_val.(bool)
   if !is_bool do return fail(interp, "check condition must be a Boolean")
@@ -1108,7 +1133,7 @@ eval_check :: proc(interp: ^Interpreter, node: Node_Idx, env: ^Env) -> (Value, b
   if !cond_bool {
     if has_msg {
       msg_val, mok := eval_slot(interp, interp.ast.extra_children[n.children_start + 1], env)
-      if mok do msg_val, mok = await_value(interp, msg_val)
+      if mok do msg_val, mok = concrete_value(interp, msg_val)
       if mok {
         if s, is_str := msg_val.(string); is_str do return fail(interp, s)
       }

src/examples_test.odin

@@ -36,6 +36,7 @@ EXAMPLE_CASES := []Example_Case{
   {"check-and-invariants.hb", "100"},
   {"comparison-and-logic.hb", "{ordered: true, both: true, either: true, mixed: false}"},
   {"context-permissions.hb", "{ambient: {io: nothing}, io_denied: {}, replaced: {}, still_ambient: {io: nothing}}"},
+  {"cyclic-data.hb", `{round_trip: "Alice", mutual: true, second_hop: "Carol", reordered: 3, same_shape: true}`},
   {"files-symlink.hb", `"optiona.txt"`},
   {"functions.hb", "121"},
   {"functions-and-holes.hb", "{section: 11, explicit: 49, nested: 507, stored: 42, asserted: 9}"},