Diagnostics for an explicit CONCAT/JOIN on an anchor without a record

CHANGELOG.md

@@ -25,8 +25,25 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   embedded DSL (`Rtl.concat`, `Rtl.join`).
 - Conformance: curated positive extra `named_key`; semantic cases `concat_named_key` and
   `join_named_key`; negative case `concat_empty_key_name`.
+- **Diagnostics for a forgotten `REC`.** An explicit `CONCAT`/`JOIN` (written on the anchor's own
+  content spec) whose anchor has no record, or none of whose provided items has a record, used to
+  be skipped silently — the anchor just vanished from the recordset. Both cases are now reported
+  through `TableInterpreter.diagnostics()` (`anchor has no record — REC missing?`,
+  `none of the provided items has a record — REC missing on the provider side?`) and raise under
+  `withStrictPreconditions(true)`. Inherited actions (row/subrow/subtable/table-level `actSpecs`)
+  and anchors whose record was folded away by an earlier `CONCAT` are not reported. The semantics
+  of the operations is unchanged: a violated precondition still has no effect.
+- API: `InterpretationAction.inherited()` (new record component; the three-argument constructor is
+  kept and means `inherited = false`); `WorkingState.report(anchor, operation, message)`,
+  `WorkingState.isConcatenated(item)`, `WorkingState.allConcatenated()` — the set `C` of
+  concatenated-away anchors, the counterpart of `J` for `CONCAT`.
+- Tests: the task corpus (`RtlTask*Test`, `AtpTask*Test`) and the semantic conformance runner now
+  assert that interpretation produces no diagnostics.
 
 ### Changed
+- `InterpretationAction` gained a fourth record component `boolean inherited`: positional
+  deconstruction patterns (`instanceof InterpretationAction(var a, var p, var o)`) need a fourth
+  binding; construction through the three-argument constructor is unaffected.
 - `ActionSpec`: the record component `Set<Integer> keyPositions` is replaced by `RecordKey key`;
   `ConcatOperation(RecordKey key)` and `JoinOperation(RecordKey key)` likewise, and
   `WorkingState.applyConcat/applyJoin` take a `RecordKey`. The previous `ActionSpec`

docs/api.md

@@ -370,8 +370,8 @@ Recordset rs = new TableInterpreter()
 | `withMissingValueHandler(MissingValueHandler h)` | Handling of missing attribute values (default: `NULL_HANDLER`). |
 | `withTransformations(List<RecordsetTransformation> t)` | Post-processing transformations. |
 | `withAnonymousAttributeTemplate(String template)` | Name template for unnamed attributes; `%i` → index. Default: `"$a_%i"`. |
-| `withStrictPreconditions(boolean strict)` | A violated `CONCAT`/`JOIN` precondition (e.g. a named attribute shared by two concatenated records) raises an `IllegalStateException` instead of having no effect. Default: `false`. |
-| `List<Diagnostic> diagnostics()` | The `CONCAT`/`JOIN` actions skipped during the most recent `interpret(...)` because a precondition was violated — each with the anchor, the operation and the reason. Empty when nothing was skipped. |
+| `withStrictPreconditions(boolean strict)` | Any diagnostic (a violated `CONCAT`/`JOIN` precondition, e.g. a named attribute shared by two concatenated records, or a missing record — see `diagnostics()`) raises an `IllegalStateException` instead of the action having no effect. Default: `false`. |
+| `List<Diagnostic> diagnostics()` | The `CONCAT`/`JOIN` actions skipped during the most recent `interpret(...)` — each with the anchor, the operation and the reason: a key mismatch, a named attribute shared by two concatenated records, an empty record product, and — for actions written on the anchor's own content spec (`InterpretationAction.inherited() == false`) — `anchor has no record — REC missing?` or `none of the provided items has a record — REC missing on the provider side?`. Inherited actions skip anchors without records silently. Empty when nothing was skipped. |
 
 ---
 

docs/examples.md

@@ -644,7 +644,10 @@ each is an anchor of its own.
   column names `x`, `y` are *values* named `var` — they will travel into the records.
 - **Data rows** `[ … ]+`: the key cell is **delimited** `(VAL: …){';'}` — one item per token:
   - `COL->AVP` names each token `id` (the `ATTR` in the same column);
-  - `()->REC` gives each token a record of its own, `⟨id:a⟩`, `⟨id:b⟩`, `⟨id:c⟩`;
+  - `()->REC` gives each token a record of its own, `⟨id:a⟩`, `⟨id:b⟩`, `⟨id:c⟩` — `JOIN` multiplies
+    *records*, so without it the token has nothing to multiply, drops out of the recordset, and
+    the interpreter reports `JOIN skipped at …: anchor has no record — REC missing?` through
+    `TableInterpreter.diagnostics()`;
   - `RT*->JOIN` multiplies the token's record by the records of all cells to its right.
 - Each number cell `[VAL: 'value'->AVP, COL->REC]` is named `value` and builds the record
   `⟨value:1, var:x⟩` with the column name above it (`COL`, cardinality 1, row-major → the header).

docs/model/itm.md

@@ -282,6 +282,14 @@ Seven **working-state update operations** populate or modify the working state:
     A violated precondition of `O_concat^K` / `O_join^K` has no effect and is recorded as a
     `Diagnostic` (`WorkingState.diagnostics()`, surfaced as `TableInterpreter.diagnostics()`);
     `new WorkingState(true)` / `TableInterpreter.withStrictPreconditions(true)` raise instead.
+    The interpreter also reports, through the same channel, the "not applicable" cases of an
+    *explicit* `CONCAT`/`JOIN` — one written on the anchor's own content spec: an anchor without a
+    record (`anchor has no record — REC missing?`) and provided items none of which has a record.
+    Inherited actions (`actSpecs` of a table/subtable/row/subrow/cell scope) reach anchors that were
+    never meant to carry records, so for them these cases are routine and stay silent. An anchor
+    whose record was folded away by an earlier concatenation is routine as well: the working state
+    keeps such anchors in the set `C` (`WorkingState.isConcatenated`, `allConcatenated`), the
+    counterpart of `J` for `O_concat`.
 
     **Implementation note — named keys.** The manuscript defines `K` over positions only
     (`K ⊆ ℕ₀`). The implementation additionally accepts key *attribute names* (`RecordKey`:

docs/rtl-reference.md

@@ -422,6 +422,14 @@ they diverge — `CONCAT` yields one wider record, `JOIN` yields one record per
 (see Examples 2 and 6). Up to jRegTab 0.5.x the folding operation was spelled `JOIN(K)`; a pattern
 written for 0.5.x must replace `JOIN(K)` by `CONCAT(K)`.
 
+Both operate on *records*: a `CONCAT`/`JOIN` on an anchor that has no `REC` has no effect — the
+anchor simply does not reach the recordset — and is reported through
+`TableInterpreter.diagnostics()` (`anchor has no record — REC missing?`), as is an explicit
+`CONCAT`/`JOIN` none of whose provided items has a record. Add `()->REC` when the anchor's own
+value is the whole record (Example 6). Only actions written on the anchor's own content spec are
+reported; actions inherited from a row/subrow/subtable/table level are applied to every cell and
+skip such anchors silently.
+
 **Choosing the key positions `K`.** `K` is any number of 0-based positions in the item-based
 record — `0` is the anchor, the following positions are the items in the order the `REC`
 providers supplied them. At every position in `K` all records being concatenated must agree, and
@@ -456,7 +464,7 @@ Examples by operation:
 [VAL: 'AIRLINE'->AVP]                   // AVP with a literal attribute (Illustrative example)
 [VAL : RT->REC, BW&STR*->CONCAT(0)]     // CONCAT(0): fold the rows below with the same key into one record (Task 16)
 [VAL : RT*->REC, BW&STR*->CONCAT(0,'A')] // CONCAT(0,'A'): the same, with the field named A as part of the key (concat_named_key)
-[(VAL: COL->AVP, RT*->JOIN){';'}]       // JOIN: one record per token × per cell to the right (Example 6)
+[(VAL: COL->AVP, ()->REC, RT*->JOIN){';'}] // JOIN: one record per token × per cell to the right (Example 6)
 [VAL: -AV->PREFIX(', ')]                // PREFIX: prepend the value above, separator ", " (Task 116)
 [BLANK ? VAL#'H': -LT&!BLANK->FILL | …] // FILL: copy the nearest non-blank cell to the left (Task 107)
 ```

src/main/java/ru/icc/regtab/atp/match/SemanticConstructor.java

@@ -222,7 +222,7 @@ InterpretationAction instantiateAction(CellDerivedItem anchor, ActionSpec action
                 }
                 providers.add(provider);
             }
-            return new InterpretationAction(anchor, providers, operation);
+            return new InterpretationAction(anchor, providers, operation, actionSpec.inherited());
         }
 
         private ItemProvider toItemProvider(ProviderSpec spec, boolean lenient) {

src/main/java/ru/icc/regtab/interpret/TableInterpreter.java

@@ -87,7 +87,10 @@ public TableInterpreter withStrictPreconditions(boolean strict) {
 
     /**
      * Diagnostics of the most recent {@link #interpret(InterpretableTable)} call: every
-     * {@code CONCAT} / {@code JOIN} action that was skipped because its precondition was violated.
+     * {@code CONCAT} / {@code JOIN} action that had no effect — a violated precondition (key
+     * mismatch, a named attribute shared by two concatenated records, an empty record product),
+     * or, for actions written on the anchor's own content spec (not inherited), an anchor without
+     * a record or provided items none of which has a record ({@code REC} missing).
      * Empty if nothing was skipped (or before the first call).
      */
     public List<Diagnostic> diagnostics() {
@@ -193,9 +196,43 @@ private void applyAction(WorkingState ws, InterpretationAction action) {
             // Empty items (e.g. lenient inherited provider on incompatible anchor) → skip
             case AvpOperation ignored  -> { if (!items.isEmpty()) ws.applyAvp(anchor, items); }
             case RecOperation ignored  -> ws.applyRec((CellDerivedItem) anchor, items);
-            case ConcatOperation op -> { if (!items.isEmpty()) ws.applyConcat((CellDerivedItem) anchor, items, op.key()); }
-            case JoinOperation op   -> { if (!items.isEmpty()) ws.applyJoin((CellDerivedItem) anchor, items, op.key()); }
+            case ConcatOperation op -> {
+                if (items.isEmpty()) break;
+                checkRecords(ws, action, items, "CONCAT");
+                ws.applyConcat((CellDerivedItem) anchor, items, op.key());
+            }
+            case JoinOperation op -> {
+                if (items.isEmpty()) break;
+                checkRecords(ws, action, items, "JOIN");
+                ws.applyJoin((CellDerivedItem) anchor, items, op.key());
+            }
+        }
+    }
+
+    /**
+     * Makes the silent "not applicable" cases of {@code CONCAT}/{@code JOIN} visible for
+     * <em>explicit</em> actions: an anchor without a record, or provided items none of which has
+     * a record — both usually a forgotten {@code REC}. Inherited actions reach anchors that were
+     * never meant to carry records, so for them these cases are routine and not reported. An
+     * anchor whose record was folded away by an earlier {@code CONCAT} (ι ∈ C) is routine as well:
+     * its own {@code CONCAT} is applied after the one that consumed it.
+     */
+    private static void checkRecords(WorkingState ws, InterpretationAction action,
+                                     List<? extends Item> items, String operation) {
+        if (action.inherited()) return;
+        CellDerivedItem anchor = (CellDerivedItem) action.anchor();
+        if (!ws.hasRec(anchor)) {
+            if (!ws.isConcatenated(anchor)) {
+                ws.report(anchor, operation, "anchor has no record — REC missing?");
+            }
+            return;
+        }
+        for (Item item : items) {
+            if (item instanceof CellDerivedItem c && c != anchor && (ws.hasRec(c) || ws.isConcatenated(c))) {
+                return;
+            }
         }
+        ws.report(anchor, operation, "none of the provided items has a record — REC missing on the provider side?");
     }
 
     // --- Phase 3: Recordset extraction ---

src/main/java/ru/icc/regtab/itm/semantics/WorkingState.java

@@ -20,6 +20,10 @@
  * have been consumed by a join; they stay in {@code rec} (a later join may consume the same
  * records again, irrespective of action order) but are excluded from recordset extraction.
  * {@link #allRec()} therefore returns the <em>live</em> anchors {@code dom(rec) \ J} only.
+ * {@code C} — the <em>concatenated-away anchors</em> — are items whose records have been folded
+ * into another anchor's record by a concatenation and removed from {@code rec}; the set is kept
+ * only so that a later action on such an anchor can be told apart from an anchor that never had
+ * a record (see {@link #isConcatenated(CellDerivedItem)}).
  */
 public final class WorkingState {
 
@@ -32,6 +36,8 @@ public final class WorkingState {
     private final Map<CellDerivedItem, List<List<Item>>> rec = new LinkedHashMap<>();
     /** J: joined-away anchors (identity semantics, like the items themselves). */
     private final Set<CellDerivedItem> joined = Collections.newSetFromMap(new IdentityHashMap<>());
+    /** C: anchors whose records were folded into another record by O_concat and removed from rec. */
+    private final Set<CellDerivedItem> concatenated = Collections.newSetFromMap(new IdentityHashMap<>());
     /** Preconditions violated during completion; the operations had no effect. */
     private final List<Diagnostic> diagnostics = new ArrayList<>();
     private final boolean strictPreconditions;
@@ -67,6 +73,8 @@ public List<List<Item>> rec(CellDerivedItem item) {
     public boolean hasRec(CellDerivedItem item) { return rec.containsKey(item); }
     /** ι ∈ J. */
     public boolean isJoined(CellDerivedItem item) { return joined.contains(item); }
+    /** ι ∈ C: the anchor's record was concatenated into another anchor's record and removed. */
+    public boolean isConcatenated(CellDerivedItem item) { return concatenated.contains(item); }
 
     public Map<Item, String> allVal() { return Collections.unmodifiableMap(val); }
     public Map<Item, String> allAttr() { return Collections.unmodifiableMap(attr); }
@@ -89,9 +97,25 @@ public Map<CellDerivedItem, List<List<Item>>> allRec() {
     /** J: the joined-away anchors. */
     public Set<CellDerivedItem> allJoined() { return Collections.unmodifiableSet(joined); }
 
+    /** C: the concatenated-away anchors. */
+    public Set<CellDerivedItem> allConcatenated() { return Collections.unmodifiableSet(concatenated); }
+
     /** Preconditions violated so far (the corresponding operations had no effect). */
     public List<Diagnostic> diagnostics() { return Collections.unmodifiableList(diagnostics); }
 
+    /**
+     * Records that an operation is not applicable to its anchor and has no effect: adds a
+     * {@link Diagnostic}; under strict preconditions raises an {@link IllegalStateException}
+     * with the same text instead. Used by the operations themselves and by the interpreter
+     * for checks that depend on information the working state does not have (e.g. whether an
+     * action is explicit or inherited).
+     */
+    public void report(CellDerivedItem anchor, String operation, String message) {
+        Diagnostic d = new Diagnostic(anchor, operation, message);
+        diagnostics.add(d);
+        if (strictPreconditions) throw new IllegalStateException(d.toString());
+    }
+
     /**
      * Derived function: assoc(iota) = a iff avp(iota) = (a, v).
      */
@@ -204,14 +228,14 @@ public void applyConcat(CellDerivedItem anchor, List<? extends Item> items, Reco
             List<Item> otherRec = rec.get(other).getFirst();
             String problem = keyMismatch(anchorRec, otherRec, key);            // (ii)
             if (problem != null) {
-                skip(anchor, "CONCAT", problem);
+                report(anchor, "CONCAT", problem);
                 return;
             }
             result.addAll(dropK(otherRec, key));
         }
         String duplicate = duplicateAttribute(result);                  // (iii)
         if (duplicate != null) {
-            skip(anchor, "CONCAT", "named attribute '" + duplicate
+            report(anchor, "CONCAT", "named attribute '" + duplicate
                     + "' occurs in more than one of the concatenated records");
             return;
         }
@@ -221,6 +245,7 @@ public void applyConcat(CellDerivedItem anchor, List<? extends Item> items, Reco
         for (CellDerivedItem other : others) {
             rec.remove(other);
             joined.remove(other);
+            concatenated.add(other);
         }
     }
 
@@ -261,7 +286,7 @@ public void applyJoin(CellDerivedItem anchor, List<? extends Item> items, Record
             }
         }
         if (result.isEmpty()) {
-            skip(anchor, "JOIN", "none of the " + dropped + " record pairs satisfies the key/attribute conditions; "
+            report(anchor, "JOIN", "none of the " + dropped + " record pairs satisfies the key/attribute conditions; "
                     + "the anchor keeps its records");
         } else {
             rec.put(anchor, result);
@@ -364,12 +389,6 @@ private String duplicateAttribute(List<Item> sequence) {
         return null;
     }
 
-    private void skip(CellDerivedItem anchor, String operation, String message) {
-        Diagnostic d = new Diagnostic(anchor, operation, message);
-        diagnostics.add(d);
-        if (strictPreconditions) throw new IllegalStateException(d.toString());
-    }
-
     // --- Consistency checks ---
 
     /**

src/main/java/ru/icc/regtab/itm/semantics/action/InterpretationAction.java

@@ -11,15 +11,28 @@
  * Interpretation action (def:interpretation-action): a triple (anchor, providers, operation)
  * where anchor is the item being interpreted, providers yield related items,
  * and operation updates the working state.
+ *
+ * @param inherited {@code true} if the action was inherited from the {@code actSpecs} of an
+ *                  enclosing scope (table/subtable/row/subrow/cell level) rather than written on
+ *                  the anchor's own content spec. Does not affect the semantics of the operation;
+ *                  it only decides whether a {@code CONCAT}/{@code JOIN} on an anchor without a
+ *                  record is reported as a diagnostic (explicit actions) or skipped silently
+ *                  (inherited actions, for which such anchors are routine).
  */
 public record InterpretationAction(
         Item anchor,
         List<ItemProvider> providers,
-        WorkingStateOperation operation
+        WorkingStateOperation operation,
+        boolean inherited
 ) {
     public InterpretationAction {
         Objects.requireNonNull(anchor, "anchor");
         providers = List.copyOf(Objects.requireNonNull(providers, "providers"));
         Objects.requireNonNull(operation, "operation");
     }
+
+    /** An explicit action ({@code inherited = false}). */
+    public InterpretationAction(Item anchor, List<ItemProvider> providers, WorkingStateOperation operation) {
+        this(anchor, providers, operation, false);
+    }
 }

src/test/java/ru/icc/regtab/atp/AtpTaskBase.java

@@ -43,9 +43,11 @@ private void runVariant(int variantId, Path taskDir, Path tasksRoot) throws Exce
                 .orElseThrow(() -> new AssertionError(
                         "ATP Task" + taskId() + " pattern did not match variant " + variantId));
 
-        Recordset actual = pattern.transform(new TableInterpreter()
-                .withStrategy(SchemaConstructionStrategy.RECORD_FIRST)
-                .interpret(itm));
+        TableInterpreter interpreter = new TableInterpreter()
+                .withStrategy(SchemaConstructionStrategy.RECORD_FIRST);
+        Recordset actual = pattern.transform(interpreter.interpret(itm));
+        assertTrue(interpreter.diagnostics().isEmpty(),
+                () -> "ATP Task" + taskId() + " variant " + variantId + ": " + interpreter.diagnostics());
 
         RecordsetMatchOptions matchOpts = TaskMatchOptionsLoader.load(tasksRoot, taskId());
         Path expectedPath = taskDir.resolve("expected_" + variantId + ".csv");