Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,34 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Fixed

- **Zero-width subrows no longer break the matcher.** An explicit subrow `{…}` whose cell
patterns all matched zero cells (e.g. `{ [BLANK]* }` in a row without blank cells) used to
throw `IllegalArgumentException: colEnd must be >= colStart` in the middle of a row and to
silently fail ("pattern did not match") at the end of a row, because a trailing pattern was
never attempted once the row was exhausted. Such a subrow now matches the empty sequence,
as `*` does in regular expressions: it is tried at the end of the row as well, a repeated
empty match (`{ [BLANK]* }+`, `{ [BLANK]* }*`, `{ [BLANK]* }{n}`) counts as a single empty
iteration and never loops, and an empty subrow is not materialized in the interpretable
table (it covers no cells, so the spatial index of 0.5.2 is unaffected). The same applies to
a subtable whose row patterns are all optional. Found while evaluating RTL on ATBench
(regtab-eval-on-atbench, `reports/full-run.md` §5); the workaround
`{ [BLANK]* [!BLANK ? VAL: COL->AVP] [BLANK]* }+` keeps its outcome, and the natural form
`{ [!BLANK ? VAL: COL->AVP] }+ { [BLANK]* }` now works.
- **`{1}` and `{0}` quantifiers are accepted.** `Quantifier.exactly(n)` required `n ≥ 2` and
the RTL compiler let the raw `IllegalArgumentException: EXACTLY requires n >= 2` escape.
`{1}` is now equivalent to no quantifier and `{0}` to zero occurrences (an empty match);
only a negative `n` is rejected. An invalid `{n}` (negative, or out of `int` range) is
reported as an `RtlCompileException` with the source position instead of a runtime exception.
- API: `MatchedSubrow` / `MatchedSubtable` allow an empty interval (`colEnd == colStart - 1`,
`rowEnd == rowStart - 1`) and gain `empty(...)`, `isEmpty()`, `width()` / `height()`;
`Quantifier.exactly(n)` accepts `n ≥ 0`.
- Conformance: semantic cases `subrow_zero_width_mid`, `subrow_zero_width_tail`,
`subrow_zero_width_repeated` (`+`), `subrow_zero_width_repeated_star` (`*`),
`quantifier_exactly_one`, `quantifier_exactly_zero`; negative case
`quantifier_exactly_negative`; curated positive case `quantifier_small_n`.

## [0.7.0] - 2026-08-29

### Added
Expand Down
23 changes: 23 additions & 0 deletions conformance/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,29 @@ compound specification).
> Changed in jRegTab 0.5.0. Earlier versions trimmed each substring and silently
> dropped empty ones; patterns relying on that must add `=TRIM` to the delimited atom.

## Zero-width subrows and `{n}` with n < 2

An explicit subrow (or subtable) whose children are all optional may consume no cells at
all, e.g. `{ [BLANK]* }` in a row without blank cells. It then matches the *empty sequence*,
as `*` does in regular expressions:

- the empty match is tried at the end of the row as well, so a trailing `{ [BLANK]* }`
does not make the row fail;
- a repeated empty match (`{ [BLANK]* }+`, `{ [BLANK]* }*`, `{ [BLANK]* }{3}`) counts as a
single empty iteration — implementations must not loop;
- an empty subrow covers no cells and must not appear in the interpretable table.

The quantifier `{n}` accepts any `n ≥ 0`: `{1}` is equivalent to no quantifier, `{0}` to
zero occurrences (an empty match). Only a negative `n` is a compile error. Positive case
`quantifier_small_n` pins the canonical form; the executable checks are
`semantic/subrow_zero_width_mid`, `semantic/subrow_zero_width_tail`,
`semantic/subrow_zero_width_repeated` (`+`), `semantic/subrow_zero_width_repeated_star` (`*`),
`semantic/quantifier_exactly_one`, `semantic/quantifier_exactly_zero` and the negative case
`quantifier_exactly_negative`.

> Changed in jRegTab after 0.7.0. Earlier versions raised an error for a zero-width subrow in the
> middle of a row, silently failed on one at the end of a row, and rejected `{0}` / `{1}`.

In jRegTab items 1–4 of the contract are executed by
`ru.icc.regtab.conformance.RtlConformanceTest` and item 5 by
`ru.icc.regtab.conformance.RtlSemanticConformanceTest`;
Expand Down
5 changes: 4 additions & 1 deletion conformance/VERSION
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,7 @@ generated: 2026-08-29
sources: RtlTask001..150 + curated extras
note: key K of CONCAT(K)/JOIN(K) may name attributes (CONCAT(0, 1, 'A', 'B'), JOIN('k')), resolved per record;
canonical form sorts positions, then names, single-quoted; curated extra named_key and
semantic cases concat_named_key, join_named_key, negative concat_empty_key_name added
semantic cases concat_named_key, join_named_key, negative concat_empty_key_name added;
zero-width subrows match the empty sequence (never repeated) and {n} accepts n in {0, 1}:
curated extra quantifier_small_n, semantic cases subrow_zero_width_{mid,tail,repeated,repeated_star},
quantifier_exactly_{one,zero}, negative quantifier_exactly_negative added
1 change: 1 addition & 0 deletions conformance/negative/quantifier_exactly_negative.rtl
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[ [VAL]{-1} ]
1 change: 1 addition & 0 deletions conformance/positive/quantifier_small_n.expected.rtl
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[ [ VAL ]{1} [ VAL ]{0} { [ BLANK ]* }{1} ]
1 change: 1 addition & 0 deletions conformance/positive/quantifier_small_n.rtl
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[ [VAL]{1} [VAL]{0} { [BLANK]* }{1} ]
2 changes: 2 additions & 0 deletions conformance/semantic/quantifier_exactly_one/expected.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
x,1
y,2
3 changes: 3 additions & 0 deletions conformance/semantic/quantifier_exactly_one/input.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
A,B
x,1
y,2
2 changes: 2 additions & 0 deletions conformance/semantic/quantifier_exactly_one/pattern.rtl
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[ [ATTR]+ ]
[ [VAL: COL->AVP, ROW*->REC] { [BLANK]* }{1} [VAL: COL->AVP] ]+
2 changes: 2 additions & 0 deletions conformance/semantic/quantifier_exactly_zero/expected.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
x,1
y,2
3 changes: 3 additions & 0 deletions conformance/semantic/quantifier_exactly_zero/input.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
A,B
x,1
y,2
2 changes: 2 additions & 0 deletions conformance/semantic/quantifier_exactly_zero/pattern.rtl
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[ [ATTR]+ ]
[ [VAL: COL->AVP, ROW*->REC] { [BLANK]* }{0} [VAL: COL->AVP] ]+
2 changes: 2 additions & 0 deletions conformance/semantic/subrow_zero_width_mid/expected.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
x,1
y,2
3 changes: 3 additions & 0 deletions conformance/semantic/subrow_zero_width_mid/input.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
A,B
x,1
y,2
2 changes: 2 additions & 0 deletions conformance/semantic/subrow_zero_width_mid/pattern.rtl
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[ [ATTR]+ ]
[ [VAL: COL->AVP, ROW*->REC] { [BLANK]* } [VAL: COL->AVP] ]+
2 changes: 2 additions & 0 deletions conformance/semantic/subrow_zero_width_repeated/expected.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
x,1
y,2
3 changes: 3 additions & 0 deletions conformance/semantic/subrow_zero_width_repeated/input.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
A,B
x,1
y,2
2 changes: 2 additions & 0 deletions conformance/semantic/subrow_zero_width_repeated/pattern.rtl
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[ [ATTR]+ ]
[ [VAL: COL->AVP, ROW*->REC] { [BLANK]* }+ [VAL: COL->AVP] { [BLANK]* }+ ]+
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
x,1
y,2
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
A,B
x,1
y,2
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[ [ATTR]+ ]
[ [VAL: COL->AVP, ROW*->REC] { [BLANK]* }* [VAL: COL->AVP] { [BLANK]* }* ]+
2 changes: 2 additions & 0 deletions conformance/semantic/subrow_zero_width_tail/expected.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
x,1
y,2
3 changes: 3 additions & 0 deletions conformance/semantic/subrow_zero_width_tail/input.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
A,B
x,1
y,2
2 changes: 2 additions & 0 deletions conformance/semantic/subrow_zero_width_tail/pattern.rtl
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[ [ATTR]+ ]
[ [VAL: COL->AVP, ROW*->REC] [VAL: COL->AVP] { [BLANK]* } ]+
2 changes: 1 addition & 1 deletion docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -318,7 +318,7 @@ For custom conditions use `new ItemFilterConditionSpec.Custom("description", (an
| `zeroOrOne()` | `?` | Zero or one. |
| `oneOrMore()` | `+` | One or more. |
| `zeroOrMore()` | `*` | Zero or more. |
| `exactly(int n)` | `{n}` | Exactly n (n ≥ 2). |
| `exactly(int n)` | `{n}` | Exactly n (n ≥ 0; `{1}` ≡ no quantifier, `{0}` ≡ empty match). |

---

Expand Down
2 changes: 1 addition & 1 deletion docs/model/atp.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ row pattern, its subtable pattern, and the top-level table pattern.
|---|---|
| `?` | zero or one occurrence |
| `1` (default) | exactly one occurrence |
| `{n}` | exactly `n` occurrences (`n ≥ 2`) |
| `{n}` | exactly `n` occurrences (`n ≥ 0`; `{1}` ≡ no quantifier, `{0}` ≡ zero occurrences, i.e. an empty match) |
| `+` | one or more occurrences |
| `*` | zero or more occurrences |

Expand Down
9 changes: 8 additions & 1 deletion docs/rtl-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,11 +104,18 @@ cell patterns:
| `?` | 0 or 1 |
| `*` | 0 or more |
| `+` | 1 or more |
| `{n}` | exactly *n* |
| `{n}` | exactly *n* (*n* ≥ 0; `{1}` is the same as no quantifier, `{0}` matches nothing, i.e. an empty match) |

The `+` on the outer row pattern above means "one or more data rows"; the `{2}` means "exactly
two value cells".

**Empty matches.** An explicit subrow (or subtable) whose children are all optional may
match zero cells (rows) — for instance `{ [BLANK]* }` in a row without blank cells. Such a
subrow matches the empty sequence, exactly like `*` in a regular expression: it never fails
the row, it is tried at the end of the row as well, and a repeated empty match (`{ [BLANK]* }+`,
`{ [BLANK]* }*`, `{ [BLANK]* }{3}`) counts as a single empty iteration rather than looping.
An empty subrow covers no cells and therefore never appears in the interpretable table.

**Inherited action specs** — `[acts]` placed at the table, subtable, row, or subrow level are
inherited by all descendant cells. Inherited actions are merged with any local actions on the
cell's `contSpec`. Incompatible inherited actions (e.g. `COL->AVP` on an `ATTR` anchor) are
Expand Down
3 changes: 3 additions & 0 deletions src/main/java/ru/icc/regtab/atp/AtpMatcher.java
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,9 @@ public static Optional<InterpretableTable> match(
}

private static void applyMatchedStructure(TableSyntax syntax, MatchResult result) {
// Empty (zero-width) matches are not materialized: they cover no cells.
int[] starts = result.matchedSubtables().stream()
.filter(m -> !m.isEmpty())
.mapToInt(m -> m.rowStart())
.distinct()
.sorted()
Expand All @@ -97,6 +99,7 @@ private static void applyMatchedStructure(TableSyntax syntax, MatchResult result
}

result.matchedSubrows().stream()
.filter(m -> !m.isEmpty())
.sorted(Comparator.comparingInt(MatchedSubrow::rowIndex)
.thenComparingInt(MatchedSubrow::colStart))
.forEach(m -> syntax.defineSubrow(m.rowIndex(), m.colStart(), m.colEnd()));
Expand Down
26 changes: 24 additions & 2 deletions src/main/java/ru/icc/regtab/atp/match/MatchedSubrow.java
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,13 @@

/**
* Matched subrow pattern and the cell interval it consumes within a row.
* <p>
* The interval {@code [colStart, colEnd]} is inclusive. A subrow pattern whose cell
* patterns all matched zero cells (e.g. {@code { [BLANK]* }} in a row without blank
* cells) yields an <em>empty</em> match: {@code colEnd == colStart - 1}, where
* {@code colStart} is the position at which the empty match occurred (equal to the
* number of columns when it occurred at the end of the row). Empty matches are
* recorded here for completeness but are never materialized as ITM subrows.
*/
public record MatchedSubrow(SubrowPattern pattern, int rowIndex, int colStart, int colEnd) {

Expand All @@ -17,8 +24,23 @@ public record MatchedSubrow(SubrowPattern pattern, int rowIndex, int colStart, i
if (colStart < 0) {
throw new IllegalArgumentException("colStart must be non-negative: " + colStart);
}
if (colEnd < colStart) {
throw new IllegalArgumentException("colEnd must be >= colStart: " + colEnd + " < " + colStart);
if (colEnd < colStart - 1) {
throw new IllegalArgumentException("colEnd must be >= colStart - 1: " + colEnd + " < " + (colStart - 1));
}
}

/** An empty (zero-width) match of {@code pattern} at column {@code col} of row {@code rowIndex}. */
public static MatchedSubrow empty(SubrowPattern pattern, int rowIndex, int col) {
return new MatchedSubrow(pattern, rowIndex, col, col - 1);
}

/** {@code true} if this match consumed no cells. */
public boolean isEmpty() {
return colEnd < colStart;
}

/** Number of cells consumed (0 for an empty match). */
public int width() {
return colEnd - colStart + 1;
}
}
25 changes: 23 additions & 2 deletions src/main/java/ru/icc/regtab/atp/match/MatchedSubtable.java
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,12 @@

/**
* Matched subtable pattern and the row interval it consumes.
* <p>
* The interval {@code [rowStart, rowEnd]} is inclusive. A subtable pattern whose row
* patterns all matched zero rows yields an <em>empty</em> match:
* {@code rowEnd == rowStart - 1}, where {@code rowStart} is the position at which the
* empty match occurred. Empty matches are recorded for completeness but never
* introduce a subtable boundary in the ITM.
*/
public record MatchedSubtable(SubtablePattern pattern, int rowStart, int rowEnd) {

Expand All @@ -14,8 +20,23 @@ public record MatchedSubtable(SubtablePattern pattern, int rowStart, int rowEnd)
if (rowStart < 0) {
throw new IllegalArgumentException("rowStart must be non-negative: " + rowStart);
}
if (rowEnd < rowStart) {
throw new IllegalArgumentException("rowEnd must be >= rowStart: " + rowEnd + " < " + rowStart);
if (rowEnd < rowStart - 1) {
throw new IllegalArgumentException("rowEnd must be >= rowStart - 1: " + rowEnd + " < " + (rowStart - 1));
}
}

/** An empty (zero-height) match of {@code pattern} at row {@code row}. */
public static MatchedSubtable empty(SubtablePattern pattern, int row) {
return new MatchedSubtable(pattern, row, row - 1);
}

/** {@code true} if this match consumed no rows. */
public boolean isEmpty() {
return rowEnd < rowStart;
}

/** Number of rows consumed (0 for an empty match). */
public int height() {
return rowEnd - rowStart + 1;
}
}
46 changes: 34 additions & 12 deletions src/main/java/ru/icc/regtab/atp/match/SyntaxMatcher.java
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,12 @@

/**
* Syntactic layer matching exactly following the formal algorithms from the paper.
* <p>
* Empty matches: a subrow (or subtable) pattern whose children all matched zero
* elements matches the empty sequence, like {@code *} in regular expressions. Such an
* iteration is recorded as an empty {@link MatchedSubrow}/{@link MatchedSubtable}, is
* never repeated by {@code +}/{@code *}/{@code {n}}, and is tried at the end of the
* sequence as well, so a trailing {@code { [BLANK]* }} matches a row without blanks.
*/
public final class SyntaxMatcher {

Expand Down Expand Up @@ -52,7 +58,6 @@ private static <P, E> MatchOutcome matchPatterns(
int rowIndex) {

int i = elementIndex;
int n = elements.size();

for (int j = 0; j < patterns.size(); j++) {
P pattern = patterns.get(j);
Expand All @@ -61,16 +66,24 @@ private static <P, E> MatchOutcome matchPatterns(
int max = quantifier.max();
Deque<StackEntry> stack = new ArrayDeque<>();

while (stack.size() < max && i < n) {
// No "i < n" guard: a pattern that can match empty (a subrow/subtable whose
// children are all optional) must be tried at the end of the sequence too;
// cell and row dispatchers fail on their own when i >= n.
while (stack.size() < max) {
MatchSnapshot saved = state.snapshot();
MatchOutcome dispatched = dispatchPattern(pattern, elements, i, state, structureKind, rowIndex);
if (dispatched.success()) {
stack.push(new StackEntry(i, saved));
i = dispatched.nextIndex();
} else {
if (!dispatched.success()) {
state.restore(saved);
break;
}
stack.push(new StackEntry(i, saved));
if (dispatched.nextIndex() == i) {
// Empty iteration: as in regex engines, an empty match is never repeated
// and satisfies any remaining lower bound ((a*)+ and (a*){3} match "").
min = 0;
break;
}
i = dispatched.nextIndex();
}

if (stack.size() < min) {
Expand Down Expand Up @@ -143,7 +156,9 @@ private static MatchOutcome dispatchSubtablePattern(
return MatchOutcome.failure(rowIndex);
}

state.matchedSubtables.add(new MatchedSubtable(pattern, rowIndex, inner.nextIndex() - 1));
state.matchedSubtables.add(inner.nextIndex() == rowIndex
? MatchedSubtable.empty(pattern, rowIndex)
: new MatchedSubtable(pattern, rowIndex, inner.nextIndex() - 1));
return inner;
}

Expand Down Expand Up @@ -192,11 +207,18 @@ private static MatchOutcome dispatchSubrowPattern(
return MatchOutcome.failure(cellIndex);
}

state.matchedSubrows.add(new MatchedSubrow(
pattern,
rowIndex,
cells.get(cellIndex).col(),
cells.get(inner.nextIndex() - 1).col()));
int next = inner.nextIndex();
if (next == cellIndex) {
// Zero-width subrow: record where it matched (cells.size() at the end of the row).
int col = cellIndex < cells.size() ? cells.get(cellIndex).col() : cells.size();
state.matchedSubrows.add(MatchedSubrow.empty(pattern, rowIndex, col));
} else {
state.matchedSubrows.add(new MatchedSubrow(
pattern,
rowIndex,
cells.get(cellIndex).col(),
cells.get(next - 1).col()));
}
return inner;
}

Expand Down
9 changes: 6 additions & 3 deletions src/main/java/ru/icc/regtab/atp/spec/Quantifier.java
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,10 @@ public enum Kind {
ZERO_OR_ONE,
/** {@code 1} — exactly one occurrence (default). */
ONE,
/** {@code {n}} — exactly n occurrences (n ≥ 2). */
/**
* {@code {n}} — exactly n occurrences (n ≥ 0): {@code {1}} is equivalent to no
* quantifier, {@code {0}} to zero occurrences (an empty match).
*/
EXACTLY,
/** {@code +} — one or more occurrences. */
ONE_OR_MORE,
Expand All @@ -35,8 +38,8 @@ public enum Kind {

public Quantifier {
Objects.requireNonNull(kind, "kind");
if (kind == Kind.EXACTLY && n < 2) {
throw new IllegalArgumentException("EXACTLY requires n >= 2, got: " + n);
if (kind == Kind.EXACTLY && n < 0) {
throw new IllegalArgumentException("EXACTLY requires n >= 0, got: " + n);
}
}

Expand Down
Loading
Loading