Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "pyregtab"
version = "0.5.0"
version = "0.5.1"
edition = "2021"
description = "Native core of pyRegTab: RTL compiler, ATP matcher and table interpreter"
license = "MIT"
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ and interprets the match into a relational **recordset**:
TableSyntax → RtlCompiler/TablePattern → AtpMatcher → TableInterpreter → Recordset
```

**pyRegTab 0.5.0 ≙ jRegTab 0.5.0** (same API, same semantics, same test
**pyRegTab 0.5.1 ≙ jRegTab 0.5.1** (same API, same semantics, same test
corpus), including the
embedded RTL DSL `pyregtab.dsl` — a port of jRegTab's `ru.icc.regtab.dsl`
(added upstream in jRegTab 0.3.0). Python-side extras on top of the Java API:
Expand Down Expand Up @@ -143,7 +143,7 @@ Rust (`pyregtab._core`, built with [PyO3](https://pyo3.rs) and

## Testing

`pytest tests` runs (1 908 tests):
`pytest tests` runs (1 925 tests):

- the full benchmark suite — tasks 001–150 (Foofah, RegTab, Baikal),
every fixture variant, **both** via RTL patterns and via ATP patterns
Expand Down
4 changes: 2 additions & 2 deletions conformance/UPSTREAM
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
commit: 035ff1a139e885e4cea85aa66a33e89a6b30f8c9
tag: v0.5.0
commit: c12633763b309fd00f65d8b236a4ab91795303b4
tag: v0.5.1
path: conformance/
3 changes: 3 additions & 0 deletions conformance/semantic/anch_named_attrs/expected.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
"Dato","Lokaler","Klasse"
"20.05","AU","0"
"11.06","A2.1","1"
3 changes: 3 additions & 0 deletions conformance/semantic/anch_named_attrs/input.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
"Dato","Lokaler","Klasse"
"20.05","AU","0"
"11.06","A2.1","1"
1 change: 1 addition & 0 deletions conformance/semantic/anch_named_attrs/options.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{ "expectedHasHeader": true }
3 changes: 3 additions & 0 deletions conformance/semantic/anch_named_attrs/pattern.rtl
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
<ANCH(1)>
[ [ATTR]+ ]
[ COL->AVP [VAL] [VAL: ROW*->REC] [VAL] ]+
4 changes: 4 additions & 0 deletions conformance/semantic/anch_named_inline_delim/expected.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
"Dato","Lokaler","Klasse"
"20.05","AU","0"
"20.05"," C1.1","0"
"11.06","A2.1","1"
3 changes: 3 additions & 0 deletions conformance/semantic/anch_named_inline_delim/input.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
"Dato","Lokaler","Klasse"
"20.05","AU, C1.1","0"
"11.06","A2.1","1"
1 change: 1 addition & 0 deletions conformance/semantic/anch_named_inline_delim/options.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{ "expectedHasHeader": true }
2 changes: 2 additions & 0 deletions conformance/semantic/anch_named_inline_delim/pattern.rtl
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[ [ATTR]+ ]
[ COL->AVP [VAL] [(VAL: ROW*->REC(1)){','}] [VAL] ]+
3 changes: 3 additions & 0 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,9 @@ byte-identical to `RtlCompiler.compile` for lambda-free patterns. See the
with `pattern.with_transformations(...)` or to an interpreter with
`.with_transformations([...])`. RTL settings `<NORM, ANCH(n), SPLIT("s")>` and
inline `REC(n)` / `REC('s')` parameters compile to these transformations.
`AnchorAttributeAtPosition(pos)` moves the anchor attribute — its name together with
its values — to 0-based position `pos`; it renames nothing, so every attribute-value
binding survives and only the schema order changes.

## RTL bindings

Expand Down
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,4 +109,4 @@ Requires **Python 3.10+**; binary wheels for Windows, Linux, and macOS.
---

!!! note "Status"
Current release: **0.5.0** (feature parity with jRegTab 0.5.0) · License: **MIT** · [PyPI](https://pypi.org/project/pyregtab/) · [GitHub](https://github.com/regtab/pyregtab)
Current release: **0.5.1** (feature parity with jRegTab 0.5.1) · License: **MIT** · [PyPI](https://pypi.org/project/pyregtab/) · [GitHub](https://github.com/regtab/pyregtab)
2 changes: 1 addition & 1 deletion docs/model/atp.md
Original file line number Diff line number Diff line change
Expand Up @@ -331,7 +331,7 @@ chosen operation (see [ITM — Interpretation actions](itm.md#interpretation-act
| Operation | Python factory | Notes |
|---|---|---|
| `REC` | `ActionSpec.rec(providers…)` | anchor → first field; providers supply remaining fields |
| `REC(n)` | `ActionSpec.rec(int n, providers…)` | adds `AnchorAttributeAtPosition` post-step |
| `REC(n)` | `ActionSpec.rec(int n, providers…)` | adds `AnchorAttributeAtPosition` post-step — moves the anchor attribute (name with its values) to position *n* |
| `REC('s')` | `ActionSpec.rec(String delim, providers…)` | adds `DelimitedFieldSplit` post-step |
| `AVP` | `ActionSpec.avp(provider)` | associates VAL anchor with ATTR item |
| `AVP "name"` | `ActionSpec.avp("ATTR_NAME")` | context-derived ATTR constant |
Expand Down
5 changes: 3 additions & 2 deletions docs/model/itm.md
Original file line number Diff line number Diff line change
Expand Up @@ -317,7 +317,8 @@ satisfy the constraints of the chosen operation (Tab. I in the paper):
| AVP with literal | `ActionSpec.avp("ATTR_NAME")` | Context-derived ATTR constant |

`ActionSpec.rec(int anchorPos, providers…)` adds an `AnchorAttributeAtPosition`
post-processing step (RTL: `REC(n)`).
post-processing step (RTL: `REC(n)`) — it moves the anchor attribute, name and
values together, to position `anchorPos` of the schema.
`ActionSpec.rec(String splitDelimiter, providers…)` adds a `DelimitedFieldSplit`
step (RTL: `REC('s')`).

Expand Down Expand Up @@ -431,7 +432,7 @@ The extracted recordset may be further post-processed by optional operations:
| Delimited field split | `DelimitedFieldSplit` | `ActionSpec.rec(String delimiter, …)` — RTL `REC('s')` |
| Field splitting | `FieldSplitting` | explicit split spec |
| Whitespace normalisation | `WhitespaceNormalization` | `with_transformations(…)` |
| Anchor attribute at position | `AnchorAttributeAtPosition` | `ActionSpec.rec(int pos, …)` — RTL `REC(n)` |
| Anchor attribute at position | `AnchorAttributeAtPosition` | `ActionSpec.rec(int pos, …)` — RTL `REC(n)` / `<ANCH(n)>`; moves the anchor attribute (name with its values) to position `pos`, preserving every attribute-value binding |

---

Expand Down
18 changes: 16 additions & 2 deletions docs/rtl-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,15 +131,29 @@ post-extraction `RecordsetTransformation`s on the resulting `TablePattern`.
| Setting | Effect |
|---|---|
| `NORM` | Apply whitespace normalisation to all field values after extraction |
| `ANCH(n)` | Use position *n* in the first record as the attribute name for all records |
| `ANCH(n)` | Move the anchor attribute to 0-based position *n* in the schema |
| `SPLIT("s")` | Split all field values by delimiter *s* after extraction |

Example: `<NORM, ANCH(2)> [ … ]` — normalise and anchor at position 2.

The anchor is the first attribute of the extracted schema, and `ANCH(n)` moves that
**attribute** — its name travels together with its values, so the attribute-value binding
of every record is untouched and only the order of the schema changes. A position of 0,
a position beyond the schema, or a single-attribute schema leaves the recordset as is.

The rule is the same for named attributes (produced by `AVP`) and for the anonymous
`$a_i` names the interpreter invents: an anonymous name is **not** renumbered, it moves
with its attribute. A schema `$a_1, $a_2, $a_3` under `ANCH(2)` therefore becomes
`$a_2, $a_3, $a_1` — the values sit in the same positions as before, and the name shows
which attribute was moved.

!!! note "Inline equivalents"
The same two transformations can be requested *inline* on a `REC` action:
`REC(n)` is equivalent to the `ANCH(n)` setting, and `REC('s')` is equivalent to
`SPLIT("s")`. Inline forms are by far the more common in practice (see Tasks 02, 03).
The position of the inline form in the pattern does not matter — `REC(n)` is picked up
anywhere, including inside a delimited content specification such as
`[(VAL: ROW*->REC(1)){','}]`, and always yields the same transformation.
The compiler merges inline and prefix forms and raises `RtlCompileError` if they
conflict (e.g. `ANCH(1)` together with `REC(2)`).

Expand Down Expand Up @@ -397,7 +411,7 @@ provSpecs -> op
|---|---|---|
| `REC` | `prov->REC` | Anchor item → record entry; provider supplies additional field values |
| `REC` | `()->REC` | Anchor item → single-field record (no additional providers; useful after `SUFFIX`/`PREFIX`/`FILL` has enriched the anchor value) |
| `REC(n)` | `prov->REC(n)` | Same + use attribute at position *n* as the record's attribute name |
| `REC(n)` | `prov->REC(n)` | Same + move the anchor attribute (name with its values) to position *n* |
| `REC('s')` | `prov->REC('s')` | Same + split field values by delimiter *s* |
| `AVP` | `prov->AVP` | Associate anchor (VAL) with an attribute from the provider (ATTR) |
| `JOIN` | `prov->JOIN` | Join item-based records: all items included, then dedup by named attribute (K=∅) |
Expand Down
Loading
Loading