Test very long SPSS strings

Author: TonisOrmisson

docs/pyspssio-spike.md

@@ -29,7 +29,7 @@ that require explicit tests or upstream work include:
 1. variable-set round trips;
 2. independent print and write formats;
 3. ordered document text;
-4. legacy code-page files and long-string edge cases.
+4. legacy code-page files and unusual very-long-string boundaries beyond the covered 340-byte UTF-8 fixture.
 
 Until a feature is proved, the adapter must report it as unsupported rather
 than silently losing it.

docs/sav-profile.md

@@ -37,6 +37,7 @@ of this boundary. It records the pinned source commit and installed engine versi
 | SPSS semantic | Pinned pyspssio status | Behaviour |
 | --- | --- | --- |
 | File label | Supported through the pinned fork | The adapter persists it in the dataset catalog and writes it through the IBM I/O identifier-string API. |
+| Very-long UTF-8 strings | Supported | The SAV and ZSAV fixture preserves a 340-byte, multi-byte UTF-8 string through the SQL catalog and back. |
 | Ordered document text | Unobservable | The engine exposes file-to-file document copying, but not reading or creating document text. Import records documents-unobservable; export requires that audited loss to be accepted. |
 | Print and write formats independently | Supported as raw IBM I/O tuples | The adapter stores both tuples separately and writes them without collapsing either value. |
 | Variable sets | Supported through raw IBM I/O | The adapter stores source sets in the extension catalog and writes them through the raw dictionary setter. Invalid target definitions fail before data are written. |

tests/test_sav_sqlite.py

@@ -179,3 +179,32 @@ def test_raw_dictionary_bridge_preserves_distinct_formats_sets_and_attribute_arr
         assert attribute_values(variable_attribute_pairs(reader, "answer")) == {
             "Array": ["red", "blue"],
         }
+
+
+@pytest.mark.parametrize("suffix", [".sav", ".zsav"])
+def test_very_long_string_round_trips_through_sqlite_and_export(tmp_path, suffix: str) -> None:
+    payload = "ü" * 170
+    payload_width = len(payload.encode("utf-8"))
+    source = tmp_path / f"long-source{suffix}"
+    destination = tmp_path / f"long-destination{suffix}"
+    database_path = tmp_path / f"long-{suffix[1:]}.sqlite"
+    database = f"sqlite:///{database_path}"
+    pyspssio.write_sav(
+        str(source), pd.DataFrame({"comment": [payload, "short"]}),
+    )
+    assert pyspssio.read_metadata(str(source))["var_types"]["comment"] == payload_width
+
+    openstatspec.import_sav(source, database_url=database, dataset_id="long")
+    connection = sqlite3.connect(database_path)
+    assert connection.execute(
+        "select string_width from variable_catalog where dataset_id = ? and source_name = ?",
+        ("long", "comment"),
+    ).fetchone() == (payload_width,)
+
+    openstatspec.export_sav(
+        database_url=database, dataset_id="long", destination=destination,
+        allow_loss=_REQUIRED_ENGINE_LOSS,
+    )
+    frame, metadata = pyspssio.read_sav(str(destination), convert_datetimes=False)
+    assert metadata["var_types"]["comment"] == payload_width
+    assert frame["comment"].tolist() == [payload, "short"]