Skip to content

Use polymake's JSON format, require polymake 4 - #29

Open
fingolfin wants to merge 6 commits into
masterfrom
polymake-json
Open

fingolfin wants to merge 6 commits into
masterfrom
polymake-json

Conversation

@fingolfin

@fingolfin fingolfin commented Aug 16, 2026

Copy link
Copy Markdown
Member

Resolves #22.

Net: +787 / −1224 lines.

What was there

polymaking spoke to polymake through the pre-4 plain format: it wrote files in
it, and pm_script_arg.pl stringified results so GAP could scrape them off
standard output. Rebuilding typed values from that text needed
ObjectConverters, a table mapping 58 keywords to hand-written parsers -- one
of which guessed whether a block was a matrix or a list of sets by checking for
a {.

What it is now

lib/pm.pl evaluates each requested property, serializes it with polymake's
own serializer, and writes one JSON file that GAP reads with the json
package. A file rather than stdout, because polymake's own output shares that
channel and one stray line would corrupt the document.

Values are decoded by polymake's type, not by keyword, so properties
polymaking has never heard of decode correctly too. What survives is only what
the type genuinely cannot tell us:

  • stripping the homogenizing 1 (POINTS, VERTICES, REL_INT_POINT,
    VALID_POINT, VERTEX_BARYCENTER) -- VERTICES and FACETS are both
    Matrix<Rational> and only one is a list of points;
  • two node indices that are plain integers (TOP_NODE, BOTTOM_NODE).

The 0-based to 1-based shift is derivable: Set<Int>, IncidenceMatrix and
GraphAdjacency are index containers, while Array<Int> (e.g.
FACET_DEGREES) holds values. That is what retires
ConvertPolymakeMatrixOrListOfSetsToGAP{,PlusOne}.

So convert.gd/gi (20 parsers), ObjectConverters.gd/gi (58 entries),
workaround_maps.gd/gi and pm_script_arg.pl are gone; json.gd/gi and
pm.pl replace them.

Consequences

  • Files are written as JSON, so polymake never converts them and never says
    so. That is the remainder of Allow to silence certain polymake messages to stderr #23 -- verified: loading a polymaking-written
    file produces completely empty stderr.
  • Nested properties are addressable: Polymake(poly, "HASSE_DIAGRAM.FACES").
    This is the "fully support HASSE_DIAGRAM again" the issue asks for. FACES,
    ADJACENCY and GRAPH remain as aliases.
  • Polymake(poly,"GRAPH") works again -- on master it fails outright with
    POSITION_SUBSTRING: <string> must be a string.
  • Keywords are evaluated independently, so one failure no longer discards
    the other results.
  • Floating point properties return a GAP float instead of a rational
    approximation with a warning.

Things worth knowing before reviewing

Two findings changed the design, and both came from probing polymake 4.15
rather than from the docs:

Serializing a nested big object is a trap. It emits only the properties
that happen to be computed and storable, and carries no _type at all:

GRAPH         => {"CONNECTED":true}                       # ADJACENCY missing
HASSE_DIAGRAM => {"ADJACENCY":…,"DECORATION":…,…}          # FACES missing

Requested individually the same values are complete and typed. So polymaking
always asks for leaf properties, and GRAPH is an alias for GRAPH.ADJACENCY.

DIMS is genuinely gone. polymake 4.15 warns available properties insufficient to compute 'DIMS' and returns null, so it is not aliased. The
existing fail expectation in the test suite was recording exactly that.

Also worth noting: Rational and Integer serialize as JSON strings, even
when integral, and Vector<Integer> gives ["8","12","6"] while Array<Int>
gives [8,12,6] -- so the scalar decoder accepts both spellings. Sparse
matrices carry a trailing {"cols": n}; cube(3)->FACETS is sparse, so this
is a common path, not an exotic one.

Breaking changes

  • polymake 4.0 or newer is required. polymake 3 cannot read the files
    polymaking now writes, so this is a hard error on the first call rather than
    an obscure later failure. The version is detected lazily; loading the package
    still spawns no processes. Users needing polymake 3 should stay on polymaking
    0.8.9.
  • json is a new needed dependency.
  • AppendToPolymakeObject(poly, name, value) takes a GAP value, not a
    string to append verbatim; JSON is not append-friendly, so the file is
    rewritten from the properties set so far. Consequently polymaking refuses to
    write to a file it did not create, instead of corrupting it.
  • ConvertMatrixToPolymakeString, ObjectConverters and the
    ConvertPolymake... functions are removed.
  • NamesKnownPropertiesOfPolymakeObject returns names in a different order
    (it always returned RecNames, i.e. unordered).

Testing

  • Full suite: 0 failures in 4 files, under the default backend and under cdd
    and beneath_beyond.
  • New tst/json.tst unit-tests the decoder against the exact shapes polymake
    emits: string and native scalars, big integers, sparse vectors and matrices,
    the trailing cols marker, index shifting vs. plain integer arrays,
    dehomogenization, and the writer.
  • New integration checks that HASSE_DIAGRAM.FACES agrees with FACES, that
    GRAPH has the documented shape, and that the file on disk really is
    polymake JSON.
  • Every decoded value was diffed against what the old converters produced for
    the same polytope before the old code was deleted.
  • makedoc.g builds with no warnings and no unresolved references.

🤖 Generated with Claude Code

@codecov

codecov Bot commented Aug 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.75000% with 49 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.44%. Comparing base (661f50f) to head (702efbb).

Files with missing lines Patch % Lines
lib/construct.gi 81.60% 23 Missing ⚠️
lib/json.gi 92.90% 10 Missing ⚠️
lib/environment.gi 89.87% 8 Missing ⚠️
lib/userpref.gi 72.22% 5 Missing ⚠️
lib/Objects.gi 85.71% 3 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           master      #29       +/-   ##
===========================================
+ Coverage   72.23%   83.44%   +11.20%     
===========================================
  Files          15       11        -4     
  Lines         868      767      -101     
===========================================
+ Hits          627      640       +13     
+ Misses        241      127      -114     
Files with missing lines Coverage Δ
lib/Objects.gd 100.00% <100.00%> (ø)
lib/construct.gd 100.00% <100.00%> (ø)
lib/environment.gd 100.00% <100.00%> (ø)
lib/json.gd 100.00% <100.00%> (ø)
lib/Objects.gi 71.18% <85.71%> (+11.75%) ⬆️
lib/userpref.gi 82.17% <72.22%> (-5.12%) ⬇️
lib/environment.gi 89.10% <89.87%> (+1.60%) ⬆️
lib/json.gi 92.90% <92.90%> (ø)
lib/construct.gi 81.48% <81.60%> (+6.93%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Base automatically changed from portable-tests to master August 16, 2026 21:58
fingolfin and others added 5 commits August 17, 2026 00:05
polymaking spoke to polymake through the pre-4 plain file format: it wrote
files in it, and lib/pm_script_arg.pl stringified results so that GAP could
scrape them back out of standard output. Reconstructing typed values from that
text needed a table mapping each of 58 keywords to a hand written parser, one
of which had to guess whether a block was a matrix or a list of sets by looking
for a brace.

polymake 4 has its own JSON format, and the json package can read it, so do
that instead. lib/pm.pl evaluates each requested property, serializes it with
polymake's own serializer and writes one JSON file, which GAP reads back. Going
through a file rather than standard output means results can never be confused
with anything polymake prints.

Values are now decoded by polymake's type rather than by keyword, so properties
polymaking has never heard of decode correctly too. Only the conventions that
genuinely depend on the keyword are left: stripping the homogenizing 1 from
points, and the two node indices that the type system cannot flag as indices.
Sets, incidence matrices and adjacency lists are recognised as index containers
by their type, so the 0-based to 1-based shift no longer needs a table either.

Consequences:

- Files are written as JSON, so polymake never converts them and never warns
  that it did. This is the rest of issue #23.
- Nested properties can be named directly: Polymake(poly, "HASSE_DIAGRAM.FACES")
  rather than the keyword rewriting in workaround_maps.gi. This restores full
  access to the Hasse diagram. DIMS is not among them; polymake 4 cannot
  compute it from a polytope's Hasse diagram and says so.
- Polymake(poly, "GRAPH") works again. It used to fail with
  POSITION_SUBSTRING: <string> must be a string.
- Keywords are evaluated independently, so one failing no longer discards the
  results of the others.
- Floating point properties such as MINIMAL_VERTEX_ANGLE return a GAP float
  rather than a rational approximation of one.

polymake before 4.0 cannot read the files polymaking now writes, so it is
rejected with a clear error on the first call rather than left to fail
obscurely. The version is detected lazily; loading the package still spawns no
processes.

AppendToPolymakeObject now takes a GAP value rather than a string to append
verbatim, since JSON is not append friendly: the file is rewritten from the
properties set so far. As a result polymaking refuses to write to a file it did
not create, rather than corrupting it.

Fixes #22.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Also avoid the two argument form of Set, declared only in GAP 4.11, in the
decoder; the version bump makes it available, but there is no reason to depend
on it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The version gate is a hard error on a path users can hit, but nothing exercised
it. Drive it through POLYMAKING_STATE rather than a stub binary, so the test
needs no subprocess.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A SparseMatrix may mix sparse rows with dense ones, and an all-zero row
serializes as an empty record; both are easy to get wrong. The shapes here were
taken from polymake 4.15 output rather than invented.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The remark only made sense to readers who knew what polymaking used to write.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
hap and hapcryst are needed-dependency consumers of polymaking, and neither
pins an upper bound, so 0.9.0 will be paired with existing releases of both.
Measured against them, the rewrite broke hapcryst's test suite entirely (9
failures, 0 before) and hap's IsAspherical.

Three things they rely on come back:

- POLYMAKE_COMMAND and POLYMAKE_DATA_DIR are set again, for code that reads
  them. Whether the *user* bound them before loading is recorded once, in
  POLYMAKING_LEGACY_SET, so our own values are not mistaken for theirs and
  the preferences stay the source of truth. A post restore hook refreshes
  them, so unlike in 0.8.9 they do not go stale with a saved workspace.

- ConvertMatrixToPolymakeString and the two argument AppendToPolymakeObject
  work together as before. Nothing appends verbatim to a JSON file, so the
  former now passes the section along as a record rather than as a string;
  code composing the two, which is how both packages use them, is unaffected.
  Appending a bare string reports what to use instead.

- The polymake 2.3 era type names that CheckAppVerTypList accepts are mapped
  to their polymake 4 spellings. hapcryst asks for RationalPolytope, which
  polymake 4 knows as Polytope<Rational> and would otherwise reject.

The deprecation notices for the second group are issued at InfoObsolete level
2 rather than 1: warning by default would change the output of the packages
this is meant to keep working, and so break their tests.

With this, hapcryst is back to 0 failures and hap's IsAspherical answers
again, both unmodified.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Drop support for polymake before version 4, and switch to "native" JSON file format support

1 participant