Skip to content

Let one polymake process serve the whole session - #31

Open
fingolfin wants to merge 3 commits into
polymake-jsonfrom
persistent-polymake
Open

Let one polymake process serve the whole session#31
fingolfin wants to merge 3 commits into
polymake-jsonfrom
persistent-polymake

Conversation

@fingolfin

Copy link
Copy Markdown
Member

Stacked on #29.

Where the time actually goes

I claimed earlier that --config-path "" was most of the per-call overhead.
That was read from the polymake source and never measured, and it is wrong:

per call
bare startup, no rules loaded 0.12 s
load a Polytope, --config-path "" 0.85 s
load a Polytope, the user's own config 0.80 s
load a Polytope, a private config we create 0.82 s

Config accounts for ~40 ms. The other ~0.7 s is loading the rules of an
application, and it is paid whatever the config path is. A per-session config
directory -- --config-path user=DIR, the writable form -- buys about 4%.

So the only way to stop paying it is to stop restarting polymake.

What this does

polymake - reads instructions from standard input, so polymaking now keeps
one such process per session and asks it to evaluate each call, driven through
InputOutputLocalProcess (in GAP core since 4.7.8, so no new dependency).

Results still go to a file, exactly as in #29, so the pipe only ever
carries a marker line. That is the design point that makes this simple: there
is no framing protocol to get wrong, and no chance of a large result filling
the pipe buffer and deadlocking.

pm.pl gained two entry points, polymaking_setup and polymaking_eval, and
calls them itself when given arguments. One-shot mode is therefore unchanged,
and stays the fallback whenever the process cannot be started or has died.

Measured

session one process one per call
10 calls 2.6 s 9.8 s 3.8x
40 calls 3.5 s 36.0 s 10.3x
hapcryst's test suite, unmodified 5.1 s 19.1 s 3.7x

Marginal cost of a call: ~29 ms rather than ~870 ms.

Things that had to be got right

  • Repeated preferences. In a long lived process, adding the same rule
    preference again makes polymake warn_print that one is already in effect --
    once per call, into the captured stderr. They are now applied once per
    application.
  • load resolves differently. Under --script a bare load($file) works;
    in shell mode it does not, and User::load is undefined. Fully qualified as
    Polymake::User::load.
  • Closed streams raise. WriteLine on a closed stream errors rather than
    returning fail, so the retry never fired. The whole exchange is now guarded
    and a polymake that has gone away is replaced rather than reported.
  • Lifecycle. The stream cannot survive into another session, so it is
    dropped after restoring a workspace (the same concern as CreatePolymakeObject(); fails the second time when using GAP workspace files #17) and closed at
    exit.
  • Quoting. Filenames, keywords and preference strings are escaped before
    they are interpolated into perl string literals.

Testing

  • Full suite: 0 failures in 5 files.
  • New tst/persistent.tst checks that both modes give identical answers for
    the same object, that the process starts only when wanted, that killing it
    mid-session is recovered from transparently, and the quoting helpers.
  • hapcryst: 0 failures, unmodified, in both modes.

Set PolymakePersistent to false for the old behaviour.

🤖 Generated with Claude Code

Starting polymake costs about 0.85s per call, and measuring where it goes shows
only ~40ms of that is the autoconfiguration that --config-path "" forces to
re-run; the rest is loading the rules of an application, which happens whatever
the configuration. So the way to stop paying it is to stop restarting polymake.

polymake reads instructions from standard input when invoked as `polymake -`,
so polymaking now keeps one such process per session, driven through
InputOutputLocalProcess, and asks it to evaluate each call. Results continue to
go to a file, so the pipe only ever carries a marker line; there is no framing
protocol to get wrong and no risk of a large result filling the pipe buffer.

pm.pl gained two entry points, polymaking_setup and polymaking_eval, and calls
them itself when given arguments, so one-shot mode is unchanged and remains the
fallback whenever the process cannot be started or has died. Preferences are
applied once per application rather than once per call, since in a long lived
process polymake objects to being told the same preference twice.

Measured, with a session that makes 40 calls: 3.5s rather than 36s. The
marginal cost of a call falls from ~870ms to ~29ms. hapcryst's test suite,
unmodified, goes from 19.1s to 5.1s.

The stream does not survive into another session, so it is dropped after
restoring a workspace and closed at exit. Talking to a closed stream raises an
error rather than returning fail, so the exchange is guarded and a polymake
that has gone away is replaced rather than reported.

Set the new PolymakePersistent preference to false to start polymake afresh for
every call.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Aug 17, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.29851% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.03%. Comparing base (702efbb) to head (c18663c).

Files with missing lines Patch % Lines
lib/environment.gi 89.43% 13 Missing ⚠️
Additional details and impacted files
@@                Coverage Diff                @@
##           polymake-json      #31      +/-   ##
=================================================
+ Coverage          83.44%   85.03%   +1.59%     
=================================================
  Files                 11       11              
  Lines                767      889     +122     
=================================================
+ Hits                 640      756     +116     
- Misses               127      133       +6     
Files with missing lines Coverage Δ
lib/userpref.gi 83.57% <100.00%> (+1.40%) ⬆️
lib/environment.gi 89.62% <89.43%> (+0.51%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

fingolfin and others added 2 commits August 17, 2026 09:52
polymake compiles a --script file with its own namespace pragma, which rejects
`our`, so the hash tracking which preferences have been applied blew up with

  reference to an undeclared variable %polymaking_applied

A file scoped lexical works in both modes. Reading the script into the
persistent process happens without that pragma, which is why only the
one-shot path was affected, and only when a rule preference was actually set --
so the CI convex hull backend jobs caught it and everything else passed.

Exercise preferences in both modes from tst/persistent.tst, so the default job
covers this rather than only the backend matrix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The config path and quiet flag are fixed when a polymake process starts, and a
rule preference cannot be withdrawn once applied, so with a persistent process
changing any of them had no effect until something else happened to restart it.
Setting PolymakeQuiet to false mid-session, say, went unnoticed. The settings a
process was started with are now recorded, and it is replaced when they no
longer match.

Also let polymaking_eval reopen the stderr file rather than rely on the handle
polymaking_setup established. Every call unlinks that file before running, and
a persistent process holds it open, so from the second call on GAP would be
reading a path the process no longer writes to. In practice polymake says its
piece once per session and nothing is observably lost -- I could not construct
a case that shows a difference -- but the reopen makes each call start from a
live file rather than depending on that.

For the same reason, polymake announces a credit once per session rather than
once per call, so with a persistent process these are seen once; noted in the
manual and in CHANGES.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@fingolfin
fingolfin force-pushed the persistent-polymake branch from e65f3d5 to c18663c Compare August 17, 2026 08:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant