Skip to content

Experiment: measure full and reduced Windows Perl archive costs - #272

Closed
lamemustafa wants to merge 5 commits into
masterfrom
tapish-codex/windows-perl-inventory-20260909
Closed

lamemustafa wants to merge 5 commits into
masterfrom
tapish-codex/windows-perl-inventory-20260909

Conversation

@lamemustafa

@lamemustafa lamemustafa commented Sep 9, 2026

Copy link
Copy Markdown
Owner

Windows Perl cache extraction is a recurring setup cost. This experiment inventories the actual pinned installation, qualifies a conservative three-directory reduction against fresh native builds, then measures full/reduced archive packing and extraction with the existing GNU tar/zstd commands.

Current archive candidate: c721bc64f403e073ebfd18a018a0319af827e715. This remains a draft experiment, not a production prerequisite change. Standard CI and application code are unchanged. The completed cold-native workflow stage is replaced by an archive-only stage; it does not rerun native tests. Both the inventory and archive jobs explicitly disable production cache saving.

The archive stage preserves the original installation in a sibling backup and runs six alternating full/reduced extraction trials on a disposable Windows runner. It uses the existing GNU tar path and compression flags, records tool/source/image hashes, validates each archived file's size/hash and TAR termination, then verifies every extracted file and Perl identity. It refuses links, traversal, replacement directories, malformed padding and unexpected bytes. Cleanup is limited to an explicitly created extraction destination; original bytes are never overwritten by a replacement. Hard termination can bypass finally, so acceptance requires terminal job success and restoration.json true. Archive payloads stay on the disposable runner; only measurement metadata and bounded logs are uploaded.

Evidence already established:

  • Installed inventory run 34325707801 at 39bf0b2 measured 20,861 files / 1,033,835,215 bytes in the exact pinned cache.
  • Cold native run 34332893760 at 67fa357 passed both full/reduced cases with matching original installation, source hashes and runner image. Artifacts 10097282576 / 10097197040 match API digests. Both rebuilt OpenSSL/SQLite, passed the exact existing encryption/wrong-key test, and restored installation/source hashes. Targets were already cold; positive native timing units prove builds ran. Reduced contents: 15,665 files / 622,626,257 bytes. This historical source qualifies that scoped build/test path, not current-head archive timings or production adoption.
  • Seven mutations of the retained cold-build evidence were rejected by offline validation, including false restoration, source/image mismatch, invalid artifact/native evidence and an unexecuted exact test.

Rectifies #273: the first Windows archive run 34337875700 failed before measuring extraction because Python text-mode writing translated the manifest newline to CRLF. GNU tar treated the carriage return as part of the installation path. The fix writes exact UTF-8 bytes without a trailing newline, matching the existing cache action; a regression checks the literal manifest bytes. The failed run restored installation/source/tool hashes and is retained as non-timing evidence. Independent review of this two-file correction is clean; corrected Windows run 34364606308 now passes all six round trips and full installation/source/tool restoration.

Archive preflight at the current candidate:

  • python3 scripts/measure-perl-archive.test.py: nine parser/filesystem controls pass.
  • python3 scripts/measure-perl-cold.test.py: four reused directory-move controls pass.
  • Actionlint, workflow consistency and PR diff whitespace checks pass.
  • Independent review is clean against all four committed changed-file hashes. Review found a buffered TAR-padding gap before dispatch; block-sized reads, physical end-position checks and explicit zero end-block validation corrected it. Both the original padding control and a real failing zstd frame reject; valid PAX padding and a real local zstd round trip pass.
  • Current Windows archive run34364606308 passed: full/reduced archives265,189,302/162,184,522 bytes (38.842% smaller); extraction full23.250/19.234/22.219s, reduced12.985/12.734/24.860s; medians22.219/12.985s. The last reduced trial is slower than all full trials. All six extracted file sets and Perl identity verified, with full restoration. Artifact10109607090 matches its API digest. Seven mutations of retained evidence reject.
  • Separate normal PR CI34364612141 failed macOS native and Required checks; Windows native and both bundle jobs passed. This draft is not merge-ready; archive success does not supersede the failed required gate.

Measurement limits: packing/extraction commands use warmed local filesystem caches on one runner. Hash checks occur outside measured extraction. Directory metadata can reflect the temporary removals; retained file content is checked exactly. Archive bytes and local extraction times do not measure GitHub transfer, full CI latency or all production consumers. The failed master seed and its warm-comparison/adoption hold remain separate; this experiment performs no crash investigation.

Reuse/deletion: reuses the pinned prerequisite action, inventory, directory-move and bounded-log helpers, and existing runner archive tools. No runtime dependency is added. The completed native workflow stage is replaced rather than repeated. Net experiment addition against the PR base: 867 lines; no production code is deleted. Without this stage, a smaller installed tree could be mistaken for a smaller/faster cache. Project-authored code and controls are recorded in docs/provenance.md under the repository license.

Migration/sync impact: none. Security impact: no Tally, DSC, credential or customer-data behavior change. Rollback removes the branch-only harness; no production cache or data migration is introduced.

  • New commands validate inputs and surface errors.
  • Prior full/reduced cold-native correctness and restoration verified on Windows.
  • Current archive round trips, restoration and scoped timings verified on Windows.
  • Current normal PR required checks pass; macOS native failure remains with its owning task.

@lamemustafa lamemustafa changed the title Experiment: inventory installed pinned Windows Perl Experiment: qualify reduced Windows Perl with fresh native builds Sep 9, 2026
@lamemustafa lamemustafa changed the title Experiment: qualify reduced Windows Perl with fresh native builds Experiment: measure full and reduced Windows Perl archive costs Sep 9, 2026
@lamemustafa lamemustafa added the type:rectify Regression rectification label Sep 9, 2026
@lamemustafa

Copy link
Copy Markdown
Owner Author

Closing this experiment.

It measured full vs reduced Windows Perl archive costs and was never a production change — it remains a draft with no review threads. The measurement it captured is preserved in the branch and in this PR's history; reopen or re-cut it if the packaging question comes back.

Closed as an experiment concluded, not as work rejected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:rectify Regression rectification

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant