Performance Sprint | Graypaper 0.7.2 (Tracking Issue)

[emielsebastiaan]

Performance Sprint | Graypaper 0.7.2 (Tracking Issue)

TODO

• CURRENT BENCHMARK METHODOLOGY FOCUSSES ON M1 WORK
• CONSIDER ADDING BENCHMARK METHODOLOGY FOR M2 WORK (DOOM / FPS)
• REMOVE FLUFFYLABS PUBLIC BENCHMARK REFERENCES (REPO IS ARCHIVED)

Summary

This tracking issue covers the PyJAMaz performance sprint for Graypaper 0.7.2.

The sprint is organized as a sequence of tailored pull requests. Each PR is expected to ship with a versioned release beginning at 0.2.1 and continuing through the final sprint release. Every change should record what changed, why it changed, and how it affected benchmarked performance.

Definition of done:

• all planned sprint PRs are linked and tracked here
• every sprint release has internal benchmark results recorded
• every sprint release has FluffyLabs benchmark references recorded
• notable improvements, regressions, and follow-up work are summarized at the end

Scope

In scope:

• performance-focused changes for Graypaper 0.7.2
• tailored PRs with release-by-release measurement
• internal benchmark tracking
• public benchmark tracking via FluffyLabs

Out of scope:

• unrelated refactors without a performance objective
• feature work not required for the performance sprint
• benchmark claims without recorded evidence

Sprint Status

• Graypaper version: 0.7.2
• Planned release sequence start: 0.2.1
• Current repo app version at time of writing: 0.2.0
• Sprint status: Planned
• Tracking owner: @emielsebastiaan
• Benchmark environment owner: @emielsebastiaan

Sprint Checklist

• Confirm sprint scope and PR sequence
• Ship 0.2.1 with benchmark evidence
• Ship 0.2.2 with benchmark evidence
• Ship 0.2.3 with benchmark evidence
• Ship additional sprint releases as needed
• Run FluffyLabs benchmark for each sprint release
• Summarize cumulative impact and follow-up work

Benchmark Sources

Internal benchmark families:

• Fallback
• Safrole
• Storage Light
• Storage
• Preimages Light
• Preimages
• Fuzzy Light
• Fuzzy

Public benchmark references:

• FluffyLabs Benchmark repository: https://github.com/FluffyLabs/jam-testing
• FluffyLabs Benchmark site: https://fluffylabs.dev/jam-testing/

Release Tracking

Release 0.2.1 / PR #266

• Status: Draft
• PR link: PVMCode caching #266
• Release tag: v0.2.1
• Change description: PVMCode Caching
• Rationale: TODO
• Expected performance hypothesis: TODO
• Risk / regression notes: TODO
• Benchmark artifacts: TODO

Internal Benchmark Results

Benchmark	Runtime (ms)	Relative Enhancement (%)	Notes
Fallback	TODO	TODO	TODO
Safrole	TODO	TODO	TODO
Storage Light	TODO	TODO	TODO
Storage	TODO	TODO	TODO
Preimages Light	TODO	TODO	TODO
Preimages	TODO	TODO	TODO
Fuzzy Light	TODO	TODO	TODO
Fuzzy	TODO	TODO	TODO

FluffyLabs Benchmark Results

• Repository run reference: TODO
• Public result link: TODO
• Summary of public benchmark outcome: TODO
• Comparability notes: TODO

Release Conclusion

• Decision: TODO (keep, rework, follow-up)
• Notes: TODO

Release 0.2.2 / PR #TODO

• Status: Planned
• PR link: TODO
• Release tag: TODO
• Change description: TODO
• Rationale: TODO
• Expected performance hypothesis: TODO
• Risk / regression notes: TODO
• Benchmark artifacts: TODO

Internal Benchmark Results

Benchmark	Runtime (ms)	Relative Enhancement (%)	Notes
Fallback	TODO	TODO	TODO
Safrole	TODO	TODO	TODO
Storage Light	TODO	TODO	TODO
Storage	TODO	TODO	TODO
Preimages Light	TODO	TODO	TODO
Preimages	TODO	TODO	TODO
Fuzzy Light	TODO	TODO	TODO
Fuzzy	TODO	TODO	TODO

FluffyLabs Benchmark Results

• Repository run reference: TODO
• Public result link: TODO
• Summary of public benchmark outcome: TODO
• Comparability notes: TODO

Release Conclusion

• Decision: TODO (keep, rework, follow-up)
• Notes: TODO

Release 0.2.3 / PR #TODO

• Status: Planned
• PR link: TODO
• Release tag: TODO
• Change description: TODO
• Rationale: TODO
• Expected performance hypothesis: TODO
• Risk / regression notes: TODO
• Benchmark artifacts: TODO

Internal Benchmark Results

Benchmark	Runtime (ms)	Relative Enhancement (%)	Notes
Fallback	TODO	TODO	TODO
Safrole	TODO	TODO	TODO
Storage Light	TODO	TODO	TODO
Storage	TODO	TODO	TODO
Preimages Light	TODO	TODO	TODO
Preimages	TODO	TODO	TODO
Fuzzy Light	TODO	TODO	TODO
Fuzzy	TODO	TODO	TODO

FluffyLabs Benchmark Results

• Repository run reference: TODO
• Public result link: TODO
• Summary of public benchmark outcome: TODO
• Comparability notes: TODO

Release Conclusion

• Decision: TODO (keep, rework, follow-up)
• Notes: TODO

Additional Release Template

Copy this section for each additional sprint PR and release after 0.2.3.

Release 0.2.xx / PR #TODO

• Status: Planned
• PR link: TODO
• Release tag: TODO
• Change description: TODO
• Rationale: TODO
• Expected performance hypothesis: TODO
• Risk / regression notes: TODO
• Benchmark artifacts: TODO

Internal Benchmark Results

Benchmark	Runtime (ms)	Relative Enhancement (%)	Notes
Fallback	TODO	TODO	TODO
Safrole	TODO	TODO	TODO
Storage Light	TODO	TODO	TODO
Storage	TODO	TODO	TODO
Preimages Light	TODO	TODO	TODO
Preimages	TODO	TODO	TODO
Fuzzy Light	TODO	TODO	TODO
Fuzzy	TODO	TODO	TODO

FluffyLabs Benchmark Results

• Repository run reference: TODO
• Public result link: TODO
• Summary of public benchmark outcome: TODO
• Comparability notes: TODO

Release Conclusion

• Decision: TODO (keep, rework, follow-up)
• Notes: TODO

Cumulative Sprint Summary

• Final sprint release: TODO
• Total number of sprint PRs: TODO
• Overall result: TODO

Internal Benchmark Summary

Benchmark	Baseline Release	Final Release	Baseline Runtime (ms)	Final Runtime (ms)	Net Relative Enhancement (%)	Notes
Fallback	v0.2.0	TODO	557	TODO	TODO	TODO
Safrole	v0.2.0	TODO	1462	TODO	TODO	TODO
Storage Light	v0.2.0	TODO	3737	TODO	TODO	TODO
Storage	v0.2.0	TODO	4316	TODO	TODO	TODO
Preimages Light	v0.2.0	TODO	3613	TODO	TODO	TODO
Preimages	v0.2.0	TODO	3932	TODO	TODO	TODO
Fuzzy Light	v0.2.0	TODO	8472	TODO	TODO	TODO
Fuzzy	v0.2.0	TODO	20500	TODO	TODO	TODO

Sprint Takeaways

• Largest wins: TODO
• Neutral changes: TODO
• Regressions: TODO
• Follow-up candidates: TODO
• Deferred work: TODO

Notes for Maintainers

• Record timings in ms.
• Record relative enhancement in %.
• If a change helps one benchmark family while hurting another, note that explicitly.
• If benchmark environments differ across releases, document the differences in the notes.
• Do not merge performance claims into the summary without linking them back to the corresponding release entry.