feat: testplane profiler - #1314
Conversation
Canonical ProfilerResultV1 DTO and schema exports for internal consumers.
Internal profiler level and configuration types without enabling the public option.
Bounded top-K storage with parent-safe retention and focused tests.
Bounded streaming moments, reservoirs, percentiles, weighted merge, and focused tests.
Runtime interfaces, strict no-op path, clocks, async activity tracking, and startup probe.
Span lifecycle, ALS context, retention interaction, and child interval union with matching tests.
Aggregate and metric updates plus bounded resource sampling with matching tests.
Fragment deltas, runtime-owned cloning, hostile input validation, and matching tests.
Source-scoped ordering, deduplication, gaps, pending caps, and matching tests.
Synthetic root, parent repair, timing formulas, compact references, and their behavioral tests.
Versioned thresholds, robust outlier detection, and deterministic finding order.
Lifecycle, major-phase, discovery, and unattributed-time analyzers with matching tests.
File, module, and listener analyzers with their positive and negative tests.
Slow-test, hook, and browser-command analyzers with matching tests.
Worker, session, CPU, and event-loop resource analyzers with matching tests.
Descriptor-safe redaction and structural sanitization for every public delivery path.
Serialized timeline and whole-result budgets with explicit truncation fallbacks.
Optional JSON delivery through a unique temporary file and atomic rename.
Grouped ANSI-aware presentation emitted through one logger call with renderer tests.
One-result guard, finalization pipeline, immutable payload, and fail-open deliveries with tests.
Node-compatible listener observation while preserving registration, removal, and emission semantics.
Translate event observer callbacks into profiler envelopes and individual listener spans.
Initialize config observation and a dormant no-op-compatible manager.
Composable CommonJS and ESM module observation shared with selectivity, with regression tests.
Profile browser preparation and command execution with bounded safe attributes and focused tests.
Pool queue, limiter state, acquisition, release, and session lifecycle observation with tests.
Master attempts and worker hook, body, cleanup, and retry spans with execution tests.
Discovery, parsing, module scopes, cache waits, and module-graph collection with tests.
Process messages, master ingestion, flush ordering, late fragments, and registry tests.
Worker clock handshake, facade RPC, fragment batching, and worker-side tests.
Propagate the optional runtime through master and worker runner boundaries.
Bounded runnable, resource, and long-task telemetry relayed across the browser environment.
Add top-level operations, teardown, signals, and exit-code preservation.
Early CLI bootstrap and command wrappers with preserved success and failure exit semantics.
Enable public config, result delivery, exports, and end-to-end tests.
commit: |
✅ Testplane E2E run succeed
|
✅ Testplane browser-env run succeed
|
| configuredSessionsPerBrowser?: Record<string, number>; | ||
| } | ||
|
|
||
| export interface ProfilerResultV1 { |
There was a problem hiding this comment.
This object is emitted as event and saved to file
| collector: bucket, | ||
| seen: state.seen, | ||
| retained, | ||
| rule: `top ${RETENTION_POLICY_V1.operationLimits[bucket]} by wall time; keep ancestors of retained operations`, |
There was a problem hiding this comment.
Rules, reasons, evidence, confidence and such are saved as human-readable values so we could display clear strings in profiler UI with at least as possible logic
| @@ -0,0 +1,49 @@ | |||
| import type { EnabledProfilerLevel } from "../schema"; | |||
|
|
|||
| export const RETENTION_POLICY_V1 = Object.freeze({ | |||
There was a problem hiding this comment.
Object configures "how much slowest objects of each type we should keep"
| evicted?: T; | ||
| } | ||
|
|
||
| export class TopK<T> { |
There was a problem hiding this comment.
classic min-heap topK implementation on array
The least element is stored at the root
| private _max = -Infinity; | ||
| private _mean = 0; | ||
| private _m2 = 0; | ||
| private readonly _reservoir: number[] = []; |
|
|
||
| this._socket.on(BrowserEventNames.runBrowserCommand, this._handleRunBrowserCommand(browser)); | ||
| this._socket.on(BrowserEventNames.profilerFragment, payload => { | ||
| this._profiler.recordMeasurement("browser.runnable", payload.wallMs, { |
There was a problem hiding this comment.
Recording somewhat same values on browser env, as we do on node.js runtime, but it lacks:
- detailed hook spans
- test body
- commands
- event loop utilization/delay
- CPU
We also have just one browser runnable here, which would just help to decide between "slow test" and "slow transport"
There was a problem hiding this comment.
After testplane profiler finalization, process might already have error code, but then get to another process.exit, which messes up exit code.
In this module we calculate origin exit code
There was a problem hiding this comment.
In "profile CLI commands" we wrap other CLI commands to same profiler flow
| }), | ||
| }), | ||
|
|
||
| profiler: section({ |
There was a problem hiding this comment.
Describe profiler root section
| input: ".testplane/failed.json", | ||
| }, | ||
| profiler: { | ||
| level: 0, |
There was a problem hiding this comment.
Disabled by default in order to not waste extra CPU cycles in CI
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 35bab54a27
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| private async _deliverResult(result: Readonly<ProfilerResultV1>): Promise<void> { | ||
| try { | ||
| printProfilerResult(result, this._console); |
There was a problem hiding this comment.
When profiler.level is enabled for data-producing CLI commands such as config, list-browsers, or list-tests, those commands still write their JSON/plain result to stdout inside the profiled action, and this unconditional finalization step appends the human-readable profiler report to the same stream. That makes otherwise successful command output invalid for consumers that parse stdout as JSON; route the profiler summary to stderr or suppress it for these commands unless explicitly requested.
Useful? React with 👍 / 👎.
1 participant
No description provided.