Skip to content

feat(beholder): capture console messages and page errors#925

Merged
YusukeHirao merged 1 commit into
devfrom
worktree-giggly-sniffing-panda
Jul 22, 2026
Merged

feat(beholder): capture console messages and page errors#925
YusukeHirao merged 1 commit into
devfrom
worktree-giggly-sniffing-panda

Conversation

@YusukeHirao

Copy link
Copy Markdown
Member

Summary

  • Capture console.log/warn/error/etc. and uncaught exceptions / unhandled promise rejections during page scraping, for internal pages only (isExternal: false), via page.on('console') / page.on('pageerror').
  • Return them as ScrapeResult.consoleLogs (ConsoleLogEntry[]), mirroring the existing resources/pageUrl pattern rather than embedding into PageData — matches how sub-resources are already collected as a page-scoped array, not a per-page-data field.
  • #fetchData's navigation/extraction body is now wrapped in try { ... } finally { await Promise.all(pendingConsoleWork); }, so async jsonValue() argument resolution is guaranteed to settle before the function returns or throws (previously-considered per-return-statement awaits would have missed the two thrown-error exit paths and let a @retryable retry contaminate the array with a stale attempt's late resolutions).
  • ConsoleLogEntry is exported from the package's public entry point alongside ResourceEntry/NetworkLog.

Scope: @d-zero/beholder only. This is a producer-side addition (ScrapeResult gains a new required field); no consumer in this repo constructs ScrapeResult literals, so nothing else in tools needed updating. A downstream consumer (nitpicker, separate repo) does construct ScrapeResult-shaped literals in a few places and will need a small follow-up there — tracked separately, out of scope for this PR.

Test plan

  • yarn build (via NX_WORKSPACE_ROOT_PATH) — no type errors
  • yarn lint — 0 errors (pre-existing unrelated warnings only)
  • yarn test — 113 files / 1612 tests pass, including new to-console-log-entry.spec.ts / to-page-error-entry.spec.ts covering: per-type propagation, mixed success/failure args() resolution, dispose() invocation, location presence/absence, and pageerror for both Error and non-Error throw values
  • Note: scraper.ts's own listener wiring (onConsole/onPageError, the try/finally flush) has no direct unit test — Vitest's esbuild-based transform cannot parse this file due to the pre-existing @retryable decorator on a private method (#fetchData); confirmed by reproducing SyntaxError: Private field '#fetchData' must be declared in an enclosing class when attempting to add a spec file. This is a pre-existing toolchain limitation (no spec file has ever imported scraper.ts), not introduced by this change.

Collect console.log/warn/error/etc. and uncaught exceptions /
unhandled promise rejections via page.on('console')/('pageerror'),
for internal pages only. Returned as ScrapeResult.consoleLogs
(ConsoleLogEntry[]), mirroring the existing resources array pattern
rather than embedding into PageData.
@YusukeHirao
YusukeHirao requested a review from yusasa16 as a code owner July 22, 2026 07:25
@YusukeHirao
YusukeHirao merged commit 0f046c9 into dev Jul 22, 2026
6 checks passed
@YusukeHirao
YusukeHirao deleted the worktree-giggly-sniffing-panda branch July 22, 2026 07:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant