Skip to content

feat(migrate): sample more pages (incl. institutional) + report organized per page - #214

Merged
JonasJesus42 merged 2 commits into
mainfrom
migrate-page-sampling
Aug 17, 2026
Merged

feat(migrate): sample more pages (incl. institutional) + report organized per page#214
JonasJesus42 merged 2 commits into
mainfrom
migrate-page-sampling

Conversation

@JonasJesus42

@JonasJesus42 JonasJesus42 commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

From your point that it only captured home/PLP/PDP, and that you'll need the institutional pages — with a single report separated by page.

Changes

  • --sample <spec> (default plp=2,pdp=2,other=3,search=1, capped at 15 pages): samples extra pages per kind from the classified sitemap. other = institutional; institutional URLs are preferred (about/contact/policies/…) over deep-category pages that also fall through to other.
  • Report organized per pageindex.md / index.html / report.html now show a Global components section (captured once) + one section per captured page (home / PLP / PDP / institutional). One report, everything separated by page.
  • category-auto no longer lands on an institutional page (institutional keywords filtered from the PLP pick — issue migrate: platform-aware PDP discovery for non-VTEX stores #200).

Verification

Unit tests for parseSample/pickSpread/sampleFromSitemap + the PLP institutional filter. Full suite green (1107), tsc + biome clean, build OK. Live electrolux.com.ec: 3 → 10 pages (home + 3 PLP + 3 PDP + 3 other), report split per page.

Follow-up (noted)

Per-page VTEX block-tree + content (institutional pages' props) — the block tree is currently read on home only; reading it per page is the natural next step to link component ↔ block ↔ content across all pages.

Off main.

🤖 Generated with Claude Code


Summary by cubic

Samples more pages by kind (including institutional) and organizes the migrate report by page. Also reads the VTEX IO block tree and CMS content on every captured page and merges them, so content is not limited to home anymore.

  • CLI: adds --sample <spec> (default plp=2,pdp=2,other=3,search=1, capped at 15). Samples per kind from the classified sitemap; skips home and already-resolved URLs; for other we prefer institutional URLs (about/contact/policies/…).
  • Report: index.md/index.html/report.html now show a Global components section plus one section per captured page, with stable component folder mapping by role+selector.
  • VTEX IO: reads the block tree + props on every captured page and merges by treePath; downloads and dedupes content images across pages under a 200-image budget; persists blocks.json and content-assets.json after capture and caches them in capture.json.
  • PLP pick: filters institutional keywords so category-auto lands on a real PLP.
  • Tests/docs: unit tests for parseSample/pickSpread/sampleFromSitemap and the PLP filter; docs/migrate.md documents per-page VTEX capture and --sample. No breaking changes.

Written for commit d1e0e35. Summary will update on new commits.

Review in cubic

JonasJesus42 and others added 2 commits August 17, 2026 19:46
…ized per page

migrate captured only home + 1 PLP + 1 PDP. Now it samples extra pages per kind
from the classified sitemap and organizes the report by page.

- **`--sample <spec>`** (default `plp=2,pdp=2,other=3,search=1`, capped at 15
  pages): samples pages of each kind from the sitemap. `other` = institutional;
  institutional URLs (about/contact/policies/…) are preferred over deep-category
  pages that also fall through to `other`.
- **Report organized per page**: index.md / index.html / report.html now show a
  "Global components" section (captured once) + one section per captured page
  (home / PLP / PDP / institutional), so a human sees everything separated by
  page in a single report.
- **`category-auto`** no longer lands on an institutional page — institutional
  keywords are filtered out of the PLP pick (issue #200).

Pure helpers `parseSample`/`pickSpread`/`sampleFromSitemap` unit-tested; PLP
institutional-filter test added. Full suite green (1107), tsc + biome clean,
build OK. Live electrolux.com.ec: 3 → 10 pages (home + 3 PLP + 3 PDP + 3 other),
report split per page.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…just home

The block tree + content was read only on the home page, so PLP/PDP/institutional
content was missing. It's now read on EVERY captured page and merged by treePath
(dedupe), with content-image download accumulated across pages under a global
budget (200). Block reading + content download moved from Phase 1 into
capturePages (Phase 3); blocks.json + content-assets.json are written after
capture and cached in capture.json for resume.

Live electrolux.com.ec (7 pages): blocks.json 509 → 5242 blocks (4051 with CMS
content), covering store.home + store.product + store.search/category; 173
content images. Full suite green (1107), tsc + biome clean, build OK.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@JonasJesus42
JonasJesus42 merged commit 486f081 into main Aug 17, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant