feat(migrate): sample more pages (incl. institutional) + report organized per page - #214
Merged
Conversation
…ized per page migrate captured only home + 1 PLP + 1 PDP. Now it samples extra pages per kind from the classified sitemap and organizes the report by page. - **`--sample <spec>`** (default `plp=2,pdp=2,other=3,search=1`, capped at 15 pages): samples pages of each kind from the sitemap. `other` = institutional; institutional URLs (about/contact/policies/…) are preferred over deep-category pages that also fall through to `other`. - **Report organized per page**: index.md / index.html / report.html now show a "Global components" section (captured once) + one section per captured page (home / PLP / PDP / institutional), so a human sees everything separated by page in a single report. - **`category-auto`** no longer lands on an institutional page — institutional keywords are filtered out of the PLP pick (issue #200). Pure helpers `parseSample`/`pickSpread`/`sampleFromSitemap` unit-tested; PLP institutional-filter test added. Full suite green (1107), tsc + biome clean, build OK. Live electrolux.com.ec: 3 → 10 pages (home + 3 PLP + 3 PDP + 3 other), report split per page. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…just home The block tree + content was read only on the home page, so PLP/PDP/institutional content was missing. It's now read on EVERY captured page and merged by treePath (dedupe), with content-image download accumulated across pages under a global budget (200). Block reading + content download moved from Phase 1 into capturePages (Phase 3); blocks.json + content-assets.json are written after capture and cached in capture.json for resume. Live electrolux.com.ec (7 pages): blocks.json 509 → 5242 blocks (4051 with CMS content), covering store.home + store.product + store.search/category; 173 content images. Full suite green (1107), tsc + biome clean, build OK. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
From your point that it only captured home/PLP/PDP, and that you'll need the institutional pages — with a single report separated by page.
Changes
--sample <spec>(defaultplp=2,pdp=2,other=3,search=1, capped at 15 pages): samples extra pages per kind from the classified sitemap.other= institutional; institutional URLs are preferred (about/contact/policies/…) over deep-category pages that also fall through toother.index.md/index.html/report.htmlnow show a Global components section (captured once) + one section per captured page (home / PLP / PDP / institutional). One report, everything separated by page.category-autono longer lands on an institutional page (institutional keywords filtered from the PLP pick — issue migrate: platform-aware PDP discovery for non-VTEX stores #200).Verification
Unit tests for
parseSample/pickSpread/sampleFromSitemap+ the PLP institutional filter. Full suite green (1107), tsc + biome clean, build OK. Liveelectrolux.com.ec: 3 → 10 pages (home + 3 PLP + 3 PDP + 3 other), report split per page.Follow-up (noted)
Per-page VTEX block-tree + content (institutional pages' props) — the block tree is currently read on home only; reading it per page is the natural next step to link component ↔ block ↔ content across all pages.
Off
main.🤖 Generated with Claude Code
Summary by cubic
Samples more pages by kind (including institutional) and organizes the migrate report by page. Also reads the VTEX IO block tree and CMS content on every captured page and merges them, so content is not limited to home anymore.
--sample <spec>(defaultplp=2,pdp=2,other=3,search=1, capped at 15). Samples per kind from the classified sitemap; skips home and already-resolved URLs; forotherwe prefer institutional URLs (about/contact/policies/…).index.md/index.html/report.htmlnow show a Global components section plus one section per captured page, with stable component folder mapping by role+selector.blocks.jsonandcontent-assets.jsonafter capture and caches them incapture.json.category-autolands on a real PLP.parseSample/pickSpread/sampleFromSitemapand the PLP filter;docs/migrate.mddocuments per-page VTEX capture and--sample. No breaking changes.Written for commit d1e0e35. Summary will update on new commits.