Conversation
… API) Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
Queries always pass through (no cached query path), ids normalized on every lookup, paranoid-scope rules, Notion-stamp-based staleness with a minute-resolution recheck, wall-clock deadlines, class-level pacer, shared advisory lock, daily reconcile with trash disambiguation. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
Paranoid-safe upserts, nil-safe created_at, HTTParty header arrays, deadline checks after progress, reconcile rewritten, page_title backfill, Lead.all on the new scope, sync_database empty guard. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
…cer and Retry-After handling Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
Deleting it defeated test_helper's process-wide default for every later test. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
…rces/databases; scopes on database_id Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
…or tables Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
…d comment Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
… replace_level Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
… soft deletes Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
…ne and minute-resolution recheck Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
…Ts, error pass-through Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
…ve fill On a hit, serve the level from NotionBlock rows with local cursor pagination. On a miss or stale marker, proxy the caller's single cursor page live (page_size 100) and store it via Mirror.replace_level / store_blocks, completing the level only on an uncursored, non-paginated response. Unknown parents pass through live without storing. The brief's test setup called Mirror.upsert_page with an unbraced hash-rocket literal, which Ruby 3.1 parses as an empty positional-arg keyword splat against upsert_page's (obj, fetched_at:) signature and raises ArgumentError before the controller is ever exercised; wrapped it in explicit braces so it's a single positional hash, matching every other caller in the codebase. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
… and Fetched-At header Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
Implements query_data_source, search, create_page, update_page, append_block_children, update_block, and delete_block in Api::Notion::ProxyController. Queries and search always hit Notion live and warm the mirror with returned pages/data sources; page writes upsert the response; block writes mark the owning page's blocks_stale_at. Also rescues ActionDispatch::Http::Parameters::ParseError so a malformed JSON body with Content-Type: application/json returns a Notion-style 400 instead of falling through to the generic 500 handler. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
…line-bounded tree refresh Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
…sh/access-lost disambiguation Stacks::Notion::Backfill does a one-off resumable full load (feed, then schemas for unfetched data sources, then seed trees), recording phase progress in SourceSync(:notion_backfill).cursor so a killed run resumes. Stacks::Notion::Reconcile does a daily full-feed walk and disambiguates unseen pages/data sources (trashed vs access-lost) within a shared request budget, and reports sweep drift. Both share Sweep's advisory lock. Adds backfill/reconcile rake tasks under stacks:notion. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
… first-run feed, per-walk deadline Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
…ixture Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
…ols over the mirror Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
…dget test, rake coverage Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
…fy_parity Proves the proxy at /api/notion/v1/* mirrors Notion one-to-one: for a fixed set of real ids, the proxy's cold and warm responses equal Notion's own (ignoring request_id/request_status), block-children levels match cursor by cursor, and live query/search results match by id. Ran live against api.notion.com: 9/9 checks passed, no proxy or mirror fixes needed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
…tools Adds docs/notion-mirror-deploy.md covering env vars, deploy order, and behavioural deviations from a pure Notion mirror (cache headers, blocks children page_size, cursor restart on mid-pagination staleness, backfill re-run semantics, sweep access-lost handling, first-sweep feed cap, and Sentry reporting for 5xx/401). Also updates McpEndpointTest's tools/list assertion to include the three notion-fetch/notion-search/notion-query-data-sources tools registered by this branch — the full suite caught this as a real regression. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
…rd-deletes what it wrote delete_all on a paranoid relation only soft-deletes; leftover rows collided on the unique notion_id index in later test runs. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
Six review fixes, all about a web request never hanging and never leaking a non-Notion response out of the proxy: - TreeFetcher's deadline now bites BETWEEN cursor pages of a level, not only between levels. One 1,000-block level is 10 paced requests, which blew past notion-fetch's 6s deadline and rack-timeout's 15s. fetch_level returns [blocks, requests, complete]; an incomplete level keeps the pages we paid for but stamps no marker, so the next walk resumes it. - Stacks::Notion gets default_timeout 10 and wraps Net::/Errno::/SocketError/ HTTParty transport failures as one Notion-shaped 503 (idempotent GETs still get their one retry first). - A saturated pacer now fails fast: Stacks::Notion.new(max_wait: 5) refuses to claim a slot further out than 5s and raises a synthetic Notion-shaped 429 with Retry-After instead of pinning a Puma thread. The synthetic 429 is flagged so the retry path does not sleep out its own backlog. The proxy and the three MCP tools opt in. - The proxy declares its own rescue_from StandardError (first, so the RequestError/ParseError handlers still win) and renders a Notion-shaped 500, and parses page_size defensively: page_size[]=1 is an Array with no #to_i. - A page row with page_fetched_at nil predates this branch, was written by the old sync with icon/cover/file payloads stripped, and is therefore NOT one-to-one: serve it as a miss so the refetch rewrites it. - The live parity test's ENV["NOTION_RPS"] ||= "2" was dead against test_helper's "1000"; set and restore it explicitly. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
- Deploy doc step 1 gains the scope-switch verification (Leads ~1018, HOM ~94, and what a drop to 0 after the first sync_notion means). - Deploy doc step 4 gains the reason stacksbot's read switch must wait for phase == "done": pre-mirror rows serve as misses until the backfill's feed walk rewrites them. - The spec's "Deviations from one-to-one" list is now actually all of them (8-16): sweep caps and tiers, access_lost on refresh, reconcile's unseen detection, page_size=100 on a miss, cursor shape by cache state, legacy rows, the Notion-shaped 500, and the synthetic 429. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacks becomes a one-to-one caching reverse proxy of Notion's REST API so stacksbot reads Notion through Stacks instead of spending Notion's per-connection and per-workspace request budget directly.
Stacks::Notionupgraded in place: Notion-Version2026-03-11, HTTParty throughout, class-level pacer (NOTION_RPS), Retry-After handling, HTTP timeout, transport errors normalized to a Notion-shaped 503, and a pacer fail-fast (synthetic 429 withRetry-After) so a cold burst cannot pin Puma threads.notion_pagesextended (database_id,data_source_id, freshness columns), newnotion_blocks,notion_data_sources,notion_databases.NotionPage.lead/.human_operating_manualnow select ondatabase_id(the new API version reports rows underdata_source_id);sync_databasereconciles ondatabase_idwith soft deletes; deadstatus_historyremoved./api/notion/v1/*proxy (X-Api-Key, same paths/bodies/responses as Notion): cache hits for pages, blocks, data sources, databases, and block-children levels; queries, searches, and writes always pass through (paced) and warm/invalidate the cache; Notion errors pass through unchanged;X-Stacks-Cache/X-Stacks-Fetched-Atheaders.notion-fetch(markdown via the one renderer),notion-search,notion-query-data-sources.stacks:notion:sweepevery 10 min (search feed → properties, deadline-bounded tree refresh, minute-resolution recheck),stacks:notion:backfill(one-off, resumable),stacks:notion:reconcile(daily trash / access-lost disambiguation). One shared advisory lock.NOTION_LIVE=1 bin/rails test test/live/notion_parity_test.rb, plusstacks:notion:verify_parity): proxy responses compared with Notion's own for pages, a database row, database, data source, every block-children cursor page cold then warm, a filtered query, a search, and a search-seeded row. Passed 9/9 on four separate live runs; output indocs/notion-mirror-parity-2026-09-10.txt.Spec:
docs/superpowers/specs/2026-09-10-notion-mirror-design.md(with the full deviations list). Plan:docs/superpowers/plans/2026-09-10-notion-mirror.md. Deploy:docs/notion-mirror-deploy.md— migrate before release, add the two Scheduler jobs, run the backfill tophase == "done", verify the Lead/HOM scope counts, run parity against prod, then switch stacksbot's reads.Phase 3 (webhook receiver, corpus projection into
documents) is a separate plan.Test plan
bin/rails test --exclude "/EtlRakeTest/"→ 1327 runs, 4123 assertions, only the knownAdminUserTest20:00–24:00 ET time-zone flake failingNotionPage.lead.count≈ 1018 andNotionPage.human_operating_manual.count≈ 94 before and after the firststacks:sync_notion🤖 Generated with Claude Code
https://claude.ai/code/session_01LWRCs8SgcVwVQBNTQEP59V