Configurable indexing of gated (members-only and paid) Ghost content

[aileen]

Problem

Ghost sites with members-only or paid posts get no search coverage for that content. The Content API returns html: null for gated posts, so they produce no Algolia records at all. Publishers currently have no way to decide whether or how gated content appears in their index.

Discovered while investigating live-smoke evidence: main.ghost.is returns html: null for some posts (there they are empty public posts, but the same API behaviour applies to gated posts). Actions runs 32368632595 and 32372419243 of the Live Ghost content smoke workflow fail schema-drift on those items.

Proposal: an excerpt-first option

A configurable option letting publishers choose how gated Ghost content is indexed, with three levels:

Default — current behaviour. No change for existing users.

Metadata-only mode. A gated post becomes one metadata-only Algolia record: title, custom_excerpt/excerpt, feature image, tags, authors, and url. This leaks nothing beyond what Ghost's own public site already shows for gated posts (title, excerpt, and a call to action). The mechanism largely exists: the fallback record shipped for #220 in #237 already emits exactly this shape for Ghost content with no extraction fragments, and visibility is already an optional projection field in the shipped allowlist, so result UIs can badge or filter gated results today. The main coupling is that #221 and #222 must decide whether html: null maps to the fallback-record path, is skipped, or stays a validation failure — this request argues for making that a publisher-facing choice.

Full-content mode — a much larger decision. Indexing complete gated content requires the Admin API, a far heavier credential in CLI configuration and the Netlify environment, and creates a real exposure path: Algolia indexes are usually queried with a public search key, so highlights and snippets would leak paid prose unless the operator uses secured API keys or a separate protected index. This should not be conflated with the excerpt-level option and needs its own threat model if pursued.

Open questions

1. What does the Content API actually return for a members-only post's computed excerpt and custom_excerpt? main.ghost.is could not answer this — its null-html posts are all public. If the computed excerpt is gated too, the metadata-only record rests on custom_excerpt plus metadata.
2. How the option is expressed: CLI configuration and Netlify environment parity with existing patterns such as contentTypes and ALGOLIA_CONTENT_PROJECTION, and its interaction with the html: null mapping decision in Adopt Ghost content projection in the CLI #221/Adopt Ghost content projection in post webhooks #222.
3. Whether gated pages, not just posts, are in scope.

Relations

• Feeds a future decision under PRD PRD: Deliver the maintained HTML-to-Algolia extraction pipeline #213; not ready to implement as filed.
• Builds on the fallback record mechanism from Release the deep fragmenter record policy #220 (shipped in Add the deep fragmenter record-building interface #237).
• Couples to the html: null handling decision in Adopt Ghost content projection in the CLI #221 and Adopt Ghost content projection in post webhooks #222.
• Adjacent to, but separate from, the pending live-smoke schema amendment for null-html items.