[Release notes] Product scoped amend and rebuilt registry

docs/cli/changelog/cmd-backfill.md

@@ -8,7 +8,7 @@ For each product in scope the command:
 2. Parses each `## {version}` section into typed entries. Unrecognized `### …` subsections whose body is bullets become `Other`-typed entries; prose subsections are preserved in the bundle description.
 3. Writes bundle YAML to `{output}/{product}/changelog/bundles/{version}.yaml`.
 4. Writes per-entry YAML files: `{pr}.yaml` for entries that reference a pull request, `note-{slug}.yaml` for entries with no PR reference.
-5. Writes `notes-{target}.json` per version listing all `note-*.yaml` files for that target (the notes registry).
+5. Writes `notes-{target}.json` per version listing all `note-*.yaml` files for that target (the notes registry). This is a **local** backfill artifact, not the CDN dual-write layout (`notes-{product}-{version}.json` plus legacy `notes-{version}.json`).
 6. Prints a per-product report including entry counts and no-PR rates.
 
 The primary use case is measuring how much published release-notes content cannot be traced back to a PR — determining how much historical content needs the PR-less "note" format from [docs-eng-team#789](https://github.com/elastic/docs-eng-team/issues/789).

docs/cli/changelog/cmd-note.md

@@ -59,9 +59,14 @@ products:
 ## After creation
 
 Upload is the same as for other changelog YAML files.
-An index at `changelog/{org}/{repo}/notes-{version}.json` lists every changelog "note" file that applies to each version.
+The scrubber writes two indexes per note:
 
-If the release bundle for that product and version or date has already shipped when you upload, the scrubber generates an amend file so the changelog reaches published docs without a manual rerun.
+- `changelog/{org}/{repo}/notes-{product}-{version}.json` — notes for that product and version. `changelog bundle` reads this key first.
+- `changelog/{org}/{repo}/notes-{version}.json` — the union of notes for that version across products, kept for older CLI pins.
+
+`{changelog}` does not read these indexes. It loads published bundle YAML (and amend sidecars listed in `bundle/{product}/registry.json`).
+
+If the release bundle for that product and version or date has already shipped when you upload, the scrubber generates an amend sidecar for **that product**, then rebuilds that product's `bundle/{product}/registry.json` so `{changelog}` `:cdn:` pages pick it up. Other products that share the version are left alone.
 
 If there is no existing or planned bundle for that product and version or date, you can create a bundle from a path list that contains all the relevant changelogs. Refer to [Bundle by file paths](/cli/changelog/bundle.md#changelog-bundle-files).
 

docs/development/changelog-bundle-registry.md

@@ -71,13 +71,18 @@ narrowed reconciliation to the bundle tree):
 - **Amend-notes sidecars** — `bundle/{product}/{parent}.amend-notes.yaml`, also **public bucket
   only**, authored by the scrubber Lambda's `NoteAmendReconciler`. When a note is uploaded after
   its release bundle has already shipped, the reconciler generates one aggregate sidecar per
-  published bundle that lists all such late notes. The Lambda rebuilds it from current state on
-  every reconcile, so redelivered events never produce duplicate amends. `{changelog}` `:cdn:` and
-  `changelog render` merge this sidecar into the parent the same way as numbered `.amend-{N}`
-  files, after those numbered amends. The `.amend-notes` suffix is **reserved** — do not create
-  files with that suffix manually; see [](/cli/changelog/bundle-amend.md).
-- **Notes index** — `changelog/{org}/{repo}/notes-{version}.json`, one per version, **public
-  bucket only**, produced by the scrubber Lambda's `NotesIndexReconciler`. See
+  published bundle that lists all such late notes **for that product**. The Lambda rebuilds it
+  from current state on every reconcile, so redelivered events never produce duplicate amends.
+  After a write, skip-unchanged, or delete of that sidecar, the same pass rebuilds
+  `bundle/{product}/registry.json` and the bundle shallow map so `{changelog}` `:cdn:` can
+  discover it. Other products at the same version are not walked and their sidecars are not
+  deleted. `{changelog}` `:cdn:` and `changelog render` merge this sidecar into the parent the
+  same way as numbered `.amend-{N}` files, after those numbered amends. The `.amend-notes`
+  suffix is **reserved** — do not create files with that suffix manually; see
+  [](/cli/changelog/bundle-amend.md).
+- **Notes index** — `changelog/{org}/{repo}/notes-{product}-{version}.json` (product-scoped) plus a dual-written
+  legacy `changelog/{org}/{repo}/notes-{version}.json` (version union), **public bucket only**, produced by
+  the scrubber Lambda's `NotesIndexReconciler`. See
   [Notes-index format](#notes-index-format) below.
 - **Changelog-entry index** — `changelog/{org}/{repo}/{branch}/registry.json`, a **legacy
   client-authored pass-through**: the current `changelog upload` never writes one, but manifests
@@ -123,32 +128,50 @@ zero-bundle state. The reconciler deliberately restores the former.
 
 ## Notes-index format [notes-index-format]
 
-For each release version that has at least one note, the scrubber Lambda writes a notes index at
-`changelog/{org}/{repo}/notes-{version}.json`. Its schema (`schema_version: 1`):
+For each product and release version that has at least one note, the scrubber Lambda writes a
+product-scoped notes index at `changelog/{org}/{repo}/notes-{product}-{version}.json`. It also
+dual-writes a legacy version-union index at `changelog/{org}/{repo}/notes-{version}.json` so
+older `changelog bundle` clients that still GET the version-only key keep working. The product
+and version segments are concatenated into the new filename; consumers must not parse that slug
+apart.
+
+`changelog bundle` GETs the product-scoped key first and falls back to the version-only key
+only on HTTP 404. An empty product-scoped index does not fall back. `{changelog}` does not read
+these indexes; it loads bundle YAML from the product registry.
+
+Product-scoped schema (`schema_version: 1`):
 
 ```json
 {
   "schema_version": 1,
+  "product": "elasticsearch",
+  "version": "9.3.0",
   "notes": [
     { "path": "main/note-esql-oom.yml", "bundle_seq": 2 },
     { "path": "main/note-cve-2026-1234.yml", "bundle_seq": 1 }
   ]
 }
 ```
 
+The legacy `notes-{version}.json` body omits `product` and `version` and lists the union of note
+paths for that version across products.
+
 | Field | Meaning |
 |---|---|
 | `schema_version` | Schema version. Currently `1`. |
+| `product` / `version` | Set on product-scoped indexes; omitted from the legacy version-union body. |
 | `notes[].path` | Pool-relative path of the note within `changelog/{org}/{repo}/`. The leading segment before the first `/` is the branch. |
 | `notes[].bundle_seq` | Derived reporting field: `0` = no bundle published for this version yet, `1` = note shipped in the original bundle, `2` = note carried by the Lambda-generated `.amend-notes` sidecar. |
 
 `bundle_seq` is derived — it is never authored and never a latch. The Lambda recomputes it on every
 reconcile by comparing the notes index against the set of entries in the published bundle and its
 amend sidecars.
 
-A 404 on a notes index means "no notes published for this version". An empty `notes` array never
-appears — the index is deleted rather than emptied, following the same
-[absent ≠ empty](#absent-empty) rule as the bundle registry.
+A 404 on both the product-scoped index and the legacy version-union index means "no notes
+published for this product and version". An empty `notes` array never appears on a successfully
+reconciled index — the index is deleted rather than emptied, following the same
+[absent ≠ empty](#absent-empty) rule as the bundle registry. Until older clients stop reading
+`notes-{version}.json`, the Lambda keeps that key while any note still declares the version.
 
 ## Shallow per-tree change maps [shallow-maps]
 
@@ -298,6 +321,12 @@ entry still missing after its retry budget fails the bundle (an incomplete relea
 ship silently). `CdnChangelogEntryFetcher` reuses a shared `HttpClient` in production and disposes an
 owned client only when a test injects a handler, mirroring `CdnChangelogFetcher`.
 
+When CDN sourcing is on, the same fetcher also merges changelog notes for each concrete
+`--output-products` pair: it GETs `changelog/{org}/{repo}/notes-{product}-{version}.json` first
+and falls back to `notes-{version}.json` only on HTTP 404. See
+[Notes-index format](#notes-index-format). Checksum dedup keeps a single copy if two products at
+the same version still share the legacy index.
+
 ## Consumer: `{changelog}` directive `cdn:` mode (implemented)
 
 ### Syntax

src/Elastic.Documentation.Configuration/ReleaseNotes/CdnChangelogEntryFetcher.cs

@@ -487,23 +487,18 @@ private static Uri CombineSegments(Uri baseUri, IReadOnlyList<string> segments)
 	}
 
 	/// <summary>
-	/// <summary>
-	/// Fetches all <c>note-*.yml</c> entries for <paramref name="org"/>/<paramref name="repo"/> at
-	/// <paramref name="version"/> from the CDN. Reads the <c>notes-{version}.json</c> index to enumerate
-	/// the pool-relative note paths; a missing index means no notes (not an error). A listed note that
-	/// cannot be fetched is a hard error — the index is an authoritative promise that the note exists.
+	/// Fetches all <c>note-*.yml</c> entries for <paramref name="org"/>/<paramref name="repo"/> and
+	/// <paramref name="product"/> at <paramref name="version"/> from the CDN. Reads
+	/// <c>notes-{product}-{version}.json</c> first; on HTTP 404 only, falls back to the legacy
+	/// <c>notes-{version}.json</c>. A missing index (both 404) means no notes (not an error).
+	/// An empty product-scoped index does not fall back. A listed note that cannot be fetched
+	/// is a hard error — the index is an authoritative promise that the note exists.
 	/// </summary>
-	/// <param name="baseUri">CDN base URI.</param>
-	/// <param name="org">Repository org (e.g. <c>elastic</c>).</param>
-	/// <param name="repo">Repository name (e.g. <c>kibana</c>).</param>
-	/// <param name="version">Release version string (e.g. <c>9.0.0</c>).</param>
-	/// <param name="emitError">Called once per hard error; caller decides how to surface it.</param>
-	/// <param name="ctx">Cancellation token.</param>
-	/// <returns>The fetched note entries, keyed by pool-relative path (<c>main/note-foo.yml</c>).</returns>
 	public async Task<IReadOnlyList<CdnChangelogEntry>> FetchNotesAsync(
 		Uri baseUri,
 		string org,
 		string repo,
+		string product,
 		string version,
 		Action<string> emitError,
 		Cancel ctx
@@ -515,31 +510,87 @@ Cancel ctx
 			return [];
 		}
 
-		var indexUri = CombineSegments(baseUri, ["changelog", org, repo, $"notes-{version}.json"]);
-		NotesIndex? index;
+		if (!ChangelogKeys.IsValidProduct(product))
+		{
+			emitError($"Invalid product '{product}' for notes fetch: must be a non-empty ASCII product segment.");
+			return [];
+		}
+
+		var productIndexUri = CombineSegments(baseUri, ["changelog", org, repo, $"notes-{product}-{version}.json"]);
+		var (productMissing, productIndex) = await TryLoadNotesIndexAsync(
+			productIndexUri,
+			org,
+			repo,
+			version,
+			emitError,
+			ctx
+		).ConfigureAwait(false);
+		if (productIndex is not null)
+			return await FetchListedNotesAsync(baseUri, org, repo, version, productIndex, emitError, ctx).ConfigureAwait(false);
+		if (!productMissing)
+			return [];
+
+		var legacyIndexUri = CombineSegments(baseUri, ["changelog", org, repo, $"notes-{version}.json"]);
+		var (legacyMissing, legacyIndex) = await TryLoadNotesIndexAsync(legacyIndexUri, org, repo, version, emitError, ctx).ConfigureAwait(
+			false
+		);
+		if (legacyIndex is not null)
+			return await FetchListedNotesAsync(baseUri, org, repo, version, legacyIndex, emitError, ctx).ConfigureAwait(false);
+		if (legacyMissing)
+		{
+			_logger.LogDebug(
+				"Notes index for {Org}/{Repo}/{Product}@{Version} not found at {ProductUri} or {LegacyUri}; no notes to bundle",
+				org,
+				repo,
+				product,
+				version,
+				productIndexUri,
+				legacyIndexUri
+			);
+		}
+		return [];
+	}
+
+	/// <summary>
+	/// Loads a notes index. Returns <c>missing: true</c> only on HTTP 404. A present but empty
+	/// or unparseable body is <c>missing: false</c> with a null index so callers do not fall back.
+	/// </summary>
+	private async Task<(bool Missing, NotesIndex? Index)> TryLoadNotesIndexAsync(
+		Uri indexUri,
+		string org,
+		string repo,
+		string version,
+		Action<string> emitError,
+		Cancel ctx
+	)
+	{
 		try
 		{
 			var (notFound, content) = await FetchTextOrNotFoundAsync(indexUri, 1, ctx).ConfigureAwait(false);
 			if (notFound)
-			{
-				_logger.LogDebug(
-					"Notes index for {Org}/{Repo}@{Version} not found at {Uri}; no notes to bundle",
-					org,
-					repo,
-					version,
-					indexUri
-				);
-				return [];
-			}
-			index = JsonSerializer.Deserialize(content, NotesIndexJsonContext.Default.NotesIndex);
+				return (true, null);
+
+			var index = JsonSerializer.Deserialize(content, NotesIndexJsonContext.Default.NotesIndex);
+			return (false, index);
 		}
 		catch (Exception ex) when (ex is not OperationCanceledException)
 		{
 			emitError($"Could not fetch notes index for {org}/{repo}@{version} from {indexUri}: {ex.Message}");
-			return [];
+			return (false, null);
 		}
+	}
 
-		if (index is null || index.Notes.Count == 0)
+	private async Task<IReadOnlyList<CdnChangelogEntry>> FetchListedNotesAsync(
+		Uri baseUri,
+		string org,
+		string repo,
+		string version,
+		NotesIndex index,
+		Action<string> emitError,
+		Cancel ctx
+	)
+	{
+		if (index.Notes is not { Count: > 0 })
 			return [];
 
 		var repoLabel = $"{org}/{repo}";
@@ -548,7 +599,6 @@ Cancel ctx
 		{
 			ctx.ThrowIfCancellationRequested();
 
-			// Pool-relative path is "{branch}/note-{name}.yml"; split on first '/' only.
 			var poolRelativePath = noteEntry.Path;
 			var slash = poolRelativePath.IndexOf('/', StringComparison.Ordinal);
 			if (slash <= 0 || slash == poolRelativePath.Length - 1)
@@ -578,7 +628,6 @@ Cancel ctx
 				continue;
 			}
 
-			// The notes index asserts this note exists — a miss is a real pipeline error.
 			emitError(
 				$"Note '{poolRelativePath}' for {repoLabel}@{version} is listed in the notes index but could not be fetched from {noteUri}: {lastError}. " +
 					"Ensure the note was uploaded and scrubbed; if it persists check the changelog scrubber pipeline."

src/Elastic.Documentation.Configuration/ReleaseNotes/ChangelogKeys.cs

@@ -93,17 +93,27 @@ public static string ChangelogFileKey(string org, string repo, string branch, st
 	public static string ChangelogRegistryKey(string poolGroup) => $"{ChangelogPrefix}{poolGroup}/{RegistryFileName}";
 
 	/// <summary>
-	/// The notes-index key for one release version within a repo: <c>changelog/{org}/{repo}/notes-{version}.json</c>.
-	/// Repo-level and branch-agnostic — all notes for a version, regardless of which branch they were authored on.
+	/// The legacy notes-index key for one release version within a repo:
+	/// <c>changelog/{org}/{repo}/notes-{version}.json</c>.
+	/// Repo-level and branch-agnostic — the union of notes for that version across every product.
 	/// </summary>
 	/// <remarks>
 	/// The slug in the key is the release version (e.g. <c>9.3.0</c> or <c>2026-05-15</c>).
 	/// Previously this parameter was named <c>target</c> to match the obsolete <c>target:</c> YAML field;
 	/// it was renamed to <c>version</c> when that field was replaced by <c>versions:</c> on notes.
-	/// The key layout (<c>notes-{slug}.json</c>) is unchanged — no migration is needed.
+	/// Older <c>changelog bundle</c> clients still GET this key; the scrubber dual-writes it
+	/// alongside <see cref="NotesIndexKey(string, string, string, string)"/>.
 	/// </remarks>
 	public static string NotesIndexKey(string org, string repo, string version) => $"{ChangelogPrefix}{org}/{repo}/notes-{version}.json";
 
+	/// <summary>
+	/// The product-scoped notes-index key:
+	/// <c>changelog/{org}/{repo}/notes-{product}-{version}.json</c>.
+	/// Constructed from the two segments; callers must not parse the slug back apart.
+	/// </summary>
+	public static string NotesIndexKey(string org, string repo, string product, string version) =>
+		$"{ChangelogPrefix}{org}/{repo}/notes-{product}-{version}.json";
+
 	/// <summary>
 	/// The S3 prefix that covers all branches and notes indexes of one repo: <c>changelog/{org}/{repo}/</c>.
 	/// Used by the notes reconciler to list the full repo tree.
@@ -112,8 +122,9 @@ public static string ChangelogFileKey(string org, string repo, string branch, st
 
 	/// <summary>
 	/// Returns true when <paramref name="key"/> is a notes-index key of the form
-	/// <c>changelog/{org}/{repo}/notes-{target}.json</c> (exactly two group segments, then
-	/// a <c>notes-</c>-prefixed JSON file with a non-empty target slug).
+	/// <c>changelog/{org}/{repo}/notes-{slug}.json</c> (exactly two group segments, then
+	/// a <c>notes-</c>-prefixed JSON file with a non-empty slug). Covers both the legacy
+	/// <c>notes-{version}.json</c> shape and the product-scoped <c>notes-{product}-{version}.json</c> shape.
 	/// </summary>
 	public static bool IsNotesIndex(string key)
 	{

src/Elastic.Documentation.Configuration/ReleaseNotes/NotesIndex.cs

@@ -41,14 +41,17 @@ public sealed record NoteIndexEntry
 }
 
 /// <summary>
-/// Notes index published at <c>changelog/{org}/{repo}/notes-{version}.json</c>.
-/// Lists all <c>note-*.yml</c> fragments for one release version,
+/// Notes index published at <c>changelog/{org}/{repo}/notes-{product}-{version}.json</c>
+/// (and dual-written to the legacy <c>notes-{version}.json</c> key).
+/// Lists all <c>note-*.yml</c> fragments for one product and release version,
 /// across every branch of the repo.
 /// </summary>
 /// <remarks>
 /// Contents are paths, not bodies — the note files remain the single source of truth.
 /// A stale index can only omit or over-list, never serve stale prose. Bundling a version
 /// is therefore 1 GET for the index + one GET per listed note.
+/// <see cref="Product"/> and <see cref="Version"/> are set on product-scoped writes and
+/// omitted from legacy version-union bodies so older deserializers keep working.
 /// </remarks>
 public sealed record NotesIndex
 {
@@ -58,6 +61,18 @@ public sealed record NotesIndex
 	/// <summary>Current schema version constant.</summary>
 	public const int CurrentSchemaVersion = 1;
 
+	/// <summary>
+	/// Bundle product id this index belongs to. Present on product-scoped keys; omitted on
+	/// the legacy version-union body.
+	/// </summary>
+	public string? Product { get; init; }
+
+	/// <summary>
+	/// Release version this index belongs to. Present on product-scoped keys; omitted on
+	/// the legacy version-union body.
+	/// </summary>
+	public string? Version { get; init; }
+
 	/// <summary>
 	/// Notes for this version. Each entry carries the pool-relative path, origin branch,
 	/// and a derived <c>bundle_seq</c>.