Support notes-product-version.json indices

docs/cli/changelog/cmd-backfill.md

@@ -8,7 +8,7 @@ For each product in scope the command:
 2. Parses each `## {version}` section into typed entries. Unrecognized `### …` subsections whose body is bullets become `Other`-typed entries; prose subsections are preserved in the bundle description.
 3. Writes bundle YAML to `{output}/{product}/changelog/bundles/{version}.yaml`.
 4. Writes per-entry YAML files: `{pr}.yaml` for entries that reference a pull request, `note-{slug}.yaml` for entries with no PR reference.
-5. Writes `notes-{target}.json` per version listing all `note-*.yaml` files for that target (the notes registry).
+5. Writes `notes-{target}.json` per version listing all `note-*.yaml` files for that target (the notes registry). This is a **local** backfill artifact, not the CDN dual-write layout (`notes-{product}-{version}.json` plus legacy `notes-{version}.json`).
 6. Prints a per-product report including entry counts and no-PR rates.
 
 The primary use case is measuring how much published release-notes content cannot be traced back to a PR — determining how much historical content needs the PR-less "note" format from [docs-eng-team#789](https://github.com/elastic/docs-eng-team/issues/789).

docs/cli/changelog/cmd-note.md

@@ -59,7 +59,12 @@ products:
 ## After creation
 
 Upload is the same as for other changelog YAML files.
-An index at `changelog/{org}/{repo}/notes-{version}.json` lists every changelog "note" file that applies to each version.
+The scrubber writes two indexes per note:
+
+- `changelog/{org}/{repo}/notes-{product}-{version}.json` — notes for that product and version. `changelog bundle` reads this key first.
+- `changelog/{org}/{repo}/notes-{version}.json` — the union of notes for that version across products, kept for older CLI pins.
+
+`{changelog}` does not read these indexes. It loads published bundle YAML (and amend sidecars listed in `bundle/{product}/registry.json`).
 
 If the release bundle for that product and version or date has already shipped when you upload, the scrubber generates a `{parent}.amend-notes.yaml` sidecar so the changelog reaches published docs without a manual rerun. Don't run `changelog bundle-amend --add` for that file. Refer to [](/data/release-notes/bundle.md#changelog-bundle-notes-after-ship).
 

docs/development/changelog-bundle-registry.md

@@ -80,8 +80,9 @@ narrowed reconciliation to the bundle tree):
   events. Authors cannot issue those deletes through docs-builder today: `changelog upload`
   does not delete objects, and `changelog remove` is local-only. For the author-facing
   add and exclude path, see [](/data/release-notes/bundle.md#changelog-bundle-notes-after-ship).
-- **Notes index** — `changelog/{org}/{repo}/notes-{version}.json`, one per version, **public
-  bucket only**, produced by the scrubber Lambda's `NotesIndexReconciler`. See
+- **Notes index** — `changelog/{org}/{repo}/notes-{product}-{version}.json` (product-scoped) plus a dual-written
+  legacy `changelog/{org}/{repo}/notes-{version}.json` (version union), **public bucket only**, produced by
+  the scrubber Lambda's `NotesIndexReconciler`. See
   [Notes-index format](#notes-index-format) below.
 - **Changelog-entry index** — `changelog/{org}/{repo}/{branch}/registry.json`, a **legacy
   client-authored pass-through**: the current `changelog upload` never writes one, but manifests
@@ -127,22 +128,38 @@ zero-bundle state. The reconciler deliberately restores the former.
 
 ## Notes-index format [notes-index-format]
 
-For each release version that has at least one note, the scrubber Lambda writes a notes index at
-`changelog/{org}/{repo}/notes-{version}.json`. Its schema (`schema_version: 1`):
+For each product and release version that has at least one note, the scrubber Lambda writes a
+product-scoped notes index at `changelog/{org}/{repo}/notes-{product}-{version}.json`. It also
+dual-writes a legacy version-union index at `changelog/{org}/{repo}/notes-{version}.json` so
+older `changelog bundle` clients that still GET the version-only key keep working. The product
+and version segments are concatenated into the new filename; consumers must not parse that slug
+apart.
+
+`changelog bundle` GETs the product-scoped key first and falls back to the version-only key
+only on HTTP 404. An empty product-scoped index does not fall back. `{changelog}` does not read
+these indexes; it loads bundle YAML from the product registry.
+
+Product-scoped schema (`schema_version: 1`):
 
 ```json
 {
   "schema_version": 1,
+  "product": "elasticsearch",
+  "version": "9.3.0",
   "notes": [
     { "path": "main/note-esql-oom.yml", "bundle_seq": 2 },
     { "path": "main/note-cve-2026-1234.yml", "bundle_seq": 1 }
   ]
 }
 ```
 
+The legacy `notes-{version}.json` body omits `product` and `version` and lists the union of note
+paths for that version across products.
+
 | Field | Meaning |
 |---|---|
 | `schema_version` | Schema version. Currently `1`. |
+| `product` / `version` | Set on product-scoped indexes; omitted from the legacy version-union body. |
 | `notes[].path` | Pool-relative path of the note within `changelog/{org}/{repo}/`. The leading segment before the first `/` is the branch. |
 | `notes[].bundle_seq` | Derived reporting field: `0` = no bundle published for this version yet, `1` = note shipped in the original bundle, `2` = note carried by the Lambda-generated `.amend-notes` sidecar. |
 
@@ -151,9 +168,11 @@ reconcile by comparing the notes index against the set of entries in the publish
 amend sidecars. Do not hand-edit `notes-{version}.json` or `.amend-notes` sidecars. Authors cannot
 delete pool objects through docs-builder today.
 
-A 404 on a notes index means "no notes published for this version". An empty `notes` array never
-appears — the index is deleted rather than emptied, following the same
-[absent ≠ empty](#absent-empty) rule as the bundle registry.
+A 404 on both the product-scoped index and the legacy version-union index means "no notes
+published for this product and version". An empty `notes` array never appears on a successfully
+reconciled index — the index is deleted rather than emptied, following the same
+[absent ≠ empty](#absent-empty) rule as the bundle registry. Until older clients stop reading
+`notes-{version}.json`, the Lambda keeps that key while any note still declares the version.
 
 ## Shallow per-tree change maps [shallow-maps]
 
@@ -303,6 +322,12 @@ entry still missing after its retry budget fails the bundle (an incomplete relea
 ship silently). `CdnChangelogEntryFetcher` reuses a shared `HttpClient` in production and disposes an
 owned client only when a test injects a handler, mirroring `CdnChangelogFetcher`.
 
+When CDN sourcing is on, the same fetcher also merges changelog notes for each concrete
+`--output-products` pair: it GETs `changelog/{org}/{repo}/notes-{product}-{version}.json` first
+and falls back to `notes-{version}.json` only on HTTP 404. See
+[Notes-index format](#notes-index-format). Checksum dedup keeps a single copy if two products at
+the same version still share the legacy index.
+
 ## Consumer: `{changelog}` directive `cdn:` mode (implemented)
 
 ### Syntax

src/Elastic.Documentation.Configuration/ReleaseNotes/CdnChangelogEntryFetcher.cs

@@ -487,23 +487,18 @@ private static Uri CombineSegments(Uri baseUri, IReadOnlyList<string> segments)
 	}
 
 	/// <summary>
-	/// <summary>
-	/// Fetches all <c>note-*.yml</c> entries for <paramref name="org"/>/<paramref name="repo"/> at
-	/// <paramref name="version"/> from the CDN. Reads the <c>notes-{version}.json</c> index to enumerate
-	/// the pool-relative note paths; a missing index means no notes (not an error). A listed note that
-	/// cannot be fetched is a hard error — the index is an authoritative promise that the note exists.
+	/// Fetches all <c>note-*.yml</c> entries for <paramref name="org"/>/<paramref name="repo"/> and
+	/// <paramref name="product"/> at <paramref name="version"/> from the CDN. Reads
+	/// <c>notes-{product}-{version}.json</c> first; on HTTP 404 only, falls back to the legacy
+	/// <c>notes-{version}.json</c>. A missing index (both 404) means no notes (not an error).
+	/// An empty product-scoped index does not fall back. A listed note that cannot be fetched
+	/// is a hard error — the index is an authoritative promise that the note exists.
 	/// </summary>
-	/// <param name="baseUri">CDN base URI.</param>
-	/// <param name="org">Repository org (e.g. <c>elastic</c>).</param>
-	/// <param name="repo">Repository name (e.g. <c>kibana</c>).</param>
-	/// <param name="version">Release version string (e.g. <c>9.0.0</c>).</param>
-	/// <param name="emitError">Called once per hard error; caller decides how to surface it.</param>
-	/// <param name="ctx">Cancellation token.</param>
-	/// <returns>The fetched note entries, keyed by pool-relative path (<c>main/note-foo.yml</c>).</returns>
 	public async Task<IReadOnlyList<CdnChangelogEntry>> FetchNotesAsync(
 		Uri baseUri,
 		string org,
 		string repo,
+		string product,
 		string version,
 		Action<string> emitError,
 		Cancel ctx
@@ -515,31 +510,87 @@ Cancel ctx
 			return [];
 		}
 
-		var indexUri = CombineSegments(baseUri, ["changelog", org, repo, $"notes-{version}.json"]);
-		NotesIndex? index;
+		if (!ChangelogKeys.IsValidProduct(product))
+		{
+			emitError($"Invalid product '{product}' for notes fetch: must be a non-empty ASCII product segment.");
+			return [];
+		}
+
+		var productIndexUri = CombineSegments(baseUri, ["changelog", org, repo, $"notes-{product}-{version}.json"]);
+		var (productMissing, productIndex) = await TryLoadNotesIndexAsync(
+			productIndexUri,
+			org,
+			repo,
+			version,
+			emitError,
+			ctx
+		).ConfigureAwait(false);
+		if (productIndex is not null)
+			return await FetchListedNotesAsync(baseUri, org, repo, version, productIndex, emitError, ctx).ConfigureAwait(false);
+		if (!productMissing)
+			return [];
+
+		var legacyIndexUri = CombineSegments(baseUri, ["changelog", org, repo, $"notes-{version}.json"]);
+		var (legacyMissing, legacyIndex) = await TryLoadNotesIndexAsync(legacyIndexUri, org, repo, version, emitError, ctx).ConfigureAwait(
+			false
+		);
+		if (legacyIndex is not null)
+			return await FetchListedNotesAsync(baseUri, org, repo, version, legacyIndex, emitError, ctx).ConfigureAwait(false);
+		if (legacyMissing)
+		{
+			_logger.LogDebug(
+				"Notes index for {Org}/{Repo}/{Product}@{Version} not found at {ProductUri} or {LegacyUri}; no notes to bundle",
+				org,
+				repo,
+				product,
+				version,
+				productIndexUri,
+				legacyIndexUri
+			);
+		}
+		return [];
+	}
+
+	/// <summary>
+	/// Loads a notes index. Returns <c>missing: true</c> only on HTTP 404. A present but empty
+	/// or unparseable body is <c>missing: false</c> with a null index so callers do not fall back.
+	/// </summary>
+	private async Task<(bool Missing, NotesIndex? Index)> TryLoadNotesIndexAsync(
+		Uri indexUri,
+		string org,
+		string repo,
+		string version,
+		Action<string> emitError,
+		Cancel ctx
+	)
+	{
 		try
 		{
 			var (notFound, content) = await FetchTextOrNotFoundAsync(indexUri, 1, ctx).ConfigureAwait(false);
 			if (notFound)
-			{
-				_logger.LogDebug(
-					"Notes index for {Org}/{Repo}@{Version} not found at {Uri}; no notes to bundle",
-					org,
-					repo,
-					version,
-					indexUri
-				);
-				return [];
-			}
-			index = JsonSerializer.Deserialize(content, NotesIndexJsonContext.Default.NotesIndex);
+				return (true, null);
+
+			var index = JsonSerializer.Deserialize(content, NotesIndexJsonContext.Default.NotesIndex);
+			return (false, index);
 		}
 		catch (Exception ex) when (ex is not OperationCanceledException)
 		{
 			emitError($"Could not fetch notes index for {org}/{repo}@{version} from {indexUri}: {ex.Message}");
-			return [];
+			return (false, null);
 		}
+	}
 
-		if (index is null || index.Notes.Count == 0)
+	private async Task<IReadOnlyList<CdnChangelogEntry>> FetchListedNotesAsync(
+		Uri baseUri,
+		string org,
+		string repo,
+		string version,
+		NotesIndex index,
+		Action<string> emitError,
+		Cancel ctx
+	)
+	{
+		if (index.Notes is not { Count: > 0 })
 			return [];
 
 		var repoLabel = $"{org}/{repo}";
@@ -548,7 +599,6 @@ Cancel ctx
 		{
 			ctx.ThrowIfCancellationRequested();
 
-			// Pool-relative path is "{branch}/note-{name}.yml"; split on first '/' only.
 			var poolRelativePath = noteEntry.Path;
 			var slash = poolRelativePath.IndexOf('/', StringComparison.Ordinal);
 			if (slash <= 0 || slash == poolRelativePath.Length - 1)
@@ -578,7 +628,6 @@ Cancel ctx
 				continue;
 			}
 
-			// The notes index asserts this note exists — a miss is a real pipeline error.
 			emitError(
 				$"Note '{poolRelativePath}' for {repoLabel}@{version} is listed in the notes index but could not be fetched from {noteUri}: {lastError}. " +
 					"Ensure the note was uploaded and scrubbed; if it persists check the changelog scrubber pipeline."

src/Elastic.Documentation.Configuration/ReleaseNotes/ChangelogKeys.cs

@@ -93,17 +93,27 @@ public static string ChangelogFileKey(string org, string repo, string branch, st
 	public static string ChangelogRegistryKey(string poolGroup) => $"{ChangelogPrefix}{poolGroup}/{RegistryFileName}";
 
 	/// <summary>
-	/// The notes-index key for one release version within a repo: <c>changelog/{org}/{repo}/notes-{version}.json</c>.
-	/// Repo-level and branch-agnostic — all notes for a version, regardless of which branch they were authored on.
+	/// The legacy notes-index key for one release version within a repo:
+	/// <c>changelog/{org}/{repo}/notes-{version}.json</c>.
+	/// Repo-level and branch-agnostic — the union of notes for that version across every product.
 	/// </summary>
 	/// <remarks>
 	/// The slug in the key is the release version (e.g. <c>9.3.0</c> or <c>2026-05-15</c>).
 	/// Previously this parameter was named <c>target</c> to match the obsolete <c>target:</c> YAML field;
 	/// it was renamed to <c>version</c> when that field was replaced by <c>versions:</c> on notes.
-	/// The key layout (<c>notes-{slug}.json</c>) is unchanged — no migration is needed.
+	/// Older <c>changelog bundle</c> clients still GET this key; the scrubber dual-writes it
+	/// alongside <see cref="NotesIndexKey(string, string, string, string)"/>.
 	/// </remarks>
 	public static string NotesIndexKey(string org, string repo, string version) => $"{ChangelogPrefix}{org}/{repo}/notes-{version}.json";
 
+	/// <summary>
+	/// The product-scoped notes-index key:
+	/// <c>changelog/{org}/{repo}/notes-{product}-{version}.json</c>.
+	/// Constructed from the two segments; callers must not parse the slug back apart.
+	/// </summary>
+	public static string NotesIndexKey(string org, string repo, string product, string version) =>
+		$"{ChangelogPrefix}{org}/{repo}/notes-{product}-{version}.json";
+
 	/// <summary>
 	/// The S3 prefix that covers all branches and notes indexes of one repo: <c>changelog/{org}/{repo}/</c>.
 	/// Used by the notes reconciler to list the full repo tree.
@@ -112,8 +122,9 @@ public static string ChangelogFileKey(string org, string repo, string branch, st
 
 	/// <summary>
 	/// Returns true when <paramref name="key"/> is a notes-index key of the form
-	/// <c>changelog/{org}/{repo}/notes-{target}.json</c> (exactly two group segments, then
-	/// a <c>notes-</c>-prefixed JSON file with a non-empty target slug).
+	/// <c>changelog/{org}/{repo}/notes-{slug}.json</c> (exactly two group segments, then
+	/// a <c>notes-</c>-prefixed JSON file with a non-empty slug). Covers both the legacy
+	/// <c>notes-{version}.json</c> shape and the product-scoped <c>notes-{product}-{version}.json</c> shape.
 	/// </summary>
 	public static bool IsNotesIndex(string key)
 	{

src/Elastic.Documentation.Configuration/ReleaseNotes/NotesIndex.cs

@@ -41,14 +41,17 @@ public sealed record NoteIndexEntry
 }
 
 /// <summary>
-/// Notes index published at <c>changelog/{org}/{repo}/notes-{version}.json</c>.
-/// Lists all <c>note-*.yml</c> fragments for one release version,
+/// Notes index published at <c>changelog/{org}/{repo}/notes-{product}-{version}.json</c>
+/// (and dual-written to the legacy <c>notes-{version}.json</c> key).
+/// Lists all <c>note-*.yml</c> fragments for one product and release version,
 /// across every branch of the repo.
 /// </summary>
 /// <remarks>
 /// Contents are paths, not bodies — the note files remain the single source of truth.
 /// A stale index can only omit or over-list, never serve stale prose. Bundling a version
 /// is therefore 1 GET for the index + one GET per listed note.
+/// <see cref="Product"/> and <see cref="Version"/> are set on product-scoped writes and
+/// omitted from legacy version-union bodies so older deserializers keep working.
 /// </remarks>
 public sealed record NotesIndex
 {
@@ -58,6 +61,18 @@ public sealed record NotesIndex
 	/// <summary>Current schema version constant.</summary>
 	public const int CurrentSchemaVersion = 1;
 
+	/// <summary>
+	/// Bundle product id this index belongs to. Present on product-scoped keys; omitted on
+	/// the legacy version-union body.
+	/// </summary>
+	public string? Product { get; init; }
+
+	/// <summary>
+	/// Release version this index belongs to. Present on product-scoped keys; omitted on
+	/// the legacy version-union body.
+	/// </summary>
+	public string? Version { get; init; }
+
 	/// <summary>
 	/// Notes for this version. Each entry carries the pool-relative path, origin branch,
 	/// and a derived <c>bundle_seq</c>.