Upgrade Dynatrace RUM: script-tag loading, runtime config, fetch tooling

CHANGELOG-FRONTIER.md

@@ -1,3 +1,17 @@
+## 8.17.0
+
+- Upgrade Dynatrace RUM agent to version 1.329+ and enable new RUM experience
+  - Added feature flag `frontier_snow_dynatraceNewRUM` to control RUM version selection (gradual rollout); the existing `frontier_snow_dynatraceRUM` flag still selects the loading mechanism
+  - New RUM is loaded exclusively via `<script>` tags (never inlined). This fixes a 500 error (`Could not find matching close tag for "<%"`) caused by EJS `include()` compiling the agent JS as a template — the new agent's minified code contains the `<%` sequence — and avoids re-shipping 300–460 KB on every page view
+  - Mapped each mechanism to a Dynatrace snippet format for the new RUM (treatment names kept for old-RUM compatibility):
+    - `asyncCS-script` → OneAgent JavaScript tag + SRI, loaded `async`
+    - `asyncCS-inline` → OneAgent JavaScript tag + SRI, loaded `sync` (closest analog to the old sync inline bootstrap; the new RUM has no small-bootstrap format, so this no longer inlines)
+    - `global-cdn` → JavaScript tag (`_complete.js`), loaded `async`
+  - SRI (Subresource Integrity) integrity hashes and `data-dtconfig` are emitted on the OneAgent tags; new RUM includes enhanced data collection (owasp=1, uxrgce=1)
+  - Snow can supply fresh values at runtime via `locals.dynatrace.*` (`cdnUrls`/`cdnIntegrity`/`cdnConfig`/`cdnCompleteUrls`), published to S3 as `dynatrace-rum-config.json`; publish-time fallbacks are baked into `dynatrace.ejs`
+  - `fetch-dynatrace-scripts.js` no longer fetches `/inlineCode` or writes `_inline_*_new.ejs`; old-RUM `_inline_*.ejs` bootstrap files are unchanged
+  - When the new flag is OFF: uses the existing RUM version and URLs (backward compatible)
+
 ## 8.16.1
 
 - Convert layout.ejs includes from the removed EJS preprocessor syntax (`<% include x %>`) to the function form (`<%- include('x') %>`) ahead of the company-wide EJS 3 upgrade

packages/react-scripts/layout/views/partials/dynatrace.ejs

@@ -3,8 +3,11 @@
 
 if (typeof getFeatureFlag !== 'undefined') {
   const dynatraceFlag = getFeatureFlag('frontier_snow_dynatraceRUM', {appName: process.env.APP_NAME});
+  const dynatraceNewFlag = getFeatureFlag('frontier_snow_dynatraceNewRUM', {appName: process.env.APP_NAME});
   const env = envType();
+  const useNewRUM = dynatraceNewFlag.isOn;
 
+  // Old RUM version URLs
   const cdnUrls = {
     int: 'https://js-cdn.dynatrace.com/jstag/15c157a40ab/bf99293tkn/3faf90e849295814_complete.js',
     beta: 'https://js-cdn.dynatrace.com/jstag/15c157a40ab/bf99293tkn/c4242bb1eb216374_complete.js',
@@ -17,12 +20,91 @@ if (typeof getFeatureFlag !== 'undefined') {
     prod: 'https://edge.fscdn.org/assets/components/hf/assets/js/monitoring/dynatrace-20221104-prod.min-44bb9345beec8a984c2fca40385b4f41.js'
   }
 
+  // New RUM 1.329+ — Snow provides fresh values via locals.dynatrace (from CDN config file);
+  // fallback to values current at react-scripts publish time.
+  const cdnUrlsNew = (locals && locals.dynatrace && locals.dynatrace.cdnUrls) || {
+    int: 'https://js-cdn.dynatrace.com/jstag/15c157a40ab/sri/ruxitagent_ICA15789NPQRTUVXfghqrux_10341260622154106.js',
+    beta: 'https://js-cdn.dynatrace.com/jstag/15c157a40ab/sri/ruxitagent_ICA7NQVfghqrux_10341260622154106.js',
+    prod: 'https://js-cdn.dynatrace.com/jstag/15c157a40ab/sri/ruxitagent_ICA7NQVfghqrux_10341260622154106.js'
+  }
+
+  const cdnIntegrityNew = (locals && locals.dynatrace && locals.dynatrace.cdnIntegrity) || {
+    int: 'sha256-xCOvz0J22cpI/MIdcbEaVWANEPMHmZAKpeD8zTuMY/s=',
+    beta: 'sha256-eGB6PLc8ndZISmjwbbi4xZhE9nbarKf48bRKxVdha+8=',
+    prod: 'sha256-eGB6PLc8ndZISmjwbbi4xZhE9nbarKf48bRKxVdha+8='
+  }
+
+  const cdnConfigNew = (locals && locals.dynatrace && locals.dynatrace.cdnConfig) || {
+    int: 'app=3faf90e849295814|cors=1|owasp=1|featureHash=ICA15789NPQRTUVXfghqrux|msl=153600|srsr=10000|nsfnv=1|reportUrl=https://bf99293tkn.bf.dynatrace.com/bf|srvr=%5C%2Fidentity%5C%2Fsettings|rdnt=2|uxrgce=1|cuc=t0rtn87t|srms=2,1,0,0%2Ftextarea%2Cinput%2Cselect%2Coption;0%2Fdatalist;0%2Fform%20button;0%2F%5Bdata-dtrum-input%5D;0%2F.data-dtrum-input;1%2F%5Edata%28%28%5C-.%2B%24%29%7C%24%29|mel=100000|dpvc=1|md=mdcc1=cfs_anid,mdcc2=bFS.User.profile.cisId,mdcc3=bs.pageName,mdcc4=bFS.attrs.anonId,mdcc5=bFS.attrs.country,mdcc6=bFS.attrs.cisId|lastModification=1785320826721|mdp=mdcc4,mdcc5|tp=500,50,0|srbbv=2|agentUri=https://js-cdn.dynatrace.com/jstag/15c157a40ab/sri/ruxitagent_ICA15789NPQRTUVXfghqrux_10341260622154106.js',
+    beta: 'app=c4242bb1eb216374|cors=1|owasp=1|featureHash=ICA7NQVfghqrux|msl=153600|srsr=10000|nsfnv=1|reportUrl=https://bf99293tkn.bf.dynatrace.com/bf|srvr=%5C%2Fidentity%5C%2Fsettings|rdnt=2|uxrgce=1|cuc=t0rtn87t|srms=2,1,0,0%2Ftextarea%2Cinput%2Cselect%2Coption;0%2Fdatalist;0%2Fform%20button;0%2F%5Bdata-dtrum-input%5D;0%2F.data-dtrum-input;1%2F%5Edata%28%28%5C-.%2B%24%29%7C%24%29|mel=100000|dpvc=1|md=mdcc1=cfs_anid,mdcc2=bFS.attrs.cisId,mdcc3=bFS.User.profile.cisId,mdcc4=fx-ratelimit-remaining,mdcc5=bs.campaign,mdcc6=bs.channel,mdcc7=bs.pageName,mdcc8=bs.pageURL,mdcc9=bs.pageType,mdcc10=bs.visitorID|lastModification=1785320826721|tp=500,50,0|srbbv=2|agentUri=https://js-cdn.dynatrace.com/jstag/15c157a40ab/sri/ruxitagent_ICA7NQVfghqrux_10341260622154106.js',
+    prod: 'app=a8e5edd77f861ace|cors=1|owasp=1|featureHash=ICA7NQVfghqrux|msl=153600|srsr=10000|vcx=1500|nsfnv=1|reportUrl=https://bf99293tkn.bf.dynatrace.com/bf|srvr=%5C%2Fidentity%5C%2Fsettings|rdnt=2|uxrgce=1|vcit=8000|cuc=t0rtn87t|srms=2,2,1,|mdl=mdcc11=20|mel=100000|dpvc=1|md=mdcc1=cfs_anid,mdcc2=bFS.attrs.cisId,mdcc3=bFS.User.profile.cisId,mdcc4=bs.pageName,mdcc5=bs.pageType,mdcc6=bs.pageURL,mdcc7=bs.visitorID,mdcc8=bs.campaign,mdcc9=bs.channel,mdcc10=fx-ratelimit-remaining,mdcc11=bdocument.referrer,mdcc12=dutm_campaign,mdcc13=dutm_source,mdcc14=dutm_medium|lastModification=1785320826721|tp=500,50,0|srbbv=2|agentUri=https://js-cdn.dynatrace.com/jstag/15c157a40ab/sri/ruxitagent_ICA7NQVfghqrux_10341260622154106.js'
+  }
+
+  // New RUM JavaScript tag (combined code+config _complete.js) from the Dynatrace global CDN.
+  // Snow provides fresh values via locals.dynatrace.cdnCompleteUrls; fallback to publish-time values.
+  const cdnCompleteUrlsNew = (locals && locals.dynatrace && locals.dynatrace.cdnCompleteUrls) || {
+    int: 'https://js-cdn.dynatrace.com/jstag/15c157a40ab/bf99293tkn/3faf90e849295814_complete.js',
+    beta: 'https://js-cdn.dynatrace.com/jstag/15c157a40ab/bf99293tkn/c4242bb1eb216374_complete.js',
+    prod: 'https://js-cdn.dynatrace.com/jstag/15c157a40ab/bf99293tkn/a8e5edd77f861ace_complete.js'
+  }
+
+  // Old-RUM inline bootstrap include path: a small sync stub that begins capture, then async-loads
+  // the full library. The new RUM has no equivalent small-bootstrap format, so this include is used
+  // for old RUM only. (The agent JS must NOT be run through EJS include() — see below.)
+  const inlinePathOld = './partials/dynatrace/_inline_' + env;
+
+  // NOTE: the new RUM is loaded exclusively via script tags built from cdnUrlsNew / cdnConfigNew /
+  // cdnIntegrityNew / cdnCompleteUrlsNew. We deliberately do NOT include() the inline agent for the
+  // new RUM: EJS compiles included files as templates, and the new agent's minified JS contains the
+  // two-character EJS open-delimiter sequence (a less-than sign immediately followed by a percent
+  // sign) inside a character-class string, which EJS misreads as an unterminated scriptlet and
+  // fails with "Could not find matching close tag". Emitting a script tag sidesteps that entirely.
   if (dynatraceFlag.treatment === 'asyncCS-script') { %>
-    <script type="text/javascript" src="<%= edgeUrls[env] %>" crossorigin="anonymous"></script>
-    <% } else if (dynatraceFlag.treatment === 'asyncCS-inline') { %>
-    <%- include(`./partials/dynatrace/_inline_${env}`) %>
+    <% if (useNewRUM) { %>
+      <!-- New RUM: OneAgent JS tag + SRI, loaded async (non-blocking; blind window until the agent
+           downloads and executes — earliest user actions / XHRs may be missed) -->
+      <script type="text/javascript" src="<%= cdnUrlsNew[env] %>" data-dtconfig="<%= cdnConfigNew[env] %>" integrity="<%= cdnIntegrityNew[env] %>" crossorigin="anonymous" async></script>
+    <% } else { %>
+      <!-- Old RUM: load from Edge CDN asynchronously -->
+      <script type="text/javascript" src="<%= edgeUrls[env] %>" crossorigin="anonymous" async></script>
+    <% } %>
+  <% } else if (dynatraceFlag.treatment === 'asyncCS-inline') { %>
+    <% if (useNewRUM) { %>
+      <!-- New RUM: there is no small-bootstrap snippet format. The closest analog to the old sync
+           inline bootstrap is the OneAgent JS tag + SRI loaded SYNCHRONOUSLY: full early capture,
+           parse-blocks only on the first uncached load, then free on warm cache (the versioned URL is
+           cacheable up to one year). NOTE: this no longer inlines the agent — the treatment name is
+           retained for old-RUM compatibility; see DYNATRACE_RUM_UPDATE.md for the remap rationale. -->
+      <script type="text/javascript" src="<%= cdnUrlsNew[env] %>" data-dtconfig="<%= cdnConfigNew[env] %>" integrity="<%= cdnIntegrityNew[env] %>" crossorigin="anonymous"></script>
+    <% } else { %>
+      <!-- Old RUM: small inline bootstrap that begins capture then async-loads the full library -->
+      <%- include(inlinePathOld) %>
+    <% } %>
   <% } else if (dynatraceFlag.treatment === 'global-cdn') { %>
+    <% if (useNewRUM) { %>
+      <!-- New RUM: JavaScript tag (combined code+config _complete.js) from the Dynatrace global CDN,
+           loaded async (single request; the combined file is cached ~1 hour) -->
+      <script type="text/javascript" src="<%= cdnCompleteUrlsNew[env] %>" crossorigin="anonymous" async></script>
+    <% } else { %>
+      <!-- Old RUM: combined _complete.js from the Dynatrace global CDN (synchronous) -->
       <script type="text/javascript" src="<%= cdnUrls[env] %>" crossorigin="anonymous"></script>
+    <% } %>
   <% } %>
-
+  <script type="text/javascript">
+    // Report the RUM loading mechanism once the agent is available. For the async treatments
+    // window.dtrum isn't defined yet when this inline script runs, so poll briefly rather than
+    // firing once and silently dropping the signal (which lost it for asyncCS-script/global-cdn).
+    (function () {
+      if (typeof window === 'undefined') return;
+      var treatment = "<%= dynatraceFlag.treatment %>";
+      var attempts = 0;
+      (function report() {
+        if (window.dtrum && typeof window.dtrum.sendSessionProperties === 'function') {
+          window.dtrum.sendSessionProperties({rum_loading_mechanism: treatment});
+        } else if (++attempts <= 50) {
+          setTimeout(report, 200); // up to ~10s for the async agent to load
+        }
+      })();
+    })();
+  </script>
 <% } %>

packages/react-scripts/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@fs/react-scripts",
-  "version": "8.16.1",
+  "version": "8.17.0",
   "upstreamVersion": "5.0.1",
   "description": "Configuration and scripts for Create React App.",
   "repository": {

packages/react-scripts/scripts/DYNATRACE_RUM_UPDATE.md

@@ -0,0 +1,177 @@
+# Updating Dynatrace RUM Scripts
+
+This guide explains how to update the Dynatrace RUM agent to a new version using the automated fetch script.
+
+## Quick Start
+
+```bash
+export DYNATRACE_API_TOKEN="your-api-token"
+node packages/react-scripts/scripts/fetch-dynatrace-scripts.js
+```
+
+The script will fetch the latest RUM scripts and CDN URLs for all three environments (int, beta, prod).
+
+## Prerequisites
+
+### 1. Get a Dynatrace API Token
+
+1. Log into your Dynatrace environment: https://bjm35087.live.dynatrace.com
+2. Go to Account → Access Tokens
+3. Create a new API token with these scopes:
+   - **Read RUM manual insertion tags** (API v2) - for fetching scripts
+   - **Real user monitoring JavaScript tag management** (API v1) - optional, for additional RUM data
+
+### 2. Know Your Entity IDs
+
+The script uses these default entity IDs:
+- **Int**: `APPLICATION-3FAF90E849295814`
+- **Beta**: `APPLICATION-C4242BB1EB216374`
+- **Prod**: `APPLICATION-A8E5EDD77F861ACE`
+
+To find entity IDs yourself:
+1. Log into Dynatrace
+2. Go to Applications → select your app → Settings
+3. The entity ID is shown in the URL or settings page
+
+## How to Use
+
+### Step 1: Fetch the Latest Scripts
+
+```bash
+export DYNATRACE_API_TOKEN="your-token-here"
+node packages/react-scripts/scripts/fetch-dynatrace-scripts.js
+```
+
+The script will:
+1. ✅ **Update the new-RUM fallback values** in `dynatrace.ejs` (`cdnUrlsNew`, `cdnIntegrityNew`, `cdnConfigNew`, `cdnCompleteUrlsNew`)
+2. ✅ **Write & publish** `dynatrace-rum-config.json` to S3 so Snow can supply fresh values via `locals.dynatrace.*` without a react-scripts redeploy
+3. 🔒 **Capture the SRI integrity hashes** and `data-dtconfig` for each environment
+
+> **The new RUM agent is never inlined.** The script no longer fetches `/inlineCode` or writes `_inline_*_new.ejs` files. Reason: EJS `include()` compiles the included file as a template, and the new agent's minified JS contains the two-character EJS open-delimiter sequence (a `<` immediately followed by a `%`) inside a string, which EJS misreads as an unterminated scriptlet ("Could not find matching close tag"). It also re-ships 300–460 KB on every page view. The new RUM loads exclusively via `<script>` tags. The old-RUM `_inline_*.ejs` bootstrap files are unaffected.
+
+### Step 2: Verify the updated values in dynatrace.ejs
+
+The script updates `dynatrace.ejs` **automatically** (see Step 1) — there is nothing to copy by hand. It rewrites four per-environment fallback objects, and the values differ per environment: `int` uses a different feature hash than `beta`/`prod`, so its integrity hash and `data-dtconfig` differ too (there is **no** single integrity hash shared across environments).
+
+```javascript
+const cdnUrlsNew         = (locals && locals.dynatrace && locals.dynatrace.cdnUrls)         || { int: '…', beta: '…', prod: '…' }
+const cdnIntegrityNew    = (locals && locals.dynatrace && locals.dynatrace.cdnIntegrity)    || { int: 'sha256-…', beta: 'sha256-…', prod: 'sha256-…' }
+const cdnConfigNew       = (locals && locals.dynatrace && locals.dynatrace.cdnConfig)       || { int: '…', beta: '…', prod: '…' }
+const cdnCompleteUrlsNew = (locals && locals.dynatrace && locals.dynatrace.cdnCompleteUrls) || { int: '…', beta: '…', prod: '…' }
+```
+
+To confirm, run `git diff` on `dynatrace.ejs` and check that all three environments show the new agent version. The script also prints the downloaded agent version and each environment's CDN URL / integrity / config to stdout for reference.
+
+### Step 3: Test All Mechanisms
+
+Two flags drive the experiment (see [dynatrace.ejs](./partials/dynatrace.ejs)):
+
+- **`frontier_snow_dynatraceRUM`**: Selects the loading mechanism (`asyncCS-script`, `asyncCS-inline`, or `global-cdn`)
+- **`frontier_snow_dynatraceNewRUM`**: Selects the RUM version (old or new)
+
+#### Treatment → snippet-format mapping
+
+The three treatment names are **fixed** (apps still on old RUM depend on them), so for the new RUM we remap each treatment to a Dynatrace snippet format. This is intentional and documented here.
+
+| Treatment | Old RUM (unchanged) | New RUM | Why |
+|---|---|---|---|
+| `asyncCS-script` | Edge CDN `<script async>` | **OneAgent JS tag + SRI, `async`** | Non-blocking; 1-year cached, integrity-verified. Measures the early-capture "blind window." |
+| `asyncCS-inline` | small inline bootstrap (`_inline_*.ejs`) | **OneAgent JS tag + SRI, `sync`** | New RUM has **no small-bootstrap format**; the sync SRI tag is the closest analog (full early capture; parse-blocks only on first uncached load, free on warm cache). **No longer inlines.** |
+| `global-cdn` | `_complete.js` (sync) | **JavaScript tag (`_complete.js`), `async`** | Combined code+config, single request, ~1-hour cache. The caching-strategy comparison point. |
+
+`defer` is intentionally not tested: for a monitoring agent it strictly enlarges the un-instrumented early window for a negligible page-speed gain over `async`.
+
+What to verify per treatment (DevTools → Network / Elements):
+
+```text
+asyncCS-script (new) → <script src=".../sri/ruxitagent_…js" data-dtconfig integrity … async>
+asyncCS-inline (new) → same SRI tag but WITHOUT async (synchronous)
+global-cdn     (new) → <script src=".../{appId}_complete.js" async>   (no integrity/data-dtconfig; config is baked in)
+```
+
+To gauge data loss between sync/async, compare RUM aggregates across treatments (user actions per session, captured XHR/fetch actions, JS error capture rate, and CWV) — or run a synthetic page that fires an early click + XHR + error and confirm each is captured under sync vs async.
+
+### Step 4: Commit and Release
+
+```bash
+# Stage changes (dynatrace.ejs fallback values are auto-updated by the fetch script)
+git add packages/react-scripts/layout/views/partials/dynatrace.ejs
+
+# Commit with descriptive message
+git commit -m "Update Dynatrace RUM to latest version
+
+- Update new-RUM fallback values (cdnUrls/cdnIntegrity/cdnConfig/cdnCompleteUrls) via fetch-dynatrace-scripts.js
+- Republish dynatrace-rum-config.json to S3
+- New RUM version includes enhanced data collection (owasp=1, uxrgce=1)"
+
+# Bump version in package.json (usually minor bump)
+# Example: 8.16.0 → 8.17.0
+sed -i '' 's/"version": "8\.16\.0"/"version": "8.17.0"/' packages/react-scripts/package.json
+
+# Update CHANGELOG-FRONTIER.md
+# Add entry at the top with the new version and upgrade details
+
+# Commit version and changelog
+git add packages/react-scripts/package.json CHANGELOG-FRONTIER.md
+git commit -m "Bump react-scripts to 8.17.0 with Dynatrace RUM upgrade"
+
+# Push when ready
+git push origin dynatrace-rum-upgrade
+```
+
+## Understanding the Output
+
+The API returns complete script tags like:
+
+```html
+<script type="text/javascript" 
+  src="https://js-cdn.dynatrace.com/jstag/15c157a40ab/sri/ruxitagent_ICA7NQVfghqrux_10337260504112723.js" 
+  data-dtconfig="app=3faf90e849295814|..." 
+  integrity="sha256-6PrnNTp1/05LJTqnB9OOhKEVMId1XHk4Xa1x3cPF2Ec=" 
+  crossorigin="anonymous">
+</script>
+```
+
+Key parts:
+- **`src`**: The CDN URL for the external script
+- **`data-dtconfig`**: Configuration string (includes app ID, parameters, etc.)
+- **`integrity`**: SRI hash for script verification
+- **app ID**: The environment-specific application identifier (e.g., `app=3faf90e849295814`)
+
+## Files Modified During Update
+
+```
+packages/react-scripts/layout/views/partials/dynatrace.ejs   (fallback values auto-updated)
+packages/react-scripts/scripts/dynatrace-rum-config.json     (regenerated; also published to S3)
+packages/react-scripts/package.json (version bump)
+CHANGELOG-FRONTIER.md (add entry)
+```
+
+> Note: the new RUM does **not** generate `_inline_*_new.ejs` files (the agent is loaded via `<script>` tags, never inlined). The old-RUM `_inline_*.ejs` bootstrap files remain and are not touched by this script.
+
+## Troubleshooting
+
+**API returns 401 Unauthorized**
+- Check that `DYNATRACE_API_TOKEN` environment variable is set
+- Verify the token has the correct scopes (RUM manual insertion tags read)
+
+**API returns 400 Bad Request**
+- Ensure entity IDs include the `APPLICATION-` prefix
+- Entity IDs must be uppercase
+
+**API returns 404 Not Found**
+- Verify the API endpoint URL is correct: `/api/v2/rum/oneAgentJavaScriptTagWithSri/{entityId}`
+- Check that you're using the `.live.dynatrace.com` domain, not `.apps.dynatrace.com`
+
+## Related Documentation
+
+- [Dynatrace RUM API Documentation](https://docs.dynatrace.com/docs/dynatrace-api/environment-api/rum/rum-manual-insertion-tags)
+- [dynatrace.ejs](./partials/dynatrace.ejs) - Main RUM configuration file
+- [fetch-dynatrace-scripts.js](./fetch-dynatrace-scripts.js) - The fetch script itself
+
+## Version History
+
+- **8.17.0**: First release with dual-flag support (old vs new RUM version)
+  - Introduced `frontier_snow_dynatraceNewRUM` feature flag for gradual rollout
+  - Updated to RUM 1.329+ with enhanced data collection (owasp=1, uxrgce=1)
+  - Added SRI integrity hash support