Skip to content

Add Foundry Toolbox resource and deploy pipeline - #17742

Merged
Eric Erhardt (eerhardt) merged 31 commits into
mainfrom
davidfowl/foundry-toolbox
Sep 8, 2026
Merged

Add Foundry Toolbox resource and deploy pipeline#17742
Eric Erhardt (eerhardt) merged 31 commits into
mainfrom
davidfowl/foundry-toolbox

Conversation

@davidfowl

@davidfowl David Fowler (davidfowl) commented May 31, 2026

Copy link
Copy Markdown
Collaborator

Description

Adds first-class Microsoft Foundry Toolbox support to Aspire.Hosting.Foundry. Toolboxes are Foundry data-plane resources that bundle tools behind one MCP endpoint; they do not have an ARM or Bicep representation.

Usage

var project = builder.AddFoundry("foundry")
    .AddProject("project");
var search = builder.AddAzureSearch("search");

var toolbox = project.AddToolbox("field-tools")
    .WithDescription("Tools for field technicians.")
    .WithWebSearchTool("web-search", "Search the public web.")
    .WithMcpTool(
        "inventory",
        "https://inventory.example.com/mcp",
        new FoundryToolboxMcpToolOptions
        {
            ServerDescription = "Inventory MCP server.",
            ApprovalPolicy = new()
            {
                Global = FoundryToolboxMcpGlobalApprovalMode.Always
            }
        })
    .WithAISearchTool("knowledge-base", search, "docs");

builder.AddProject<Projects.Api>("api")
    .WithReference(toolbox)
    .WaitFor(toolbox);

Existing Toolboxes can be validated without mutation using RunAsExisting(), PublishAsExisting(), or AsExisting(). A specific immutable version can be pinned through AddToolbox options.

MCP endpoints must be reachable from the Foundry data plane over HTTPS. Localhost, loopback, URI credentials, inline headers, and connection-authenticated MCP tools are not supported. MCP approval policies are discovery metadata; consuming applications remain responsible for enforcing approval before invocation.

Deployment behavior

  • Reconciles Toolboxes after the parent Foundry project and referenced deployment resources are ready.
  • Supports read-only existing-resource validation in run and/or deploy mode.
  • Uses Aspire ownership metadata and a deterministic XxHash3 configuration fingerprint.
  • Preserves tool descriptions, MCP server metadata, and global or custom tool-name/read-only approval filters.
  • Reuses the current immutable version when the desired configuration matches.
  • Re-promotes an existing matching version after an interrupted deployment, or creates and promotes a new version when configuration changes.
  • Uses check-act-check ownership validation around creation, reuse, and promotion, failing on contradictory concurrent changes rather than replaying immutable-version creation.
  • Uses the unversioned consumer endpoint by default; callers can explicitly pin an existing immutable version.
  • Gives Azure AI Search connections deterministic identities and applies the required Search roles.

Connection properties

Property Description
Uri Default consumer endpoint, or a version-specific endpoint when pinned
ProjectEndpoint Parent Foundry project endpoint
Name Toolbox name
ApiVersion Toolbox data-plane API version
Version Explicitly pinned immutable version, when configured
FoundryFeatures Required Foundry-Features request-header value
AuthorizationScope Microsoft Entra authorization scope

Coverage

  • 169 Foundry unit tests covering lifecycle modes, authoring payloads, hashing, ownership, concurrency, endpoint validation, connection properties, and deployment ordering.
  • Polyglot validation for the exported TypeScript APIs and option DTOs.
  • Azure deployment E2E coverage in swedencentral that provisions Web Search, MCP, and Azure AI Search tools; validates persisted metadata and Search index configuration; performs an authenticated MCP initialize / tools/list handshake; and verifies a second deployment preserves the default version, hash, and version count.

Checklist

  • Is this feature complete?
    • Yes. Ready for review.
    • No. Follow-up changes expected.
  • Are you including unit tests for the changes and scenario tests if relevant?
    • Yes
    • No
  • Did you add public API?
    • Yes
      • If yes, did you have an API Review for it?
        • Yes
        • No
      • Did you add <remarks /> and <code /> elements on your triple slash comments?
        • Yes
        • No
    • No
  • Does the change make any security assumptions or guarantees?
    • Yes
    • No

Adds first-class Microsoft Foundry Toolbox support to Aspire.Hosting.Foundry.
Toolboxes are a Foundry data-plane construct (no ARM/Bicep) that bundle tools
for agents. Aspire now models them as a resource that is created via the
AgentToolboxes data plane API at deploy time, after the parent project's
endpoint becomes ready.

Public API:
- FoundryToolboxResource (parented to AzureCognitiveServicesProjectResource,
  implements IResourceWithConnectionString)
- AddToolbox extension on the project resource builder
- WithWebSearchTool, WithMcpTool, WithAISearchTool tool-definition extensions

Deploy is wired as a PipelineStepAnnotation so it runs alongside Azure ARM
provisioning: a publish-mode deploy step and a run-mode before-start step that
both call AgentToolboxes.CreateToolboxVersionAsync once the project endpoint
is reachable. Retries with Polly handle the brief window where the freshly
provisioned project endpoint is still warming up.

Polyglot (TypeScript): AddToolboxForPolyglot and WithMcpToolForPolyglot use
AspireExport/AspireUnion so the TS apphost can call addToolbox, withMcpTool,
withWebSearchTool, and withAISearchTool with TS-friendly union types.

Validated end-to-end against a real Azure AI Foundry account: the deploy hook
fired in 3.4s after the project reached Running, and the toolbox was confirmed
present on the data plane (toolboxes API returned field-tools, default_version
1). 102/102 tests pass in Aspire.Hosting.Foundry.Tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented May 31, 2026

Copy link
Copy Markdown
Contributor

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 17742

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/microsoft/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 17742"

David Fowler (davidfowl) and others added 3 commits May 30, 2026 19:20
Temporary instrumentation in AsHostedAgent_ResolvesToolboxConnectionString to capture python app + hosted agent + project env vars and annotations into the assertion failure message. To be reverted once the CI vs local divergence is identified.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ion test

AsHostedAgent_ResolvesToolboxConnectionString was failing on CI (Windows + Ubuntu) with `InvalidOperationException: No output for APPLICATION_INSIGHTS_CONNECTION_STRING on resource my-project`.

AzureHostedAgentResource.GetResolvedEnvironmentVariablesAsync walks every env var callback on the target resource. On CI the python app picks up extra WithReference(project)-style callbacks that splat AzureCognitiveServicesProjectResource.GetConnectionProperties() (Uri / ConnectionString / ApplicationInsightsConnectionString) as MY_PROJECT_* env vars. Those references resolve through BicepOutputReference, which throws if the named output is not seeded on the test resource.

The test is only asserting on ConnectionStrings__field-tools, so unrelated extra env vars should not break it. Seed APPLICATION_INSIGHTS_CONNECTION_STRING (in addition to the existing endpoint output) so every project output reachable through GetConnectionProperties() can be resolved. Also drops the temporary diagnostic dump that was wrapped around the assertion.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Two follow-ups to the Foundry Toolbox MCP integration:

1. FoundryToolboxMcpToolDefinition now exposes AuthorizationTokenExpression
   and a case-insensitive Headers dictionary. ToProjectsAgentToolAsync resolves
   both at deploy time and forwards them to ResponseTool.CreateMcpTool so
   private MCP servers behind a bearer token or custom headers can be wired
   up declaratively. The new WithMcpTool(..., configure) overloads thread an
   optional configuration callback so callers don't have to grab the resource
   to mutate the tool definition.

2. FoundryToolboxResource now installs a PipelineConfigurationAnnotation that
   walks each MCP tool's EndpointExpression via IValueWithReferences and
   wires the toolbox deploy-compute step to depend on the deploy-compute step
   of every referenced compute resource that lives in the app model. This
   removes the manual WaitFor(...) ordering caveat for the common case where
   an MCP tool points at a sibling project/container/app service. Run-mode
   waits are filtered to resources that implement both IComputeResource and
   IResourceWithWaitSupport so we don't hang on Azure-only or model-only
   resources that never publish a Running state.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

Re-running the failed jobs in the CI workflow for this pull request because 1 job was identified as retry-safe transient failures in the CI run attempt.
GitHub was asked to rerun all failed jobs for that attempt, and the rerun is being tracked in the rerun attempt.
The job links below point to the failed attempt jobs that matched the retry-safe transient failure rules.

Adds an end-to-end playground showcasing the new polyglot auth/headers
support on FoundryToolboxMcpToolOptions:

* WithMcpToolForPolyglot now accepts an optional FoundryToolboxMcpToolOptions
  (AuthorizationToken, Headers), projected onto the existing .NET configure
  callback so polyglot AppHosts can wire bearer-token / custom-header MCP
  servers without dropping to C#.
* playground/FoundryAgentToolboxTs/ - a TypeScript AppHost (apphost.mts)
  that wires a Foundry agent + Toolbox to a custom Node MCP server which
  validates Authorization: Bearer <token>. A secret mcp-bearer-token
  parameter is fed into both the MCP server (env var) and the toolbox
  tool definition (authorizationToken), with an extra x-app-source
  custom header to exercise the headers feature.
* The MCP server uses @modelcontextprotocol/sdk with the streamable HTTP
  transport, exposes lookup_employee and record_note tools, and was
  smoke-tested end-to-end (init handshake, tools/list, tools/call) with
  bearer auth enforced.

NuGet.config and 13.5.0-dev pins are temporary until this PR ships in
a release of Aspire.Hosting.Foundry; the README documents the dev-loop
workflow and the cleanup steps.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@davidfowl

Copy link
Copy Markdown
Collaborator Author

Polyglot auth/headers support + TypeScript playground

The latest commit adds end-to-end coverage for bearer-token / custom-header MCP servers from a polyglot AppHost, and a TS playground that exercises it.

What changed

  • WithMcpToolForPolyglot now accepts an optional FoundryToolboxMcpToolOptions (AuthorizationToken, Headers), projected onto the same configure callback the .NET overloads use. Polyglot AppHosts can now wire an MCP server that requires Authorization: Bearer <token> (and arbitrary extra headers) without dropping to C#.
  • New playground at playground/FoundryAgentToolboxTs/ containing:
    • A TypeScript AppHost (apphost.mts) that wires a Foundry agent + Toolbox to a custom Node MCP server.
    • A mcp-bearer-token secret parameter fed into both the MCP server (validation, via env var) and the toolbox tool definition (outbound authorizationToken), plus an extra x-app-source custom header to exercise the new headers feature.
    • A Node-based MCP server (mcp-server/src/server.ts) using @modelcontextprotocol/sdk with the streamable HTTP transport. Exposes lookup_employee and record_note tools.

Validated locally

  • npx tsc --noEmit on apphost.mts is clean against the regenerated polyglot bindings.
  • aspire publish --non-interactive completes all 10 steps. Foundry / project bicep is emitted; toolbox + MCP tool config is intentionally NOT in the published bicep (toolbox is a pure data-plane resource — see caveats).
  • Standalone MCP server bearer-auth smoke test passed 6/6:
    • GET /health no auth → 200
    • POST /mcp no auth → 401
    • POST /mcp wrong bearer → 401
    • POST /mcp authed initialize → 200 + Mcp-Session-Id header + SSE-framed init response (protocolVersion: 2025-06-18, serverInfo: {name: "directory-mcp", version: "1.0.0"})
    • notifications/initialized (202) + tools/list → 200, both tools returned with full JSON Schema
    • tools/call lookup_employee("grace hopper") → 200, returns the seeded directory record

Caveats (worth being honest about)

  • Toolbox is data-plane only. The MCP tool configuration (URL, auth, headers) does not appear in the published Bicep, so aspire publish alone is not enough to prove a full end-to-end flow — it confirms the AppHost graph and the Foundry/project provisioning. A aspire deploy against a real subscription is required for the Foundry data plane to actually register and invoke the tool.
  • Foundry data plane cannot reach localhost MCP servers. In aspire start mode the data plane runs in Azure, so it cannot call http://localhost:*. You can still verify all resources are healthy and that the MCP server enforces bearer auth. For a real call, either deploy with aspire deploy or front the local MCP server with a tunnel (dev tunnels / ngrok).
  • Run mode requires Azure. FoundryToolboxResource always adds a pre-start deploy-{Name}-before-start pipeline step depending on run-mode-azure-provision, so aspire start against this playground will prompt for a subscription. This matches the existing FoundryAgentBasic playground.
  • Local-only wiring in the playground. NuGet.config + 13.5.0-dev pins in aspire.config.json point at locally-built artifacts because this PR's polyglot API is not yet in a shipped release. The README documents the cleanup: once Add Foundry Toolbox resource and deploy pipeline #17742 ships, delete NuGet.config and reset both packages back to "" so the playground follows the standard convention used by playground/TypeScriptAppHost.

How to try it

From the repo root:

dotnet pack src/Aspire.Hosting.Foundry/Aspire.Hosting.Foundry.csproj    -c Debug --no-restore
dotnet pack src/Aspire.Hosting.JavaScript/Aspire.Hosting.JavaScript.csproj -c Debug --no-restore

cd playground/FoundryAgentToolboxTs/mcp-server && npm install && cd ..
aspire restore --apphost apphost.mts
aspire publish --non-interactive   # or: aspire start

@davidfowl

Copy link
Copy Markdown
Collaborator Author

PR Testing Report

PR Information

  • PR Number: Add Foundry Toolbox resource and deploy pipeline #17742
  • Title: Foundry Toolbox: TS apphost + bearer-auth MCP polyglot binding
  • Head Commit: 65205f17c2… (branch davidfowl/foundry-toolbox)
  • Tested At: 2026-05-31 (Windows 11 / PowerShell)
  • Mode: Local (no container) — isolated temp dir, host gh auth preserved
  • Tester: GitHub Copilot CLI (autopilot, model: claude-opus-4.7-xhigh)

CLI Version Verification

  • Expected (PR head): 65205f17c2…
  • Installed CLI: 13.5.0-pr.17742.g65205f17
  • Status: ✅ Verified — short SHA g65205f17 matches PR head
  • Install path: C:\Users\davifowl\AppData\Local\Temp\aspire-pr-test-7a335772\dogfood\pr-17742\bin\aspire.exe
  • Hive path: …\hives\pr-17742\packages (110 .nupkg)
  • Install method: get-aspire-cli-pr.ps1 17742 -InstallPath … -SkipExtension -SkipPath (no HOME/USERPROFILE/APPDATA overrides)

Changes Analyzed

Files changed (highlights)

  • src/Aspire.Hosting.Foundry/FoundryToolboxMcpToolDefinition.cs — DTO definition; new [AspireDto] options surface (AuthorizationToken, Headers).
  • src/Aspire.Hosting.Foundry/FoundryToolboxResource.cs (and extensions) — WithMcpTool(name, urlExpr, options) polyglot binding wiring authorizationToken + headers through ResponseTool.CreateMcpTool.
  • playground/FoundryAgentToolboxTs/ — new TS apphost playground exercising the polyglot binding end-to-end, including a Node MCP server protected by a bearer token validated middleware-side and a token parameter wired into both sides.
  • Code-generation projects (Aspire.Hosting.CodeGeneration.TypeScript / Aspire.TypeSystem) — emit the new options DTO surface so TS apphosts can pass { authorizationToken, headers }.

Change categories

  • Hosting integration changes (src/Aspire.Hosting.Foundry/*)
  • Polyglot/TypeScript codegen surface
  • Playground (TS apphost sample)
  • CLI changes
  • Dashboard changes
  • Client/Component changes

Test Scenarios Executed

S1 — CLI version pin matches PR head

Coverage: Verification
Result: ✅ Passed

aspire --version13.5.0-pr.17742.g65205f17. Short SHA matches PR head. aspire.config.json versions were pinned to the same string for downstream scenarios.


S2 — aspire-empty publish smoke test (baseline)

Coverage: Happy path / regression baseline
Result: ✅ Passed (4/4 pipeline steps)

Created a fresh aspire-empty C# file-based apphost (apphost.cs referencing Aspire.AppHost.Sdk@13.5.0-pr.17742.g65205f17) and ran aspire publish --non-interactive. All four publish pipeline steps completed; this proves the PR CLI is fully functional on the host before exercising Foundry-specific paths.

Non-interactive flags discovered/required for aspire-empty: --language csharp --non-interactive --localhost-tld false --suppress-agent-init. Without --language it still prompts and fails in non-interactive mode.


S3 — TS Foundry Toolbox playground: restore + publish

Coverage: Happy path — primary target of this PR
Result: ✅ Passed (restore exit 0, publish 10/10 steps)

Copied playground/FoundryAgentToolboxTs/ to …\scenario-foundry\FoundryPlay\ verbatim. Rewrote aspire.config.json to pin versions to 13.5.0-pr.17742.g65205f17, and authored a local NuGet.config whose packageSourceMapping pointed Aspire.* at the PR hive (see Recommendations).

The playground's apphost.mts is the exact code path the PR introduces:

const bearerToken = builder.addParameter('mcp-bearer-token', { secret: true });

const mcpServer = builder.addNodeApp('mcp-server', './mcp-server/server.mjs', './mcp-server')
  .withHttpEndpoint({ env: 'PORT' })
  .withEnvironment('MCP_BEARER_TOKEN', bearerToken);

const foundry = builder.addFoundry('foundry');
const project = foundry.addProject('project');

const toolbox = project.addToolbox('toolbox');
toolbox.withMcpTool('internal-tools', mcpServer.getEndpoint('http'), {
  authorizationToken: refExpr`${bearerToken}`,
  headers: { 'x-app-source': refExpr`foundry-toolbox-ts-sample` },
});

Run:

${env:Parameters__mcp-bearer-token} = "dogfood-secret-token-abc123"   # required for non-interactive
& $cliPath restore --apphost .\apphost.mts     # exit 0
& $cliPath publish --non-interactive --apphost .\apphost.mts          # 10/10 steps, exit 0

Artifacts generated (aspire-output/):

  • main.bicep — top-level deployment template (Foundry account + project + AI connection)
  • foundry/foundry.bicepMicrosoft.CognitiveServices/accounts (Foundry account) + identity / role assignments
  • project/project.bicep — Foundry project + model deployment + AI connection

What does not show up in Bicep — and why that is expected: the toolbox's mcp-tool is registered against the Foundry data plane at deploy time (via ResponseTool.CreateMcpTool), not via an ARM resource. This is by design — Foundry MCP tools are not first-class ARM objects. Verification that the PR's wiring works is therefore "publish completed without surfacing the new options as errors and the authorizationToken / headers references resolved cleanly through the codegen layer."

Similarly, the mcp-server Node app is not in the publish artifacts because the apphost has no addAzureContainerAppEnvironment call. See Recommendations — the playground should optionally model a compute env so the MCP server gets deployed alongside Foundry for a true E2E.


S4 — Unhappy path: nonexistent apphost

Coverage: Unhappy path / argument validation
Result: ✅ Passed (clean failure)

& $cliPath restore --apphost .\does-not-exist.mts   # exit 7
& $cliPath publish --apphost .\does-not-exist.mts   # exit 7

Both commands returned a clear, user-readable error stating the project file does not exist. No stack trace leakage. Exit code is consistent (7) across commands.


S5 — Unhappy path: non-interactive publish without secret parameter value

Coverage: Unhappy path / required-input validation
Result: ✅ Passed (clean failure with exit 6) — with a minor UX nit (see below)

Ran aspire publish --non-interactive against a freshly-copied playground directory with no Parameters__mcp-bearer-token env var and no prior user-secrets state. Publish stopped at process-parameters with:

❌ The  option must be specified when running in non-interactive mode.
❌ An unexpected error occurred: Required input  was not provided in non-interactive mode.

Exit code 6. Behavior is correct — non-interactive runs fail loudly rather than hanging waiting for input.

UX nit (worth filing as a follow-up): the parameter name is missing from the error message ("The option must be specified" instead of "The mcp-bearer-token option must be specified"). It looks like the formatter swallows parameter names that contain hyphens, or the name is being resolved from the wrong field. Minor but user-facing.

Side observation about parameter caching: after a single successful publish (S3) saved the parameter value to per-apphost user-secrets state, subsequent publishes in the same directory succeed without the env var. This is expected and matches the existing parameter-prompting model; just worth being aware of when reproducing this S5 — copy the apphost to a fresh directory to escape the cache.

Summary

# Scenario Coverage Result Exit
S1 CLI version pin matches PR head Verification ✅ Pass
S2 aspire-empty publish baseline Happy path ✅ Pass (4/4) 0
S3 TS Foundry Toolbox playground restore + publish Happy path (PR target) ✅ Pass (10/10) 0
S4 --apphost <nonexistent> for restore + publish Unhappy path ✅ Pass (clean error) 7
S5 Non-interactive publish without secret param Unhappy path ✅ Pass (clean failure, exit 6) 6

Overall Result

✅ PR VERIFIED

The new WithMcpTool(name, urlExpr, options) polyglot binding on Aspire.Hosting.Foundry works end-to-end from a TypeScript apphost through aspire restore and aspire publish against the PR's CLI build. The authorizationToken and headers options flow cleanly through the codegen layer; publish produces correct Foundry Bicep artifacts; unhappy paths fail loudly with consistent exit codes.

Recommendations / follow-ups

  1. Playground packageSourceMapping papercut. Aspire.Hosting.Foundry pulls a deep transitive Aspire surface (Aspire.Hosting.Azure.*, Aspire.Hosting.CodeGeneration.TypeScript, and notably Aspire.TypeSystem which lacks the .Hosting. prefix). When pinning a playground to a hive via NuGet.config packageSourceMapping, narrow patterns like Aspire.Hosting.Foundry or even Aspire.Hosting.* fail. Use <package pattern="Aspire.*" /> to be safe. Worth a docs note next to the existing playground README, or a small aspire-restore-time hint.
  2. Sample apphost should optionally model a compute env. The TS playground's mcp-server Node app is defined but never makes it into the publish artifacts because there's no addAzureContainerAppEnvironment on the builder. Documenting (or wiring) a compute env in the sample would let a real user actually deploy the MCP alongside Foundry and exercise the bearer-token round-trip from the Foundry data plane. Today the sample proves the binding works; it doesn't prove an actually-reachable MCP gets called.
  3. Missing parameter name in non-interactive error message. S5 surfaced The option must be specified when running in non-interactive mode. with the parameter name blank. Likely a small formatter bug; consider a regression test on a hyphenated parameter name.
  4. Run-mode caveat (already documented in PR/README, repeating for visibility). Foundry data-plane → localhost:<port> is not reachable. Run-mode exercising of the agent against the MCP requires a tunnel (dev tunnels / ngrok). The PR is honest about this; publish-mode against ACA with public ingress is the supported E2E. Per author's PR comments, the deploy-time ordering between the toolbox mcp-tool registration and the MCP compute container is also a known gap.

Evidence

All evidence files live under C:\Users\davifowl\AppData\Local\Temp\aspire-pr-test-7a335772\:

  • scenario-empty/empty-publish.log — S2 baseline publish output
  • scenario-foundry/restore3.log — S3 restore output
  • scenario-foundry/publish.log, publish2.log — S3 publish output (10/10 steps)
  • scenario-foundry/FoundryPlay/aspire-output/{main,foundry/foundry,project/project}.bicep — generated artifacts
  • scenario-unhappy-restore.log, scenario-unhappy-publish.log — S4 nonexistent-apphost output
  • scenario-clean/publish-noparam.log — S5 clean "missing required input" failure

@davidfowl

Copy link
Copy Markdown
Collaborator Author

Follow-up: in-repo dogfood test of playground/FoundryAgentToolboxTs

Re-tested the PR against an unmodified clone of this branch, using the in-repo playground exactly as playground/FoundryAgentToolboxTs/README.md documents (no temp-dir, no PR CLI download, just dotnet pack + the host's installed aspire CLI). End-to-end the bearer-auth path of the new polyglot binding works:

Setup (matches the README)

# from repo root, with the local SDK
.\.dotnet\dotnet.exe pack src\Aspire.Hosting.Foundry\Aspire.Hosting.Foundry.csproj -c Debug
.\.dotnet\dotnet.exe pack src\Aspire.Hosting.JavaScript\Aspire.Hosting.JavaScript.csproj -c Debug

# clear cached extracted packages so 13.5.0-dev re-extracts from the fresh nupkgs
Remove-Item -Recurse -Force $env:USERPROFILE\.nuget\packages\aspire.hosting.foundry\13.5.0-dev -EA SilentlyContinue
Remove-Item -Recurse -Force $env:USERPROFILE\.nuget\packages\aspire.hosting.javascript\13.5.0-dev -EA SilentlyContinue

cd playground\FoundryAgentToolboxTs
${env:Parameters__mcp-bearer-token} = "in-repo-dogfood-token-xyz"

aspire restore --apphost apphost.mts   # exit 0
aspire publish --apphost apphost.mts --non-interactive   # 10/10 steps, generates main/foundry/project bicep
aspire start   --apphost apphost.mts   # dashboard + mcp-server up

The playground already ships a NuGet.config mapping Aspire.Hosting.Foundry and Aspire.Hosting.JavaScript to ..\..\artifacts\packages\Debug\Shipping and an aspire.config.json pinning both to 13.5.0-dev, so the host's aspire 13.4.0 CLI just shells out to NuGet to pick them up — no version-drift workaround needed.

MCP bearer-auth round trip (the actual PR feature)

# 1. health (no auth) -> 200
curl http://localhost:<port>/health
# {"status":"ok"}

# 2. /mcp without Authorization -> 401
curl -X POST http://localhost:<port>/mcp -H 'Content-Type: application/json' -d '{"jsonrpc":"2.0","method":"initialize",...}'
# {"error":"unauthorized","message":"Missing or invalid bearer token."}

# 3. /mcp with correct bearer + initialize -> 200, SSE handshake completes
curl -X POST http://localhost:<port>/mcp \
  -H 'Authorization: Bearer in-repo-dogfood-token-xyz' \
  -H 'Accept: application/json, text/event-stream' \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","method":"initialize","params":{...},"id":1}'
# < mcp-session-id: <uuid>
# event: message
# data: {"result":{"protocolVersion":"2025-06-18","capabilities":{"tools":{"listChanged":true}},"serverInfo":{"name":"directory-mcp","version":"1.0.0"}},...}

# 4. /mcp tools/call lookup_employee with the session -> tool runs
curl -X POST http://localhost:<port>/mcp \
  -H 'Authorization: Bearer in-repo-dogfood-token-xyz' \
  -H 'Mcp-Session-Id: <uuid>' \
  -H 'Accept: application/json, text/event-stream' \
  -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","method":"tools/call","params":{"name":"lookup_employee","arguments":{"name":"Ada Lovelace"}},"id":3}'
# event: message
# data: {"result":{"content":[{"type":"text","text":"Name: Ada Lovelace\nTitle: Principal Engineer\nOffice: London\nEmail: ada@contoso.example"}]},...}

# 5. /mcp with WRONG bearer -> 401
# {"error":"unauthorized","message":"Missing or invalid bearer token."}

So the bearer-auth path the PR adds (FoundryToolboxMcpToolOptions.AuthorizationToken) is exercised end-to-end against the same token the AppHost wired into both sides of the connection. The Foundry data plane itself can't reach localhost (well-documented caveat in the README), so this validates the wiring without provisioning Azure.

Dashboard

7 resources show up, including the new FoundryToolboxResource (field-tools) in Waiting for project — confirms the toolbox + WithMcpTool(..., { authorizationToken: ... }) deserialized correctly through the TS AppHost generator and bound into the application model.

Resource Type State
mcp-server Executable (npm) Running, Healthy, http://localhost:<port>
mcp-server-installer Executable (npm install) Finished, exit 0
foundry FoundryResource Starting (needs Azure creds)
project AzureCognitiveServicesProjectResource Starting
field-tools FoundryToolboxResource Waiting for project
foundry-roles AzureRoleAssignmentResource Starting
chat FoundryDeploymentResource Starting

Note for the earlier comment

My earlier dogfood report recommended that the docs call out a packageSourceMapping papercut for external testers building synthetic test fixtures. That recommendation does not apply to in-repo developers — the existing playground/FoundryAgentToolboxTs/NuGet.config already covers this case, and dotnet pack + aspire start/publish from the playground dir is the supported in-repo workflow. The papercut only matters if someone outside the repo tries to construct their own minimal test against the PR-built packages.

Result

✅ Verified end-to-end: pack → restore → publish (10/10 steps, bicep emitted) → start → dashboard healthy → MCP bearer auth round-trip with tools/call returning the expected payload.

@davidfowl

Copy link
Copy Markdown
Collaborator Author

Follow-up: two dogfood findings from a full Azure E2E

I took the playground all the way through to a real aspire deploy against
Azure and an actual tools/list round-trip from a Foundry agent through the
deployed toolbox. Two findings came out of it — one I'm fixing in this PR,
one is a Foundry-side gap.

Finding 1 — polyglot AppHostServer cross-ALC IPersistableModel<T> mismatch (fixed in this PR)

The original FoundryToolboxWebSearchToolDefinition.ToProjectsAgentToolAsync
and FoundryToolboxMcpToolDefinition.ToProjectsAgentToolAsync both went
OpenAI.Responses.ResponseToolModelReaderWriter.Write<T>(...)
BinaryDataModelReaderWriter.Read<ProjectsAgentTool>(...) to translate
the OpenAI Responses tool shape into the Azure.AI.Projects.Agents tool shape.

In the standalone .NET AppHost case this is fine, all assemblies load into
the default ALC and the type identity round-trips cleanly.

In the polyglot TS AppHost case (apphost.mts running under
Aspire.Hosting.RemoteHost's prebuilt server), IntegrationLoadContext
loads probe assemblies version-by-version. Today the prebuilt AppHostServer
ships System.ClientModel 1.10.0 in the default ALC. Foundry's probe
brings System.ClientModel 1.11.0. The load policy in
IntegrationLoadContext.LoadProbedAssembly defers OpenAI 2.10.0 to the
default ALC (because the probed version isn't higher than the default), but
loads SCM 1.11.0 into the probe ALC — so OpenAI is bound against SCM 1.10.0
and the rest of the probe sees SCM 1.11.0. ResponseTool ends up
implementing IPersistableModel<WebSearchTool> from SCM 1.10.0, but the
MRW.Write<T>(...) call inside the probe ALC checks for
IPersistableModel<> from SCM 1.11.0. Types don't match, so MRW falls
through to "no model writer" and throws:

System.InvalidOperationException: 'WebSearchTool' must implement IEnumerable or IPersistableModel<T>.
   at System.ClientModel.Primitives.ModelReaderWriter.Write[T](T value, ...)
   at OpenAI.Responses.ResponseTool.System.ClientModel.Primitives.IPersistableModel<ResponseTool>.Write(...)
   at Aspire.Hosting.Foundry.Toolbox.FoundryToolboxWebSearchToolDefinition.ToProjectsAgentToolAsync(...)

The exception text is misleading — WebSearchTool does implement
IPersistableModel<WebSearchTool>. It's just the wrong copy of it, from the
wrong ALC. (I confirmed this by decompiling both SCM 1.10.0 and 1.11.0 from
their respective NuGet caches: same type name, same shape, two ALCs.)

Fix in this PR: stop calling MRW.Write<ResponseTool>(...) from inside
the polyglot ALC. Both tool definitions now hand-build the wire JSON for
the OpenAI Responses MCP / web-search shape and pass it directly to
MRW.Read<ProjectsAgentTool>(...). Read stays inside a single ALC
(Azure.AI.Projects.Agents + probe-ALC SCM 1.11.0 + Utf8JsonWriter /
BinaryData from the BCL), so the type-identity issue can't bite.

Wire shapes are taken straight from the openai-dotnet Serialization.cs
files; the McpTool overload writes the full
type / server_label / server_url / authorization / headers set when
configured.

Verified end-to-end: after the fix, aspire deploy against
playground/FoundryAgentToolboxTs cleanly completes the deploy-field-tools
step, and a REST GET /toolboxes/field-tools/versions/1?api-version=v1
returns:

{
  "tools": [
    { "type": "web_search" },
    {
      "type": "mcp",
      "server_label": "directory",
      "server_url": "https://mcp-server.blackforest-b4169b0d.eastus2.azurecontainerapps.io/mcp",
      "authorization": "tunnel-e2e-token-...",
      "headers": { "x-app-source": "foundry-toolbox-ts-sample" }
    }
  ]
}

Broader follow-up worth filing separately: the prebuilt AppHostServer
shipping SCM 1.10.0 is a latent compatibility problem for any hosting
integration that goes through cross-ALC IPersistableModel<T> against
default-ALC-bound types. Two reasonable fixes upstream: (a) bump the
prebuilt host's SCM to match what current Azure SDKs target, or (b) treat
System.ClientModel as a shared assembly in
IntegrationLoadContext.GetSharedAssemblyNames() (it's currently only
Aspire.TypeSystem). I'll open a separate issue for that.

Finding 2 — Foundry data plane silently drops inline authorization / headers

The toolbox version stores the bearer token and custom header exactly as
the AppHost wired them (see the JSON above). But when Foundry actually
invokes the MCP server through the toolbox, those fields are dropped on
the floor.

Repro:

  1. Add diagnostic logging in the MCP server's bearer middleware that logs
    method, path, user-agent, and a masked authHeader per request.

  2. From a Foundry agent, list/invoke a tool through the toolbox.

  3. MCP server log:

    [auth] POST /mcp ua="<none>" authHeader="<missing>" match=false
    [auth] GET  /mcp ua="<none>" authHeader="<missing>" match=false
    

Foundry returns 424 / 401 to the agent, with the response body being the
exact unauthorized error our MCP server emits — so Foundry IS reaching the
MCP server, it just isn't sending the auth headers the toolbox version
stored.

Why: ResponseTool.CreateMcpTool(authorizationToken: ..., headers: ...)
and FoundryToolboxMcpToolOptions.AuthorizationToken / Headers belong to
the OpenAI Responses MCP shape, which is the agent → MCP auth surface.
It works correctly when an agent calls an MCP server directly, or when the
consumer calls the toolbox itself as an MCP server using an AAD token.

For the toolbox-deployed → downstream MCP hop, the Foundry toolbox
proxy ignores those inline fields. The documented auth surface for that
hop is project_connection_id referencing a key-auth project connection
(see the JS/Python samples in
the toolbox doc).
The .NET ResponseTool.CreateMcpTool overload doesn't surface
project_connection_id yet, so there's no OpenAI.Responses-level way for
FoundryToolboxMcpToolDefinition to wire that today.

This is a Foundry-side gap, not something to fix in this PR. Two
suggestions for follow-up:

  1. Document the limitation on the new FoundryToolboxMcpToolOptions
    members (noting that they DO work for consumer-side WithReference
    patterns where the same AppHost wires both ends).
  2. Once the OpenAI .NET SDK exposes project_connection_id for
    CreateMcpTool, surface it as something like
    withMcpTool(..., { projectConnection: keyAuthConnection }), where
    keyAuthConnection is a FoundryProjectConnectionResource the AppHost
    creates and binds to the bearer parameter.

Repro environment

Piece Value
Playground playground/FoundryAgentToolboxTs (this branch)
Foundry account foundry-a3lhnn2n4wpfk.services.ai.azure.com, project project
Toolbox field-tools (toolbox_50aa9be1ce950b7c9bacebc7dd2d3c58a9e2c508), version 1
MCP server Node @modelcontextprotocol/sdk HTTP transport at https://mcp-server.blackforest-b4169b0d.eastus2.azurecontainerapps.io/mcp
Deploy CLI aspire 13.4.0+64efcefb + this branch's Aspire.Hosting.Foundry in the NuGet cache

… AppHost

The natural FoundryToolbox{WebSearch,Mcp}ToolDefinition.ToProjectsAgentToolAsync
implementation goes through ResponseTool.Create*().AsAgentTool(), which internally
calls ModelReaderWriter.Write<T> on an OpenAI.Responses tool. In a normal .NET
process this works. In the polyglot (TypeScript) AppHostServer, IntegrationLoadContext
loads probe assemblies into a separate ALC, and the prebuilt host's System.ClientModel
1.10.0 ends up bound to OpenAI in the default ALC while the integration's
System.ClientModel 1.11.0 lives in the probe ALC. The two SCMs surface as distinct
CLR assemblies, so the IPersistableModel<WebSearchTool> interface check inside
ModelReaderWriter.Write fails with a misleading 'must implement IEnumerable or
IPersistableModel<T>' exception.`Fix: stop calling MRW.Write across the ALC boundary. Both definitions now hand-build
the OpenAI Responses wire JSON and feed it to MRW.Read<ProjectsAgentTool>(...,
AzureAIProjectsAgentsContext.Default), which stays inside a single ALC (BCL +
Azure.AI.Projects.Agents + probe-ALC SCM). Wire shapes match openai-dotnet's
McpTool.Serialization.cs and WebSearchTool.Serialization.cs.`Also widens the polyglot WithMcpToolForPolyglot union to accept ReferenceExpression
so TS apphosts can use refExpr to compose URLs like \
efExpr\\/mcp\\.`Validated end-to-end against playground/FoundryAgentToolboxTs deployed to Azure:
aspire deploy completes deploy-field-tools cleanly and the resulting toolbox
version contains both {type:web_search} and the full MCP tool with auth + headers.`Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…pology

Iterates on playground/FoundryAgentToolboxTs to exercise the full bearer-auth
MCP path against a real Foundry data plane:`- apphost.mts: dual run/publish topology. In run mode the local MCP server is
  fronted by an Aspire dev tunnel so the Foundry data plane can reach it; in
  publish mode it deploys as a NodeApp container with public ingress and the
  toolbox points at its public ACA endpoint. Documents the Foundry data-plane
  inline-auth gap inline.
- mcp-server/src/server.ts: bearer auth middleware + request logging so we
  can verify whether the Foundry proxy is forwarding the Authorization header.
- scripts/invoke-toolbox.mjs: standalone Node consumer that creates a Foundry
  agent against the deployed toolbox and runs a thread end-to-end. Useful for
  smoke-testing toolbox deploys without the dashboard.
- NuGet.config / aspire.config.json: add Aspire.Hosting.DevTunnels and
  Aspire.Hosting.Azure.AppContainers package mappings.`Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

CLI E2E Tests unknown — 110 passed, 0 failed, 2 unknown (commit 454e116)

View all recordings
Status Test Recording Job Artifacts
AddPackageInteractiveWhileAppHostRunningDetached Recording #78770028690 Logs
AddPackageWhileAppHostRunningDetached Recording #78770028690 Logs
AgentCommands_AllHelpOutputs_AreCorrect Recording #78770028854 Logs
AgentInitCommand_DefaultSelection_InstallsDefaultSkills Recording #78770028854 Logs
AgentInitCommand_MigratesDeprecatedConfig Recording #78770028854 Logs
AgentInitCommand_NonInteractive_BundleOnlySkillsBeyondCliCatalog_AreInstallable Recording #78770028854 Logs
AgentMcpListStructuredLogsReturnsLogsFromStarterApp Recording #78770028684 Logs
AgentMcpListStructuredLogsReturnsLogsFromStarterApp_DevLocalhost Recording #78770028684 Logs
AgentMcpListStructuredLogsReturnsLogsFromStarterApp_Isolated Recording #78770028684 Logs
AllPublishMethodsBuildDockerImages Recording #78770028687 Logs
AspireAddAndStartWorkAgainstLegacyAppHostTs Recording #78770028773 Logs
AspireAddPackageVersionToDirectoryPackagesProps Recording #78770028868 Logs
AspireInitSingleFileAppHostRunsViaDotnetRunAppHost Recording #78770028700 Logs
AspireInitWithExistingAppHostDirRecreatesMissingNuGetConfigAndPreservesFiles Recording #78770028941 Logs
AspireInitWithSolutionFileGeneratesAppHostThatBuildsAgainstChannelHive Recording #78770028941 Logs
AspireStartUpdatesStaleTypeScriptAppHostPath Recording #78770028736 Logs
AspireUpdateRemovesAppHostPackageVersionFromDirectoryPackagesProps Recording #78770028868 Logs
AspireUpdateRemovesOrphanAppHostPackageVersionWhenSdkAlreadyCurrent Recording #78770028868 Logs
Banner_DisplayedOnFirstRun Recording #78770028483 Logs
Banner_DisplayedWithExplicitFlag Recording #78770028483 Logs
Banner_NotDisplayedWithNoLogoFlag Recording #78770028483 Logs
CertificatesClean_RemovesCertificates Recording #78770028546 Logs
CertificatesTrust_WithNoCert_CreatesAndTrustsCertificate Recording #78770028546 Logs
CertificatesTrust_WithUntrustedCert_TrustsCertificate Recording #78770028546 Logs
ConfigSetGet_CreatesNestedJsonFormat Recording #78770028619 Logs
CreateAndRunAspireStarterProject Recording #78770028839 Logs
CreateAndRunAspireStarterProjectWithBundle Recording #78770028446 Logs
CreateAndRunEmptyAppHostProject Recording #78770028757 Logs
CreateAndRunJavaEmptyAppHostProject Recording #78770028500 Logs
CreateAndRunJsReactProject Recording #78770028775 Logs
CreateAndRunPolyglotAppHostWithDevLocalhostUrls Recording #78770028839 Logs
CreateAndRunPythonReactProject Recording #78770028848 Logs
CreateAndRunTypeScriptEmptyAppHostProject Recording #78770028464 Logs
CreateAndRunTypeScriptStarterProject Recording #78770028802 Logs
CreateJavaAppHostWithViteApp Recording #78770028821 Logs
CreateTypeScriptAppHostWithViteApp_AllowsGuestAppPackageManagerToDiffer Recording #78770028626 Logs
CreateTypeScriptAppHostWithViteApp_UsesConfiguredToolchain Recording #78770028626 Logs
DashboardRunWithAgentMcpListTracesReturnsNoTraces Recording #78770028542 Logs
DashboardRunWithAgentMcpListTracesReturnsNoTraces_DevLocalhost Recording #78770028542 Logs
DashboardRunWithOtelTracesReturnsNoTraces Recording #78770028542 Logs
DashboardRunWithOtelTracesReturnsNoTraces_DevLocalhost Recording #78770028542 Logs
DeployK8sBasicApiService Recording #78770028547 Logs
DeployK8sWithExternalHelmChart Recording #78770028803 Logs
DeployK8sWithGarnet Recording #78770028686 Logs
DeployK8sWithMongoDB Recording #78770028484 Logs
DeployK8sWithMySql Recording #78770028770 Logs
DeployK8sWithPostgres Recording #78770028589 Logs
DeployK8sWithRabbitMQ Recording #78770028582 Logs
DeployK8sWithRedis Recording #78770028794 Logs
DeployK8sWithSqlServer Recording #78770028711 Logs
DeployK8sWithValkey Recording #78770028806 Logs
DeployTypeScriptAppToKubernetes Recording #78770028828 Logs
DescribeCommandResolvesReplicaNames Recording #78770028544 Logs
DescribeCommandShowsRunningResources Recording #78770028544 Logs
DetachFormatJsonProducesValidJson Recording #78770028915 Logs
DetachFormatJsonProducesValidJsonWhenRestartingExistingInstance Recording #78770028915 Logs
DoPublishAndDeployListStepsWork Recording #78770028807 Logs
DocsCommand_RendersInteractiveMarkdownFromLocalSource Recording #78770028617 Logs
DoctorCommand_DetectsDeprecatedAgentConfig Recording #78770028854 Logs
DoctorCommand_TypeScriptAppHostReportsMissingConfiguredToolchain Recording #78770028718 Logs
DoctorCommand_WithSslCertDir_ShowsTrusted Recording #78770028718 Logs
DoctorCommand_WithoutSslCertDir_ShowsPartiallyTrusted Recording #78770028718 Logs
GatewayWithoutExternalEndpoint_FailsPublishWithGuidance Recording #78770028733 Logs
GeneratedAspireDevScript_StartsWatchMode_WithConfiguredToolchain Recording #78770028626 Logs
GlobalMigration_HandlesCommentsAndTrailingCommas Recording #78770028619 Logs
GlobalMigration_HandlesMalformedLegacyJson Recording #78770028619 Logs
GlobalMigration_PreservesAllValueTypes Recording #78770028619 Logs
GlobalMigration_SkipsWhenNewConfigExists Recording #78770028619 Logs
GlobalSettings_MigratedFromLegacyFormat Recording #78770028619 Logs
IngressWithoutExternalEndpoint_FailsPublishWithGuidance Recording #78770028733 Logs
InitTypeScriptAppHost_AugmentsExistingViteRepoInWorkspaceSubdirectory Recording #78770028626 Logs
InteractiveCSharpInitCreatesExpectedFiles Recording #78770028789 Logs
InvalidAppHostPathWithComments_IsHealedOnRun Recording #78770028817 Logs
JavaScriptHostingApisRunFromTypeScriptAppHost Recording #78770028687 Logs
LatestCliCanStartStableChannelAppHost Recording #78770028839 Logs
LatestCliCanStartStableChannelTypeScriptAppHost Recording #78770028839 Logs
LegacySettingsMigration_AdjustsRelativeAppHostPath Recording #78770028736 Logs
LogsCommandShowsResourceLogs Recording #78770028865 Logs
OtelLogsReturnsStructuredLogsFromStarterApp Recording #78770028587 Logs
OtelLogsReturnsStructuredLogsFromStarterAppIsolated Recording #78770028587 Logs
PsCommandListsRunningAppHost Recording #78770028452 Logs
PsFormatJsonOutputsOnlyJsonToStdout Recording #78770028452 Logs
PublishJavaScriptPatternsGeneratesExpectedDockerComposeArtifacts Recording #78770028852 Logs
PublishWithConfigureEnvFileUpdatesEnvOutput Recording #78770028852 Logs
PublishWithDockerComposeServiceCallbackSucceeds Recording #78770028852 Logs
PublishWithoutOutputPathUsesAppHostDirectoryDefault Recording #78770028852 Logs
ResourceCommand_FailedExecution_DisplaysAppHostLogPathAndLogContainsEntries Recording #78770028470 Logs
ResourceCommand_SetAndDeleteParameterUpdatesDescribeOutput Recording #78770028470 Logs
RestoreGeneratesSdkFiles Recording #78770028844 Logs
RestoreGeneratesSdkFiles_WithConfiguredToolchain Recording #78770028934 Logs
RestoreRefreshesGeneratedSdkAfterAddingIntegration Recording #78770028934 Logs
RestoreSupportsConfigOnlyHelperPackageAndCrossPackageTypes Recording #78770028623 Logs
RunFromParentDirectory_UsesExistingConfigNearAppHost Recording #78770028480 Logs
RunReportsSyntaxErrorsForDotNetAppHost Recording #78770028823 Logs
RunReportsSyntaxErrorsForTypeScriptAppHost Recording #78770028823 Logs
SecretCrudOnDotNetAppHost Recording #78770028629 Logs
SecretCrudOnTypeScriptAppHost Recording #78770028471 Logs
StagingChannel_ConfigureAndVerifySettings_ThenSwitchChannels Recording #78770028624 Logs
StartAndWaitForTypeScriptSqlServerAppHostWithNativeAssets Recording #78770028490 Logs
StartReportsSyntaxErrorsForDotNetAppHost Recording #78770028823 Logs
StartReportsSyntaxErrorsForTypeScriptAppHost Recording #78770028823 Logs
StopAllAppHostsFromAppHostDirectory Recording #78770028513 Logs
StopJavaPolyglotAppHostUsingApphostDirectory Recording #78770028866 Logs
StopNonInteractiveSingleAppHost Recording #78770028513 Logs
StopTypeScriptPolyglotAppHostUsingApphostDirectory Recording #78770028510 Logs
StopWithNoRunningAppHostExitsSuccessfully Recording #78770028690 Logs
TypeScriptAppHostRunDoesNotDeadlockWhenLazyOptionsInvokeAsyncCallback Recording #78770028464 Logs
UnAwaitedChainsCompileWithAutoResolvePromises Recording #78770028934 Logs
UpdateProjectChannelToStable_CSharpEmptyAppHost_PreservesAspireConfigChannel Recording #78770028555 Logs
UpdateProjectChannelToStable_CSharpSingleFileInit_PreservesAspireConfigChannel Recording #78770028555 Logs
UpdateProjectChannelToStable_TypeScriptSingleFileInit_PreservesAspireConfigChannel Recording #78770028555 Logs
UpdateProjectChannelToStable_TypeScript_PreviewsStablePackagesAndPreservesChannel Recording #78770028555 Logs

📹 Recordings uploaded automatically from CI run #26728980717

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add ownership-aware, idempotent Toolbox version reconciliation; remove unsupported inline MCP authentication; add unit, polyglot, and Azure deployment coverage; and remove the temporary authentication playground.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 1, 2026 16:19
@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

Review tier: Balanced
Findings: 5 Medium severity · 2 Low severity

New issues introduced by this change (7)
Severity Finding
Medium severity src/​Aspire.Hosting.Foundry/​Toolbox/​FoundryToolboxBuilderExtensions.csAddConnection(search) generates a new connection-{Guid} physical name on every AppHost process…
Medium severity src/​Aspire.Hosting.Foundry/​Toolbox/​FoundryToolboxReconciler.cs — This has a time-of-check/time-of-use race: two deploys can both observe no toolbox, then create…
Medium severity src/​Aspire.Hosting.Foundry/​Toolbox/​FoundryToolboxResource.cs — Waiting for a local compute resource to reach Running does not make its endpoint reachable from…
Medium severity src/​Aspire.Hosting.Foundry/​Toolbox/​FoundryToolboxToolDefinition.cs — The public name supplied to WithWebSearchTool is not serialized into this wire payload; it only…
Medium severity src/​Aspire.Hosting.Foundry/​Toolbox/​FoundryToolboxToolDefinition.csWithAISearchTool accepts a tool name, but this SDK object is created without assigning its…
Low severity src/​Aspire.Hosting.Foundry/​Toolbox/​FoundryToolboxBuilderExtensions.cs — This adds the primary public Toolbox entry point and new connection properties, but…
Low severity tests/​Aspire.Deployment.EndToEnd.Tests/​FoundryHostedAgentDeploymentTests.cs — The deployment test exercises only Web Search, so it cannot catch the Azure AI Search path's…
What changed in this PR

Adds first-class Microsoft Foundry Toolbox resources, tool definitions, connection properties, and deployment reconciliation.

Changes:

  • Adds Toolbox builder APIs for Web Search, MCP, and Azure AI Search.
  • Integrates Toolbox reconciliation into run and deployment pipelines.
  • Adds unit, polyglot, deployment E2E, and playground coverage.
File Description
FoundryToolboxBuilderExtensions.cs Adds Toolbox fluent APIs.
FoundryToolboxOptions.cs Defines polyglot options.
FoundryToolboxResource.cs Models lifecycle, connections, and pipeline steps.
FoundryToolboxReconciler.cs Implements version reconciliation.
FoundryToolboxToolDefinition.cs Converts supported tool definitions.
ToolboxTests.cs Tests resources, tools, and pipelines.
FoundryToolboxReconcilerTests.cs Tests reconciliation behavior.
FoundryHostedAgentDeploymentTests.cs Adds Azure deployment coverage.
TypeScript/​apphost.mts Exercises generated Toolbox APIs.
FoundryAgentBasic.AppHost/​AppHost.cs Adds a playground example.

Comment thread src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxBuilderExtensions.cs Outdated
Comment thread src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxReconciler.cs
Comment thread src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxResource.cs
Comment thread src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxToolDefinition.cs Outdated
Comment thread src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxToolDefinition.cs Outdated
Comment thread tests/Aspire.Deployment.EndToEnd.Tests/FoundryHostedAgentDeploymentTests.cs Outdated
Make search connections deterministic, serialize tool names, close the initial-version promotion race, reject local MCP endpoints, document Toolbox usage, and extend Azure Search deployment coverage.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 1, 2026 16:50
@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

Review tier: Balanced
Findings: 2 Medium severity · 3 Low severity

New issues introduced by this change (5)
Severity Finding
Medium severity src/​Aspire.Hosting.Foundry/​Toolbox/​FoundryToolboxBuilderExtensions.csindexName cannot be optional here: the Foundry Azure AI Search tool contract requires…
Medium severity src/​Aspire.Hosting.Foundry/​Toolbox/​FoundryToolboxResource.cs — These sets model Aspire resource identities but use ordinal comparison. Resource names are…
Low severity src/​Aspire.Hosting.Foundry/​README.md — The new Toolbox feature has no corresponding official Toolbox link in the README's Additional…
Low severity tests/​Aspire.Hosting.Foundry.Tests/​ToolboxTests.cs — This assertion only proves that ModelReaderWriter returned an object; it does not verify the…
Low severity tests/​PolyglotAppHosts/​Aspire.Hosting.Foundry/​TypeScript/​apphost.mts — This fixture omits the options argument, so the new [AspireDto] projection and the advertised…
Issues resolved since last review (7)
Severity Finding
Low severity tests/​Aspire.Deployment.EndToEnd.Tests/​FoundryHostedAgentDeploymentTests.cs — The deployment test exercises only Web Search, so it cannot catch the Azure AI Search path's… View resolved comment
Low severity src/​Aspire.Hosting.Foundry/​Toolbox/​FoundryToolboxBuilderExtensions.cs — This adds the primary public Toolbox entry point and new connection properties, but… View resolved comment
Medium severity src/​Aspire.Hosting.Foundry/​Toolbox/​FoundryToolboxToolDefinition.csWithAISearchTool accepts a tool name, but this SDK object is created without assigning its… View resolved comment
Medium severity src/​Aspire.Hosting.Foundry/​Toolbox/​FoundryToolboxToolDefinition.cs — The public name supplied to WithWebSearchTool is not serialized into this wire payload; it only… View resolved comment
Medium severity src/​Aspire.Hosting.Foundry/​Toolbox/​FoundryToolboxResource.cs — Waiting for a local compute resource to reach Running does not make its endpoint reachable from… View resolved comment
Medium severity src/​Aspire.Hosting.Foundry/​Toolbox/​FoundryToolboxReconciler.cs — This has a time-of-check/time-of-use race: two deploys can both observe no toolbox, then create… View resolved comment
Medium severity src/​Aspire.Hosting.Foundry/​Toolbox/​FoundryToolboxBuilderExtensions.csAddConnection(search) generates a new connection-{Guid} physical name on every AppHost process… View resolved comment
Suppressed comments (2)

Previously missed (1) — in code that hasn't changed since the last review.

playground/FoundryAgentBasic/FoundryAgentBasic.AppHost/AppHost.cs:11

  • This comment says the toolbox is created only at deploy time, but this AppHost also reconciles it during aspire run via the background before-start path. Describe both modes so the playground does not contradict the resource lifecycle.
// Add a Foundry Toolbox with a single WebSearch tool. The toolbox is created on the Foundry data
// plane at deploy time via AgentToolboxes.CreateToolboxVersionAsync.

src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxBuilderExtensions.cs:216

  • This bypasses the existing Azure AI Search connection overload's role wiring (ConnectionBuilderExtensions.cs:238-245). The Foundry project identity therefore receives neither SearchIndexDataReader nor SearchServiceContributor, so the toolbox can be created but Azure AI Search tool calls will fail authorization. Apply the same role assignments before creating the deterministic connection.
        var connection = projectBuilder.AddSearchConnection(connectionName, search.Resource);

Comment thread src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxBuilderExtensions.cs Outdated
Comment thread src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxResource.cs Outdated
Comment thread src/Aspire.Hosting.Foundry/README.md
Comment thread tests/Aspire.Hosting.Foundry.Tests/ToolboxTests.cs Outdated
Comment thread tests/PolyglotAppHosts/Aspire.Hosting.Foundry/TypeScript/apphost.mts Outdated
Require Azure AI Search index names and role assignments, align resource-name comparison, cover MCP serialization and polyglot options, and complete Toolbox documentation.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 1, 2026 17:06
@github-actions

This comment has been minimized.

Give the deployment test an isolated ten-minute budget for Azure Search role assignments to propagate before reporting a focused timeout.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 2, 2026 15:03
@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Mixed-tool readiness can report success without discovering any tools from configured MCP servers.

Review tier: Balanced
Findings: None

Suppressed comments (1)

src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxResource.cs:433

  • Mixed toolboxes can be marked Running before any configured MCP server is usable. This list drops every FoundryToolboxMcpToolDefinition, so discovery of a built-in tool satisfies the probe even when an MCP server contributes zero tools. Foundry exposes MCP tools with a {serverLabel}. prefix; require at least one discovered tool for each configured server label (while retaining exact checks for built-ins), and cover the mixed-tool case.
        var requiredToolNames = _tools
            .Where(tool => tool is not FoundryToolboxMcpToolDefinition)
            .Select(tool => tool.Name)
            .ToArray();

Keep mixed Toolboxes in readiness until built-in tools are present and each configured MCP server label contributes at least one discovered tool.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 2, 2026 15:26
@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

The new Azure data-plane reconciliation, concurrency handling, and RBAC behavior warrant final human validation.

Review tier: Balanced
Findings: None

@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Retrying the failed CI jobs for this pull request from the CI run attempt. The rerun is being tracked in the rerun attempt.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Tommaso Stocchi (@tommasodotNET). I re-reviewed the current head (7205ea9f) against the gaps I raised earlier. The lifecycle, approval-policy, reconciliation, RBAC, readiness, and MCP discovery changes address my concerns. Current CI is green, and I found no blocking regressions.

Approved from my side. Nice work closing this out.

Comment thread src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxResource.cs Outdated
Comment thread src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxToolDefinition.cs Outdated
Comment thread src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxResource.cs Outdated
Comment thread src/Aspire.Hosting.Foundry/Project/ConnectionBuilderExtensions.cs
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 7, 2026 07:06
@github-actions

github-actions Bot commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Tests selector

5 / 99 PR test projects · 3 PR jobs · 2 advisory-only targets, from 19 changed files.

Selected PR test projects (5 / 99)

Aspire.Cli.EndToEnd.Tests, Aspire.Hosting.Azure.Kubernetes.Tests, Aspire.Hosting.Azure.Tests, Aspire.Hosting.Foundry.Tests, Aspire.Playground.Tests

Selected PR jobs (3)

extension-e2e, polyglot, typescript-api-compat

Advisory workflow impact (2)

  • Aspire.Deployment.EndToEnd.Tests (deployment workflow-only)
  • deployment-e2e (schedule/dispatch-only)

How these were chosen — grouped by what changed

🔧 src/Aspire.Hosting.Foundry/FoundryResource.cs (changed source)
1 directly: Aspire.Hosting.Foundry.Tests
2 via the project graph: Aspire.Hosting.Azure.Kubernetes.Tests (2 hops), Aspire.Hosting.Azure.Tests

📄 playground/FoundryAgentBasic/FoundryAgentBasic.AppHost/AppHost.cs (changed)
1 directly: Aspire.Playground.Tests

🔧 src/Aspire.Hosting.Foundry/HostedAgent/AzureHostedAgentResource.cs (changed source)
1 directly: Aspire.Hosting.Foundry.Tests

🔧 src/Aspire.Hosting.Foundry/HostedAgent/HostedAgentBuilderExtension.cs (changed source)
1 directly: Aspire.Hosting.Foundry.Tests

🔧 src/Aspire.Hosting.Foundry/Project/ConnectionBuilderExtensions.cs (changed source)
1 directly: Aspire.Hosting.Foundry.Tests

🔧 src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxBuilderExtensions.cs (changed source)
1 directly: Aspire.Hosting.Foundry.Tests

🔧 src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxOptions.cs (changed source)
1 directly: Aspire.Hosting.Foundry.Tests

🔧 src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxReadinessProbe.cs (changed source)
1 directly: Aspire.Hosting.Foundry.Tests

🔧 src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxReconciler.cs (changed source)
1 directly: Aspire.Hosting.Foundry.Tests

🔧 src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxResource.cs (changed source)
1 directly: Aspire.Hosting.Foundry.Tests

🔧 src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxToolDefinition.cs (changed source)
1 directly: Aspire.Hosting.Foundry.Tests

🧪 tests/Aspire.Deployment.EndToEnd.Tests/Aspire.Deployment.EndToEnd.Tests.csproj (changed test)
1 directly: Aspire.Deployment.EndToEnd.Tests

🧪 tests/Aspire.Deployment.EndToEnd.Tests/FoundryHostedAgentDeploymentTests.cs (changed test)
1 directly: Aspire.Deployment.EndToEnd.Tests

🧪 tests/Aspire.Hosting.Foundry.Tests/FoundryToolboxReconcilerTests.cs (changed test)
1 directly: Aspire.Hosting.Foundry.Tests

🧪 tests/Aspire.Hosting.Foundry.Tests/Helpers/SequenceHttpMessageHandler.cs (changed test)
1 directly: Aspire.Hosting.Foundry.Tests

🧪 tests/Aspire.Hosting.Foundry.Tests/HostedAgentExtensionTests.cs (changed test)
1 directly: Aspire.Hosting.Foundry.Tests

🧪 tests/Aspire.Hosting.Foundry.Tests/ToolboxTests.cs (changed test)
1 directly: Aspire.Hosting.Foundry.Tests

🧪 tests/Shared/Hex1bAutomatorTestHelpers.cs (changed test)
1 via the project graph: Aspire.Cli.EndToEnd.Tests

Job reasons

Job Triggered by
deployment-e2e tests/Aspire.Deployment.EndToEnd.Tests/Aspire.Deployment.EndToEnd.Tests.csproj, tests/Aspire.Deployment.EndToEnd.Tests/FoundryHostedAgentDeploymentTests.cs
extension-e2e src/Aspire.Hosting.Foundry/FoundryResource.cs, src/Aspire.Hosting.Foundry/HostedAgent/AzureHostedAgentResource.cs, src/Aspire.Hosting.Foundry/HostedAgent/HostedAgentBuilderExtension.cs, src/Aspire.Hosting.Foundry/Project/ConnectionBuilderExtensions.cs, src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxBuilderExtensions.cs, src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxOptions.cs, src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxReadinessProbe.cs, src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxReconciler.cs, src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxResource.cs, src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxToolDefinition.cs
• affected project Aspire.Hosting.Foundry
polyglot tests/PolyglotAppHosts/Aspire.Hosting.Foundry/TypeScript/apphost.mts
typescript-api-compat affected project Aspire.Hosting.Foundry

Selection computed for commit 7d4af08.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Existing-mode readiness rejects valid skill-only Foundry Toolboxes.

Review tier: Balanced
Findings: None

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

src/Aspire.Hosting.Foundry/Toolbox/FoundryToolboxReadinessProbe.cs:104

  • Existing-mode readiness incorrectly requires tools/list to return at least one tool when there are no modeled expectations. Foundry supports skill-only Toolboxes, where skills are exposed through resources/list; AsExisting() therefore times out and marks a valid skill-only Toolbox as failed. When both expectation lists are empty, a successful MCP handshake should be sufficient, or the probe must also check resources/list.

@eerhardt
Eric Erhardt (eerhardt) merged commit 5eaf431 into main Sep 8, 2026
178 checks passed
@eerhardt
Eric Erhardt (eerhardt) deleted the davidfowl/foundry-toolbox branch September 8, 2026 17:48
@microsoft-github-policy-service microsoft-github-policy-service Bot added this to the 13.6 milestone Sep 8, 2026
@aspire-repo-bot

Copy link
Copy Markdown
Contributor

Pull request created: #1628

Generated by PR Documentation Check · auto · 89.4 AIC · ⌖ 11.8 AIC · ⊞ 19.7K

@aspire-repo-bot

Copy link
Copy Markdown
Contributor

📝 Documentation has been drafted in microsoft/aspire.dev#1628 targeting release/13.6.

Documented the new Foundry Toolbox resource added in this PR. Updated azure-ai-foundry-host.mdx with a new "Add a Toolbox" section (C#/TypeScript examples, parameter table, MCP reachability/approval-policy notes, existing-resource usage) and azure-ai-foundry-connect.mdx with a new "Foundry Toolbox resource" connection-properties subsection (Uri, ProjectEndpoint, Name, ApiVersion, Version, FoundryFeatures, AuthorizationScope).

Note

This draft PR needs human review before merging.

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

🔍 CI Failure Analysis: Transient Infrastructure Failure

The CI build failed due to transient infrastructure issues.

Failed jobs:

If a rerun was not already requested automatically, visit the workflow run page to rerun the failed jobs manually.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants