Skip to content

Add hosted voice agent support - #9697

Draft
JianW (v1212) wants to merge 32 commits into
Azure:mainfrom
v1212:users/wujia/hosted-voice-agent-support
Draft

Add hosted voice agent support#9697
JianW (v1212) wants to merge 32 commits into
Azure:mainfrom
v1212:users/wujia/hosted-voice-agent-support

Conversation

@v1212

Copy link
Copy Markdown
Contributor

Summary

  • add Hosted Voice wrapper authoring and deployment on top of the existing Hosted Agent / invocations_ws lifecycle
  • resolve and pin the deployed Hosted target through the azd uses dependency graph
  • validate target project, status, Voice Bridge metadata, and invocations_ws/1.0.0
  • support azd ai agent init -m agent.manifest.yaml from the Hosted Voice sample
  • support initialization into both new and existing azd projects

Experience

$env:AZD_AI_AGENT_ENABLE_PROMPT_VOICE = "true"
azd ai agent init -m agent.manifest.yaml
azd up

The sample descriptor supplies the agent name, source, runtime, entry point, bundled dependency mode, protocol, and Voice Bridge metadata. Users select the subscription, Foundry project, and model deployment through the normal init prompts.

Validation

  • go fix ./...
  • go test ./...
  • go vet ./...
  • cspell over the extension
  • Scenario 1: init from packaged sample, deploy, show, doctor, typed /help, real audio, cleanup
  • Scenario 2: add sample to an existing azd project, preserve existing services, service-specific deploy, typed /help, real audio, cleanup
  • West US 2 runtime: STT Hello., target response, and output audio

Stacking

This draft includes and depends on #9655. Once #9655 merges, this PR should contain only the Hosted Voice commits.

Follow-up

  • ownership-aware composite azd down cleanup

Copilot AI added 19 commits August 19, 2026 14:33
…ice-agent-unified-api

# Conflicts:
#	cli/azd/extensions/azure.ai.agents/README.md
…ice-agent-advanced-settings-unified

# Conflicts:
#	cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json
…vanced-settings-unified' into users/wujia/hosted-voice-agent-support

# Conflicts:
#	cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_api/models.go
#	cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/map.go
#	cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/parse.go
#	cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/parse_voice_test.go
#	cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go
#	cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go
#	cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent_test.go
#	cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
20 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@v1212

Copy link
Copy Markdown
Contributor Author

Final validation status

All automated CI checks are green, including:

  • extension tests
  • cspell
  • go-fix
  • golangci-lint on Linux and Windows
  • provider verification
  • Linux, macOS, and Windows builds
  • Linux/macOS/Windows ARM64 cross-build validation
  • release artifact generation
  • SDL analysis
  • Azure SDK check enforcer: All checks passed

Manual regression validation also passed:

  • Scenario 1: init -m from sample -> target/wrapper deploy -> show/doctor -> typed /help -> real audio -> cleanup
  • Scenario 2: init -m into existing azd project -> existing services preserved -> service-specific deploy -> typed /help -> real audio -> cleanup
  • target-owned /help returned Commands: through the wrapper
  • audio STT recognized Hello. and returned output audio

Review status:

  • unresolved review threads: 0
  • inline review comments: 0
  • pending reviews: 0

The PR remains Draft and therefore still requires normal human review/approval before merge.

Copilot AI added 7 commits August 25, 2026 09:32
…vanced-settings-unified' into users/wujia/sync-hosted-voice-latest

# Conflicts:
#	cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_api/models.go
#	cli/azd/extensions/azure.ai.agents/internal/pkg/agents/agent_yaml/map.go
#	cli/azd/extensions/azure.ai.agents/internal/project/agent_definition.go
#	cli/azd/extensions/azure.ai.agents/internal/project/service_target_agent.go
#	cli/azd/extensions/azure.ai.agents/schemas/azure.ai.agent.json
@v1212

Copy link
Copy Markdown
Contributor Author

Synced with latest #9634 Voice API surface

PR #9697 now contains the latest #9634 head (4c2447bb3) through the updated #9655 stack.

Hosted Voice adaptation completed:

  • migrated wrapper create/update to the replacement unified Voice API surface
  • retained strict kind=voice, model_type=hosted_agent, and pinned target_agent
  • retained target status/project/protocol/Voice Bridge validation
  • retained init-from-sample and existing-project workflows
  • added explicit output voice locale precedence regression coverage

Validation:

  • go test ./...: PASS
  • go vet ./...: PASS
  • cspell: PASS
  • Linux/Windows golangci-lint: PASS
  • cross-platform build/test pipeline: PASS
  • West US 2 deploy against current TTL project: PASS
  • wrapper /help returned target-owned Commands: response
  • real audio: STT Hello., non-empty response, 5 output audio frames
  • wrapper and target cleaned up

Updated private package:
D:\jwshare\adc-hosted-agent\ws-endpoint\hosted-voice-agent-private-test-share-win-amd64.zip

SHA256:
344fff0c60221e848b1021899a71a95708b37f428f4a6a79d49fcb05a7f2fbfa

Private bundle commit: aba916c6adcf6a58e8ff4d25857bf3d182898872.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ext-agents azure.ai.agents extension

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants