feat: add client tool search support by haoshan98 · Pull Request #186 · vllm-project/agentic-api

haoshan98 · 2026-08-20T13:33:28Z

Summary

Add client-executed tool search support to the Responses API.

Model tool_search_call as a first-class item for blocking and streaming responses.
Normalize public tool_search declarations into upstream-compatible function tools.
Support deferred standalone functions and namespace members.
Preserve public tool-search state across replay, persistence, and compaction.
Add provider-parity cassettes for OpenAI, direct upstream, HTTP/SSE, and WebSocket flows.

Signed-off-by: haoshan98 <haoshanw@gmail.com>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 447f734c49

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-08-20T13:39:46Z

+        let rows = item::get_items_by_conversation_in_tx(&mut tx, conversation_id).await?;
+        let latest_item_id = rows.last().map(|row| row.id.as_str());
+        let conversation_turns = response::get_conversation_turns_in_tx(&mut tx, conversation_id).await?;
+        let latest_response = latest_item_id.and_then(|latest_item_id| {


Preserve metadata for zero-item conversation turns

When a conversation turn persists no items—most notably a generate:false prewarm with input: []—latest_item_id is None, so this expression always discards the response's metadata. A subsequent request using the same conversation_id therefore fails to inherit the prewarmed tool-search declarations and loaded-tool state; if older items exist, it can instead inherit stale metadata from the prior non-empty turn. The latest conversation response must remain identifiable even when its history_item_ids is empty.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-08-20T13:39:46Z

+        if self.parallel_tool_calls == Some(true) {
+            return Err(ToolError::Config(
+                "parallel_tool_calls must be false when tool search is active".to_owned(),
+            ));


Force serial calls when tool search is active

When the client omits the optional parallel_tool_calls field, this check accepts the request and the private upstream request also leaves the field absent, allowing the backend's enabled/default behavior to produce multiple or mixed tool calls. The translator permits only one tool-search call and rejects a second as an invalid upstream response, turning an otherwise valid request into a 502. Active tool search should normalize None to Some(false) rather than only rejecting explicit true.

Useful? React with 👍 / 👎.

Signed-off-by: haoshan98 <haoshanw@gmail.com>

maralbahari · 2026-08-28T05:18:15Z

+) -> DbResult<Option<Response>> {
+    let escaped_item_id = item_id.replace('!', "!!").replace('%', "!%").replace('_', "!_");
+    let history_suffix = format!("%\"{escaped_item_id}\"]");
+    sqlx::query_as::<_, Response>(


I understand why we need metadata from the response associated with the captured conversation version: compaction can remove the tool_search_call/tool_search_output history, and selecting the latest response would be unsafe if another turn is persisted after rehydration.

However, this query reconstructs the response-to-item relationship by applying LIKE to the serialized history_item_ids JSON. This is unindexed, depends on the exact JSON encoding, and suggests that the storage schema is missing an explicit relationship.

Could we model this association directly—for example, with a response_id/turn_id on stored items or a normalized response_items relation—and resolve it using an indexed lookup? At minimum, ConversationSnapshot could retain the last item ID already loaded by rehydrate_snapshot, avoiding the additional get_id_by_conversation_sequence query. I would keep the exact-version behavior but avoid introducing CRUD based on textual matching over JSON.

Signed-off-by: haoshan98 <haoshanw@gmail.com>

This reverts commit ef79009. Signed-off-by: haoshan98 <haoshanw@gmail.com>

Signed-off-by: haoshan98 <haoshanw@gmail.com>

maralbahari · 2026-09-01T09:39:10Z

        || payload.conversation_id.is_some()
        || payload.input.contains_compaction()
        || payload.input.has_compaction_trigger()
+        || has_tool_search_state


why tool search state needs to be part of should_execute? sounds unnecessary

maralbahari · 2026-09-01T09:47:24Z

    }

-    run_gateway_tool_loop(ctx, exec_ctx, auth, stream_upstream, stream).await
+    run_gateway_tool_loop(ctx, registry, exec_ctx, auth, stream_upstream, stream).await


registry is being passed down to multiple-functions in src/executor without that function even needing to access registry all they do is to pass it down to another function. perhaps just pass to exactly where it is needed. it's hard to track now.
like the example in this function. or persist_if_needed

maralbahari · 2026-09-01T09:50:06Z

-    };
-    let mut combined_output: Vec<OutputItem> = registry
+    let mut registry = registry
+        .build_prepared_with_handlers(ctx.enriched_request.tools.as_mut(), &mut executors)


this function builds the registry here so only the entries (tool lookup table) is created only here but why is it receiving registry:ToolRegistry instance argument?

maralbahari · 2026-09-01T09:56:02Z

+        ensure_request_prepared(request, self.tool_search.is_some())
+    }
+
+    pub(crate) fn normalize_response_output(


this function should be part of src/executor module. not sure its relevency to registry?

maralbahari · 2026-09-01T09:58:01Z

+    /// Prepare the request's private inference projection, build the normal
+    /// dispatch table, and retain the public tool-search projection in this
+    /// request-scoped registry.
+    pub(crate) fn prepare_request(


I think this function can also exist in executor module after registry is created by executor module then only call registry.install_tool_search_state() instead of creating registry instance in a function that is part of registry.

Signed-off-by: haoshan98 <haoshanw@gmail.com>

franciscojavierarceo · 2026-09-04T01:41:31Z

+        conversation_id: &str,
+        version: ConversationVersion,
+    ) -> StoreResult<Option<ResponseMetadata>> {
+        let ConversationVersion::LastSequence(sequence) = version else {


ConversationVersion::Empty returns no metadata here, but a WebSocket generate:false prewarm with empty input persists response metadata without advancing the item sequence. the next conversation request therefore loses the prewarmed tool-search state, or can inherit stale metadata from an earlier item-bearing turn. we should track the latest response independently of item sequence so zero-item turns remain addressable.

franciscojavierarceo · 2026-09-04T01:41:32Z

+            |member| matches!(member, CodexNamespaceMember::Function(function) if function.defer_loading == Some(true)),
+        ),
+        ResponsesTool::ToolSearch(_)
+        | ResponsesTool::Mcp(_)


MCP and custom declarations with defer_loading:true are accepted but treated as immediately available, so the model can invoke them before the client returns them from tool search. the Responses tool schema supports deferred MCP and custom tools. we should implement withholding for these variants or reject unsupported deferred kinds before discovery and normalization.

franciscojavierarceo · 2026-09-04T01:41:32Z

+                .unwrap_or(DEFAULT_DESCRIPTION)
+                .to_owned(),
+        );
+        normalized.parameters = Some(


a declared tool_search.parameters schema without top-level "type":"object", such as oneOf, is silently replaced with the built-in query schema. other object schemas are forced into strict:true lowering without strict-schema validation. this can produce arguments that do not match the client’s declaration. we should preserve the declared schema or return a clear 400 when the upstream lowering cannot support it.

franciscojavierarceo · 2026-09-04T01:41:32Z

+        .and_then(Value::as_str)
+        .map_or(ResponseStatus::Completed, |status| status.parse().unwrap_or_default());
+    let discard_unfinished = matches!(status, ResponseStatus::Error | ResponseStatus::Incomplete);
+    for item in value.get("output").and_then(Value::as_array).into_iter().flatten() {


this validates each tool-search call independently, so an upstream response containing two completed calls is exposed and persisted. continuation then rehydrates call1, call2 before either client output and rejects call2 because call1 remains unresolved. we should reject more than one search call before exposing the response, or support multiple pending calls throughout replay.

franciscojavierarceo · 2026-09-04T01:41:32Z

+        status: ResponseStatus,
+        unfinished_stream_item_ids: &HashSet<String>,
+    ) -> Result<(), ToolError> {
+        let discard_unfinished = matches!(status, ResponseStatus::Error | ResponseStatus::Incomplete);


ResponseStatus::Incomplete is treated like an error here, which drops incomplete or still-in-progress search calls from the terminal response and persistence. streaming clients can observe response.output_item.added and then receive a terminal response without that item, while blocking clients never see it. we should preserve the public item with its documented incomplete status while keeping it non-replayable as a completed call.

franciscojavierarceo · 2026-09-04T01:41:32Z

+        "SELECT * FROM responses \
+         WHERE conversation_id = $1 \
+           AND previous_response_id IS NULL \
+           AND history_item_ids LIKE $2 ESCAPE '!' \


this leading-wildcard LIKE over serialized history_item_ids cannot use an index and depends on the exact JSON encoding. compacted continuation metadata lookup therefore grows linearly with the number of stored conversation turns. we should store an indexed last_item_id or normalize the response-to-item relationship.

LOGO127 · 2026-09-06T15:38:22Z

AI-assisted compatibility note while preparing #253, tested against your exact head 4fd4e163e0c3d641c89dc16fd555e26de814bb18.

I added a local synthetic image-preservation fixture without modifying any tracked production files on your branch. It passes 34/34 cases there. The identical fixture on main 74c6b2de7725e7217695ee391cf5a4e59b6efa4d passes 25/34; the nine failures are namespace history across HTTP JSON/SSE/WebSocket and previous-response/conversation/manual-replay paths:

Tool declaration sent upstream: agentic_ns__functions__view_image.
Replayed call on main: name: "view_image", namespace: "functions".
Replayed call on this PR: name: "agentic_ns__functions__view_image", without the public namespace field.

This is a tool-name consistency check, not a claim that the image bytes themselves disappear. Your CodexNamespaceHandler::resolve_input covers the normalization missing on main.

The fixture also checks mixed text/images and ordering, raw HTTP proxy and stateless replay, retained user images through explicit/automatic compaction, and structured function/custom image outputs through the following inference request and a third turn restoring/replaying those outputs. Conversation cases explicitly redeclare tools; they do not assume request settings are inherited. Shared solid-color PNG fixtures are separately decoded and pixel-checked with Pillow.

I am treating this as an existing dependency rather than opening a competing namespace implementation. Would you prefer the complementary image regression tests as a follow-up after this PR lands? The fixture is still local/unpublished, and #253's required live vision-model/vLLM check remains pending; these deterministic checks do not certify the entire PR or demonstrate model image understanding.

Signed-off-by: haoshan98 <haoshanw@gmail.com>

haoshan98 added 2 commits August 20, 2026 12:33

Support tool search

7cc61b3

Signed-off-by: haoshan98 <haoshanw@gmail.com>

Cassette Tests

447f734

Signed-off-by: haoshan98 <haoshanw@gmail.com>

haoshan98 requested review from bbrowning, franciscojavierarceo, jiahuei, leseb, maralbahari, noobHappylife, qandrew and tjtanaa as code owners August 20, 2026 13:33

chatgpt-codex-connector Bot reviewed Aug 20, 2026

View reviewed changes

haoshan98 added 2 commits August 20, 2026 13:59

Merge branch 'main' into tool-search

91d94de

Signed-off-by: haoshan98 <haoshanw@gmail.com>

Updates

1e2c1bb

Signed-off-by: haoshan98 <haoshanw@gmail.com>

maralbahari reviewed Aug 24, 2026

View reviewed changes

Comment thread crates/agentic-server-core/src/tool/registry.rs Outdated

Comment thread crates/agentic-server-core/src/tool/mcp/handler.rs Outdated

maralbahari reviewed Aug 25, 2026

View reviewed changes

Comment thread crates/agentic-server-core/src/tool/normalize.rs Outdated

haoshan98 added 7 commits August 25, 2026 10:12

Merge branch 'main' into tool-search

69bdd59

Signed-off-by: haoshan98 <haoshanw@gmail.com>

Fix

bc3c4e7

Signed-off-by: haoshan98 <haoshanw@gmail.com>

Updates

c0dd0a4

Signed-off-by: haoshan98 <haoshanw@gmail.com>

Enhance validation

2baaf2d

Signed-off-by: haoshan98 <haoshanw@gmail.com>

Keep compaction trigger

2e31d1f

Signed-off-by: haoshan98 <haoshanw@gmail.com>

Merge branch 'main' into tool-search

0477abb

ToolRegistry-centered redesign

a2fe3fa

Signed-off-by: haoshan98 <haoshanw@gmail.com>

maralbahari reviewed Aug 28, 2026

View reviewed changes

haoshan98 added 4 commits August 28, 2026 07:52

Updates

ef79009

Signed-off-by: haoshan98 <haoshanw@gmail.com>

Revert "Updates"

e860626

This reverts commit ef79009. Signed-off-by: haoshan98 <haoshanw@gmail.com>

Merge branch 'main' into tool-search

453a552

Signed-off-by: haoshan98 <haoshanw@gmail.com>

Updates

8fd35ea

Signed-off-by: haoshan98 <haoshanw@gmail.com>

haoshan98 force-pushed the tool-search branch from 1fedfbb to 8fd35ea Compare August 28, 2026 10:14

maralbahari reviewed Sep 1, 2026

View reviewed changes

Updates tool search state pass through

4fd4e16

Signed-off-by: haoshan98 <haoshanw@gmail.com>

franciscojavierarceo reviewed Sep 4, 2026

View reviewed changes

haoshan98 added 2 commits September 7, 2026 08:14

Merge branch 'main' into tool-search

1895a28

Signed-off-by: haoshan98 <haoshanw@gmail.com>

Merge branch 'main' into tool-search

3276cdc

Signed-off-by: haoshan98 <haoshanw@gmail.com>

Uh oh!

Conversation

haoshan98 commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Aug 20, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Aug 20, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LOGO127 commented Sep 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

haoshan98 commented Aug 20, 2026 •

edited

Loading