feat(rest): let a tool see the response headers it asks for (Link pagination, rate limits) - #558
Merged
Merged
Conversation
A REST tool got the body and nothing else. Right for almost every call, wrong for list endpoints that paginate through a Link header (GitHub, GitLab, Sentry, Shopify): the model could pass a cursor in but never learned the next one, so every such tool was one page long. Rate-limit headers were the other thing adapters kept wishing they could show. A tool now opts in with endpointMapping.exposeHeaders, a list of header names. The engine returns those (lower-cased) next to the body through a new executeWithMeta; execute() is unchanged and every tool that did not opt in stays on it. The executor attaches them to the result as _headers, and reads Link for rel="next" into _pagination with nextUrl and, when the URL carries a recognisable parameter, nextCursor and its name. _pagination is absent on the last page: absence is the signal. Placement: after the response transform, so a select cannot drop them; an object body is extended, anything else is wrapped as data. The audit log keeps the bare body. The cache stores body and headers together in a small envelope and still reads entries written before it existed. Closes #557
The tool test page went through executeConnectorCall, which only ever saw the body, so a tool that asked for headers looked different there than to a model. Same helper, same shape.
GitHub's next link carries both after= and page=. The generic list chose after, but a tool mapping page: "$page" cannot send that back. The parameters a tool feeds from its own inputs now come first, so nextCursor is a value the model can pass straight back. Seen against the live API while testing locally.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #557. Came out of reviewing the Sentry adapter (#551).
What changes
endpointMapping.exposeHeaders: string[](REST only, opt in per tool). Names are matched case-insensitively and returned lower-cased.RestEngine.executeWithMeta()returns{ body, headers };execute()is now a thin wrapper and behaves exactly as before. Tools withoutexposeHeadersnever touch the new path._headers, and parses aLinkheader'srel="next"into_pagination: { nextUrl, nextCursor?, cursorParam?, prevUrl? }. Nonextrelation → no_paginationat all, which is the "last page" signal.selectcan't drop them); an object body is extended, anything else is wrapped asdata.docs/tool-definition.md.Not in this PR: updating existing adapters to use it. Sentry (#551), GitHub and Shopify are the obvious candidates once this is in.
Tests:
response-headers.util.spec(8),rest.engine.spec+2,dynamic-mcp-tools.spec+8 (opt-in vs not, last page, non-object body, transform ordering, audit excludes headers, cache round trip incl. legacy entries). mcp-server + engines suites: 336 green. tsc + eslint clean.