From f86049e9bdd06c9ba48df2f9a9d1b04e70764e23 Mon Sep 17 00:00:00 2001 From: "carpentry-heartbeat[bot]" Date: Mon, 24 Aug 2026 17:22:40 +0200 Subject: [PATCH 1/3] Keep HEAD a HEAD across a 301, 302 or 303 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `redirect-verb` rewrote every method to GET on a 301, 302 or 303, so a `Client.head` through a redirect issued a GET at the target and `drain-stream` downloaded the whole body — the transfer the caller explicitly asked to avoid. `Client.head` at a 301 to `/get` returned the body `ok`; it now returns nothing. RFC 9110 §15.4.2 and §15.4.3 sanction one rewrite only ("For historical reasons, a user agent MAY change the request method from POST to GET for the subsequent request"), and §15.4.4 says a 303 is followed with "a GET or HEAD request if using HTTP". `curl -I -L` stays a HEAD throughout. GET and HEAD now keep their method; every other method still becomes a GET, and 307/308 are untouched. The same branch dropped the body and Content-Length but left Content-Type, so the follow-up GET went out advertising a body it no longer had. §15.4's redirect checklist names the whole set: "If the request method has been changed to GET or HEAD, remove content-specific header fields, including (but not limited to) Content-Encoding, Content-Language, Content-Location, Content-Type, Content-Length, Digest, Last-Modified." `remove-content-headers` strips exactly that list. --- README.md | 6 ++++-- docs/Client.html | 9 +++++---- http-client.carp | 44 +++++++++++++++++++++++++++++-------------- test/http-client.carp | 36 +++++++++++++++++++++++++++++++++++ 4 files changed, 75 insertions(+), 20 deletions(-) diff --git a/README.md b/README.md index 02521c2..6f4a9f6 100644 --- a/README.md +++ b/README.md @@ -180,8 +180,10 @@ those characters are emitted unchanged. All return `(Result Response String)` (or `(Result ResponseStream String)` for the streaming variants). All methods follow HTTP redirects automatically (up to `Client.default-max-redirects`, -which is 10). For 301/302/303 responses the method is changed to GET and the body is -dropped. For 307/308 responses the original method and body are preserved. Use the +which is 10). For 301/302/303 responses a GET or HEAD keeps its method; any other +method becomes a GET, and the body and the headers describing it (`Content-Type`, +`Content-Length`, …) are dropped. For 307/308 responses the original method and body +are preserved. Use the `-with-max-redirects` variants to control the limit, or pass 0 to disable. A relative `Location` is resolved against the URL of the hop that produced it, diff --git a/docs/Client.html b/docs/Client.html index eb8ba00..fd2293f 100644 --- a/docs/Client.html +++ b/docs/Client.html @@ -549,8 +549,8 @@

sends an HTTP request to the given URL. Returns (Result Response String).

Follows up to default-max-redirects HTTP redirects automatically. For 301, -302, and 303 responses the method is changed to GET. For 307 and 308 the -original method is preserved.

+302, and 303 responses a GET or HEAD keeps its method and any other method +becomes a GET. For 307 and 308 the original method is preserved.

See request-with-max-redirects to control the redirect limit.

@@ -667,8 +667,9 @@

sends an HTTP request and returns a ResponseStream, following up to max-redirects HTTP redirects.

-

For 301, 302, and 303 responses the method is changed to GET and the body is -dropped. For 307 and 308 responses the original method and body are preserved.

+

For 301, 302, and 303 responses a GET or HEAD keeps its method; any other +method becomes a GET, and the body and the headers describing it are dropped. +For 307 and 308 responses the original method and body are preserved.

Pass 0 to disable redirect following.

diff --git a/http-client.carp b/http-client.carp index cdad0c2..21fd130 100644 --- a/http-client.carp +++ b/http-client.carp @@ -316,10 +316,14 @@ to follow. Used by `request`, `request-stream`, and convenience methods.") (or (= code 301) (or (= code 302) (or (= code 303) (or (= code 307) (= code 308)))))) + ; RFC 9110 §15.4.2–§15.4.4 method rewriting. (hidden redirect-verb) (private redirect-verb) (defn redirect-verb [code verb] - (if (or (= code 307) (= code 308)) @verb @"GET")) + (cond + (or (= code 307) (= code 308)) @verb + (or (= verb "GET") (= verb "HEAD")) @verb + @"GET")) ; uri@0.5.1 parses `/` as an empty path, so URI.resolve inherits the base ; path; drop this once a uri after 0.5.1 is pinned (carpentry-org/uri#35). @@ -357,14 +361,25 @@ to follow. Used by `request`, `request-stream`, and convenience methods.") (Result.Error _) @location (Result.Success resolved) (URI.str &resolved))))))) - (hidden remove-content-length) - (private remove-content-length) - (defn remove-content-length [headers] + ; the fields RFC 9110 §15.4 drops when a redirect changes the method. + (hidden content-header?) + (private content-header?) + (defn content-header? [name] + (Array.contains? + &[@"content-encoding" + @"content-language" + @"content-location" + @"content-type" + @"content-length" + @"digest" + @"last-modified"] + &(String.ascii-to-lower name))) + + (hidden remove-content-headers) + (private remove-content-headers) + (defn remove-content-headers [headers] (Map.kv-reduce - &(fn [acc k v] - (if (= &(String.ascii-to-lower k) "content-length") - acc - (Map.put acc k v))) + &(fn [acc k v] (if (content-header? k) acc (Map.put acc k v))) (the (Map String (Array String)) {}) headers)) @@ -449,7 +464,7 @@ to follow. Used by `request`, `request-stream`, and convenience methods.") (when-do (/= &new-verb &cur-verb) (set! cur-body @"") (set! cur-headers - (remove-content-length &cur-headers))) + (remove-content-headers &cur-headers))) (set! cur-verb new-verb)) (set! remaining (Int.dec remaining)))))) (do @@ -475,8 +490,9 @@ to follow. Used by `request`, `request-stream`, and convenience methods.") (doc request-stream-with-max-redirects "sends an HTTP request and returns a `ResponseStream`, following up to `max-redirects` HTTP redirects. -For 301, 302, and 303 responses the method is changed to GET and the body is -dropped. For 307 and 308 responses the original method and body are preserved. +For 301, 302, and 303 responses a GET or HEAD keeps its method; any other +method becomes a GET, and the body and the headers describing it are dropped. +For 307 and 308 responses the original method and body are preserved. Pass 0 to disable redirect following.") (defn request-stream-with-max-redirects [verb url headers body max-redirects] @@ -541,8 +557,8 @@ Returns `(Result Response String)`. Pass 0 to disable redirect following.") "sends an HTTP request to the given URL. Returns `(Result Response String)`. Follows up to `default-max-redirects` HTTP redirects automatically. For 301, -302, and 303 responses the method is changed to GET. For 307 and 308 the -original method is preserved. +302, and 303 responses a GET or HEAD keeps its method and any other method +becomes a GET. For 307 and 308 the original method is preserved. See `request-with-max-redirects` to control the redirect limit.") (defn request [verb url headers body] @@ -749,7 +765,7 @@ See `RequestConfig` for timeout and redirect details.") (when-do (/= &new-verb &cur-verb) (set! cur-body @"") (set! cur-headers - (remove-content-length &cur-headers))) + (remove-content-headers &cur-headers))) (set! cur-verb new-verb)) (set! remaining (Int.dec remaining)))))) (do diff --git a/test/http-client.carp b/test/http-client.carp index 9a58434..4ed2efc 100644 --- a/test/http-client.carp +++ b/test/http-client.carp @@ -26,6 +26,12 @@ (defn get-body [url] (match (Client.get url) (Result.Success r) @(Response.body &r) _ @"")) +; Reports the error text, so a failed request cannot pass for an empty body. +(defn head-status-and-body [url] + (match (Client.head url) + (Result.Success r) (fmt "%d [%s]" @(Response.code &r) (Response.body &r)) + (Result.Error e) e)) + (defn count-occurrences [s needle] (let-do [n 0 rest @s @@ -196,6 +202,36 @@ _ false) "307 redirect preserves POST method") + (assert-equal test + "200 []" + &(head-status-and-body + "http://127.0.0.1:8791/redirect-to?url=/get&status_code=301") + "301 redirect keeps HEAD a HEAD") + + (assert-equal test + "200 []" + &(head-status-and-body + "http://127.0.0.1:8791/redirect-to?url=/get&status_code=302") + "302 redirect keeps HEAD a HEAD") + + (assert-equal test + "200 []" + &(head-status-and-body + "http://127.0.0.1:8791/redirect-to?url=/get&status_code=303") + "303 redirect keeps HEAD a HEAD") + + (assert-equal test + 0 + (count-occurrences + &(match (Client.post + "http://127.0.0.1:8791/redirect-to?url=/headers&status_code=303" + {@"Content-Type" [@"application/json"]} + "{\"a\": 1}") + (Result.Success r) @(Response.body &r) + (Result.Error e) e) + "Content-") + "a redirect that changes the method drops the headers describing the body") + (assert-true test (match (Client.request "GET" "http://127.0.0.1:8791/redirect-to?url=/headers&status_code=302" From e163413786a119cd365f33515a028077727532ff Mon Sep 17 00:00:00 2001 From: "carpentry-heartbeat[bot]" Date: Mon, 24 Aug 2026 23:00:45 +0200 Subject: [PATCH 2/3] Pin the stripped header set by name, not by substring count The `Content-` counter could not see `Digest` or `Last-Modified` by construction, so two of the seven names in `content-header?` survived deletion with the suite green. It also fed the error text of a failed request into the same counter, so the assertion read 0 whether the headers were stripped or the request never left the process. `surviving-headers` sends every name in `content-header?` plus an unrelated `X-Keep-Me` control, and reports the names that reached the target prefixed with the status code, so a transport error cannot pass. Deleting any one of the seven now fails the suite; the 307 companion pins the other direction, that a redirect keeping its method strips nothing. The header lookup matches `": "` in the dump rather than anchoring on a leading newline: carp-fmt rewrites a bare "\n" literal into a real newline in the source. --- test/http-client.carp | 43 ++++++++++++++++++++++++++++++++++--------- 1 file changed, 34 insertions(+), 9 deletions(-) diff --git a/test/http-client.carp b/test/http-client.carp index 4ed2efc..f598e26 100644 --- a/test/http-client.carp +++ b/test/http-client.carp @@ -32,6 +32,33 @@ (Result.Success r) (fmt "%d [%s]" @(Response.code &r) (Response.body &r)) (Result.Error e) e)) +; The probe headers that reached the target, prefixed with the status code so +; a failed request cannot pass for a stripped header set. +(defn surviving-headers [status-code] + (let [names [@"CONTENT-TYPE" + @"Content-Encoding" + @"Content-Language" + @"Content-Location" + @"Content-Length" + @"Digest" + @"Last-Modified" + @"X-Keep-Me"]] + (match (Client.post + &(fmt + "http://127.0.0.1:8791/redirect-to?url=/headers&status_code=%d" + status-code) + (Map.from-array + (Array.copy-map &(fn [n] (Pair.init @n [@"probe"])) &names)) + "{\"a\": 1}") + (Result.Error e) e + (Result.Success r) + (let-do [dump (Response.body &r) + kept []] + (foreach [n &names] + (when (String.contains-string? dump &(fmt "%s: " n)) + (set! kept (Array.push-back kept @n)))) + (fmt "%d %s" @(Response.code &r) &(String.join "," &kept)))))) + (defn count-occurrences [s needle] (let-do [n 0 rest @s @@ -221,17 +248,15 @@ "303 redirect keeps HEAD a HEAD") (assert-equal test - 0 - (count-occurrences - &(match (Client.post - "http://127.0.0.1:8791/redirect-to?url=/headers&status_code=303" - {@"Content-Type" [@"application/json"]} - "{\"a\": 1}") - (Result.Success r) @(Response.body &r) - (Result.Error e) e) - "Content-") + "200 X-Keep-Me" + &(surviving-headers 303) "a redirect that changes the method drops the headers describing the body") + (assert-equal test + "200 CONTENT-TYPE,Content-Encoding,Content-Language,Content-Location,Content-Length,Digest,Last-Modified,X-Keep-Me" + &(surviving-headers 307) + "a redirect that keeps the method keeps the headers describing the body") + (assert-true test (match (Client.request "GET" "http://127.0.0.1:8791/redirect-to?url=/headers&status_code=302" From 66e20a4186dc960a7ebc0cf2319f0206e42cd1a8 Mon Sep 17 00:00:00 2001 From: "carpentry-heartbeat[bot]" Date: Tue, 25 Aug 2026 04:22:52 +0200 Subject: [PATCH 3/3] Cover the cookie-jar redirect loop the same change edited The redirect loop exists twice, in request-stream- and in request-stream-with-jar-, and every redirect fixture reached only the first: reverting the jar loop's verb computation or its header strip to the pre-PR expression left the suite at 129 passed / 0 failed. Three rows through Client.head-with-jar and Client.post-with-jar pin the jar copy against the same fixtures, so the two loops cannot drift silently. The helpers are factored rather than duplicated so the jar variants inherit the status-code guard that keeps a failed request from passing for a stripped header set. --- test/http-client.carp | 88 +++++++++++++++++++++++++++++++------------ 1 file changed, 63 insertions(+), 25 deletions(-) diff --git a/test/http-client.carp b/test/http-client.carp index f598e26..33f9cc4 100644 --- a/test/http-client.carp +++ b/test/http-client.carp @@ -27,37 +27,59 @@ (match (Client.get url) (Result.Success r) @(Response.body &r) _ @"")) ; Reports the error text, so a failed request cannot pass for an empty body. -(defn head-status-and-body [url] - (match (Client.head url) +(defn status-and-body [res] + (match res (Result.Success r) (fmt "%d [%s]" @(Response.code &r) (Response.body &r)) (Result.Error e) e)) +(defn head-status-and-body [url] (status-and-body (Client.head url))) + +(defn head-with-jar-status-and-body [url] + (let-do [jar (CookieJar.create)] + (status-and-body (Client.head-with-jar url &jar)))) + +(def probe-names + [@"CONTENT-TYPE" + @"Content-Encoding" + @"Content-Language" + @"Content-Location" + @"Content-Length" + @"Digest" + @"Last-Modified" + @"X-Keep-Me"]) + +(defn probe-map [] + (Map.from-array + (Array.copy-map &(fn [n] (Pair.init @n [@"probe"])) &probe-names))) + +(defn probe-url [status-code] + (fmt + "http://127.0.0.1:8791/redirect-to?url=/headers&status_code=%d" + status-code)) + ; The probe headers that reached the target, prefixed with the status code so ; a failed request cannot pass for a stripped header set. +(defn kept-headers [res] + (match res + (Result.Error e) e + (Result.Success r) + (let-do [dump (Response.body &r) + kept []] + (foreach [n &probe-names] + (when (String.contains-string? dump &(fmt "%s: " n)) + (set! kept (Array.push-back kept @n)))) + (fmt "%d %s" @(Response.code &r) &(String.join "," &kept))))) + (defn surviving-headers [status-code] - (let [names [@"CONTENT-TYPE" - @"Content-Encoding" - @"Content-Language" - @"Content-Location" - @"Content-Length" - @"Digest" - @"Last-Modified" - @"X-Keep-Me"]] - (match (Client.post - &(fmt - "http://127.0.0.1:8791/redirect-to?url=/headers&status_code=%d" - status-code) - (Map.from-array - (Array.copy-map &(fn [n] (Pair.init @n [@"probe"])) &names)) - "{\"a\": 1}") - (Result.Error e) e - (Result.Success r) - (let-do [dump (Response.body &r) - kept []] - (foreach [n &names] - (when (String.contains-string? dump &(fmt "%s: " n)) - (set! kept (Array.push-back kept @n)))) - (fmt "%d %s" @(Response.code &r) &(String.join "," &kept)))))) + (kept-headers (Client.post &(probe-url status-code) (probe-map) "{\"a\": 1}"))) + +(defn surviving-headers-with-jar [status-code] + (let-do [jar (CookieJar.create)] + (kept-headers + (Client.post-with-jar &(probe-url status-code) + (probe-map) + "{\"a\": 1}" + &jar)))) (defn count-occurrences [s needle] (let-do [n 0 @@ -247,6 +269,12 @@ "http://127.0.0.1:8791/redirect-to?url=/get&status_code=303") "303 redirect keeps HEAD a HEAD") + (assert-equal test + "200 []" + &(head-with-jar-status-and-body + "http://127.0.0.1:8791/redirect-to?url=/get&status_code=301") + "301 redirect keeps HEAD a HEAD on the cookie-jar path") + (assert-equal test "200 X-Keep-Me" &(surviving-headers 303) @@ -257,6 +285,16 @@ &(surviving-headers 307) "a redirect that keeps the method keeps the headers describing the body") + (assert-equal test + "200 X-Keep-Me" + &(surviving-headers-with-jar 303) + "the same drop holds on the cookie-jar path") + + (assert-equal test + "200 CONTENT-TYPE,Content-Encoding,Content-Language,Content-Location,Content-Length,Digest,Last-Modified,X-Keep-Me" + &(surviving-headers-with-jar 307) + "the same keep holds on the cookie-jar path") + (assert-true test (match (Client.request "GET" "http://127.0.0.1:8791/redirect-to?url=/headers&status_code=302"