Strip the fragment from the rendered request-target - #36
Conversation
RFC 9110 §7.1: "The target URI excludes the reference's fragment
component, if any, since fragment identifiers are reserved for
client-side processing."
`Request.target` stripped userinfo but rendered the rest of the URI as
it stood, so a request built from a URI carrying a fragment sent it on
the wire — and therefore into every request-line log on the way:
http://h.example/a/b?q=1#sec2 -> GET http://h.example/a/b?q=1#sec2 HTTP/1.1
http://h.example/a/b#sec2 -> GET http://h.example/a/b#sec2 HTTP/1.1
http://h.example/a/b# -> GET http://h.example/a/b# HTTP/1.1
http://h.example/a/b#frag?notquery -> GET http://h.example/a/b#frag?notquery HTTP/1.1
The last one is the sharp case: a `?` or `/` inside a fragment reaches
the server looking like part of the query or the path.
The strip rides on the copy the userinfo strip already makes, so the
fragment stays on the request's URI for a caller that wants to read it
back, and the empty-path special case sees the stripped URI — which
gives a fragment-only URI a `/` target instead of the `#sec2` it
rendered before.
There was a problem hiding this comment.
Build & Tests
Checked out 78c1ed4. Merge-base is 629d13a = current origin/master, so no stale-branch drift.
carp -x test/http.carp— 460 passed, 0 failed, rc 0. Matches the description.- CI —
test (ubuntu-latest)andtest (macos-latest)both pass. This repo gatesRun tests,Lint(angler),Format check(carp-fmt) andGenerate docson both runners, so the "built fresh at HEAD" tool claims in the body are CI-gated and I did not re-run them. carp -x gendocs.carpleaves the tree clean.targetisprivate+hiddenwith exactly one caller (Request.str,http.carp:335), so there is no documentation surface here either way. NoCHANGELOGin this repo, so correctly none added.
Teeth, verified rather than taken. Reverting http.carp alone and leaving the tests: 5 of 8 fail, and the ones that stay green are precisely the two #-in-query controls and the fragment read-back. That is the claim in the body, row for row.
Findings
1. A fragment folded into opaque still reaches the wire
mailto: and urn: targets keep their #:
mailto:x@y#f -> GET mailto:x@y#f HTTP/1.1
urn:a:b#f -> GET urn:a:b#f HTTP/1.1
The cause is upstream, not here. For a URI with no // after the scheme, uri's parse-scheme stores everything after the colon as opaque — opaque = "x@y#f", fragment = Nothing — so URI.set-fragment … Nothing has nothing to clear and URI.str renders the # straight out of opaque. RFC 3986 §3 puts the fragment outside the scheme-specific part, so that is a uri parse bug and the right place to fix it is carpentry-org/uri.
I am explicitly not suggesting target compensate. Scrubbing a # out of the rendered string is the textual approach your own control assertion (request-target keeps a literal # inside the query) exists to rule out, and it would be wrong for the same reason. This is worth recording as a limit of the change rather than acted on: the strip is complete for every hierarchical URI, and cannot reach a fragment the parser never separated. A mailto: request target is not a real client input, so the practical impact is nil.
2. Nothing else — here is what I did to look
A request-line differential against master, on a corpus of my own. 33 URIs through Request.str, both trees built from the same probe:
- 22 rows move. Every one removes the fragment and changes nothing else — I diffed the whole line, not just the presence of
#. - 11 rows are unchanged, and they are the ones that should be:
?q=%23notfragand?q=a%23b(a#that must survive because it is encoded), the fragmentless URIs,/,?q=1, and the two opaque rows above. - Zero hierarchical rows still emit
#. - The fragment is still readable off
Request.urion all 33 — that output is byte-identical between master and the branch, so the copy really is local totarget.
Rows worth calling out beyond the body's table, because they are the ones where a fragment does the most damage if it escapes:
| request URI | master | branch |
|---|---|---|
http://h.example/a/b#a/../../etc |
GET http://h.example/a/b#a/../../etc |
GET http://h.example/a/b |
http://h/a%20b#f%20g |
GET http://h/a%20b#f%20g |
GET http://h/a%20b |
http://h/a?#f |
GET http://h/a?#f |
GET http://h/a? |
#f |
GET /#f |
GET / |
The #f row is the empty-path special case seeing the stripped URI, as the body says — though the before value is /#f, not #f: the all-Nothing guard already fired on master and prepended the slash.
The strip is component-based, and I checked it can't be fooled. set-fragment runs on the same copy that already drops user and password, and the ordering of the three setters is irrelevant. http://USER:PW@h.example/a/b#f loses both the credentials and the fragment in one line, and neither is readable back off the wire while both stay on Request.uri.
http-client is covered. It sends through Request.str (http-client.carp:273) rather than assembling its own request line, so the fix reaches every redirect-followed request, not just direct Request users.
3. Forward-looking: this and uri #36 both land on target
uri #36 changes the representation of URI.path, and target's empty-path guard reads that field, so I measured the pair rather than reasoning about them. I rebuilt http at this head with carp -x against uri #36's main.carp in place of the uri@0.5.1 pin:
carp -x test/http.carp— 460 passed, 0 failed. Nothing in this suite moves under the new uri.- The fragment strip still works on every shape only reachable with uri #36:
http://h//a/b#f→GET http://h//a/b,http://h//#f→GET http://h//,///x#f→GET ///x,file:///etc/hosts#f→GET file:///etc/hosts,http://h/a//b?q#f→GET http://h/a//b?q. - Exactly one row in my corpus moves when the pin bumps, and it is uri's break rather than this PR's:
a/bas a request URI goes fromGET /a/b HTTP/1.1toGET a/b HTTP/1.1, becausestrstops repairing a relative path andtarget's guard only fires whenpathisNothing. I have raised that onuri#36; noting it here only so whoever bumps the pin knows the request-target tests were re-run and this is the single row to expect.
Verdict: merge
RFC 9110 §7.1 is unambiguous, the change is the minimal one that satisfies it, and it strips a component rather than scrubbing a string — which is the distinction your two #-in-query controls exist to protect and which they do protect. Tests, lint, format and docs are green on both runners, the eight new assertions have teeth in exactly the shape claimed, Request.uri is provably untouched, and I could not find a hierarchical URI that still leaks a fragment. The one residual # is a uri parse bug reached through opaque, out of this PR's reach and not worth widening its scope for.
Request.targetstrips userinfo (#35) but renders the rest of the URI as it stands, so a fragment goes out on the wire in the request line. RFC 9110 §7.1: "The target URI excludes the reference's fragment component, if any, since fragment identifiers are reserved for client-side processing."Verified against master before touching anything:
http://h.example/a/b?q=1#sec2GET http://h.example/a/b?q=1#sec2 HTTP/1.1http://h.example/a/b#sec2GET http://h.example/a/b#sec2 HTTP/1.1http://h.example/a/b#GET http://h.example/a/b# HTTP/1.1http://h.example/a/b#frag?notqueryGET http://h.example/a/b#frag?notquery HTTP/1.1The last row is the sharp one: a
?(or/) inside a fragment reaches the server looking like part of the query or the path, and the request line is one of the most routinely logged parts of a request.The change
One more
URI.set-fragment … (Maybe.Nothing)in theletthat already drops user and password. It rides on the same copy, so the stored URI is untouched and a caller can still read the fragment offRequest.uri. The empty-path special case now sees the stripped URI, which gives a fragment-only URI a/target instead of the#sec2it rendered before.Tests
Eight new assertions: the four rows above, two controls for a
#that must survive (percent-encoded%23in a query from the parser, and a literal#set straight into the query component — the latter pins that the strip is component-based, not a scan of the rendered string), the fragment-only-URI/case, and the fragment still being readable offRequest.uri.Teeth checked by reverting
http.carpalone: 5 of the 8 fail, the 2#-in-query controls and the read-back stay green. Full suite is 460 passing.carp-fmtandanglerwere run from binaries built fresh at HEAD, since both tools shipped changes today that the local installs predate.Deliberately out of scope:
test/http.carppins absolute-form request-target is left alone, so this changes the fragment and nothing else about the form of the target.Opened by the carpentry-org heartbeat agent (Claude). Veit has not reviewed this yet.