From 6c30210f0959fd4d793f2f43ddc2e472a45d5022 Mon Sep 17 00:00:00 2001 From: "carpentry-heartbeat[bot]" Date: Sat, 29 Aug 2026 02:38:37 +0200 Subject: [PATCH 1/2] Decode multipart bodies from bytes so binary uploads survive (#40) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `Multipart.parse` took a `&String`, and a Carp `String` is a bare `char*`, so a body carrying a NUL byte ended at that NUL before any parsing started. The closing delimiter went with it and the call returned `Result.Success []` — an upload that succeeded and carried nothing. Every PNG, PDF, zip and JPEG has NUL bytes in it. The scan is now byte-based end to end. `Multipart.parse-bytes` takes an `(Array Byte)` and returns `BinaryPart`s whose bodies are bytes; `at?`, `skip-table` and `index-from` work on bytes, keeping #39's Boyer-Moore-Horspool shift table. `Multipart.parse` keeps its signature and is now a thin wrapper over `parse-bytes`, so there is one algorithm rather than two, and its docstring says plainly that a `String` body truncates. --- README.md | 1 + docs/Accept.html | 5 + docs/AcceptEncoding.html | 5 + docs/AcceptLanguage.html | 5 + docs/Auth.html | 5 + docs/BinaryPart.html | 609 +++++++++++++++++++++++++++++++++++++ docs/ByteRange.html | 5 + docs/ByteRangeSpec.html | 5 + docs/CacheControl.html | 5 + docs/ContentRange.html | 5 + docs/Cookie.html | 5 + docs/Credentials.html | 5 + docs/ETag.html | 5 + docs/ETagList.html | 5 + docs/Form.html | 5 + docs/FormPart.html | 5 + docs/HttpDate.html | 5 + docs/MediaRange.html | 5 + docs/MediaType.html | 5 + docs/Multipart.html | 47 ++- docs/Precondition.html | 5 + docs/Request.html | 8 + docs/Response.html | 5 + docs/SameSite.html | 5 + docs/Status.html | 5 + docs/TransferEncoding.html | 5 + docs/Weighted.html | 5 + docs/http_index.html | 5 + docs/index.html | 5 + gendocs.carp | 1 + http.carp | 187 ++++++++---- test/http.carp | 158 ++++++++++ 32 files changed, 1065 insertions(+), 71 deletions(-) create mode 100644 docs/BinaryPart.html diff --git a/README.md b/README.md index 8c01d02..1b71e1f 100644 --- a/README.md +++ b/README.md @@ -243,6 +243,7 @@ Status.not-found ; => 404 | `Credentials` | one authentication scheme with its token68 or auth-params | | `Multipart` | `multipart/form-data` body decoder | | `FormPart` | a single decoded multipart part (name, filename, content-type, body) | +| `BinaryPart` | a decoded multipart part whose body is bytes, for binary uploads | | `CacheControl` | `Cache-Control` directive parser (RFC 7234 §5.2) | | `TransferEncoding` | Chunked transfer-encoding decoder | | `HttpDate` | HTTP-date parser and formatter (RFC 9110 §5.6.7) | diff --git a/docs/Accept.html b/docs/Accept.html index fe74c8b..1d24976 100644 --- a/docs/Accept.html +++ b/docs/Accept.html @@ -62,6 +62,11 @@ FormPart +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/AcceptEncoding.html b/docs/AcceptEncoding.html index fa71380..967f418 100644 --- a/docs/AcceptEncoding.html +++ b/docs/AcceptEncoding.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/AcceptLanguage.html b/docs/AcceptLanguage.html index 34ed2c4..7c4c298 100644 --- a/docs/AcceptLanguage.html +++ b/docs/AcceptLanguage.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/Auth.html b/docs/Auth.html index f9b4c8f..c21e890 100644 --- a/docs/Auth.html +++ b/docs/Auth.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/BinaryPart.html b/docs/BinaryPart.html new file mode 100644 index 0000000..dae903a --- /dev/null +++ b/docs/BinaryPart.html @@ -0,0 +1,609 @@ + + + + + + + + + +
    + +
    +

    + BinaryPart +

    +
    +

    is a single part decoded from a multipart/form-data body by +Multipart.parse-bytes. It carries the same +fields as a FormPart, but its body is an (Array Byte), so +it survives the NUL bytes every binary upload contains.

    + +
    +
    + +

    + body +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a)] (Ref (Array Byte) a)) +

    + + + +

    +

    gets the body property of a BinaryPart.

    + +

    +
    +
    + +

    + content-type +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a)] (Ref (Maybe String) a)) +

    + + + +

    +

    gets the content-type property of a BinaryPart.

    + +

    +
    +
    + +

    + copy +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a)] BinaryPart) +

    + + + +

    +

    copies a BinaryPart.

    + +

    +
    +
    + +

    + delete +

    +     +
    + instantiate +
    +

    + (Fn [BinaryPart] ()) +

    + + + +

    +

    deletes a BinaryPart. Should usually not be called manually.

    + +

    +
    +
    + +

    + filename +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a)] (Ref (Maybe String) a)) +

    + + + +

    +

    gets the filename property of a BinaryPart.

    + +

    +
    +
    + +

    + init +

    +     +
    + instantiate +
    +

    + (Fn [String, (Maybe String), (Maybe String), (Array Byte)] BinaryPart) +

    + + + +

    +

    creates a BinaryPart.

    + +

    +
    +
    + +

    + name +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a)] (Ref String a)) +

    + + + +

    +

    gets the name property of a BinaryPart.

    + +

    +
    +
    + +

    + prn +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a)] String) +

    + + + +

    +

    converts a BinaryPart to a string.

    + +

    +
    +
    + +

    + set-body +

    +     +
    + instantiate +
    +

    + (Fn [BinaryPart, (Array Byte)] BinaryPart) +

    + + + +

    +

    sets the body property of a BinaryPart.

    + +

    +
    +
    + +

    + set-body! +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a), (Array Byte)] ()) +

    + + + +

    +

    sets the body property of a BinaryPart in place.

    + +

    +
    +
    + +

    + set-content-type +

    +     +
    + instantiate +
    +

    + (Fn [BinaryPart, (Maybe String)] BinaryPart) +

    + + + +

    +

    sets the content-type property of a BinaryPart.

    + +

    +
    +
    + +

    + set-content-type! +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a), (Maybe String)] ()) +

    + + + +

    +

    sets the content-type property of a BinaryPart in place.

    + +

    +
    +
    + +

    + set-filename +

    +     +
    + instantiate +
    +

    + (Fn [BinaryPart, (Maybe String)] BinaryPart) +

    + + + +

    +

    sets the filename property of a BinaryPart.

    + +

    +
    +
    + +

    + set-filename! +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a), (Maybe String)] ()) +

    + + + +

    +

    sets the filename property of a BinaryPart in place.

    + +

    +
    +
    + +

    + set-name +

    +     +
    + instantiate +
    +

    + (Fn [BinaryPart, String] BinaryPart) +

    + + + +

    +

    sets the name property of a BinaryPart.

    + +

    +
    +
    + +

    + set-name! +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a), String] ()) +

    + + + +

    +

    sets the name property of a BinaryPart in place.

    + +

    +
    +
    + +

    + str +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a)] String) +

    + + + +

    +

    converts a BinaryPart to a string.

    + +

    +
    +
    + +

    + to-form-part +

    +     +
    + defn +
    +

    + (Fn [(Ref BinaryPart a)] FormPart) +

    + 
    +                        (to-form-part p)
+
    +

    +

    converts a BinaryPart into a FormPart. +The body becomes a String, and is therefore cut at its first NUL byte; only +use this on parts you know are text.

    + +

    +
    +
    + +

    + update-body +

    +     +
    + instantiate +
    +

    + (Fn [BinaryPart, (Ref (Fn [(Array Byte)] (Array Byte) a) b)] BinaryPart) +

    + + + +

    +

    updates the body property of a BinaryPart using a function f.

    + +

    +
    +
    + +

    + update-content-type +

    +     +
    + instantiate +
    +

    + (Fn [BinaryPart, (Ref (Fn [(Maybe String)] (Maybe String) a) b)] BinaryPart) +

    + + + +

    +

    updates the content-type property of a BinaryPart using a function f.

    + +

    +
    +
    + +

    + update-filename +

    +     +
    + instantiate +
    +

    + (Fn [BinaryPart, (Ref (Fn [(Maybe String)] (Maybe String) a) b)] BinaryPart) +

    + + + +

    +

    updates the filename property of a BinaryPart using a function f.

    + +

    +
    +
    + +

    + update-name +

    +     +
    + instantiate +
    +

    + (Fn [BinaryPart, (Ref (Fn [String] String a) b)] BinaryPart) +

    + + + +

    +

    updates the name property of a BinaryPart using a function f.

    + +

    +
    +
    +
    + + diff --git a/docs/ByteRange.html b/docs/ByteRange.html index d71f1eb..3758a6f 100644 --- a/docs/ByteRange.html +++ b/docs/ByteRange.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/ByteRangeSpec.html b/docs/ByteRangeSpec.html index 874fe9f..f8b590a 100644 --- a/docs/ByteRangeSpec.html +++ b/docs/ByteRangeSpec.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/CacheControl.html b/docs/CacheControl.html index b81b346..3c135ad 100644 --- a/docs/CacheControl.html +++ b/docs/CacheControl.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/ContentRange.html b/docs/ContentRange.html index b211c24..4f4f0ed 100644 --- a/docs/ContentRange.html +++ b/docs/ContentRange.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/Cookie.html b/docs/Cookie.html index f565e85..30a0fd9 100644 --- a/docs/Cookie.html +++ b/docs/Cookie.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/Credentials.html b/docs/Credentials.html index 9ab9fea..699d678 100644 --- a/docs/Credentials.html +++ b/docs/Credentials.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/ETag.html b/docs/ETag.html index 852a36f..360f780 100644 --- a/docs/ETag.html +++ b/docs/ETag.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/ETagList.html b/docs/ETagList.html index e59c3fe..63bbd87 100644 --- a/docs/ETagList.html +++ b/docs/ETagList.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/Form.html b/docs/Form.html index 88fa728..b93cd71 100644 --- a/docs/Form.html +++ b/docs/Form.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/FormPart.html b/docs/FormPart.html index d369065..2d9b080 100644 --- a/docs/FormPart.html +++ b/docs/FormPart.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/HttpDate.html b/docs/HttpDate.html index 17fe73a..751e5bd 100644 --- a/docs/HttpDate.html +++ b/docs/HttpDate.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/MediaRange.html b/docs/MediaRange.html index 5df64e5..af9e33d 100644 --- a/docs/MediaRange.html +++ b/docs/MediaRange.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/MediaType.html b/docs/MediaType.html index 67f94a1..ab41821 100644 --- a/docs/MediaType.html +++ b/docs/MediaType.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/Multipart.html b/docs/Multipart.html index 481e8e0..e731eb2 100644 --- a/docs/Multipart.html +++ b/docs/Multipart.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding @@ -151,14 +156,18 @@

    decodes multipart/form-data bodies (RFC 7578) into their -FormParts. The boundary is a parameter of the Content-Type -header; Request.multipart-data reads it off the -request for you.

    +parts. The boundary is a parameter of the Content-Type header; +Request.multipart-data reads it off the request +for you.

    (match (Request.multipart-data &req)
   (Result.Success parts)
     (println* (FormPart.name (Array.unsafe-nth &parts 0)))
   (Result.Error e) (IO.errorln &e))
    +

    parse yields FormParts, whose bodies are +Strings and so stop at their first NUL byte. Uploads that are not text need +parse-bytes, which yields +BinaryParts instead.

    @@ -178,7 +187,37 @@

    decodes a multipart/form-data body with the given boundary -into its FormParts. Fails when the opening boundary delimiter is absent.

    +into its FormParts. Fails when the opening boundary delimiter +is absent.

    +

    A Carp String ends at its first NUL byte, so a binary body — any PNG, PDF or +zip upload — is cut short before parsing even begins, and usually decodes to no +parts at all. Reach for parse-bytes whenever the body might +not be text.

    + +

    +

    +
    + +

    + parse-bytes +

    +     +
    + defn +
    +

    + (Fn [(Ref (Array Byte) a), (Ref String b)] (Result (Array BinaryPart) String)) +

    + 
    +                        (parse-bytes body boundary)
+
    +

    +

    decodes a multipart/form-data body with the given +boundary into its BinaryParts, reading the body as bytes +so that binary uploads survive intact.

    +

    Fails when the opening boundary delimiter is absent, and when a part’s header +region holds a NUL byte: header field lines are US-ASCII, and reading one as a +String would cut a filename short without saying so.

    diff --git a/docs/Precondition.html b/docs/Precondition.html index ccfe454..6af5232 100644 --- a/docs/Precondition.html +++ b/docs/Precondition.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/Request.html b/docs/Request.html index cd4dc8b..0e25cf1 100644 --- a/docs/Request.html +++ b/docs/Request.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding @@ -442,6 +447,9 @@

    the boundary from the Content-Type header. Returns (Result (Array FormPart) String); fails when the request is not multipart/form-data or carries no boundary.

    +

    A Request body is a String and therefore stops at its first NUL byte, so an +upload that is not text has to be decoded with +Multipart.parse-bytes instead.

    diff --git a/docs/Response.html b/docs/Response.html index 6b94478..fe0b682 100644 --- a/docs/Response.html +++ b/docs/Response.html @@ -62,6 +62,11 @@ FormPart

    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/SameSite.html b/docs/SameSite.html index b63e1d1..e409aa6 100644 --- a/docs/SameSite.html +++ b/docs/SameSite.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/Status.html b/docs/Status.html index c47cca6..2f3b16f 100644 --- a/docs/Status.html +++ b/docs/Status.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/TransferEncoding.html b/docs/TransferEncoding.html index 6dfc20a..0c74055 100644 --- a/docs/TransferEncoding.html +++ b/docs/TransferEncoding.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/Weighted.html b/docs/Weighted.html index f45845d..bff199b 100644 --- a/docs/Weighted.html +++ b/docs/Weighted.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/http_index.html b/docs/http_index.html index 76b81c0..1c22232 100644 --- a/docs/http_index.html +++ b/docs/http_index.html @@ -58,6 +58,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/index.html b/docs/index.html index 76b81c0..1c22232 100644 --- a/docs/index.html +++ b/docs/index.html @@ -58,6 +58,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/gendocs.carp b/gendocs.carp index 9950803..e9c90ad 100644 --- a/gendocs.carp +++ b/gendocs.carp @@ -50,6 +50,7 @@ Status.not-found ; => 404 Form Multipart FormPart + BinaryPart TransferEncoding MediaType Accept diff --git a/http.carp b/http.carp index 495a035..71db98a 100644 --- a/http.carp +++ b/http.carp @@ -1785,29 +1785,54 @@ field `name`, an optional `filename` (present for file uploads), an optional content-type (Maybe String) body String]) +(doc BinaryPart "is a single part decoded from a `multipart/form-data` body by +[`Multipart.parse-bytes`](Multipart.html#parse-bytes). It carries the same +fields as a [`FormPart`](FormPart.html), but its `body` is an `(Array Byte)`, so +it survives the NUL bytes every binary upload contains.") +(deftype BinaryPart + [name String + filename (Maybe String) + content-type (Maybe String) + body (Array Byte)]) + +(defmodule BinaryPart + (doc to-form-part "converts a `BinaryPart` into a [`FormPart`](FormPart.html). +The body becomes a `String`, and is therefore cut at its first NUL byte; only +use this on parts you know are text.") + (defn to-form-part [p] + (FormPart.init @(name p) + @(filename p) + @(content-type p) + (String.from-bytes (body p))))) + (doc Multipart "decodes `multipart/form-data` bodies (RFC 7578) into their -[`FormPart`](FormPart.html)s. The boundary is a parameter of the `Content-Type` -header; [`Request.multipart-data`](Request.html#multipart-data) reads it off the -request for you. +parts. The boundary is a parameter of the `Content-Type` header; +[`Request.multipart-data`](Request.html#multipart-data) reads it off the request +for you. ``` (match (Request.multipart-data &req) (Result.Success parts) (println* (FormPart.name (Array.unsafe-nth &parts 0))) (Result.Error e) (IO.errorln &e)) -```") +``` + +[`parse`](#parse) yields [`FormPart`](FormPart.html)s, whose bodies are +`String`s and so stop at their first NUL byte. Uploads that are not text need +[`parse-bytes`](#parse-bytes), which yields +[`BinaryPart`](BinaryPart.html)s instead.") (defmodule Multipart (hidden at?) (private at?) - ; does `s`, whose byte length is `slen`, hold `sub` at byte offset `pos`? - (defn at? [s slen pos sub] - (let [subl (String.length sub)] - (if (or (< pos 0) (> (+ pos subl) slen)) + ; do the bytes `sub` occur in `bs` at byte offset `pos`? + (defn at? [bs pos sub] + (let [subl (Array.length sub)] + (if (or (< pos 0) (> (+ pos subl) (Array.length bs))) false (let-do [ok true j 0] (while (and ok (< j subl)) - (if (= (String.char-at s (+ pos j)) (String.char-at sub j)) + (if (= (Array.unsafe-nth bs (+ pos j)) (Array.unsafe-nth sub j)) (set! j (Int.inc j)) (set! ok false))) ok)))) @@ -1815,26 +1840,29 @@ request for you. (hidden skip-table) (private skip-table) ; Boyer-Moore-Horspool bad-character shifts for `needle`, indexed by byte. - (defn skip-table [needle nlen] - (let-do [t (Array.replicate 256 &nlen) + (defn skip-table [needle] + (let-do [nlen (Array.length needle) + t (Array.replicate 256 &nlen) last (Int.dec nlen)] (for [i 0 last] - (Array.aset! &t (Char.to-int (String.char-at needle i)) (- last i))) + (Array.aset! &t (Byte.to-int @(Array.unsafe-nth needle i)) (- last i))) t)) (hidden index-from) (private index-from) - ; first byte offset >= `from` where `needle` occurs in `s`, or -1. `skip` is + ; first byte offset >= `from` where `needle` occurs in `bs`, or -1. `skip` is ; the needle’s `skip-table`. - (defn index-from [s slen needle nlen skip from] + (defn index-from [bs needle skip from] (let-do [res -1 + len (Array.length bs) + nlen (Array.length needle) i from last (Int.dec nlen)] - (while (and (< res 0) (<= (+ i nlen) slen)) - (let [c (String.char-at s (+ i last))] - (if (and (= c (String.char-at needle last)) (at? s slen i needle)) + (while (and (< res 0) (<= (+ i nlen) len)) + (let [c @(Array.unsafe-nth bs (+ i last))] + (if (and (= c @(Array.unsafe-nth needle last)) (at? bs i needle)) (set! res i) - (set! i (+ i @(Array.unsafe-nth skip (Char.to-int c))))))) + (set! i (+ i @(Array.unsafe-nth skip (Byte.to-int c))))))) res)) (hidden header-value) @@ -1855,73 +1883,94 @@ request for you. (hidden parse-part) (private parse-part) - ; parses one part (`headers CRLF CRLF body`) into a FormPart. + ; parses one part (`headers CRLF CRLF body`) into a BinaryPart. (defn parse-part [raw] - (let [sep (String.index-of-string raw "\r\n\r\n") - head (if (< sep 0) @raw (String.byte-slice raw 0 sep)) - body (if (< sep 0) - @"" - (String.byte-slice raw (+ sep 4) (String.length raw))) - lines (String.split-by &head &[\newline])] - (let-do [nm @"" - fname (the (Maybe String) (Maybe.Nothing)) - ctype (the (Maybe String) (Maybe.Nothing))] - (for [i 0 (Array.length &lines)] - (let-do [line &(String.trim (Array.unsafe-nth &lines i))] - (match (header-value line "content-disposition") - (Maybe.Just cd) - (let-do [ps (MediaType.parse-params &cd)] - (set! nm (Map.get-with-default &ps "name" "")) - (set! fname (Map.get-maybe &ps "filename"))) - (Maybe.Nothing) ()) - (match (header-value line "content-type") - (Maybe.Just ct) (set! ctype (Maybe.Just ct)) - (Maybe.Nothing) ()))) - (FormPart.init nm fname ctype body)))) - - (doc parse "decodes a `multipart/form-data` `body` with the given `boundary` -into its `FormPart`s. Fails when the opening boundary delimiter is absent.") - (defn parse [body boundary] - (let [dash (String.append "--" boundary) - crlf-dash (String.append "\r\n" &dash) - cdlen (String.length &crlf-dash) - dashlen (String.length &dash) - len (String.length body) - skip (Multipart.skip-table &crlf-dash cdlen) + (let [blank (String.to-bytes "\r\n\r\n") + sep (index-from raw &blank &(skip-table &blank) 0) + head (if (< sep 0) @raw (Array.prefix raw sep)) + body (if (< sep 0) (the (Array Byte) []) (Array.suffix raw (+ sep 4)))] + (if (Array.any? &(fn [b] (= 0b @b)) &head) + (Result.Error @"multipart: NUL byte in part headers") + (let-do [lines (String.split-by &(String.from-bytes &head) &[\newline]) + nm @"" + fname (the (Maybe String) (Maybe.Nothing)) + ctype (the (Maybe String) (Maybe.Nothing))] + (for [i 0 (Array.length &lines)] + (let-do [line &(String.trim (Array.unsafe-nth &lines i))] + (match (header-value line "content-disposition") + (Maybe.Just cd) + (let-do [ps (MediaType.parse-params &cd)] + (set! nm (Map.get-with-default &ps "name" "")) + (set! fname (Map.get-maybe &ps "filename"))) + (Maybe.Nothing) ()) + (match (header-value line "content-type") + (Maybe.Just ct) (set! ctype (Maybe.Just ct)) + (Maybe.Nothing) ()))) + (Result.Success (BinaryPart.init nm fname ctype body)))))) + + (doc parse-bytes "decodes a `multipart/form-data` `body` with the given +`boundary` into its [`BinaryPart`](BinaryPart.html)s, reading the body as bytes +so that binary uploads survive intact. + +Fails when the opening boundary delimiter is absent, and when a part’s header +region holds a NUL byte: header field lines are US-ASCII, and reading one as a +`String` would cut a `filename` short without saying so.") + (defn parse-bytes [body boundary] + (let [dash (String.to-bytes &(String.append "--" boundary)) + crlf-dash (String.to-bytes &(String.append "\r\n--" boundary)) + crlf (String.to-bytes "\r\n") + dashes (String.to-bytes "--") + cdlen (Array.length &crlf-dash) + dashlen (Array.length &dash) + len (Array.length body) + skip (Multipart.skip-table &crlf-dash) ; the opening delimiter is `--boundary`, either at the very start or ; after a preamble that ends in CRLF; every later delimiter is ; `CRLF--boundary`, so searching for that avoids matching a bare ; `--boundary` that happens to occur inside a part body. - open (if (Multipart.at? body len 0 &dash) + open (if (Multipart.at? body 0 &dash) dashlen - (let [p (Multipart.index-from body len &crlf-dash cdlen &skip 0)] + (let [p (Multipart.index-from body &crlf-dash &skip 0)] (if (< p 0) -1 (+ p cdlen))))] (if (< open 0) (Result.Error @"multipart: boundary delimiter not found") - (let-do [parts (the (Array FormPart) []) + (let-do [parts (the (Array BinaryPart) []) + err (the (Maybe String) (Maybe.Nothing)) pos open done false] (while (not done) (cond - (Multipart.at? body len pos "--") (set! done true) + (Multipart.at? body pos &dashes) (set! done true) (>= pos len) (set! done true) (do - (when (Multipart.at? body len pos "\r\n") (set! pos (+ pos 2))) - (let [next (Multipart.index-from body - len - &crlf-dash - cdlen - &skip - pos)] + (when (Multipart.at? body pos &crlf) (set! pos (+ pos 2))) + (let [next (Multipart.index-from body &crlf-dash &skip pos)] (if (< next 0) (set! done true) (do - (Array.push-back! &parts - (Multipart.parse-part &(String.byte-slice body - pos - next))) + (match (Multipart.parse-part &(Array.slice body pos next)) + (Result.Success p) (Array.push-back! &parts p) + (Result.Error e) + (do (set! err (Maybe.Just e)) (set! done true))) (set! pos (+ next cdlen)))))))) - (Result.Success parts)))))) + (match err + (Maybe.Just e) (Result.Error e) + (Maybe.Nothing) (Result.Success parts)))))) + + (doc parse "decodes a `multipart/form-data` `body` with the given `boundary` +into its [`FormPart`](FormPart.html)s. Fails when the opening boundary delimiter +is absent. + +A Carp `String` ends at its first NUL byte, so a binary `body` — any PNG, PDF or +zip upload — is cut short before parsing even begins, and usually decodes to no +parts at all. Reach for [`parse-bytes`](#parse-bytes) whenever the body might +not be text.") + (defn parse [body boundary] + (match (Multipart.parse-bytes &(String.to-bytes body) boundary) + (Result.Error e) (Result.Error e) + (Result.Success ps) + (Result.Success + (Array.copy-map &(fn [p] (BinaryPart.to-form-part p)) &ps))))) (doc ByteRangeSpec "is one `range-spec` of a `Range` header in the `bytes` unit (RFC 9110 §14.1.1): `FirstLast` for `0-499`, `From` for `500-` — everything from @@ -2543,7 +2592,11 @@ Returns `(Result (Map String String) String)`.") (doc multipart-data "parses the request body as `multipart/form-data`, taking the boundary from the `Content-Type` header. Returns `(Result (Array FormPart) String)`; fails when the request is not -`multipart/form-data` or carries no boundary.") +`multipart/form-data` or carries no boundary. + +A `Request` body is a `String` and therefore stops at its first NUL byte, so an +upload that is not text has to be decoded with +[`Multipart.parse-bytes`](Multipart.html#parse-bytes) instead.") (defn multipart-data [r] (match (Request.header r "Content-Type") (Maybe.Nothing) (Result.Error @"multipart: no Content-Type header") diff --git a/test/http.carp b/test/http.carp index e5a5085..359188c 100644 --- a/test/http.carp +++ b/test/http.carp @@ -65,6 +65,53 @@ (def mp-two @"--b\r\nContent-Disposition: form-data; name=\"a\"\r\n\r\n1\r\n--b\r\nContent-Disposition: form-data; name=\"c\"\r\n\r\n2\r\n--b--\r\n") +(defn bp-parts [body boundary] + (match (Multipart.parse-bytes body boundary) + (Result.Success ps) ps + (Result.Error _) [])) +(defn bp-name [ps i] @(BinaryPart.name (Array.unsafe-nth ps i))) +(defn bp-body [ps i] @(BinaryPart.body (Array.unsafe-nth ps i))) +(defn bp-fname [ps i] + (Maybe.from @(BinaryPart.filename (Array.unsafe-nth ps i)) @"")) +(defn bp-error [body boundary] + (match (Multipart.parse-bytes body boundary) + (Result.Error e) e + (Result.Success _) @"")) +(defn bp-as-form-parts [body boundary] + (Array.copy-map &(fn [p] (BinaryPart.to-form-part p)) + &(bp-parts body boundary))) + +; a one-part body whose payload is the bytes of `pre`, a NUL, then `post` +(defn nul-part [pre post] + (Array.concat + &[(String.to-bytes + "--b\r\nContent-Disposition: form-data; name=\"f\"; filename=\"up.bin\"\r\n\r\n") + (String.to-bytes pre) + [0b] + (String.to-bytes post) + (String.to-bytes "\r\n--b--\r\n")])) + +(def mp-nul + (Array.concat + &[(String.to-bytes + "--B\r\nContent-Disposition: form-data; name=\"f\"; filename=\"a.bin\"\r\n\r\nXX") + [0b] + (String.to-bytes "Y\r\n--B--\r\n")])) + +(def mp-mixed + (Array.concat + &[(String.to-bytes + "--b\r\nContent-Disposition: form-data; name=\"a\"\r\n\r\nhello\r\n--b\r\nContent-Disposition: form-data; name=\"f\"; filename=\"x.bin\"\r\n\r\n") + [0b 1b 2b] + (String.to-bytes "\r\n--b--\r\n")])) + +(def mp-nul-filename + (Array.concat + &[(String.to-bytes + "--b\r\nContent-Disposition: form-data; name=\"f\"; filename=\"evil.php") + [0b] + (String.to-bytes ".jpg\"\r\n\r\ndata\r\n--b--\r\n")])) + ; ---- content negotiation (Accept) test helpers ---- (defn pick [accept offers] (match (Accept.negotiate accept offers) @@ -1273,6 +1320,117 @@ 0) "a delimiter that ends at the end of the body is still found") + ; ---- Multipart.parse-bytes (RFC 7578 bodies that are not text) ---- + (assert-equal test + 80 + (Array.length &mp-nul) + "the binary fixture is 80 bytes long") + (assert-equal test + 69 + (String.length &(String.from-bytes &mp-nul)) + "read as a String the binary fixture ends 11 bytes early, at its NUL") + (assert-equal test + 0 + (Array.length &(mp-parts &(String.from-bytes &mp-nul) "B")) + "Multipart.parse decodes the truncated fixture to no parts at all") + (assert-equal test + 1 + (Array.length &(bp-parts &mp-nul "B")) + "Multipart.parse-bytes decodes the same fixture to one part") + (assert-equal test + &[88b 88b 0b 89b] + &(bp-body &(bp-parts &mp-nul "B") 0) + "the part body keeps its NUL and the bytes behind it") + (assert-equal test + "a.bin" + &(bp-fname &(bp-parts &mp-nul "B") 0) + "the filename is still read off the part headers") + (assert-equal test + "XX" + (FormPart.body + &(BinaryPart.to-form-part (Array.unsafe-nth &(bp-parts &mp-nul "B") 0))) + "converting a binary part to a FormPart cuts the body at its NUL") + (assert-equal test + &[88b 88b 0b] + &(bp-body &(bp-parts &(nul-part "XX" "") "b") 0) + "a NUL immediately before the closing delimiter is kept") + (assert-equal test + &[0b 89b] + &(bp-body &(bp-parts &(nul-part "" "Y") "b") 0) + "a NUL at the very start of a part body is kept") + (assert-equal test + 1 + (Array.length &(bp-parts &(nul-part "foo--bbar" "z") "b")) + "a --boundary without a leading CRLF inside a body is not a delimiter") + (assert-equal test + 11 + (Array.length &(bp-body &(bp-parts &(nul-part "foo--bbar" "z") "b") 0)) + "and the body carrying it survives whole") + (assert-equal test + &[88b 0b 89b] + &(bp-body + &(bp-parts + &(Array.concat &[(String.to-bytes "preamble\r\n") (nul-part "X" "Y")]) + "b") + 0) + "an opening delimiter after a preamble is found") + (assert-equal test + 0 + (Array.length + &(bp-parts + &(Array.concat + &[(String.to-bytes + "--b\r\nContent-Disposition: form-data; name=\"f\"\r\n\r\nXX") + [0b] + (String.to-bytes "Y")]) + "b")) + "a body whose closing delimiter is missing yields no parts") + (assert-equal test + 0 + (Array.length + &(bp-body + &(bp-parts + &(String.to-bytes + "--b\r\nContent-Disposition: form-data; name=\"a\"\r\n\r\n\r\n--b--\r\n") + "b") + 0)) + "an empty part body parses to an empty byte array") + (assert-equal test + "" + &(bp-name + &(bp-parts &(String.to-bytes "--b\r\n\r\nbody\r\n--b--\r\n") "b") + 0) + "a part with no headers at all decodes to an empty name") + (assert-equal test + 2 + (Array.length &(bp-parts &mp-mixed "b")) + "a body mixing a text part and a binary one decodes to two parts") + (assert-equal test + "hello" + &(String.from-bytes &(bp-body &(bp-parts &mp-mixed "b") 0)) + "the text part of a mixed body is unaffected") + (assert-equal test + &[0b 1b 2b] + &(bp-body &(bp-parts &mp-mixed "b") 1) + "the binary part of a mixed body keeps its leading NUL") + (assert-equal test + "multipart: NUL byte in part headers" + &(bp-error &mp-nul-filename "b") + "a NUL inside a filename is an error, not a silently shortened name") + (assert-true test + (match (Multipart.parse-bytes &(String.to-bytes "no boundary here") "b") + (Result.Error _) true + (Result.Success _) false) + "a missing boundary delimiter is an error on the byte path too") + (assert-equal test + &(str &(mp-parts &mp-file "b")) + &(str &(bp-as-form-parts &(String.to-bytes &mp-file) "b")) + "a text file part decodes identically through parse and parse-bytes") + (assert-equal test + &(str &(mp-parts &mp-two "b")) + &(str &(bp-as-form-parts &(String.to-bytes &mp-two) "b")) + "a two-part text body decodes identically through both entry points") + ; ---- Request.multipart-data ---- (assert-equal test "world" From 7b249cdc1caf87c24bd7244a43fe83313eeb09df Mon Sep 17 00:00:00 2001 From: "carpentry-heartbeat[bot]" Date: Sat, 29 Aug 2026 08:18:07 +0200 Subject: [PATCH 2/2] Name the real fault when a part has no header block MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit parse-part fell back to treating the whole part as its header region when the part held no CRLF CRLF, and only then scanned that region for a NUL. A headerless binary part therefore failed as "NUL byte in part headers" — a cause it does not have — and since a part error aborts the scan, every part already decoded went with it. Measured before this commit: headerless TEXT part => OK n=1 [name= bodylen=0] headerless BINARY part => ERR NUL byte in part headers good part THEN headerless binary part => ERR NUL byte in part headers All three now report "part headers not terminated by CRLF CRLF". The blast radius is deliberately unchanged: a headerless part still fails the whole body. RFC 7578 requires every part to carry a Content-Disposition with a name, so a part with no header block can never be a form field, and master's answer — a nameless part whose body was silently dropped — is the "succeeded and carried nothing" shape this branch exists to remove. Skipping the part instead would discard bytes the caller sent, which is the same bug one case over. This does mean parse now diverges from master on that one malformed input class, where master returned Success with a nameless empty part. Request.multipart-data's docstring pointed binary uploads at Multipart.parse-bytes, which a Request holder cannot reach: Request.body is a String and Request.parse takes a &String, so the bytes are gone before a Request exists. It now says the bytes have to be kept before the request is parsed. --- docs/BinaryPart.html | 2 +- docs/Multipart.html | 13 +++++---- docs/Request.html | 8 +++-- http.carp | 69 +++++++++++++++++++++++++------------------- test/http.carp | 32 ++++++++++++++++---- 5 files changed, 80 insertions(+), 44 deletions(-) diff --git a/docs/BinaryPart.html b/docs/BinaryPart.html index dae903a..0c726f7 100644 --- a/docs/BinaryPart.html +++ b/docs/BinaryPart.html @@ -158,7 +158,7 @@

    is a single part decoded from a multipart/form-data body by Multipart.parse-bytes. It carries the same fields as a FormPart, but its body is an (Array Byte), so -it survives the NUL bytes every binary upload contains.

    +it survives the NUL bytes a binary upload carries.

    diff --git a/docs/Multipart.html b/docs/Multipart.html index e731eb2..a1dd459 100644 --- a/docs/Multipart.html +++ b/docs/Multipart.html @@ -187,8 +187,8 @@

    decodes a multipart/form-data body with the given boundary -into its FormParts. Fails when the opening boundary delimiter -is absent.

    +into its FormParts. Fails on the same bodies +parse-bytes does.

    A Carp String ends at its first NUL byte, so a binary body — any PNG, PDF or zip upload — is cut short before parsing even begins, and usually decodes to no parts at all. Reach for parse-bytes whenever the body might @@ -215,9 +215,12 @@

    decodes a multipart/form-data body with the given boundary into its BinaryParts, reading the body as bytes so that binary uploads survive intact.

    -

    Fails when the opening boundary delimiter is absent, and when a part’s header -region holds a NUL byte: header field lines are US-ASCII, and reading one as a -String would cut a filename short without saying so.

    +

    Fails when the opening boundary delimiter is absent, when a part’s headers are +not terminated by a CRLF CRLF — RFC 7578 requires every part to carry a +Content-Disposition, so a part with no header block at all is malformed — and +when a part’s header region holds a NUL byte: header field lines are US-ASCII, +and reading one as a String would cut a filename short without saying so.

    +

    Any of these fails the whole body, not just the offending part.

    diff --git a/docs/Request.html b/docs/Request.html index 0e25cf1..e5a777b 100644 --- a/docs/Request.html +++ b/docs/Request.html @@ -447,9 +447,11 @@

    the boundary from the Content-Type header. Returns (Result (Array FormPart) String); fails when the request is not multipart/form-data or carries no boundary.

    -

    A Request body is a String and therefore stops at its first NUL byte, so an -upload that is not text has to be decoded with -Multipart.parse-bytes instead.

    +

    A Request body is a String, and a Carp String ends at its first NUL byte, +so a binary upload is already cut short before parse is handed the +request text, and cannot be recovered from a Request afterwards. Decoding one +means keeping the raw request bytes yourself and calling +Multipart.parse-bytes on the body.

    diff --git a/http.carp b/http.carp index 71db98a..5e4d19f 100644 --- a/http.carp +++ b/http.carp @@ -1788,7 +1788,7 @@ field `name`, an optional `filename` (present for file uploads), an optional (doc BinaryPart "is a single part decoded from a `multipart/form-data` body by [`Multipart.parse-bytes`](Multipart.html#parse-bytes). It carries the same fields as a [`FormPart`](FormPart.html), but its `body` is an `(Array Byte)`, so -it survives the NUL bytes every binary upload contains.") +it survives the NUL bytes a binary upload carries.") (deftype BinaryPart [name String filename (Maybe String) @@ -1886,35 +1886,42 @@ for you. ; parses one part (`headers CRLF CRLF body`) into a BinaryPart. (defn parse-part [raw] (let [blank (String.to-bytes "\r\n\r\n") - sep (index-from raw &blank &(skip-table &blank) 0) - head (if (< sep 0) @raw (Array.prefix raw sep)) - body (if (< sep 0) (the (Array Byte) []) (Array.suffix raw (+ sep 4)))] - (if (Array.any? &(fn [b] (= 0b @b)) &head) - (Result.Error @"multipart: NUL byte in part headers") - (let-do [lines (String.split-by &(String.from-bytes &head) &[\newline]) - nm @"" - fname (the (Maybe String) (Maybe.Nothing)) - ctype (the (Maybe String) (Maybe.Nothing))] - (for [i 0 (Array.length &lines)] - (let-do [line &(String.trim (Array.unsafe-nth &lines i))] - (match (header-value line "content-disposition") - (Maybe.Just cd) - (let-do [ps (MediaType.parse-params &cd)] - (set! nm (Map.get-with-default &ps "name" "")) - (set! fname (Map.get-maybe &ps "filename"))) - (Maybe.Nothing) ()) - (match (header-value line "content-type") - (Maybe.Just ct) (set! ctype (Maybe.Just ct)) - (Maybe.Nothing) ()))) - (Result.Success (BinaryPart.init nm fname ctype body)))))) + sep (index-from raw &blank &(skip-table &blank) 0)] + (if (< sep 0) + (Result.Error @"multipart: part headers not terminated by CRLF CRLF") + (let [head (Array.prefix raw sep) + body (Array.suffix raw (+ sep 4))] + (if (Array.any? &(fn [b] (= 0b @b)) &head) + (Result.Error @"multipart: NUL byte in part headers") + (let-do [lines (String.split-by &(String.from-bytes &head) + &[\newline]) + nm @"" + fname (the (Maybe String) (Maybe.Nothing)) + ctype (the (Maybe String) (Maybe.Nothing))] + (for [i 0 (Array.length &lines)] + (let-do [line &(String.trim (Array.unsafe-nth &lines i))] + (match (header-value line "content-disposition") + (Maybe.Just cd) + (let-do [ps (MediaType.parse-params &cd)] + (set! nm (Map.get-with-default &ps "name" "")) + (set! fname (Map.get-maybe &ps "filename"))) + (Maybe.Nothing) ()) + (match (header-value line "content-type") + (Maybe.Just ct) (set! ctype (Maybe.Just ct)) + (Maybe.Nothing) ()))) + (Result.Success (BinaryPart.init nm fname ctype body)))))))) (doc parse-bytes "decodes a `multipart/form-data` `body` with the given `boundary` into its [`BinaryPart`](BinaryPart.html)s, reading the body as bytes so that binary uploads survive intact. -Fails when the opening boundary delimiter is absent, and when a part’s header -region holds a NUL byte: header field lines are US-ASCII, and reading one as a -`String` would cut a `filename` short without saying so.") +Fails when the opening boundary delimiter is absent, when a part’s headers are +not terminated by a CRLF CRLF — RFC 7578 requires every part to carry a +`Content-Disposition`, so a part with no header block at all is malformed — and +when a part’s header region holds a NUL byte: header field lines are US-ASCII, +and reading one as a `String` would cut a `filename` short without saying so. + +Any of these fails the whole body, not just the offending part.") (defn parse-bytes [body boundary] (let [dash (String.to-bytes &(String.append "--" boundary)) crlf-dash (String.to-bytes &(String.append "\r\n--" boundary)) @@ -1958,8 +1965,8 @@ region holds a NUL byte: header field lines are US-ASCII, and reading one as a (Maybe.Nothing) (Result.Success parts)))))) (doc parse "decodes a `multipart/form-data` `body` with the given `boundary` -into its [`FormPart`](FormPart.html)s. Fails when the opening boundary delimiter -is absent. +into its [`FormPart`](FormPart.html)s. Fails on the same bodies +[`parse-bytes`](#parse-bytes) does. A Carp `String` ends at its first NUL byte, so a binary `body` — any PNG, PDF or zip upload — is cut short before parsing even begins, and usually decodes to no @@ -2594,9 +2601,11 @@ the boundary from the `Content-Type` header. Returns `(Result (Array FormPart) String)`; fails when the request is not `multipart/form-data` or carries no boundary. -A `Request` body is a `String` and therefore stops at its first NUL byte, so an -upload that is not text has to be decoded with -[`Multipart.parse-bytes`](Multipart.html#parse-bytes) instead.") +A `Request` body is a `String`, and a Carp `String` ends at its first NUL byte, +so a binary upload is already cut short before [`parse`](#parse) is handed the +request text, and cannot be recovered from a `Request` afterwards. Decoding one +means keeping the raw request bytes yourself and calling +[`Multipart.parse-bytes`](Multipart.html#parse-bytes) on the body.") (defn multipart-data [r] (match (Request.header r "Content-Type") (Maybe.Nothing) (Result.Error @"multipart: no Content-Type header") diff --git a/test/http.carp b/test/http.carp index 359188c..de6c319 100644 --- a/test/http.carp +++ b/test/http.carp @@ -112,6 +112,14 @@ [0b] (String.to-bytes ".jpg\"\r\n\r\ndata\r\n--b--\r\n")])) +(def headerless-binary + (Array.concat + &[(String.to-bytes "--b\r\n\r\nbo") [0b] (String.to-bytes "dy\r\n--b--\r\n")])) + +; a well-formed first part, ready to be followed by another part’s delimiter +(def mp-first + @"--b\r\nContent-Disposition: form-data; name=\"a\"\r\n\r\nhello\r\n") + ; ---- content negotiation (Accept) test helpers ---- (defn pick [accept offers] (match (Accept.negotiate accept offers) @@ -1396,11 +1404,25 @@ 0)) "an empty part body parses to an empty byte array") (assert-equal test - "" - &(bp-name - &(bp-parts &(String.to-bytes "--b\r\n\r\nbody\r\n--b--\r\n") "b") - 0) - "a part with no headers at all decodes to an empty name") + "multipart: part headers not terminated by CRLF CRLF" + &(bp-error &(String.to-bytes "--b\r\n\r\nbody\r\n--b--\r\n") "b") + "a text part with no headers at all is rejected") + (assert-equal test + "multipart: part headers not terminated by CRLF CRLF" + &(bp-error &headerless-binary "b") + "a binary part with no headers is not diagnosed as a NUL in its headers") + (assert-equal test + "multipart: part headers not terminated by CRLF CRLF" + &(bp-error + &(Array.concat &[(String.to-bytes &mp-first) @&headerless-binary]) + "b") + "a headerless part fails the whole body, discarding the parts before it") + (assert-equal test + "multipart: part headers not terminated by CRLF CRLF" + &(match (Multipart.parse "--b\r\n\r\nbody\r\n--b--\r\n" "b") + (Result.Error e) e + (Result.Success _) @"") + "the String entry point rejects a headerless part too") (assert-equal test 2 (Array.length &(bp-parts &mp-mixed "b"))