diff --git a/README.md b/README.md index 8c01d02..1b71e1f 100644 --- a/README.md +++ b/README.md @@ -243,6 +243,7 @@ Status.not-found ; => 404 | `Credentials` | one authentication scheme with its token68 or auth-params | | `Multipart` | `multipart/form-data` body decoder | | `FormPart` | a single decoded multipart part (name, filename, content-type, body) | +| `BinaryPart` | a decoded multipart part whose body is bytes, for binary uploads | | `CacheControl` | `Cache-Control` directive parser (RFC 7234 §5.2) | | `TransferEncoding` | Chunked transfer-encoding decoder | | `HttpDate` | HTTP-date parser and formatter (RFC 9110 §5.6.7) | diff --git a/docs/Accept.html b/docs/Accept.html index fe74c8b..1d24976 100644 --- a/docs/Accept.html +++ b/docs/Accept.html @@ -62,6 +62,11 @@ FormPart +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/AcceptEncoding.html b/docs/AcceptEncoding.html index fa71380..967f418 100644 --- a/docs/AcceptEncoding.html +++ b/docs/AcceptEncoding.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/AcceptLanguage.html b/docs/AcceptLanguage.html index 34ed2c4..7c4c298 100644 --- a/docs/AcceptLanguage.html +++ b/docs/AcceptLanguage.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/Auth.html b/docs/Auth.html index f9b4c8f..c21e890 100644 --- a/docs/Auth.html +++ b/docs/Auth.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/BinaryPart.html b/docs/BinaryPart.html new file mode 100644 index 0000000..0c726f7 --- /dev/null +++ b/docs/BinaryPart.html @@ -0,0 +1,609 @@ + + + + + + + + + +
    + +
    +

    + BinaryPart +

    +
    +

    is a single part decoded from a multipart/form-data body by +Multipart.parse-bytes. It carries the same +fields as a FormPart, but its body is an (Array Byte), so +it survives the NUL bytes a binary upload carries.

    + +
    +
    + +

    + body +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a)] (Ref (Array Byte) a)) +

    + + + +

    +

    gets the body property of a BinaryPart.

    + +

    +
    +
    + +

    + content-type +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a)] (Ref (Maybe String) a)) +

    + + + +

    +

    gets the content-type property of a BinaryPart.

    + +

    +
    +
    + +

    + copy +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a)] BinaryPart) +

    + + + +

    +

    copies a BinaryPart.

    + +

    +
    +
    + +

    + delete +

    +     +
    + instantiate +
    +

    + (Fn [BinaryPart] ()) +

    + + + +

    +

    deletes a BinaryPart. Should usually not be called manually.

    + +

    +
    +
    + +

    + filename +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a)] (Ref (Maybe String) a)) +

    + + + +

    +

    gets the filename property of a BinaryPart.

    + +

    +
    +
    + +

    + init +

    +     +
    + instantiate +
    +

    + (Fn [String, (Maybe String), (Maybe String), (Array Byte)] BinaryPart) +

    + + + +

    +

    creates a BinaryPart.

    + +

    +
    +
    + +

    + name +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a)] (Ref String a)) +

    + + + +

    +

    gets the name property of a BinaryPart.

    + +

    +
    +
    + +

    + prn +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a)] String) +

    + + + +

    +

    converts a BinaryPart to a string.

    + +

    +
    +
    + +

    + set-body +

    +     +
    + instantiate +
    +

    + (Fn [BinaryPart, (Array Byte)] BinaryPart) +

    + + + +

    +

    sets the body property of a BinaryPart.

    + +

    +
    +
    + +

    + set-body! +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a), (Array Byte)] ()) +

    + + + +

    +

    sets the body property of a BinaryPart in place.

    + +

    +
    +
    + +

    + set-content-type +

    +     +
    + instantiate +
    +

    + (Fn [BinaryPart, (Maybe String)] BinaryPart) +

    + + + +

    +

    sets the content-type property of a BinaryPart.

    + +

    +
    +
    + +

    + set-content-type! +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a), (Maybe String)] ()) +

    + + + +

    +

    sets the content-type property of a BinaryPart in place.

    + +

    +
    +
    + +

    + set-filename +

    +     +
    + instantiate +
    +

    + (Fn [BinaryPart, (Maybe String)] BinaryPart) +

    + + + +

    +

    sets the filename property of a BinaryPart.

    + +

    +
    +
    + +

    + set-filename! +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a), (Maybe String)] ()) +

    + + + +

    +

    sets the filename property of a BinaryPart in place.

    + +

    +
    +
    + +

    + set-name +

    +     +
    + instantiate +
    +

    + (Fn [BinaryPart, String] BinaryPart) +

    + + + +

    +

    sets the name property of a BinaryPart.

    + +

    +
    +
    + +

    + set-name! +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a), String] ()) +

    + + + +

    +

    sets the name property of a BinaryPart in place.

    + +

    +
    +
    + +

    + str +

    +     +
    + instantiate +
    +

    + (Fn [(Ref BinaryPart a)] String) +

    + + + +

    +

    converts a BinaryPart to a string.

    + +

    +
    +
    + +

    + to-form-part +

    +     +
    + defn +
    +

    + (Fn [(Ref BinaryPart a)] FormPart) +

    + 
    +                        (to-form-part p)
+
    +

    +

    converts a BinaryPart into a FormPart. +The body becomes a String, and is therefore cut at its first NUL byte; only +use this on parts you know are text.

    + +

    +
    +
    + +

    + update-body +

    +     +
    + instantiate +
    +

    + (Fn [BinaryPart, (Ref (Fn [(Array Byte)] (Array Byte) a) b)] BinaryPart) +

    + + + +

    +

    updates the body property of a BinaryPart using a function f.

    + +

    +
    +
    + +

    + update-content-type +

    +     +
    + instantiate +
    +

    + (Fn [BinaryPart, (Ref (Fn [(Maybe String)] (Maybe String) a) b)] BinaryPart) +

    + + + +

    +

    updates the content-type property of a BinaryPart using a function f.

    + +

    +
    +
    + +

    + update-filename +

    +     +
    + instantiate +
    +

    + (Fn [BinaryPart, (Ref (Fn [(Maybe String)] (Maybe String) a) b)] BinaryPart) +

    + + + +

    +

    updates the filename property of a BinaryPart using a function f.

    + +

    +
    +
    + +

    + update-name +

    +     +
    + instantiate +
    +

    + (Fn [BinaryPart, (Ref (Fn [String] String a) b)] BinaryPart) +

    + + + +

    +

    updates the name property of a BinaryPart using a function f.

    + +

    +
    +
    +
    + + diff --git a/docs/ByteRange.html b/docs/ByteRange.html index d71f1eb..3758a6f 100644 --- a/docs/ByteRange.html +++ b/docs/ByteRange.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/ByteRangeSpec.html b/docs/ByteRangeSpec.html index 874fe9f..f8b590a 100644 --- a/docs/ByteRangeSpec.html +++ b/docs/ByteRangeSpec.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/CacheControl.html b/docs/CacheControl.html index b81b346..3c135ad 100644 --- a/docs/CacheControl.html +++ b/docs/CacheControl.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/ContentRange.html b/docs/ContentRange.html index b211c24..4f4f0ed 100644 --- a/docs/ContentRange.html +++ b/docs/ContentRange.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/Cookie.html b/docs/Cookie.html index f565e85..30a0fd9 100644 --- a/docs/Cookie.html +++ b/docs/Cookie.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/Credentials.html b/docs/Credentials.html index 9ab9fea..699d678 100644 --- a/docs/Credentials.html +++ b/docs/Credentials.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/ETag.html b/docs/ETag.html index 852a36f..360f780 100644 --- a/docs/ETag.html +++ b/docs/ETag.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/ETagList.html b/docs/ETagList.html index e59c3fe..63bbd87 100644 --- a/docs/ETagList.html +++ b/docs/ETagList.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/Form.html b/docs/Form.html index 88fa728..b93cd71 100644 --- a/docs/Form.html +++ b/docs/Form.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/FormPart.html b/docs/FormPart.html index d369065..2d9b080 100644 --- a/docs/FormPart.html +++ b/docs/FormPart.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/HttpDate.html b/docs/HttpDate.html index 17fe73a..751e5bd 100644 --- a/docs/HttpDate.html +++ b/docs/HttpDate.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/MediaRange.html b/docs/MediaRange.html index 5df64e5..af9e33d 100644 --- a/docs/MediaRange.html +++ b/docs/MediaRange.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/MediaType.html b/docs/MediaType.html index 67f94a1..ab41821 100644 --- a/docs/MediaType.html +++ b/docs/MediaType.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/Multipart.html b/docs/Multipart.html index 481e8e0..a1dd459 100644 --- a/docs/Multipart.html +++ b/docs/Multipart.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding @@ -151,14 +156,18 @@

    decodes multipart/form-data bodies (RFC 7578) into their -FormParts. The boundary is a parameter of the Content-Type -header; Request.multipart-data reads it off the -request for you.

    +parts. The boundary is a parameter of the Content-Type header; +Request.multipart-data reads it off the request +for you.

    (match (Request.multipart-data &req)
   (Result.Success parts)
     (println* (FormPart.name (Array.unsafe-nth &parts 0)))
   (Result.Error e) (IO.errorln &e))
    +

    parse yields FormParts, whose bodies are +Strings and so stop at their first NUL byte. Uploads that are not text need +parse-bytes, which yields +BinaryParts instead.

    @@ -178,7 +187,40 @@

    decodes a multipart/form-data body with the given boundary -into its FormParts. Fails when the opening boundary delimiter is absent.

    +into its FormParts. Fails on the same bodies +parse-bytes does.

    +

    A Carp String ends at its first NUL byte, so a binary body — any PNG, PDF or +zip upload — is cut short before parsing even begins, and usually decodes to no +parts at all. Reach for parse-bytes whenever the body might +not be text.

    + +

    +

    +
    + +

    + parse-bytes +

    +     +
    + defn +
    +

    + (Fn [(Ref (Array Byte) a), (Ref String b)] (Result (Array BinaryPart) String)) +

    + 
    +                        (parse-bytes body boundary)
+
    +

    +

    decodes a multipart/form-data body with the given +boundary into its BinaryParts, reading the body as bytes +so that binary uploads survive intact.

    +

    Fails when the opening boundary delimiter is absent, when a part’s headers are +not terminated by a CRLF CRLF — RFC 7578 requires every part to carry a +Content-Disposition, so a part with no header block at all is malformed — and +when a part’s header region holds a NUL byte: header field lines are US-ASCII, +and reading one as a String would cut a filename short without saying so.

    +

    Any of these fails the whole body, not just the offending part.

    diff --git a/docs/Precondition.html b/docs/Precondition.html index ccfe454..6af5232 100644 --- a/docs/Precondition.html +++ b/docs/Precondition.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/Request.html b/docs/Request.html index cd4dc8b..e5a777b 100644 --- a/docs/Request.html +++ b/docs/Request.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding @@ -442,6 +447,11 @@

    the boundary from the Content-Type header. Returns (Result (Array FormPart) String); fails when the request is not multipart/form-data or carries no boundary.

    +

    A Request body is a String, and a Carp String ends at its first NUL byte, +so a binary upload is already cut short before parse is handed the +request text, and cannot be recovered from a Request afterwards. Decoding one +means keeping the raw request bytes yourself and calling +Multipart.parse-bytes on the body.

    diff --git a/docs/Response.html b/docs/Response.html index 6b94478..fe0b682 100644 --- a/docs/Response.html +++ b/docs/Response.html @@ -62,6 +62,11 @@ FormPart

    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/SameSite.html b/docs/SameSite.html index b63e1d1..e409aa6 100644 --- a/docs/SameSite.html +++ b/docs/SameSite.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/Status.html b/docs/Status.html index c47cca6..2f3b16f 100644 --- a/docs/Status.html +++ b/docs/Status.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/TransferEncoding.html b/docs/TransferEncoding.html index 6dfc20a..0c74055 100644 --- a/docs/TransferEncoding.html +++ b/docs/TransferEncoding.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/Weighted.html b/docs/Weighted.html index f45845d..bff199b 100644 --- a/docs/Weighted.html +++ b/docs/Weighted.html @@ -62,6 +62,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/http_index.html b/docs/http_index.html index 76b81c0..1c22232 100644 --- a/docs/http_index.html +++ b/docs/http_index.html @@ -58,6 +58,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/docs/index.html b/docs/index.html index 76b81c0..1c22232 100644 --- a/docs/index.html +++ b/docs/index.html @@ -58,6 +58,11 @@ FormPart
    • +
  • + + BinaryPart + +
  • TransferEncoding diff --git a/gendocs.carp b/gendocs.carp index 9950803..e9c90ad 100644 --- a/gendocs.carp +++ b/gendocs.carp @@ -50,6 +50,7 @@ Status.not-found ; => 404 Form Multipart FormPart + BinaryPart TransferEncoding MediaType Accept diff --git a/http.carp b/http.carp index 495a035..5e4d19f 100644 --- a/http.carp +++ b/http.carp @@ -1785,29 +1785,54 @@ field `name`, an optional `filename` (present for file uploads), an optional content-type (Maybe String) body String]) +(doc BinaryPart "is a single part decoded from a `multipart/form-data` body by +[`Multipart.parse-bytes`](Multipart.html#parse-bytes). It carries the same +fields as a [`FormPart`](FormPart.html), but its `body` is an `(Array Byte)`, so +it survives the NUL bytes a binary upload carries.") +(deftype BinaryPart + [name String + filename (Maybe String) + content-type (Maybe String) + body (Array Byte)]) + +(defmodule BinaryPart + (doc to-form-part "converts a `BinaryPart` into a [`FormPart`](FormPart.html). +The body becomes a `String`, and is therefore cut at its first NUL byte; only +use this on parts you know are text.") + (defn to-form-part [p] + (FormPart.init @(name p) + @(filename p) + @(content-type p) + (String.from-bytes (body p))))) + (doc Multipart "decodes `multipart/form-data` bodies (RFC 7578) into their -[`FormPart`](FormPart.html)s. The boundary is a parameter of the `Content-Type` -header; [`Request.multipart-data`](Request.html#multipart-data) reads it off the -request for you. +parts. The boundary is a parameter of the `Content-Type` header; +[`Request.multipart-data`](Request.html#multipart-data) reads it off the request +for you. ``` (match (Request.multipart-data &req) (Result.Success parts) (println* (FormPart.name (Array.unsafe-nth &parts 0))) (Result.Error e) (IO.errorln &e)) -```") +``` + +[`parse`](#parse) yields [`FormPart`](FormPart.html)s, whose bodies are +`String`s and so stop at their first NUL byte. Uploads that are not text need +[`parse-bytes`](#parse-bytes), which yields +[`BinaryPart`](BinaryPart.html)s instead.") (defmodule Multipart (hidden at?) (private at?) - ; does `s`, whose byte length is `slen`, hold `sub` at byte offset `pos`? - (defn at? [s slen pos sub] - (let [subl (String.length sub)] - (if (or (< pos 0) (> (+ pos subl) slen)) + ; do the bytes `sub` occur in `bs` at byte offset `pos`? + (defn at? [bs pos sub] + (let [subl (Array.length sub)] + (if (or (< pos 0) (> (+ pos subl) (Array.length bs))) false (let-do [ok true j 0] (while (and ok (< j subl)) - (if (= (String.char-at s (+ pos j)) (String.char-at sub j)) + (if (= (Array.unsafe-nth bs (+ pos j)) (Array.unsafe-nth sub j)) (set! j (Int.inc j)) (set! ok false))) ok)))) @@ -1815,26 +1840,29 @@ request for you. (hidden skip-table) (private skip-table) ; Boyer-Moore-Horspool bad-character shifts for `needle`, indexed by byte. - (defn skip-table [needle nlen] - (let-do [t (Array.replicate 256 &nlen) + (defn skip-table [needle] + (let-do [nlen (Array.length needle) + t (Array.replicate 256 &nlen) last (Int.dec nlen)] (for [i 0 last] - (Array.aset! &t (Char.to-int (String.char-at needle i)) (- last i))) + (Array.aset! &t (Byte.to-int @(Array.unsafe-nth needle i)) (- last i))) t)) (hidden index-from) (private index-from) - ; first byte offset >= `from` where `needle` occurs in `s`, or -1. `skip` is + ; first byte offset >= `from` where `needle` occurs in `bs`, or -1. `skip` is ; the needle’s `skip-table`. - (defn index-from [s slen needle nlen skip from] + (defn index-from [bs needle skip from] (let-do [res -1 + len (Array.length bs) + nlen (Array.length needle) i from last (Int.dec nlen)] - (while (and (< res 0) (<= (+ i nlen) slen)) - (let [c (String.char-at s (+ i last))] - (if (and (= c (String.char-at needle last)) (at? s slen i needle)) + (while (and (< res 0) (<= (+ i nlen) len)) + (let [c @(Array.unsafe-nth bs (+ i last))] + (if (and (= c @(Array.unsafe-nth needle last)) (at? bs i needle)) (set! res i) - (set! i (+ i @(Array.unsafe-nth skip (Char.to-int c))))))) + (set! i (+ i @(Array.unsafe-nth skip (Byte.to-int c))))))) res)) (hidden header-value) @@ -1855,73 +1883,101 @@ request for you. (hidden parse-part) (private parse-part) - ; parses one part (`headers CRLF CRLF body`) into a FormPart. + ; parses one part (`headers CRLF CRLF body`) into a BinaryPart. (defn parse-part [raw] - (let [sep (String.index-of-string raw "\r\n\r\n") - head (if (< sep 0) @raw (String.byte-slice raw 0 sep)) - body (if (< sep 0) - @"" - (String.byte-slice raw (+ sep 4) (String.length raw))) - lines (String.split-by &head &[\newline])] - (let-do [nm @"" - fname (the (Maybe String) (Maybe.Nothing)) - ctype (the (Maybe String) (Maybe.Nothing))] - (for [i 0 (Array.length &lines)] - (let-do [line &(String.trim (Array.unsafe-nth &lines i))] - (match (header-value line "content-disposition") - (Maybe.Just cd) - (let-do [ps (MediaType.parse-params &cd)] - (set! nm (Map.get-with-default &ps "name" "")) - (set! fname (Map.get-maybe &ps "filename"))) - (Maybe.Nothing) ()) - (match (header-value line "content-type") - (Maybe.Just ct) (set! ctype (Maybe.Just ct)) - (Maybe.Nothing) ()))) - (FormPart.init nm fname ctype body)))) - - (doc parse "decodes a `multipart/form-data` `body` with the given `boundary` -into its `FormPart`s. Fails when the opening boundary delimiter is absent.") - (defn parse [body boundary] - (let [dash (String.append "--" boundary) - crlf-dash (String.append "\r\n" &dash) - cdlen (String.length &crlf-dash) - dashlen (String.length &dash) - len (String.length body) - skip (Multipart.skip-table &crlf-dash cdlen) + (let [blank (String.to-bytes "\r\n\r\n") + sep (index-from raw &blank &(skip-table &blank) 0)] + (if (< sep 0) + (Result.Error @"multipart: part headers not terminated by CRLF CRLF") + (let [head (Array.prefix raw sep) + body (Array.suffix raw (+ sep 4))] + (if (Array.any? &(fn [b] (= 0b @b)) &head) + (Result.Error @"multipart: NUL byte in part headers") + (let-do [lines (String.split-by &(String.from-bytes &head) + &[\newline]) + nm @"" + fname (the (Maybe String) (Maybe.Nothing)) + ctype (the (Maybe String) (Maybe.Nothing))] + (for [i 0 (Array.length &lines)] + (let-do [line &(String.trim (Array.unsafe-nth &lines i))] + (match (header-value line "content-disposition") + (Maybe.Just cd) + (let-do [ps (MediaType.parse-params &cd)] + (set! nm (Map.get-with-default &ps "name" "")) + (set! fname (Map.get-maybe &ps "filename"))) + (Maybe.Nothing) ()) + (match (header-value line "content-type") + (Maybe.Just ct) (set! ctype (Maybe.Just ct)) + (Maybe.Nothing) ()))) + (Result.Success (BinaryPart.init nm fname ctype body)))))))) + + (doc parse-bytes "decodes a `multipart/form-data` `body` with the given +`boundary` into its [`BinaryPart`](BinaryPart.html)s, reading the body as bytes +so that binary uploads survive intact. + +Fails when the opening boundary delimiter is absent, when a part’s headers are +not terminated by a CRLF CRLF — RFC 7578 requires every part to carry a +`Content-Disposition`, so a part with no header block at all is malformed — and +when a part’s header region holds a NUL byte: header field lines are US-ASCII, +and reading one as a `String` would cut a `filename` short without saying so. + +Any of these fails the whole body, not just the offending part.") + (defn parse-bytes [body boundary] + (let [dash (String.to-bytes &(String.append "--" boundary)) + crlf-dash (String.to-bytes &(String.append "\r\n--" boundary)) + crlf (String.to-bytes "\r\n") + dashes (String.to-bytes "--") + cdlen (Array.length &crlf-dash) + dashlen (Array.length &dash) + len (Array.length body) + skip (Multipart.skip-table &crlf-dash) ; the opening delimiter is `--boundary`, either at the very start or ; after a preamble that ends in CRLF; every later delimiter is ; `CRLF--boundary`, so searching for that avoids matching a bare ; `--boundary` that happens to occur inside a part body. - open (if (Multipart.at? body len 0 &dash) + open (if (Multipart.at? body 0 &dash) dashlen - (let [p (Multipart.index-from body len &crlf-dash cdlen &skip 0)] + (let [p (Multipart.index-from body &crlf-dash &skip 0)] (if (< p 0) -1 (+ p cdlen))))] (if (< open 0) (Result.Error @"multipart: boundary delimiter not found") - (let-do [parts (the (Array FormPart) []) + (let-do [parts (the (Array BinaryPart) []) + err (the (Maybe String) (Maybe.Nothing)) pos open done false] (while (not done) (cond - (Multipart.at? body len pos "--") (set! done true) + (Multipart.at? body pos &dashes) (set! done true) (>= pos len) (set! done true) (do - (when (Multipart.at? body len pos "\r\n") (set! pos (+ pos 2))) - (let [next (Multipart.index-from body - len - &crlf-dash - cdlen - &skip - pos)] + (when (Multipart.at? body pos &crlf) (set! pos (+ pos 2))) + (let [next (Multipart.index-from body &crlf-dash &skip pos)] (if (< next 0) (set! done true) (do - (Array.push-back! &parts - (Multipart.parse-part &(String.byte-slice body - pos - next))) + (match (Multipart.parse-part &(Array.slice body pos next)) + (Result.Success p) (Array.push-back! &parts p) + (Result.Error e) + (do (set! err (Maybe.Just e)) (set! done true))) (set! pos (+ next cdlen)))))))) - (Result.Success parts)))))) + (match err + (Maybe.Just e) (Result.Error e) + (Maybe.Nothing) (Result.Success parts)))))) + + (doc parse "decodes a `multipart/form-data` `body` with the given `boundary` +into its [`FormPart`](FormPart.html)s. Fails on the same bodies +[`parse-bytes`](#parse-bytes) does. + +A Carp `String` ends at its first NUL byte, so a binary `body` — any PNG, PDF or +zip upload — is cut short before parsing even begins, and usually decodes to no +parts at all. Reach for [`parse-bytes`](#parse-bytes) whenever the body might +not be text.") + (defn parse [body boundary] + (match (Multipart.parse-bytes &(String.to-bytes body) boundary) + (Result.Error e) (Result.Error e) + (Result.Success ps) + (Result.Success + (Array.copy-map &(fn [p] (BinaryPart.to-form-part p)) &ps))))) (doc ByteRangeSpec "is one `range-spec` of a `Range` header in the `bytes` unit (RFC 9110 §14.1.1): `FirstLast` for `0-499`, `From` for `500-` — everything from @@ -2543,7 +2599,13 @@ Returns `(Result (Map String String) String)`.") (doc multipart-data "parses the request body as `multipart/form-data`, taking the boundary from the `Content-Type` header. Returns `(Result (Array FormPart) String)`; fails when the request is not -`multipart/form-data` or carries no boundary.") +`multipart/form-data` or carries no boundary. + +A `Request` body is a `String`, and a Carp `String` ends at its first NUL byte, +so a binary upload is already cut short before [`parse`](#parse) is handed the +request text, and cannot be recovered from a `Request` afterwards. Decoding one +means keeping the raw request bytes yourself and calling +[`Multipart.parse-bytes`](Multipart.html#parse-bytes) on the body.") (defn multipart-data [r] (match (Request.header r "Content-Type") (Maybe.Nothing) (Result.Error @"multipart: no Content-Type header") diff --git a/test/http.carp b/test/http.carp index e5a5085..de6c319 100644 --- a/test/http.carp +++ b/test/http.carp @@ -65,6 +65,61 @@ (def mp-two @"--b\r\nContent-Disposition: form-data; name=\"a\"\r\n\r\n1\r\n--b\r\nContent-Disposition: form-data; name=\"c\"\r\n\r\n2\r\n--b--\r\n") +(defn bp-parts [body boundary] + (match (Multipart.parse-bytes body boundary) + (Result.Success ps) ps + (Result.Error _) [])) +(defn bp-name [ps i] @(BinaryPart.name (Array.unsafe-nth ps i))) +(defn bp-body [ps i] @(BinaryPart.body (Array.unsafe-nth ps i))) +(defn bp-fname [ps i] + (Maybe.from @(BinaryPart.filename (Array.unsafe-nth ps i)) @"")) +(defn bp-error [body boundary] + (match (Multipart.parse-bytes body boundary) + (Result.Error e) e + (Result.Success _) @"")) +(defn bp-as-form-parts [body boundary] + (Array.copy-map &(fn [p] (BinaryPart.to-form-part p)) + &(bp-parts body boundary))) + +; a one-part body whose payload is the bytes of `pre`, a NUL, then `post` +(defn nul-part [pre post] + (Array.concat + &[(String.to-bytes + "--b\r\nContent-Disposition: form-data; name=\"f\"; filename=\"up.bin\"\r\n\r\n") + (String.to-bytes pre) + [0b] + (String.to-bytes post) + (String.to-bytes "\r\n--b--\r\n")])) + +(def mp-nul + (Array.concat + &[(String.to-bytes + "--B\r\nContent-Disposition: form-data; name=\"f\"; filename=\"a.bin\"\r\n\r\nXX") + [0b] + (String.to-bytes "Y\r\n--B--\r\n")])) + +(def mp-mixed + (Array.concat + &[(String.to-bytes + "--b\r\nContent-Disposition: form-data; name=\"a\"\r\n\r\nhello\r\n--b\r\nContent-Disposition: form-data; name=\"f\"; filename=\"x.bin\"\r\n\r\n") + [0b 1b 2b] + (String.to-bytes "\r\n--b--\r\n")])) + +(def mp-nul-filename + (Array.concat + &[(String.to-bytes + "--b\r\nContent-Disposition: form-data; name=\"f\"; filename=\"evil.php") + [0b] + (String.to-bytes ".jpg\"\r\n\r\ndata\r\n--b--\r\n")])) + +(def headerless-binary + (Array.concat + &[(String.to-bytes "--b\r\n\r\nbo") [0b] (String.to-bytes "dy\r\n--b--\r\n")])) + +; a well-formed first part, ready to be followed by another part’s delimiter +(def mp-first + @"--b\r\nContent-Disposition: form-data; name=\"a\"\r\n\r\nhello\r\n") + ; ---- content negotiation (Accept) test helpers ---- (defn pick [accept offers] (match (Accept.negotiate accept offers) @@ -1273,6 +1328,131 @@ 0) "a delimiter that ends at the end of the body is still found") + ; ---- Multipart.parse-bytes (RFC 7578 bodies that are not text) ---- + (assert-equal test + 80 + (Array.length &mp-nul) + "the binary fixture is 80 bytes long") + (assert-equal test + 69 + (String.length &(String.from-bytes &mp-nul)) + "read as a String the binary fixture ends 11 bytes early, at its NUL") + (assert-equal test + 0 + (Array.length &(mp-parts &(String.from-bytes &mp-nul) "B")) + "Multipart.parse decodes the truncated fixture to no parts at all") + (assert-equal test + 1 + (Array.length &(bp-parts &mp-nul "B")) + "Multipart.parse-bytes decodes the same fixture to one part") + (assert-equal test + &[88b 88b 0b 89b] + &(bp-body &(bp-parts &mp-nul "B") 0) + "the part body keeps its NUL and the bytes behind it") + (assert-equal test + "a.bin" + &(bp-fname &(bp-parts &mp-nul "B") 0) + "the filename is still read off the part headers") + (assert-equal test + "XX" + (FormPart.body + &(BinaryPart.to-form-part (Array.unsafe-nth &(bp-parts &mp-nul "B") 0))) + "converting a binary part to a FormPart cuts the body at its NUL") + (assert-equal test + &[88b 88b 0b] + &(bp-body &(bp-parts &(nul-part "XX" "") "b") 0) + "a NUL immediately before the closing delimiter is kept") + (assert-equal test + &[0b 89b] + &(bp-body &(bp-parts &(nul-part "" "Y") "b") 0) + "a NUL at the very start of a part body is kept") + (assert-equal test + 1 + (Array.length &(bp-parts &(nul-part "foo--bbar" "z") "b")) + "a --boundary without a leading CRLF inside a body is not a delimiter") + (assert-equal test + 11 + (Array.length &(bp-body &(bp-parts &(nul-part "foo--bbar" "z") "b") 0)) + "and the body carrying it survives whole") + (assert-equal test + &[88b 0b 89b] + &(bp-body + &(bp-parts + &(Array.concat &[(String.to-bytes "preamble\r\n") (nul-part "X" "Y")]) + "b") + 0) + "an opening delimiter after a preamble is found") + (assert-equal test + 0 + (Array.length + &(bp-parts + &(Array.concat + &[(String.to-bytes + "--b\r\nContent-Disposition: form-data; name=\"f\"\r\n\r\nXX") + [0b] + (String.to-bytes "Y")]) + "b")) + "a body whose closing delimiter is missing yields no parts") + (assert-equal test + 0 + (Array.length + &(bp-body + &(bp-parts + &(String.to-bytes + "--b\r\nContent-Disposition: form-data; name=\"a\"\r\n\r\n\r\n--b--\r\n") + "b") + 0)) + "an empty part body parses to an empty byte array") + (assert-equal test + "multipart: part headers not terminated by CRLF CRLF" + &(bp-error &(String.to-bytes "--b\r\n\r\nbody\r\n--b--\r\n") "b") + "a text part with no headers at all is rejected") + (assert-equal test + "multipart: part headers not terminated by CRLF CRLF" + &(bp-error &headerless-binary "b") + "a binary part with no headers is not diagnosed as a NUL in its headers") + (assert-equal test + "multipart: part headers not terminated by CRLF CRLF" + &(bp-error + &(Array.concat &[(String.to-bytes &mp-first) @&headerless-binary]) + "b") + "a headerless part fails the whole body, discarding the parts before it") + (assert-equal test + "multipart: part headers not terminated by CRLF CRLF" + &(match (Multipart.parse "--b\r\n\r\nbody\r\n--b--\r\n" "b") + (Result.Error e) e + (Result.Success _) @"") + "the String entry point rejects a headerless part too") + (assert-equal test + 2 + (Array.length &(bp-parts &mp-mixed "b")) + "a body mixing a text part and a binary one decodes to two parts") + (assert-equal test + "hello" + &(String.from-bytes &(bp-body &(bp-parts &mp-mixed "b") 0)) + "the text part of a mixed body is unaffected") + (assert-equal test + &[0b 1b 2b] + &(bp-body &(bp-parts &mp-mixed "b") 1) + "the binary part of a mixed body keeps its leading NUL") + (assert-equal test + "multipart: NUL byte in part headers" + &(bp-error &mp-nul-filename "b") + "a NUL inside a filename is an error, not a silently shortened name") + (assert-true test + (match (Multipart.parse-bytes &(String.to-bytes "no boundary here") "b") + (Result.Error _) true + (Result.Success _) false) + "a missing boundary delimiter is an error on the byte path too") + (assert-equal test + &(str &(mp-parts &mp-file "b")) + &(str &(bp-as-form-parts &(String.to-bytes &mp-file) "b")) + "a text file part decodes identically through parse and parse-bytes") + (assert-equal test + &(str &(mp-parts &mp-two "b")) + &(str &(bp-as-form-parts &(String.to-bytes &mp-two) "b")) + "a two-part text body decodes identically through both entry points") + ; ---- Request.multipart-data ---- (assert-equal test "world"