Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions docs/TransferEncoding.html
Original file line number Diff line number Diff line change
Expand Up @@ -216,12 +216,14 @@ <h3 id="dechunk">
(dechunk body)
</pre>
<p class="doc">
<p>decodes a chunked transfer-encoded <code>body</code> per RFC 7230 §4.1.</p>
<p>decodes a chunked transfer-encoded <code>body</code> per RFC 9112 §7.1.</p>
<p>Each chunk is a hexadecimal size line — any chunk extension after a <code>;</code> is
ignored — followed by that many bytes of data and a CRLF, repeating until a
zero-size chunk. Optional trailer headers after the final chunk are ignored.
Returns the decoded body, or an <code>(Error String)</code> on malformed framing such as a
bad hex size or a truncated chunk.</p>
bad hex size or a truncated chunk. The size line is read strictly: it must begin
with a hexadecimal digit, and nothing but a chunk extension may follow the
size.</p>

</p>
</div>
Expand Down
38 changes: 28 additions & 10 deletions http.carp
Original file line number Diff line number Diff line change
Expand Up @@ -661,19 +661,42 @@ Use `Request.chunked?`/`Response.chunked?` to detect a chunked body, then
(set! acc (+ (* acc 16) d)))))
(if ok (Maybe.Just acc) (Maybe.Nothing))))

(private bws?)
(hidden bws?)
(defn bws? [c] (or (= c \space) (= c \tab)))

; chunk-size = 1*HEXDIG, then chunk-ext = *( BWS ";" ... ) per RFC 9112 §7.1
(private parse-size-line)
(hidden parse-size-line)
(defn parse-size-line [line]
(let-do [len (String.length line)
digits 0
i 0]
(while (and (< digits len)
(>= (hex-nibble (String.char-at line digits)) 0))
(set! digits (Int.inc digits)))
(set! i digits)
(while (and (< i len) (bws? (String.char-at line i))) (set! i (Int.inc i)))
(if (and (> digits 0)
(or (= digits len) (and (< i len) (= \; (String.char-at line i)))))
(parse-hex &(String.byte-slice line 0 digits))
(Maybe.Nothing))))

(private crlf-at?)
(hidden crlf-at?)
(defn crlf-at? [s i]
(and (= (String.char-at s i) \return)
(= (String.char-at s (Int.inc i)) \newline)))

(doc dechunk "decodes a chunked transfer-encoded `body` per RFC 7230 §4.1.
(doc dechunk "decodes a chunked transfer-encoded `body` per RFC 9112 §7.1.

Each chunk is a hexadecimal size line — any chunk extension after a `;` is
ignored — followed by that many bytes of data and a CRLF, repeating until a
zero-size chunk. Optional trailer headers after the final chunk are ignored.
Returns the decoded body, or an `(Error String)` on malformed framing such as a
bad hex size or a truncated chunk.")
bad hex size or a truncated chunk. The size line is read strictly: it must begin
with a hexadecimal digit, and nothing but a chunk extension may follow the
size.")
(defn dechunk [body]
(let-do [len (String.length body)
pos 0
Expand All @@ -684,18 +707,13 @@ bad hex size or a truncated chunk.")
(let [eol (String.find-crlf body pos len)]
(if (= eol -1)
(set! err @"malformed chunked body: unterminated chunk size line")
(let [size-line &(String.byte-slice body pos eol)
semi (String.index-of size-line \;)
hex &(String.trim
&(if (= semi -1)
@size-line
(String.byte-slice size-line 0 semi)))]
(match (parse-hex hex)
(let [size-line &(String.byte-slice body pos eol)]
(match (parse-size-line size-line)
(Maybe.Nothing)
(set! err
(fmt
"malformed chunked body: invalid chunk size '%s'"
hex))
size-line))
(Maybe.Just size)
(if (= size 0)
(set! terminated true)
Expand Down
34 changes: 34 additions & 0 deletions test/http.carp
Original file line number Diff line number Diff line change
Expand Up @@ -2195,6 +2195,40 @@
(dechunk-err? "4\r\nWiki\r\n5\r\npedia\r\n")
"dechunk errors on a missing terminating zero-size chunk")

(assert-true test
(dechunk-err? " 5\r\nhello\r\n0\r\n\r\n")
"dechunk rejects a chunk size line with leading whitespace")

(assert-true test
(dechunk-err? "\t5\r\nhello\r\n0\r\n\r\n")
"dechunk rejects a chunk size line starting with a tab")

(assert-true test
(dechunk-err? "5 \r\nhello\r\n0\r\n\r\n")
"dechunk rejects a chunk size with trailing whitespace and no extension")

(assert-true test
(dechunk-err? "5\r\r\nhello\r\n0\r\n\r\n")
"dechunk rejects a stray CR between the chunk size and the CRLF")

(assert-true test
(dechunk-err? " 0 \r\n\r\n")
"dechunk rejects a whitespace-padded terminating chunk")

(assert-true test
(dechunk-err? ";name=value\r\nhello\r\n0\r\n\r\n")
"dechunk rejects a chunk extension with no size before it")

(assert-equal test
&(dechunk-ok "5 ;name=value\r\nhello\r\n0\r\n\r\n")
"hello"
"dechunk allows the bad whitespace RFC 9112 §7.1.1 permits before an extension")

(assert-equal test
&(dechunk-ok "1C\r\nabcdefghijklmnopqrstuvwxyz01\r\n0\r\n\r\n")
"abcdefghijklmnopqrstuvwxyz01"
"dechunk decodes an uppercase hex chunk size")

(assert-equal test
&(match (Response.parse
"HTTP/1.1 200 OK\r\nTransfer-Encoding: chunked\r\n\r\n4\r\nWiki\r\n5\r\npedia\r\n0\r\n\r\n")
Expand Down