From 8aa5eabcd9db553f0592702beb4157698f443fe5 Mon Sep 17 00:00:00 2001 From: dickhardt Date: Wed, 12 Aug 2026 14:42:52 +0100 Subject: [PATCH 1/3] httpsig: vendor an RFC 8941 parser and make it the only one Signature-Input is a Dictionary of Inner Lists with parameters, the hardest shape RFC 8941 defines. It was hand-parsed here with regexes and split(), as were Signature, Signature-Key, Signature-Error and the Accept-Signature family -- and the same grammar is hand-parsed again in two other places across the AAuth family. Hand-rolled 8941 fails on the same three things every time: quoting, escaping and byte sequences. Vendor structured-headers v1.0.1 (MIT, itself zero-dependency) into src/vendor/structured-headers/ rather than depending on it. Zero dependencies is a deliberate security property of this package -- the same reason its JWT verification was written by reading jose rather than importing it. The copy is byte-identical to upstream src/ once this repository's Prettier config and .js specifiers are applied, so it stays diffable; the directory README records the version, the licence, and the command that checks it. Replace every hand-rolled parser and generator in utils/signature.ts with the vendored one, and export the parser and serializer from the package so consumers stop writing their own. Two behaviour fixes fall out: - @signature-params is now re-serialized from the parsed Inner List instead of rebuilt from extracted parts. The old reconstruction quoted every non-numeric parameter, so a signer that sent a Token-valued parameter (;alg=hmac-sha256) had it turned into a String in the signature base and failed to verify. - A covered component carrying parameters (;req, ;bs, ;sf, ;key) is now refused explicitly rather than silently signed over as a bare header name. Tightenings, all fail-closed: covered component identifiers must be Strings; `created` must be an Integer; Signature members must be Byte Sequences; Signature-Key parameters must be Strings or Tokens. No version bump -- this ships alongside a coordinated AAuth wave. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01FmiCqDjRUSx6zb1N4gZPXE --- httpsig/README.md | 67 +++ httpsig/src/fetch.ts | 12 +- httpsig/src/index.ts | 40 ++ httpsig/src/structured-fields.ts | 90 ++++ httpsig/src/types.ts | 15 + httpsig/src/utils/signature.ts | 419 ++++++++++------- httpsig/src/vendor/structured-headers/LICENSE | 21 + .../src/vendor/structured-headers/README.md | 85 ++++ .../src/vendor/structured-headers/index.ts | 10 + .../src/vendor/structured-headers/parser.ts | 434 ++++++++++++++++++ .../vendor/structured-headers/serializer.ts | 142 ++++++ .../src/vendor/structured-headers/token.ts | 22 + .../src/vendor/structured-headers/types.ts | 51 ++ httpsig/src/vendor/structured-headers/util.ts | 30 ++ httpsig/src/verify.ts | 25 +- httpsig/tests/test-structured-fields.ts | 397 ++++++++++++++++ 16 files changed, 1684 insertions(+), 176 deletions(-) create mode 100644 httpsig/src/structured-fields.ts create mode 100644 httpsig/src/vendor/structured-headers/LICENSE create mode 100644 httpsig/src/vendor/structured-headers/README.md create mode 100644 httpsig/src/vendor/structured-headers/index.ts create mode 100644 httpsig/src/vendor/structured-headers/parser.ts create mode 100644 httpsig/src/vendor/structured-headers/serializer.ts create mode 100644 httpsig/src/vendor/structured-headers/token.ts create mode 100644 httpsig/src/vendor/structured-headers/types.ts create mode 100644 httpsig/src/vendor/structured-headers/util.ts create mode 100644 httpsig/tests/test-structured-fields.ts diff --git a/httpsig/README.md b/httpsig/README.md index 7df2b87..ace0f3e 100644 --- a/httpsig/README.md +++ b/httpsig/README.md @@ -721,6 +721,73 @@ We support the two most widely recommended algorithms from the [IANA HTTP Messag - Widely supported - Perfect interoperability +## Structured Fields (RFC 8941) + +Every header this package reads or writes is an RFC 8941 Structured Field, and +the parser and serializer that handle them are exported. Use them for +neighbouring fields rather than writing another parser — `AAuth-Requirement` is +a Dictionary, `AAuth-Capabilities` a List of Tokens, and hand-rolled 8941 fails +on the same three things every time: quoting, escaping, and byte sequences. + +```ts +import { + parseDictionary, + serializeDictionary, + Token, + bareItemToString, +} from '@hellocoop/httpsig' + +// A `;` inside a quoted string does not end the parameter list. +const dict = parseDictionary( + 'requirement=interaction; url="https://resource.example/i?a=1;b=2"; code="A1B2-C3D4"', +) + +const [value, params] = dict.get('requirement') as Item +value instanceof Token // true — `interaction` is a Token, not a String +params.get('url') // 'https://resource.example/i?a=1;b=2' +params.get('code') // 'A1B2-C3D4' + +serializeDictionary( + new Map([ + [ + 'requirement', + [new Token('auth-token'), new Map([['resource-token', jwt]])], + ], + ]), +) +``` + +**Exported** + +| | | +| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------- | +| Parsing | `parseDictionary`, `parseList`, `parseItem`, `ParseError` | +| Serializing | `serializeDictionary`, `serializeList`, `serializeItem`, `serializeInnerList`, `serializeBareItem`, `serializeParameters`, `SerializeError` | +| Values | `Token`, `ByteSequence` | +| Guards | `isInnerList`, `isByteSequence`, `isValidTokenStr`, `isValidKeyStr` | +| Helper | `bareItemToString` — reads a String or a Token, refuses anything else | +| Types | `Dictionary`, `List`, `Item`, `InnerList`, `Parameters`, `BareItem` | + +Shapes: + +``` +Dictionary Map +List (Item | InnerList)[] +Item [BareItem, Parameters] +InnerList [Item[], Parameters] +Parameters Map +BareItem number | string | Token | ByteSequence | boolean +``` + +A `Token` is a bare word (`hwk`, `Ed25519`); a `string` is a quoted sf-string. +The distinction is load-bearing — `@signature-params` is covered by the +signature, so a parameter that arrives as a Token must go back out as a Token. + +The implementation is vendored from +[`structured-headers`](https://github.com/evert/structured-header) v1.0.1 (MIT) +rather than taken as a dependency, because this package has zero runtime +dependencies by design. See `src/vendor/structured-headers/README.md`. + ## Security Considerations ### Timestamp Validation diff --git a/httpsig/src/fetch.ts b/httpsig/src/fetch.ts index 2b64517..48fd1e9 100644 --- a/httpsig/src/fetch.ts +++ b/httpsig/src/fetch.ts @@ -19,6 +19,7 @@ import { import { generateSignatureBase, generateSignatureInputHeader, + generateSignatureParams, generateSignatureKeyHeader, generateSignatureHeader, generateContentDigest, @@ -266,10 +267,13 @@ export async function fetch( ) headers.set('signature-input', signatureInputHeader) - // Add signature params to component values - const componentList = components.map((c) => `"${c}"`).join(' ') - const signatureParams = `(${componentList});created=${created}` - componentValues.set('@signature-params', signatureParams) + // Add signature params to component values. Serialized from the same + // structure the Signature-Input header was built from, so the header and + // the signature base cannot disagree. + componentValues.set( + '@signature-params', + generateSignatureParams(components, created), + ) components.push('@signature-params') // Generate signature base diff --git a/httpsig/src/index.ts b/httpsig/src/index.ts index ae96f40..395f922 100644 --- a/httpsig/src/index.ts +++ b/httpsig/src/index.ts @@ -24,6 +24,46 @@ export { parseAcceptSignatureAlg, } from './utils/signature.js' +/** + * RFC 8941 Structured Field Values. + * + * Exported so that consumers parsing neighbouring structured fields -- + * AAuth-Requirement is a Dictionary, AAuth-Capabilities a List of Tokens -- + * use the implementation this package already carries instead of hand-rolling + * one. Hand-rolled 8941 fails on quoting, escaping and byte sequences every + * time; a `;` inside a quoted `url` is enough to break a naive parameter + * split. + */ +export { + parseDictionary, + parseList, + parseItem, + serializeDictionary, + serializeList, + serializeItem, + serializeInnerList, + serializeBareItem, + serializeParameters, + bareItemToString, + isInnerList, + isByteSequence, + isValidTokenStr, + isValidKeyStr, + Token, + ByteSequence, + ParseError, + SerializeError, +} from './structured-fields.js' + +export type { + Dictionary, + List, + Item, + InnerList, + Parameters, + BareItem, +} from './structured-fields.js' + export { generateKeyPair, determineAlgorithm, diff --git a/httpsig/src/structured-fields.ts b/httpsig/src/structured-fields.ts new file mode 100644 index 0000000..778d79b --- /dev/null +++ b/httpsig/src/structured-fields.ts @@ -0,0 +1,90 @@ +/** + * RFC 8941 Structured Field Values for HTTP + * + * This is the package's single structured-field implementation. Every header + * it reads or writes -- Signature, Signature-Input, Signature-Key, + * Signature-Error, Accept-Signature, Accept-Signature-Scheme, + * Accept-Signature-Alg -- goes through it, and it is exported so that + * consumers parsing neighbouring fields (AAuth-Requirement, a Dictionary; + * AAuth-Capabilities, a List of Tokens) do not write a fourth one. + * + * The implementation is vendored, not depended on: see + * `src/vendor/structured-headers/README.md` for what was taken, from which + * version, and why. This module is the seam -- nothing outside it should + * import from `vendor/` directly. + * + * Shapes, briefly: + * + * Dictionary Map + * List (Item | InnerList)[] + * Item [BareItem, Parameters] + * InnerList [Item[], Parameters] + * Parameters Map + * BareItem number | string | Token | ByteSequence | boolean + * + * A `Token` is a bare word (`hwk`, `Ed25519`); a `string` is a quoted + * sf-string. They are distinct types, and the distinction is load-bearing: + * `requirement=interaction` and `requirement="interaction"` are different + * values, and a parser that erases the difference cannot round-trip. + */ + +export { + parseDictionary, + parseList, + parseItem, + ParseError, +} from './vendor/structured-headers/parser.js' + +export { + serializeDictionary, + serializeList, + serializeItem, + serializeInnerList, + serializeBareItem, + serializeParameters, + SerializeError, +} from './vendor/structured-headers/serializer.js' + +export { Token } from './vendor/structured-headers/token.js' +export { ByteSequence } from './vendor/structured-headers/types.js' +export { + isInnerList, + isByteSequence, + isValidTokenStr, + isValidKeyStr, +} from './vendor/structured-headers/util.js' + +export type { + Dictionary, + List, + Item, + InnerList, + Parameters, + BareItem, +} from './vendor/structured-headers/types.js' + +import { BareItem } from './vendor/structured-headers/types.js' +import { Token } from './vendor/structured-headers/token.js' + +/** + * Read a Bare Item that is expected to carry text. + * + * Returns the value of a String or a Token, and throws for anything else. The + * Token case is deliberate leniency: several fields in this family are + * specified to carry Strings, but a sender that omits the quotes produces a + * Token that means the same thing to a human and parses unambiguously. Values + * that cannot be text -- Integers, Decimals, Booleans, Byte Sequences -- are + * rejected rather than stringified, so a type confusion surfaces as an error + * instead of a plausible-looking string. + */ +export function bareItemToString(value: BareItem): string { + if (typeof value === 'string') { + return value + } + if (value instanceof Token) { + return value.toString() + } + throw new TypeError( + `Expected a Structured Field String or Token, got ${typeof value}`, + ) +} diff --git a/httpsig/src/types.ts b/httpsig/src/types.ts index a65c975..443a1de 100644 --- a/httpsig/src/types.ts +++ b/httpsig/src/types.ts @@ -2,6 +2,8 @@ * Type definitions for @hellocoop/httpsig */ +import type { InnerList } from './structured-fields.js' + /** * Valid derived components from RFC 9421 Section 2.2 */ @@ -176,12 +178,25 @@ export interface VerificationResult { } export interface ParsedSignatureInput { + /** The Dictionary key this signature is under. */ label: string + /** The covered component identifiers, unquoted. */ components: string[] + /** + * The Inner List parameters, as Structured Field values: `created` is an + * Integer, a quoted `keyid` is a string, a bare one is a `Token`. + */ params: { created: number [key: string]: any } + /** + * The parsed Inner List exactly as the signer sent it. Serializing this is + * how `@signature-params` is reproduced for the signature base -- it is + * covered by the signature, so every parameter must survive the round + * trip, including ones this implementation does not act on. + */ + signatureParams: InnerList } export interface ParsedSignatureKey { diff --git a/httpsig/src/utils/signature.ts b/httpsig/src/utils/signature.ts index 4e30442..181c22d 100644 --- a/httpsig/src/utils/signature.ts +++ b/httpsig/src/utils/signature.ts @@ -2,7 +2,7 @@ * HTTP Message Signature generation and parsing utilities */ -import { base64Encode, sha256 } from './base64.js' +import { base64Encode, base64Decode, sha256 } from './base64.js' import { ParsedSignatureInput, ParsedSignatureKey, @@ -10,8 +10,53 @@ import { SignatureError, SignatureErrorCode, AcceptSignatureParams, + HwkValue, } from '../types.js' import { invalidKey, unsupportedScheme } from '../errors.js' +import { + parseDictionary, + parseList, + serializeDictionary, + serializeList, + serializeInnerList, + isInnerList, + isByteSequence, + isValidTokenStr, + bareItemToString, + Token, + ByteSequence, +} from '../structured-fields.js' +import type { + Dictionary, + InnerList, + Item, + Parameters, +} from '../structured-fields.js' + +/** + * Build the Inner List that a Signature-Input dictionary member carries: the + * covered component identifiers, parameterized with `created`. + * + * Serialized, this is also the `@signature-params` component value. Both the + * header and the signature base are produced from this one structure so they + * cannot drift apart. + */ +function buildSignatureParams( + components: string[], + created: number, +): InnerList { + const items: Item[] = components.map((component) => [ + component, + new Map() as Parameters, + ]) + return [items, new Map([['created', created]]) as Parameters] +} + +/** Wrap a parse failure in a message naming the field that failed. */ +function parseFailure(field: string, error: unknown): Error { + const detail = error instanceof Error ? error.message : String(error) + return new Error(`Invalid ${field} format: ${detail}`) +} /** * Generate signature base string from components @@ -48,8 +93,20 @@ export function generateSignatureInputHeader( components: string[], created: number, ): string { - const componentList = components.map((c) => `"${c}"`).join(' ') - return `${label}=(${componentList});created=${created}` + return serializeDictionary( + new Map([[label, buildSignatureParams(components, created)]]), + ) +} + +/** + * Generate the `@signature-params` component value: the serialized Inner List + * that the Signature-Input dictionary member for `label` carries. + */ +export function generateSignatureParams( + components: string[], + created: number, +): string { + return serializeInnerList(buildSignatureParams(components, created)) } /** @@ -64,6 +121,14 @@ export function generateSignatureKeyHeader( signatureKey: SignatureKeyType, publicJwk?: JsonWebKey, ): string { + /** One-member Dictionary: label=;. */ + const oneMember = (scheme: string, params: [string, string][]): string => + serializeDictionary( + new Map([ + [label, [new Token(scheme), new Map(params) as Parameters]], + ]) as Dictionary, + ) + if (signatureKey.type === 'hwk') { if (!publicJwk) { throw new Error('Public JWK required for hwk signature key type') @@ -79,36 +144,34 @@ export function generateSignatureKeyHeader( // Build hwk parameters from JWK. kid is deliberately not emitted: the // key is inline, so an identifier selects nothing. - const params: string[] = [ - `alg="${publicJwk.alg}"`, - `kty="${publicJwk.kty}"`, + const params: [string, string][] = [ + ['alg', publicJwk.alg], + ['kty', publicJwk.kty as string], ] - if (publicJwk.crv) params.push(`crv="${publicJwk.crv}"`) - if (publicJwk.x) params.push(`x="${publicJwk.x}"`) - if (publicJwk.y) params.push(`y="${publicJwk.y}"`) - if (publicJwk.n) params.push(`n="${publicJwk.n}"`) - if (publicJwk.e) params.push(`e="${publicJwk.e}"`) + if (publicJwk.crv) params.push(['crv', publicJwk.crv]) + if (publicJwk.x) params.push(['x', publicJwk.x]) + if (publicJwk.y) params.push(['y', publicJwk.y]) + if (publicJwk.n) params.push(['n', publicJwk.n]) + if (publicJwk.e) params.push(['e', publicJwk.e]) - return `${label}=hwk;${params.join(';')}` + return oneMember('hwk', params) } if (signatureKey.type === 'jwt') { - return `${label}=jwt;jwt="${signatureKey.jwt}"` + return oneMember('jwt', [['jwt', signatureKey.jwt]]) } if (signatureKey.type === 'jkt_jwt') { - return `${label}=jkt-jwt;jwt="${signatureKey.jwt}"` + return oneMember('jkt-jwt', [['jwt', signatureKey.jwt]]) } if (signatureKey.type === 'jwks_uri') { - const params = [ - `id="${signatureKey.id}"`, - `dwk="${signatureKey.dwk}"`, - `kid="${signatureKey.kid}"`, - ] - - return `${label}=jwks_uri;${params.join(';')}` + return oneMember('jwks_uri', [ + ['id', signatureKey.id], + ['dwk', signatureKey.dwk], + ['kid', signatureKey.kid], + ]) } // Note: x509 scheme not yet implemented @@ -130,8 +193,17 @@ export function generateSignatureHeader( label: string, signature: Uint8Array, ): string { - const encoded = base64Encode(signature) - return `${label}=:${encoded}:` + return serializeDictionary( + new Map([ + [ + label, + [ + new ByteSequence(base64Encode(signature)), + new Map() as Parameters, + ], + ], + ]) as Dictionary, + ) } /** @@ -159,52 +231,70 @@ export async function generateContentDigest(body: BodyInit): Promise { } /** - * Parse Signature-Input header + * Parse Signature-Input header: a Dictionary of Inner Lists with parameters. + * + * The Inner List is kept alongside the extracted component names, because + * `@signature-params` is the serialization of that Inner List and is covered + * by the signature. Re-deriving it from the extracted parts would risk + * dropping a parameter the signer included. */ export function parseSignatureInput(header: string): ParsedSignatureInput[] { - const results: ParsedSignatureInput[] = [] + let dictionary: Dictionary + try { + dictionary = parseDictionary(header) + } catch (error) { + throw parseFailure('Signature-Input', error) + } - // Split by comma to handle multiple signatures - const parts = header.split(',').map((p) => p.trim()) + const results: ParsedSignatureInput[] = [] - for (const part of parts) { - // Format: label=(components);params - // Note: component list can be empty, so use * instead of + - const match = part.match(/^([^=]+)=\(([^)]*)\);(.+)$/) - if (!match) { - throw new Error(`Invalid Signature-Input format: ${part}`) + for (const [label, member] of dictionary) { + if (!isInnerList(member)) { + throw new Error( + `Invalid Signature-Input format: member "${label}" is not an Inner List of covered components`, + ) } - const label = match[1].trim() - const componentsStr = match[2] - const paramsStr = match[3] - - // Parse components - const components = componentsStr - .split(/\s+/) - .map((c) => c.replace(/"/g, '')) - .filter((c) => c) - - // Parse parameters - const params: any = {} - const paramPairs = paramsStr.split(';').map((p) => p.trim()) - - for (const pair of paramPairs) { - const [key, value] = pair.split('=').map((s) => s.trim()) - if (key === 'created') { - params.created = parseInt(value, 10) - } else { - params[key] = value + const [items, parameters] = member + + const components: string[] = [] + for (const [bareItem, itemParameters] of items) { + if (typeof bareItem !== 'string') { + throw new Error( + 'Invalid Signature-Input format: a covered component identifier must be a String', + ) + } + // Component parameters (;req, ;bs, ;sf, ;key, ;name) change what + // the component value is and how the signature base line is + // written. This implementation does not produce them, so rather + // than silently signing over a base that ignores them, refuse. + if (itemParameters.size > 0) { + throw new Error( + `Unsupported component parameters on "${bareItem}" in Signature-Input`, + ) } + components.push(bareItem) } - if (!params.created) { + const params: Record = {} + for (const [key, value] of parameters) { + params[key] = value + } + + // `created` is an Integer. A signer that omits it, or sends it as a + // String, leaves the signature unbounded in time. + if (typeof params.created !== 'number') { throw new Error( 'Signature-Input missing required parameter: created', ) } - results.push({ label, components, params }) + results.push({ + label, + components, + params: params as ParsedSignatureInput['params'], + signatureParams: member, + }) } return results @@ -223,49 +313,48 @@ export function parseSignatureInput(header: string): ParsedSignatureInput[] { * - The member key is the label */ export function parseSignatureKey(header: string): ParsedSignatureKey[] { - const trimmed = header.trim() - - // Check for multiple members (commas outside of quoted strings indicate multiple dictionary members) - // Simple check: if there's a comma not inside quotes, reject - let inQuote = false - for (let i = 0; i < trimmed.length; i++) { - if (trimmed[i] === '"' && (i === 0 || trimmed[i - 1] !== '\\')) { - inQuote = !inQuote - } else if (trimmed[i] === ',' && !inQuote) { - throw new Error( - 'Invalid Signature-Key: must have exactly one dictionary member', - ) - } - } + const malformed = new Error( + 'Invalid Signature-Key: must be RFC 8941 Dictionary with format label=scheme;params', + ) - // RFC 8941 Dictionary format: label=scheme;param1="value1";param2="value2" - // Match: label=token followed by optional parameters - // Note: [\w-]+ allows hyphens in labels and scheme names (e.g., sig-b26, jkt-jwt) - const match = trimmed.match(/^([\w-]+)=([\w-]+)(.*)$/) + let dictionary: Dictionary + try { + dictionary = parseDictionary(header) + } catch { + throw malformed + } - if (!match) { + // Exactly one member: the label the Signature and Signature-Input entries + // are keyed by. More than one leaves which key signed the request + // undetermined. + if (dictionary.size !== 1) { throw new Error( - 'Invalid Signature-Key: must be RFC 8941 Dictionary with format label=scheme;params', + 'Invalid Signature-Key: must have exactly one dictionary member', ) } - const label = match[1] - const scheme = match[2] - const paramsStr = match[3] + const [label, member] = [...dictionary][0] - // Parse parameters (semicolon-separated) - const params: any = {} - if (paramsStr) { - // Note: [\w-]+ allows hyphens in parameter names (e.g., jkt-jwt) - const paramMatches = paramsStr.matchAll( - /;([\w-]+)=(?:"([^"]*)"|(\w+))/g, - ) + // The member is an Item whose bare value is the scheme Token. An Inner + // List, a String, a number, or a bare member (`sig` alone, meaning ?1) is + // not a Signature-Key. + if (isInnerList(member) || !(member[0] instanceof Token)) { + throw malformed + } - for (const paramMatch of paramMatches) { - const key = paramMatch[1] - const value = - paramMatch[2] !== undefined ? paramMatch[2] : paramMatch[3] // quoted or unquoted value - params[key] = value + const scheme = member[0].toString() + + // Scheme parameters are Strings. A Token is accepted for the same value -- + // a sender that omits the quotes around, say, `kty=EC` is unambiguous -- + // but anything that is not text is a type confusion, not a value. + const params: Record = {} + for (const [key, value] of member[1]) { + try { + params[key] = bareItemToString(value) + } catch { + throw invalidKey( + `Signature-Key ${key} parameter must be a String or Token`, + ) } } @@ -291,7 +380,9 @@ export function parseSignatureKey(header: string): ParsedSignatureKey[] { ) } - return [{ label, type: 'hwk', value: params }] + // The hwk parameters are the JWK: alg and kty are checked present + // above, the rest are the key material for whichever kty it is. + return [{ label, type: 'hwk', value: params as unknown as HwkValue }] } if (scheme === 'jwt') { @@ -364,31 +455,37 @@ export function parseSignatureKey(header: string): ParsedSignatureKey[] { export function generateSignatureErrorHeader( signatureError: SignatureError, ): string { - const parts: string[] = [`error=${signatureError.error}`] + const dictionary: Dictionary = new Map([ + ['error', [new Token(signatureError.error), new Map()] as Item], + ]) if (signatureError.required_input) { - const inputList = signatureError.required_input - .map((c) => `"${c}"`) - .join(' ') - parts.push(`required_input=(${inputList})`) + dictionary.set('required_input', [ + signatureError.required_input.map((c) => [c, new Map()] as Item), + new Map(), + ]) } - return parts.join(', ') + return serializeDictionary(dictionary) } /** * Parse Signature-Error header (RFC 8941 Dictionary) */ export function parseSignatureError(header: string): SignatureError { - const trimmed = header.trim() + let dictionary: Dictionary + try { + dictionary = parseDictionary(header) + } catch (error) { + throw parseFailure('Signature-Error', error) + } - // Parse error token - const errorMatch = trimmed.match(/error=([\w]+)/) - if (!errorMatch) { + const errorMember = dictionary.get('error') + if (errorMember === undefined || isInnerList(errorMember)) { throw new Error('Invalid Signature-Error: missing error member') } - const error = errorMatch[1] as SignatureErrorCode + const error = bareItemToString(errorMember[0]) as SignatureErrorCode const validCodes: SignatureErrorCode[] = [ 'unsupported_algorithm', 'unsupported_scheme', @@ -409,12 +506,16 @@ export function parseSignatureError(header: string): SignatureError { const result: SignatureError = { error } // Parse required_input inner list - const inputMatch = trimmed.match(/required_input=\(([^)]*)\)/) - if (inputMatch) { - result.required_input = inputMatch[1] - .split(/\s+/) - .map((c) => c.replace(/"/g, '')) - .filter((c) => c) + const inputMember = dictionary.get('required_input') + if (inputMember !== undefined) { + if (!isInnerList(inputMember)) { + throw new Error( + 'Invalid Signature-Error: required_input must be an Inner List', + ) + } + result.required_input = inputMember[0].map(([component]) => + bareItemToString(component), + ) } return result @@ -425,23 +526,24 @@ export function parseSignatureError(header: string): SignatureError { */ function generateTokenList(values: string[]): string { for (const value of values) { - if (!/^[A-Za-z*][A-Za-z0-9!#$%&'*+\-.^_`|~:/]*$/.test(value)) { + if (!isValidTokenStr(value)) { throw new Error( `Value is not a valid Structured Field Token: ${value}`, ) } } - return values.join(', ') + return serializeList(values.map((value) => [new Token(value), new Map()])) } /** * Parse an RFC 8941 List of Tokens, ignoring entries that are not tokens. */ function parseTokenList(header: string): string[] { - return header - .split(',') - .map((v) => v.trim()) - .filter((v) => /^[A-Za-z*][A-Za-z0-9!#$%&'*+\-.^_`|~:/]*$/.test(v)) + return parseList(header) + .filter((member): member is Item => !isInnerList(member)) + .map(([bareItem]) => bareItem) + .filter((bareItem): bareItem is Token => bareItem instanceof Token) + .map((token) => token.toString()) } /** @@ -492,17 +594,21 @@ export function generateAcceptSignatureHeader( params: AcceptSignatureParams, ): string { const { label = 'sig', components, alg, tag } = params - const componentList = components.map((c) => `"${c}"`).join(' ') - let header = `${label}=(${componentList})` + const parameters: Parameters = new Map() if (alg) { - header += `;alg="${alg}"` + parameters.set('alg', alg) } if (tag) { - header += `;tag="${tag}"` + parameters.set('tag', tag) } - return header + const innerList: InnerList = [ + components.map((c) => [c, new Map()] as Item), + parameters, + ] + + return serializeDictionary(new Map([[label, innerList]])) } /** @@ -510,37 +616,37 @@ export function generateAcceptSignatureHeader( * Format: label=("comp1" "comp2")[;alg="algo"][;tag="tag"] */ export function parseAcceptSignature(header: string): AcceptSignatureParams { - const trimmed = header.trim() + const malformed = new Error('Invalid Accept-Signature format') - // Match: label=(components);params - const match = trimmed.match(/^([\w-]+)=\(([^)]*)\)(.*)$/) - if (!match) { - throw new Error('Invalid Accept-Signature format') + let dictionary: Dictionary + try { + dictionary = parseDictionary(header) + } catch { + throw malformed } - const label = match[1] - const componentsStr = match[2] - const paramsStr = match[3] + if (dictionary.size !== 1) { + throw malformed + } - const components = componentsStr - .split(/\s+/) - .map((c) => c.replace(/"/g, '')) - .filter((c) => c) + const [label, member] = [...dictionary][0] + if (!isInnerList(member)) { + throw malformed + } - const result: AcceptSignatureParams = { label, components } + const result: AcceptSignatureParams = { + label, + components: member[0].map(([component]) => bareItemToString(component)), + } - if (paramsStr) { - // Parse alg string parameter - const algMatch = paramsStr.match(/;alg="([^"]*)"/) - if (algMatch) { - result.alg = algMatch[1] - } + const alg = member[1].get('alg') + if (alg !== undefined) { + result.alg = bareItemToString(alg) + } - // Parse tag string parameter - const tagMatch = paramsStr.match(/;tag="([^"]*)"/) - if (tagMatch) { - result.tag = tagMatch[1] - } + const tag = member[1].get('tag') + if (tag !== undefined) { + result.tag = bareItemToString(tag) } return result @@ -550,25 +656,24 @@ export function parseAcceptSignature(header: string): AcceptSignatureParams { * Parse Signature header */ export function parseSignature(header: string): Map { - const results = new Map() - - // Split by comma for multiple signatures - const entries = header.split(/,(?=\s*\w+=)/) + let dictionary: Dictionary + try { + dictionary = parseDictionary(header) + } catch (error) { + throw parseFailure('Signature', error) + } - for (const entry of entries) { - const trimmed = entry.trim() + const results = new Map() + for (const [label, member] of dictionary) { // Format: label=:base64: - const match = trimmed.match(/^([^=]+)=:([^:]+):$/) - if (!match) { - throw new Error(`Invalid Signature format: ${trimmed}`) + if (isInnerList(member) || !isByteSequence(member[0])) { + throw new Error( + `Invalid Signature format: member "${label}" is not a Byte Sequence`, + ) } - const label = match[1].trim() - const base64 = match[2] - - const signature = Buffer.from(base64, 'base64') - results.set(label, new Uint8Array(signature)) + results.set(label, base64Decode(member[0].toBase64())) } return results diff --git a/httpsig/src/vendor/structured-headers/LICENSE b/httpsig/src/vendor/structured-headers/LICENSE new file mode 100644 index 0000000..064d69d --- /dev/null +++ b/httpsig/src/vendor/structured-headers/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2018-2023 Bad Gateway Inc. + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/httpsig/src/vendor/structured-headers/README.md b/httpsig/src/vendor/structured-headers/README.md new file mode 100644 index 0000000..a5ab59e --- /dev/null +++ b/httpsig/src/vendor/structured-headers/README.md @@ -0,0 +1,85 @@ +# Vendored: `structured-headers` + +An RFC 8941 (Structured Field Values for HTTP) parser and serializer. + +| | | +| ------------- | ------------------------------------------ | +| Upstream | https://github.com/evert/structured-header | +| Package | `structured-headers` | +| Version taken | **1.0.1** | +| Licence | MIT — see `LICENSE` in this directory | +| Copyright | 2018-2023 Bad Gateway Inc. (Evert Pot) | + +## What was taken + +The whole parser and serializer, from upstream `src/`: + +| This directory | Upstream | +| --------------- | ------------------- | +| `parser.ts` | `src/parser.ts` | +| `serializer.ts` | `src/serializer.ts` | +| `types.ts` | `src/types.ts` | +| `token.ts` | `src/token.ts` | +| `util.ts` | `src/util.ts` | +| `index.ts` | `src/index.ts` | + +Nothing else from the package was taken: no build config, no tests, no browser +bundle. + +## Why vendored rather than depended on + +`@hellocoop/httpsig` has zero runtime dependencies, deliberately. It verifies +HTTP message signatures, so every package in its dependency closure is a +package that can silently change how a signature is checked. The same reasoning +produced the hand-written JWT verification in `src/utils/` — written by reading +`jose` rather than importing it. + +A copy that is read, reviewed, and pinned is a different risk than a version +range resolved at install time. `structured-headers` is itself zero-dependency +and MIT, so copying it costs nothing in licence terms and removes the supply +chain entirely. + +## Why the whole grammar rather than a subset + +`Signature-Input` is a Dictionary of Inner Lists with parameters, which +exercises nearly every production RFC 8941 defines — Strings with escapes, +Tokens, Integers, Byte Sequences, and parameters on both the inner items and +the Inner List itself. `Signature` is a Dictionary of Byte Sequences. +`Signature-Key` is a Dictionary of Tokens with String parameters. A partial +copy covering "just the Dictionary bits" would have to grow into the rest +anyway, and that growth is how a divergent fourth implementation gets written. + +## What was changed + +Formatting only, to match this repository's Prettier configuration (4-space +indent, no semicolons, single quotes) and its ESM-style `.js` import +specifiers. No logic, no control flow, no error messages, and no exported names +were changed, so this directory stays diffable against upstream `src/` when +upstream fixes something. + +The copy is byte-identical to upstream once the repository's Prettier +configuration is applied and the `.js` specifiers are removed. That is +checkable: + +```sh +# from the repository root, with structured-headers@1.0.1 in node_modules +mkdir -p /tmp/sh && cp node_modules/structured-headers/src/*.ts /tmp/sh/ +npx prettier --tab-width 4 --no-semi --single-quote --trailing-comma all \ + --write '/tmp/sh/*.ts' + +for f in parser serializer types token util index; do + sed '1,5d' httpsig/src/vendor/structured-headers/$f.ts \ + | sed "s/\.js'/'/g" \ + | diff -u /tmp/sh/$f.ts - || echo "DIVERGED: $f" +done +``` + +(`sed '1,5d'` drops the vendoring header comment added to each file.) + +## Upgrading + +1. Read the upstream changelog between the pinned version and the new one. +2. Re-copy `src/`, reformat, restore the `.js` import specifiers. +3. Update the version in this README. +4. Run `npm test` in `httpsig/` — `tests/test-structured-fields.ts` covers the + shapes this package depends on, including the ones naive parsers get wrong. diff --git a/httpsig/src/vendor/structured-headers/index.ts b/httpsig/src/vendor/structured-headers/index.ts new file mode 100644 index 0000000..e3f0bfb --- /dev/null +++ b/httpsig/src/vendor/structured-headers/index.ts @@ -0,0 +1,10 @@ +/** + * Vendored from structured-headers v1.0.1 (MIT). See README.md in this + * directory. Upstream: src/index.ts + */ + +export * from './serializer.js' +export * from './parser.js' +export * from './types.js' +export * from './util.js' +export { Token } from './token.js' diff --git a/httpsig/src/vendor/structured-headers/parser.ts b/httpsig/src/vendor/structured-headers/parser.ts new file mode 100644 index 0000000..a0cbdf1 --- /dev/null +++ b/httpsig/src/vendor/structured-headers/parser.ts @@ -0,0 +1,434 @@ +/** + * Vendored from structured-headers v1.0.1 (MIT). See README.md in this + * directory. Upstream: src/parser.ts + */ + +import { + Dictionary, + List, + Item, + BareItem, + Parameters, + InnerList, + ByteSequence, +} from './types.js' + +import { Token } from './token.js' + +import { isAscii } from './util.js' + +export function parseDictionary(input: string): Dictionary { + const parser = new Parser(input) + return parser.parseDictionary() +} + +export function parseList(input: string): List { + const parser = new Parser(input) + return parser.parseList() +} + +export function parseItem(input: string): Item { + const parser = new Parser(input) + return parser.parseItem() +} + +export class ParseError extends Error { + constructor(position: number, message: string) { + super(`Parse error: ${message} at offset ${position}`) + } +} + +export default class Parser { + input: string + pos: number + + constructor(input: string) { + this.input = input + this.pos = 0 + } + + parseDictionary(): Dictionary { + this.skipWS() + const dictionary = new Map() + while (!this.eof()) { + const thisKey = this.parseKey() + let member + if (this.lookChar() === '=') { + this.pos++ + member = this.parseItemOrInnerList() + } else { + member = [true, this.parseParameters()] + } + dictionary.set(thisKey, member) + this.skipOWS() + if (this.eof()) { + return dictionary + } + this.expectChar(',') + this.pos++ + this.skipOWS() + if (this.eof()) { + throw new ParseError( + this.pos, + 'Dictionary contained a trailing comma', + ) + } + } + return dictionary + } + + parseList(): List { + this.skipWS() + const members: List = [] + while (!this.eof()) { + members.push(this.parseItemOrInnerList()) + this.skipOWS() + if (this.eof()) { + return members + } + this.expectChar(',') + this.pos++ + this.skipOWS() + if (this.eof()) { + throw new ParseError( + this.pos, + 'A list may not end with a trailing comma', + ) + } + } + + return members + } + + parseItem(standaloneItem: boolean = true): Item { + if (standaloneItem) this.skipWS() + + const result: Item = [this.parseBareItem(), this.parseParameters()] + + if (standaloneItem) this.checkTrail() + return result + } + + private parseItemOrInnerList(): Item | InnerList { + if (this.lookChar() === '(') { + return this.parseInnerList() + } else { + return this.parseItem(false) + } + } + + private parseInnerList(): InnerList { + this.expectChar('(') + this.pos++ + + const innerList: Item[] = [] + + while (!this.eof()) { + this.skipWS() + if (this.lookChar() === ')') { + this.pos++ + return [innerList, this.parseParameters()] + } + + innerList.push(this.parseItem(false)) + + const nextChar = this.lookChar() + if (nextChar !== ' ' && nextChar !== ')') { + throw new ParseError( + this.pos, + 'Expected a whitespace or ) after every item in an inner list', + ) + } + } + + throw new ParseError(this.pos, 'Could not find end of inner list') + } + + private parseBareItem(): BareItem { + const char = this.lookChar() + if (char === undefined) { + throw new ParseError(this.pos, 'Unexpected end of string') + } + if (char.match(/^[-0-9]/)) { + return this.parseIntegerOrDecimal() + } + if (char === '"') { + return this.parseString() + } + if (char.match(/^[A-Za-z*]/)) { + return this.parseToken() + } + if (char === ':') { + return this.parseByteSequence() + } + if (char === '?') { + return this.parseBoolean() + } + + throw new ParseError(this.pos, 'Unexpected input') + } + + private parseParameters(): Parameters { + const parameters = new Map() + while (!this.eof()) { + const char = this.lookChar() + if (char !== ';') { + break + } + this.pos++ + this.skipWS() + const key = this.parseKey() + let value: BareItem = true + if (this.lookChar() === '=') { + this.pos++ + value = this.parseBareItem() + } + parameters.set(key, value) + } + + return parameters + } + + private parseIntegerOrDecimal(): number { + let type: 'integer' | 'decimal' = 'integer' + let sign = 1 + let inputNumber = '' + if (this.lookChar() === '-') { + sign = -1 + this.pos++ + } + + // The spec wants this check but it's unreachable code. + //if (this.eof()) { + // throw new ParseError(this.pos, 'Empty integer'); + //} + + if (!isDigit(this.lookChar())) { + throw new ParseError(this.pos, 'Expected a digit (0-9)') + } + + while (!this.eof()) { + const char = this.getChar() + if (isDigit(char)) { + inputNumber += char + } else if (type === 'integer' && char === '.') { + if (inputNumber.length > 12) { + throw new ParseError( + this.pos, + 'Exceeded maximum decimal length', + ) + } + inputNumber += '.' + type = 'decimal' + } else { + // We need to 'prepend' the character, so it's just a rewind + this.pos-- + break + } + + if (type === 'integer' && inputNumber.length > 15) { + throw new ParseError( + this.pos, + 'Exceeded maximum integer length', + ) + } + if (type === 'decimal' && inputNumber.length > 16) { + throw new ParseError( + this.pos, + 'Exceeded maximum decimal length', + ) + } + } + + if (type === 'integer') { + return parseInt(inputNumber, 10) * sign + } else { + if (inputNumber.endsWith('.')) { + throw new ParseError(this.pos, 'Decimal cannot end on a period') + } + if (inputNumber.split('.')[1].length > 3) { + throw new ParseError( + this.pos, + 'Number of digits after the decimal point cannot exceed 3', + ) + } + return parseFloat(inputNumber) * sign + } + } + + private parseString(): string { + let outputString = '' + this.expectChar('"') + this.pos++ + + while (!this.eof()) { + const char = this.getChar() + if (char === '\\') { + if (this.eof()) { + throw new ParseError(this.pos, 'Unexpected end of input') + } + const nextChar = this.getChar() + if (nextChar !== '\\' && nextChar !== '"') { + throw new ParseError( + this.pos, + 'A backslash must be followed by another backslash or double quote', + ) + } + outputString += nextChar + } else if (char === '"') { + return outputString + } else if (!isAscii(char)) { + throw new ParseError( + this.pos, + 'Strings must be in the ASCII range', + ) + } else { + outputString += char + } + } + throw new ParseError(this.pos, 'Unexpected end of input') + } + + private parseToken(): Token { + // The specification wants this check, but it's an unreachable code block. + // if (!/^[A-Za-z*]/.test(this.lookChar())) { + // throw new ParseError(this.pos, 'A token must begin with an asterisk or letter (A-Z, a-z)'); + //} + + let outputString = '' + + while (!this.eof()) { + const char = this.lookChar() + if ( + char === undefined || + !/^[:/!#$%&'*+\-.^_`|~A-Za-z0-9]$/.test(char) + ) { + return new Token(outputString) + } + outputString += this.getChar() + } + + return new Token(outputString) + } + + private parseByteSequence(): ByteSequence { + this.expectChar(':') + this.pos++ + const endPos = this.input.indexOf(':', this.pos) + if (endPos === -1) { + throw new ParseError( + this.pos, + 'Could not find a closing ":" character to mark end of Byte Sequence', + ) + } + const b64Content = this.input.substring(this.pos, endPos) + this.pos += b64Content.length + 1 + + if (!/^[A-Za-z0-9+/=]*$/.test(b64Content)) { + throw new ParseError( + this.pos, + 'ByteSequence does not contain a valid base64 string', + ) + } + + return new ByteSequence(b64Content) + } + + private parseBoolean(): boolean { + this.expectChar('?') + this.pos++ + + const char = this.getChar() + if (char === '1') { + return true + } + if (char === '0') { + return false + } + throw new ParseError( + this.pos, + 'Unexpected character. Expected a "1" or a "0"', + ) + } + + private parseKey(): string { + if (!this.lookChar()?.match(/^[a-z*]/)) { + throw new ParseError( + this.pos, + 'A key must begin with an asterisk or letter (a-z)', + ) + } + + let outputString = '' + + while (!this.eof()) { + const char = this.lookChar() + if (char === undefined || !/^[a-z0-9_\-.*]$/.test(char)) { + return outputString + } + outputString += this.getChar() + } + + return outputString + } + + /** + * Looks at the next character without advancing the cursor. + * + * Returns undefined if we were at the end of the string. + */ + private lookChar(): string | undefined { + return this.input[this.pos] + } + + /** + * Checks if the next character is 'char', and fail otherwise. + */ + private expectChar(char: string): void { + if (this.lookChar() !== char) { + throw new ParseError(this.pos, `Expected ${char}`) + } + } + + private getChar(): string { + return this.input[this.pos++] + } + private eof(): boolean { + return this.pos >= this.input.length + } + // Advances the pointer to skip all whitespace. + private skipOWS(): void { + while (true) { + const c = this.input.substr(this.pos, 1) + if (c === ' ' || c === '\t') { + this.pos++ + } else { + break + } + } + } + // Advances the pointer to skip all spaces + private skipWS(): void { + while (this.lookChar() === ' ') { + this.pos++ + } + } + + // At the end of parsing, we need to make sure there are no bytes after the + // header except whitespace. + private checkTrail(): void { + this.skipWS() + if (!this.eof()) { + throw new ParseError( + this.pos, + 'Unexpected characters at end of input', + ) + } + } +} + +const isDigitRegex = /^[0-9]$/ +function isDigit(char: string | undefined): boolean { + if (char === undefined) return false + return isDigitRegex.test(char) +} diff --git a/httpsig/src/vendor/structured-headers/serializer.ts b/httpsig/src/vendor/structured-headers/serializer.ts new file mode 100644 index 0000000..d7329b0 --- /dev/null +++ b/httpsig/src/vendor/structured-headers/serializer.ts @@ -0,0 +1,142 @@ +/** + * Vendored from structured-headers v1.0.1 (MIT). See README.md in this + * directory. Upstream: src/serializer.ts + */ + +import { + BareItem, + ByteSequence, + Dictionary, + InnerList, + Item, + List, + Parameters, +} from './types.js' + +import { Token } from './token.js' + +import { isAscii, isInnerList, isValidKeyStr } from './util.js' + +export class SerializeError extends Error {} + +export function serializeList(input: List): string { + return input + .map((value) => { + if (isInnerList(value)) { + return serializeInnerList(value) + } else { + return serializeItem(value) + } + }) + .join(', ') +} + +export function serializeDictionary(input: Dictionary): string { + return Array.from(input.entries()) + .map(([key, value]) => { + let out = serializeKey(key) + if (value[0] === true) { + out += serializeParameters(value[1]) + } else { + out += '=' + if (isInnerList(value)) { + out += serializeInnerList(value) + } else { + out += serializeItem(value) + } + } + return out + }) + .join(', ') +} + +export function serializeItem(input: Item): string { + return serializeBareItem(input[0]) + serializeParameters(input[1]) +} + +export function serializeInnerList(input: InnerList): string { + return `(${input[0].map((value) => serializeItem(value)).join(' ')})${serializeParameters(input[1])}` +} + +export function serializeBareItem(input: BareItem): string { + if (typeof input === 'number') { + if (Number.isInteger(input)) { + return serializeInteger(input) + } + return serializeDecimal(input) + } + if (typeof input === 'string') { + return serializeString(input) + } + if (input instanceof Token) { + return serializeToken(input) + } + if (input instanceof ByteSequence) { + return serializeByteSequence(input) + } + if (typeof input === 'boolean') { + return serializeBoolean(input) + } + throw new SerializeError(`Cannot serialize values of type ${typeof input}`) +} + +export function serializeInteger(input: number): string { + if (input < -999_999_999_999_999 || input > 999_999_999_999_999) { + throw new SerializeError( + 'Structured headers can only encode integers in the range range of -999,999,999,999,999 to 999,999,999,999,999 inclusive', + ) + } + return input.toString() +} + +export function serializeDecimal(input: number): string { + const out = input.toFixed(3).replace(/0+$/, '') + const signifantDigits = out.split('.')[0].replace('-', '').length + + if (signifantDigits > 12) { + throw new SerializeError( + 'Fractional numbers are not allowed to have more than 12 significant digits before the decimal point', + ) + } + return out +} + +export function serializeString(input: string): string { + if (!isAscii(input)) { + throw new SerializeError('Only ASCII strings may be serialized') + } + return `"${input.replace(/("|\\)/g, (v) => '\\' + v)}"` +} + +export function serializeBoolean(input: boolean): string { + return input ? '?1' : '?0' +} + +export function serializeByteSequence(input: ByteSequence): string { + return `:${input.toBase64()}:` +} + +export function serializeToken(input: Token): string { + return input.toString() +} + +export function serializeParameters(input: Parameters): string { + return Array.from(input) + .map(([key, value]) => { + let out = ';' + serializeKey(key) + if (value !== true) { + out += '=' + serializeBareItem(value) + } + return out + }) + .join('') +} + +export function serializeKey(input: string): string { + if (!isValidKeyStr(input)) { + throw new SerializeError( + 'Keys in dictionaries must only contain lowercase letter, numbers, _-*. and must start with a letter or *', + ) + } + return input +} diff --git a/httpsig/src/vendor/structured-headers/token.ts b/httpsig/src/vendor/structured-headers/token.ts new file mode 100644 index 0000000..82beaad --- /dev/null +++ b/httpsig/src/vendor/structured-headers/token.ts @@ -0,0 +1,22 @@ +/** + * Vendored from structured-headers v1.0.1 (MIT). See README.md in this + * directory. Upstream: src/token.ts + */ + +import { isValidTokenStr } from './util.js' + +export class Token { + private value: string + constructor(value: string) { + if (!isValidTokenStr(value)) { + throw new TypeError( + "Invalid character in Token string. Tokens must start with *, A-Z and the rest of the string may only contain a-z, A-Z, 0-9, :/!#$%&'*+-.^_`|~", + ) + } + this.value = value + } + + toString(): string { + return this.value + } +} diff --git a/httpsig/src/vendor/structured-headers/types.ts b/httpsig/src/vendor/structured-headers/types.ts new file mode 100644 index 0000000..8b2dc61 --- /dev/null +++ b/httpsig/src/vendor/structured-headers/types.ts @@ -0,0 +1,51 @@ +/** + * Vendored from structured-headers v1.0.1 (MIT). See README.md in this + * directory. Upstream: src/types.ts + */ + +import { Token } from './token.js' + +/** + * Lists are arrays of zero or more members, each of which can be an Item + * or an Inner List, both of which can be Parameterized + */ +export type List = (InnerList | Item)[] + +/** + * An Inner List is an array of zero or more Items. Both the individual Items + * and the Inner List itself can be Parameterized. + */ +export type InnerList = [Item[], Parameters] + +/** + * Parameters are an ordered map of key-value pairs that are associated with + * an Item or Inner List. The keys are unique within the scope of the + * Parameters they occur within, and the values are bare items (i.e., they + * themselves cannot be parameterized + */ +export type Parameters = Map + +/** + * Dictionaries are ordered maps of key-value pairs, where the keys are short + * textual strings and the values are Items or arrays of Items, both of which + * can be Parameterized. + * + * There can be zero or more members, and their keys are unique in the scope + * of the Dictionary they occur within. + */ +export type Dictionary = Map + +export class ByteSequence { + base64Value: string + constructor(base64Value: string) { + this.base64Value = base64Value + } + + toBase64(): string { + return this.base64Value + } +} + +export type BareItem = number | string | Token | ByteSequence | boolean + +export type Item = [BareItem, Parameters] diff --git a/httpsig/src/vendor/structured-headers/util.ts b/httpsig/src/vendor/structured-headers/util.ts new file mode 100644 index 0000000..1c1aeca --- /dev/null +++ b/httpsig/src/vendor/structured-headers/util.ts @@ -0,0 +1,30 @@ +/** + * Vendored from structured-headers v1.0.1 (MIT). See README.md in this + * directory. Upstream: src/util.ts + */ + +import { Item, InnerList, BareItem, ByteSequence } from './types.js' + +const asciiRe = /^[\x20-\x7E]*$/ +const tokenRe = /^[a-zA-Z*][:/!#$%&'*+\-.^_`|~A-Za-z0-9]*$/ +const keyRe = /^[a-z*][*\-_.a-z0-9]*$/ + +export function isAscii(str: string): boolean { + return asciiRe.test(str) +} + +export function isValidTokenStr(str: string): boolean { + return tokenRe.test(str) +} + +export function isValidKeyStr(str: string): boolean { + return keyRe.test(str) +} + +export function isInnerList(input: Item | InnerList): input is InnerList { + return Array.isArray(input[0]) +} + +export function isByteSequence(input: BareItem): input is ByteSequence { + return typeof input === 'object' && 'base64Value' in input +} diff --git a/httpsig/src/verify.ts b/httpsig/src/verify.ts index 864538a..ef60adc 100644 --- a/httpsig/src/verify.ts +++ b/httpsig/src/verify.ts @@ -21,6 +21,7 @@ import { parseSignature, generateSignatureBase, } from './utils/signature.js' +import { serializeInnerList } from './structured-fields.js' import { base64urlDecode } from './utils/base64.js' import { calculateThumbprint } from './utils/thumbprint.js' import { BoundedTtlCache } from './utils/cache.js' @@ -651,21 +652,15 @@ export async function verify( // taken from the key material only (see getAlgorithmFromJwk below), // per RFC 9421 Section 3.3.7. A signer that declares a misleading // `alg` does not change which operation the verifier performs. - const componentList = components.map((c) => `"${c}"`).join(' ') - const paramPairs = Object.entries(params) - .map(([key, value]) => { - if (typeof value === 'number') { - return `${key}=${value}` - } - // String values from parsing may already have quotes - const stringValue = String(value) - if (stringValue.startsWith('"') && stringValue.endsWith('"')) { - return `${key}=${stringValue}` - } - return `${key}="${stringValue}"` - }) - .join(';') - const signatureParams = `(${componentList});${paramPairs}` + // + // The value is re-serialized from the parsed Inner List rather than + // rebuilt from the extracted parts, so a String stays a String and a + // Token stays a Token. Rebuilding by hand is what quoted every value + // alike and would have corrupted the base for any signer that sent a + // Token-valued parameter. + const signatureParams = serializeInnerList( + signatureInput.signatureParams, + ) componentValues.set('@signature-params', signatureParams) const componentsWithParams = [...components, '@signature-params'] diff --git a/httpsig/tests/test-structured-fields.ts b/httpsig/tests/test-structured-fields.ts new file mode 100644 index 0000000..8f37c63 --- /dev/null +++ b/httpsig/tests/test-structured-fields.ts @@ -0,0 +1,397 @@ +/** + * RFC 8941 Structured Fields + * + * These cover the shapes that hand-rolled parsers get wrong. Every one of them + * has a real failure behind it: a `;` inside a quoted string, an escape inside + * a string, the Dictionary-of-Inner-Lists-with-parameters shape that + * Signature-Input uses, Byte Sequences, and the Token/String distinction that + * a regex-based parser erases. + */ + +import { test } from 'node:test' +import assert from 'node:assert' +import { + parseDictionary, + parseList, + parseItem, + serializeDictionary, + serializeList, + serializeItem, + serializeInnerList, + bareItemToString, + isInnerList, + isByteSequence, + Token, + ByteSequence, + verify, +} from '../src/index.js' +import type { BareItem, Item, InnerList, Parameters } from '../src/index.js' +import { generateSignatureBase } from '../src/utils/signature.js' +import { base64Encode } from '../src/utils/base64.js' + +/** The member of a one-member Dictionary. */ +function only(header: string) { + const dictionary = parseDictionary(header) + assert.strictEqual(dictionary.size, 1, `expected one member: ${header}`) + return [...dictionary][0] +} + +/* ------------------------------------------------------------------------- + * The AAuth-Requirement regression case + * ---------------------------------------------------------------------- */ + +test("AAuth-Requirement: the specification's literal interaction example", () => { + // This is the example a naive `split(";")` gets wrong. It is a Dictionary + // with one member whose value is the Token `interaction`, carrying two + // String parameters. + const header = + 'requirement=interaction; url="https://resource.example/interaction"; code="A1B2-C3D4"' + + const [key, member] = only(header) + assert.strictEqual(key, 'requirement') + assert.ok(!isInnerList(member)) + + const [value, params] = member as Item + assert.ok(value instanceof Token, 'requirement value is a Token') + assert.strictEqual(value.toString(), 'interaction') + assert.strictEqual( + params.get('url'), + 'https://resource.example/interaction', + ) + assert.strictEqual(params.get('code'), 'A1B2-C3D4') +}) + +test('AAuth-Requirement: a `;` inside a quoted url does not split the parameters', () => { + // The failure this whole exercise exists for. Splitting on `;` cuts the + // url in half and loses `code` entirely. + const header = + 'requirement=interaction; url="https://resource.example/i?a=1;b=2"; code="A1B2-C3D4"' + + const [, member] = only(header) + const [, params] = member as Item + + assert.strictEqual(params.get('url'), 'https://resource.example/i?a=1;b=2') + assert.strictEqual(params.get('code'), 'A1B2-C3D4') + assert.strictEqual(params.size, 2, 'exactly two parameters') +}) + +test('AAuth-Requirement: auth-token with a resource token round-trips', () => { + const header = + 'requirement=auth-token; resource-token="eyJhbGciOiJFZERTQSJ9.eyJzdWIiOiJhIn0.sig"' + const [, member] = only(header) + const [value, params] = member as Item + + assert.strictEqual((value as Token).toString(), 'auth-token') + assert.strictEqual( + params.get('resource-token'), + 'eyJhbGciOiJFZERTQSJ9.eyJzdWIiOiJhIn0.sig', + ) + + // Re-serialization is canonical: the optional space after `;` that RFC + // 8941 allows on input is not emitted on output. + assert.strictEqual( + serializeDictionary(parseDictionary(header)), + header.replace('; ', ';'), + ) +}) + +test('AAuth-Capabilities: a List of Tokens', () => { + const list = parseList('interaction, auth-token, payment') + assert.deepStrictEqual( + list.map((member) => bareItemToString((member as Item)[0])), + ['interaction', 'auth-token', 'payment'], + ) + assert.strictEqual(serializeList(list), 'interaction, auth-token, payment') +}) + +/* ------------------------------------------------------------------------- + * Strings: quoting and escaping + * ---------------------------------------------------------------------- */ + +test('String: an escaped double quote survives the round trip', () => { + const [, member] = only('a="say \\"hello\\""') + assert.strictEqual((member as Item)[0], 'say "hello"') + + assert.strictEqual( + serializeItem([`say "hello"`, new Map()]), + 'a="say \\"hello\\""'.slice(2), + ) +}) + +test('String: an escaped backslash survives the round trip', () => { + const [, member] = only('a="back\\\\slash"') + assert.strictEqual((member as Item)[0], 'back\\slash') + + assert.strictEqual( + serializeItem(['back\\slash', new Map()]), + '"back\\\\slash"', + ) +}) + +test('String: a backslash before anything else is a parse error', () => { + // RFC 8941 Section 3.3.3 permits only \\ and \" inside an sf-string. + assert.throws(() => parseDictionary('a="new\\nline"'), /backslash/) +}) + +test('String: an unterminated string is a parse error, not a truncation', () => { + assert.throws(() => parseDictionary('a="unterminated'), /Parse error/) +}) + +test('String: commas and semicolons inside a string do not end the member', () => { + const dictionary = parseDictionary('a="x,y;z", b=2') + assert.strictEqual(dictionary.size, 2) + assert.strictEqual((dictionary.get('a') as Item)[0], 'x,y;z') + assert.strictEqual((dictionary.get('b') as Item)[0], 2) +}) + +/* ------------------------------------------------------------------------- + * Tokens vs Strings + * ---------------------------------------------------------------------- */ + +test('Token vs String: they are different values, not the same text', () => { + const asToken = parseItem('interaction')[0] + const asString = parseItem('"interaction"')[0] + + assert.ok(asToken instanceof Token) + assert.strictEqual(typeof asString, 'string') + assert.notStrictEqual(asToken, asString) + + // And they serialize back differently -- which is why erasing the + // distinction breaks a signature base. + assert.strictEqual(serializeItem([asToken, new Map()]), 'interaction') + assert.strictEqual(serializeItem([asString, new Map()]), '"interaction"') +}) + +test('Token vs String: bareItemToString reads either, and refuses the rest', () => { + assert.strictEqual(bareItemToString(new Token('hwk')), 'hwk') + assert.strictEqual(bareItemToString('hwk'), 'hwk') + assert.throws(() => bareItemToString(42), /String or Token/) + assert.throws(() => bareItemToString(true), /String or Token/) +}) + +test('Token: a value that is not a valid token is a parse error', () => { + // `@method` unquoted is not a Token -- component identifiers are Strings. + assert.throws(() => parseList('@method'), /Parse error/) +}) + +/* ------------------------------------------------------------------------- + * Byte Sequences + * ---------------------------------------------------------------------- */ + +test('Byte Sequence: parses base64 between colons', () => { + const [, member] = only('sig=:aGVsbG8=:') + const [value] = member as Item + assert.ok(isByteSequence(value)) + assert.strictEqual((value as ByteSequence).toBase64(), 'aGVsbG8=') +}) + +test('Byte Sequence: base64 containing `+` and `/` is not mistaken for anything else', () => { + const b64 = 'w6/Cr8O/w6s+PDw/Pg==' + const [, member] = only(`sig=:${b64}:`) + assert.strictEqual(((member as Item)[0] as ByteSequence).toBase64(), b64) + assert.strictEqual( + serializeDictionary(parseDictionary(`sig=:${b64}:`)), + `sig=:${b64}:`, + ) +}) + +test('Byte Sequence: a missing closing colon is a parse error', () => { + assert.throws(() => parseDictionary('sig=:aGVsbG8='), /closing/) +}) + +test('Byte Sequence: non-base64 content is rejected', () => { + assert.throws(() => parseDictionary('sig=:not valid!:'), /base64/) +}) + +/* ------------------------------------------------------------------------- + * Dictionary of Inner Lists with parameters -- the Signature-Input shape + * ---------------------------------------------------------------------- */ + +test('Signature-Input: Dictionary of Inner Lists with parameters', () => { + const header = + 'sig-b26=("date" "@method" "@path" "@authority" "content-type" "content-length");created=1618884473;keyid="test-key-ed25519"' + + const [label, member] = only(header) + assert.strictEqual(label, 'sig-b26') + assert.ok(isInnerList(member), 'the member is an Inner List') + + const [items, params] = member as InnerList + assert.deepStrictEqual( + items.map(([component]) => component), + [ + 'date', + '@method', + '@path', + '@authority', + 'content-type', + 'content-length', + ], + ) + assert.strictEqual(params.get('created'), 1618884473) + assert.strictEqual(params.get('keyid'), 'test-key-ed25519') + + // Re-serializing the Inner List is how @signature-params is reproduced. + assert.strictEqual( + serializeInnerList(member as InnerList), + header.slice('sig-b26='.length), + ) +}) + +test('Signature-Input: two signatures in one Dictionary', () => { + const dictionary = parseDictionary( + 'sig1=("@method");created=1, sig2=("@path" "@authority");created=2;keyid="k"', + ) + assert.deepStrictEqual([...dictionary.keys()], ['sig1', 'sig2']) + assert.strictEqual((dictionary.get('sig2') as InnerList)[0].length, 2) +}) + +test('Signature-Input: an empty Inner List is valid', () => { + const [, member] = only('sig=();created=1767021027;alg="ed25519"') + assert.deepStrictEqual((member as InnerList)[0], []) + assert.strictEqual((member as InnerList)[1].get('alg'), 'ed25519') +}) + +test('Signature-Input: a `;` inside a quoted keyid does not split the parameters', () => { + const [, member] = only('sig=("@method");created=1;keyid="a;b";tag="c"') + const params = (member as InnerList)[1] + assert.strictEqual(params.get('keyid'), 'a;b') + assert.strictEqual(params.get('tag'), 'c') +}) + +test('Signature-Input: an Inner List item may carry its own parameters', () => { + // The grammar allows it even though this implementation refuses to sign + // over one. Proving the parser sees them is what lets the refusal be + // explicit rather than an accident. + const [, member] = only( + 'sig=("@query-param";name="q" "host";req);created=1', + ) + const [items] = member as InnerList + assert.strictEqual(items[0][1].get('name'), 'q') + assert.strictEqual(items[1][1].get('req'), true) +}) + +test('Signature-Input: a parameterized covered component is refused, not ignored', async () => { + const result = await verify({ + method: 'GET', + authority: 'api.example.com', + path: '/data', + headers: { + 'signature-key': + 'sig=hwk;alg="Ed25519";kty="OKP";crv="Ed25519";x="JrQLj5P_89iXES9-vFgrIy29clF9CC_oPPsw3c5D0bs"', + 'signature-input': + 'sig=("@method" "signature-key";req);created=1618884473', + signature: 'sig=:dGVzdA==:', + }, + }) + + assert.strictEqual(result.verified, false) + assert.match(result.error ?? '', /Unsupported component parameters/) +}) + +/* ------------------------------------------------------------------------- + * @signature-params fidelity + * ---------------------------------------------------------------------- */ + +test('@signature-params: a Token-valued parameter is reproduced as a Token', async () => { + // The old hand-rolled reconstruction quoted every non-numeric parameter, + // so a signer that sent `;alg=hmac-sha256` (a Token) had it turned into + // `;alg="hmac-sha256"` in the signature base and failed to verify. This + // signs a request with a bare Token parameter and verifies it. + const keyPair = (await crypto.subtle.generateKey( + { name: 'Ed25519' }, + true, + ['sign', 'verify'], + )) as CryptoKeyPair + const privateJwk = await crypto.subtle.exportKey('jwk', keyPair.privateKey) + const publicJwk = await crypto.subtle.exportKey('jwk', keyPair.publicKey) + privateJwk.alg = 'Ed25519' + publicJwk.alg = 'Ed25519' + + const created = Math.floor(Date.now() / 1000) + const components = ['@method', '@authority', '@path', 'signature-key'] + const signatureKey = + `sig=hwk;alg="Ed25519";kty="OKP";` + + `crv="${publicJwk.crv}";x="${publicJwk.x}"` + + // Built with the serializer, so `alg` really is a bare Token. + const innerList: InnerList = [ + components.map((c) => [c, new Map()] as Item), + new Map([ + ['created', created], + ['alg', new Token('hmac-sha256')], + ]) as Parameters, + ] + const signatureParams = serializeInnerList(innerList) + assert.ok( + signatureParams.endsWith(';alg=hmac-sha256'), + `alg must be unquoted, got: ${signatureParams}`, + ) + + const base = generateSignatureBase( + [...components, '@signature-params'], + new Map([ + ['@method', 'GET'], + ['@authority', 'api.example.com'], + ['@path', '/data'], + ['signature-key', signatureKey], + ['@signature-params', signatureParams], + ]), + ) + + const signature = new Uint8Array( + await crypto.subtle.sign( + { name: 'Ed25519' }, + keyPair.privateKey, + new TextEncoder().encode(base), + ), + ) + + const result = await verify({ + method: 'GET', + authority: 'api.example.com', + path: '/data', + headers: { + 'signature-key': signatureKey, + 'signature-input': `sig=${signatureParams}`, + signature: `sig=:${base64Encode(signature)}:`, + }, + }) + + assert.strictEqual( + result.verified, + true, + `Token-valued parameter must survive re-serialization: ${result.error}`, + ) +}) + +/* ------------------------------------------------------------------------- + * Dictionary edge cases + * ---------------------------------------------------------------------- */ + +test('Dictionary: a bare member means true', () => { + const dictionary = parseDictionary('a, b=?0, c') + assert.strictEqual((dictionary.get('a') as Item)[0], true) + assert.strictEqual((dictionary.get('b') as Item)[0], false) + assert.strictEqual((dictionary.get('c') as Item)[0], true) +}) + +test('Dictionary: a trailing comma is a parse error', () => { + assert.throws(() => parseDictionary('a=1,'), /trailing comma/) +}) + +test('Dictionary: an empty header is an empty Dictionary', () => { + assert.strictEqual(parseDictionary('').size, 0) + assert.deepStrictEqual(parseList(''), []) +}) + +test('Dictionary: Integers and Decimals keep their types', () => { + const dictionary = parseDictionary('i=42, n=-17, d=4.5') + assert.strictEqual((dictionary.get('i') as Item)[0], 42) + assert.strictEqual((dictionary.get('n') as Item)[0], -17) + assert.strictEqual((dictionary.get('d') as Item)[0], 4.5) + assert.strictEqual(serializeDictionary(dictionary), 'i=42, n=-17, d=4.5') +}) + +test('Item: trailing junk after a standalone Item is a parse error', () => { + assert.throws(() => parseItem('a b'), /Unexpected characters at end/) +}) From 3af29b70ca2a16419dd75499e7701588f8a8eda1 Mon Sep 17 00:00:00 2001 From: dickhardt Date: Wed, 12 Aug 2026 14:45:12 +0100 Subject: [PATCH 2/3] httpsig: ship the parser on its own subpath, and the vendored licence `@aauth/protocol` has zero runtime dependencies of its own and needs only the structured-field helpers, not verify() and the crypto that comes with it. Add a `./structured-fields` subpath export so a consumer can take the parser without the rest of the package. Also add `src/vendor/` to `files`. The compiled vendored code ships in dist, and MIT requires the copyright notice to travel with it. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01FmiCqDjRUSx6zb1N4gZPXE --- httpsig/package.json | 12 ++++++++++-- httpsig/src/vendor/structured-headers/README.md | 4 ++++ 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/httpsig/package.json b/httpsig/package.json index 0f83f42..e6aa605 100644 --- a/httpsig/package.json +++ b/httpsig/package.json @@ -10,10 +10,18 @@ "main": "./dist/index.js", "types": "./dist/index.d.ts", "exports": { - ".": "./dist/index.js" + ".": { + "types": "./dist/index.d.ts", + "default": "./dist/index.js" + }, + "./structured-fields": { + "types": "./dist/structured-fields.d.ts", + "default": "./dist/structured-fields.js" + } }, "files": [ - "dist/" + "dist/", + "src/vendor/" ], "keywords": [ "http", diff --git a/httpsig/src/vendor/structured-headers/README.md b/httpsig/src/vendor/structured-headers/README.md index a5ab59e..5c1c522 100644 --- a/httpsig/src/vendor/structured-headers/README.md +++ b/httpsig/src/vendor/structured-headers/README.md @@ -26,6 +26,10 @@ The whole parser and serializer, from upstream `src/`: Nothing else from the package was taken: no build config, no tests, no browser bundle. +This directory is listed in the package's `files`, so the MIT licence text +travels with every published copy of the compiled output, as the licence +requires. + ## Why vendored rather than depended on `@hellocoop/httpsig` has zero runtime dependencies, deliberately. It verifies From 645bd349e1f2a459f1697c99f3073c7d786c6559 Mon Sep 17 00:00:00 2001 From: dickhardt Date: Wed, 12 Aug 2026 16:31:13 +0100 Subject: [PATCH 3/3] httpsig: 2.1.0 The vendored RFC 8941 parser fixes two verification bugs and adds a `/structured-fields` export. Both are additive for existing callers, so minor. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01FmiCqDjRUSx6zb1N4gZPXE --- httpsig/package.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/httpsig/package.json b/httpsig/package.json index e6aa605..cfab502 100644 --- a/httpsig/package.json +++ b/httpsig/package.json @@ -1,6 +1,6 @@ { "name": "@hellocoop/httpsig", - "version": "2.0.1", + "version": "2.1.0", "description": "HTTP Message Signatures (RFC 9421) with Signature-Key header support", "repository": { "type": "git",