Automatic parsing / serialization

[dhensby]

Problem description

When dealing with headers, it's hard for a generic library to know what the serialization for an arbitrary header is going to be.

If we wish to normalise an arbitrary header without knowing it's serialization (is it a dictionary, a list, or an item?) then to parse the header reliably becomes somewhat tedious.

Further to this, serialisation becomes tricky as well. Any parser that is implemented by this libary's consumer to attempt to parse the fields in a generic way must somehow keep hold of the type that was successfully parsed and, likewise, how to re-serialize it (if needed). Serializing the values of a dictionary are more complex still as there's no concrete way to know if the item in the Map is itself an Item or and InnerList. (edit: there is an isInnerList type guard which is useful for this purpose).

The lack of concrete typing of the returned values from the parsers make inference harder as well. If the returned types were first-class classes that we could perform instanceof checks on or similar, then inferring the type once parsed in a generic manner would be easier, including the serialization of any sub items that were returned.

Proposed solution

Implementing concrete classes would facilitate a more generic parsing / serialization approach to the data that an application deals with.

For example:

interface StructuredField {
  serialize(): string;
  toString(): string; // just an alias of serialize()
}

class Dictionary {
    private readonly parsed: DictType;
    private readonly raw: string;
    constructor(input: string) {
        this.raw = input;
        this.parsed = parseDictionary(input);
    }

    toString(): string {
        return this.serialize();
    }

    serialize(): string {
        return serializeDictionary(this.parsed);
    }

    has(key: string): boolean {
        return this.parsed.has(key);
    }

    get(key: string): StructuredField {
        return this.parsed.get(key);
    }
}

With this kind of concrete implementation we can then perform some more generic parsing/serialization of dictionaries.

const dict = new Dictionary('a=1, b=2;x=1;y=2, c=(a   b    c), d');
console.log(dict.has('a')); // true
console.log(dict.has('e')); // false
console.log(dict.get('b').serialize()); // 2;x=1;y=2
console.log(dict.get('c').serialize()); // (a b c)

Above the consumer doesn't need to know anything about the items in the dictionary to serialize them, we can also see that this could extend to a more generic parser that returns the appropriate class for the structured field, and then can be acted on appropriately.

The existing parseDictionary() and similar functions effectively become factories for the respective classes (mostly for backwards compatability) and then we could introduce a generic parseHeader()/parseField() function that could follow the guidance of the spec in terms of parsing structured fields

Use case

I'm working on HTTP Message Signing, which requires parsing of structured headers in a generic way. It's not feasible for the signer/verifier logic to know exactly what type of structured header we (should) be parsing, there's also a need to be able to serialize headers or items in dictionaries, without having to know ahead of time what the type of item we are dealing with.

How does this sound as an approach?

[mxxk]

I know this is a two-year-old issue, but wanted to point out that because of syntactical overlap between top-level structured headers types (List, Dictionary, Item), it might not be possible to parse a header generically. For example, consider the following example header (from http_sf):

X-Header: foo; a=1, bar; b=2

X-Header could be parsed as a Dictionary, or a List:

> structuredHeaders.parseDictionary('a=1, bar; b=2')
Map(2) {
  'foo' => [ true, Map(1) { 'a' => 1 } ],
  'bar' => [ true, Map(1) { 'b' => 2 } ]
}

> structuredHeaders.parseList('a=1, bar; b=2')
[
  [ Token { value: 'foo' }, Map(1) { 'a' => 1 } ],
  [ Token { value: 'bar' }, Map(1) { 'b' => 2 } ]
]

Note

Having said the above, in either case, the strict re-serialization of this header is identical,

> sh.serializeList(sh.parseList('a=1, bar; b=2'))
'foo;a=1, bar;b=2'

> sh.serializeDictionary(sh.parseDictionary('a=1, bar; b=2'))
'foo;a=1, bar;b=2'

so I'm not sure if this is a compelling example... 🤔

This issue is asking about normalizing structured field values (in a generic way) for the purposes of processing HTTP message signatures, so I assume it's related to handling the sf (strict serialization) parameter documented in RFC 9421 § 2.1.1. — Strict Serialization of HTTP Structured Fields. Quoting that section (with italic emphasis added),

If the value of an HTTP field is known by the application to be a Structured Field type (as defined in [STRUCTURED-FIELDS] or its extensions or updates) and the expected type of the Structured Field is known, the signer MAY include the sf parameter in the component identifier.

The italicized portion seems to suggest that it's a prerequisite to know the type of the structured field in order to strictly re-serialize it. This is exactly what the JavaScript implementation of RFC 9421 (misskey-dev/node-http-message-signatures) does:

https://github.com/misskey-dev/node-http-message-signatures/blob/308c1e1630b77ef1c961e52e1af05a0e9df5e6d2/src/rfc9421/base.ts#L275-L278

Considering the above guidance from RFC 9421, would it ever be needed to re-serialize a structured field of an unknown type? (I would think not.)