[swift-server-dev] Proposal: Header fields as a type conforming to Encodable / Decodable

[George Leontiev]

@Cory
Using `Decodable` does not force the representation of header fields to be a dictionary. [HeaderField] can also be Decodable and the client can choose to handle the cases you bring up in the way they see fit and avoid many of the pitfalls you bring up.
`Decodable` provides a Swift-standard mechanism for the client to choose what representation of headers works for them. Want to use a basic [HeaderField] list? Great! Implementing a simple API and don’t really care about Set-Cookie or other nuances of the HTTP spec? This makes it easy! We can even provide “strategies” for repeated header field names similar to how JSONDecoder provides `dateDecodingStrategy`.
This approach also allows the framework to provide standard implementations of things like HTTPMediaType that web frameworks CAN use without preventing them from doing something else if it is necessary for their use case. 

I do agree that the implementation of something like Codable has some overhead, is a nontrivial amount of work, and is currently missing some nice features like streaming-decoding. These are things that the Swift language WILL have to address at some point but, from my perspective, coding JSON over my own RPC mechanism or coding the (admittedly nuanced) HTTP header format are different flavors of the same problem and it would be great if they were handled in a similar way.

I could also see a world where both APIs are provided. An HTTPParser which can be used to parse token-by-token or an HTTPRequestParser which supports Codable headers.

- George

> On Dec 11, 2017, at 2:43 AM, Cory Benfield <cbenfield at apple.com> wrote:
> 
> 
> 
>> On 11 Dec 2017, at 01:40, George Leontiev via swift-server-dev <swift-server-dev at swift.org <mailto:swift-server-dev at swift.org>> wrote:
>> 
>> I know this has been discussed a few times on this list, but I’d like to throw a different model for decoding headers into the ring. My approach revolves around using the Codable protocols introduced in Swift 4. An example can be found here: https://github.com/GeorgeLyon/Server <https://github.com/GeorgeLyon/Server> / https://github.com/GeorgeLyon/Server/blob/master/Sources/Server/main.swift <https://github.com/GeorgeLyon/Server/blob/master/Sources/Server/main.swift>
>> 
>> This approach is a happy medium between the swift-server/http specifying the headers and allowing the frameworks built on top of this core to be flexible in how they handle header fields. By providing a type, the framework can choose to use a [String:String] representation, or even a [(String, String)]. On the other hand, they could choose to use a Codable type with String values or even specialized types like [HTTPMediaRange] for Accept. Indeed, it may be beneficial for swift-server/http to provide default implementations of various header types like HTTPMediaRange without forcing a downstream framework into using them. These specialized types are however outside the scope of this proposal.
>> 
>> Also, decoding HTTP header fields and decoding JSON are VERY similar tasks, and it would be nice if they used the same underlying Swift mechanism so optimizations can be built once for both. 
> 
> Sadly, they aren’t really.
> 
> JSON and XML are extremely well-specified formats that, amongst other things, are pretty well defined in the form of key-value data structures. HTTP headers are not like that, though they may appear that way superficially.
> 
> Let’s get one thing out of the way quickly: any HTTP implementation that uses [String: String] as the format of HTTP headers at the lowest level is spec-nonconformant, and will fail in a number of situations where it should succeed. The most notable case is cookies: the Set-Cookie header field can appear multiple times in a response but is not a header list and cannot be joined by any character as both comma (used elsewhere in HTTP) and semicolon (used by Cookie) are meaningful reserved characters in the syntax for Set-Cookie. Any attempt to represent Set-Cookie in [String: String] is doomed to failure.
> 
> As a more general note, attempts to think about HTTP header fields as a mapping tend to fall short. You *can* construct them this way, but they have a number of extensions that make them ill-suited to this simplified representation. The Python community has landed on the CaseInsensitiveOrderedMultiDict as the most informational representation for HTTP header fields: that is, a mapping that has case-insensitive keys, retains order, and has multiple values for each field.
> 
> All of this gets even trickier as you consider the sub-problems involved here. Consider what should happen for header fields that do not define a list format, but are nonetheless sent multiple times. Consider what should happen for header field values that include binary junk. Consider what should happen for header field values that use different text encodings than other header values in the same header block. Consider what should happen for header fields that are malformed but parseable (such as my favourite ever header line that I encountered in the wild, the four-octet sequence 3A 20 0D 0A “: \r\n”). Consider what should happen in cases where header field values that should conform to a specific format fail to do so.
> 
> All of these are complex application-specific error handling concerns, and all should be tolerated by the lowest-level implementations. This would not be an issue if Codable provided meaningful benefit that was not easily provided in another form, but it doesn’t really. All high-level implementations will likely accept the lowest-common-denominator implementation of headers (something like [(String, String)] or even [(String, Data)]) and provide a number of helper methods on top of it to transform these formats from their underlying representation to the higher-level one. This grants applications that want to be resilient to bad header fields the opportunity to do so, while allowing those that can reasonably assume they’ll handle well-formed data to provide the appropriate extensions that do the transformation.
> 
> This wouldn’t matter by itself either, but then we have to add that Codable is a whole lot of overhead that doesn’t provide us much advantage over the “base representation with extensions” model. Implementing the encoder/decoder for HTTP headers is a big chunk of work that needs to be handled by every single entity that attempts to provide the low-level API, and it won’t be faster than the base version, and it won’t provide much utility.
> 
> All-in-all I think I’m -1 on this: it’s a lot of complexity for relatively minimal gain.
> 
> Cory

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-server-dev/attachments/20171216/c1c3d27b/attachment.html>