[swift-server-dev] Proposal: Header fields as a type conforming to Encodable / Decodable

[Helge Heß]

Hi George,

On 11. Dec 2017, at 02:40, George Leontiev via swift-server-dev <swift-server-dev at swift.org> wrote:
> I know this has been discussed a few times on this list, but I’d like to throw a different model for decoding headers into the ring. My approach revolves around using the Codable protocols introduced in Swift 4. An example can be found here: https://github.com/GeorgeLyon/Server / https://github.com/GeorgeLyon/Server/blob/master/Sources/Server/main.swift

That is an interesting idea for sure!

> Also, decoding HTTP header fields and decoding JSON are VERY similar tasks, and it would be nice if they used the same underlying Swift mechanism so optimizations can be built once for both. 

I don’t they are very similar tasks at all, but I have no specific issues w/ abusing them for this :->
One is about decoding arbitrary nested structures of objects w/ rather fixed base types (and potentially back-references, hahaha, no ;->),
while the other one is about decoding a predefined, fixed structure of potentially arbitrary base values.

Also Decodable doesn’t really help us much w/ the part which actually requires decoding, i.e. properly decoding Cookie or If headers.


I think your suggestion is actually two kinda separate things:
1) latching on the framework level request representation to the
   HTTP framework layer and have the framework map them
2) use Decodable for mapping.


But lets take a step back and see what we are doing in the parsing process.

Right now:

1. grab a base-buffer, we receive data into it
2. trigger the parser, passes back pointers into base-buffer,
   we copy the data into a decoding-buffer until a token
   is complete
3. we convert the Data into a String and add it to the
   HTTPRequestHead object
4. if the head is complete, we trigger processing
5. potentially convert `HTTPRequestHead` into `FrameworkRequest`
   (but maybe just wrap or use as-is)


To use Decodable we would need to do this:

1. grab a base-buffer, we receive data into it
2. trigger the parser, passes back pointers into base-buffer,
   we copy the data into a decoding-buffer until a token
   is complete
3. keep the data, add it to a structure of essentially the
   form [ ( DataSlice, DataSlice ) ] - lets call it ‘HTTPRawRequestHead'
4. if the head is complete, call the decoder, convert the
   raw-request into the `FrameworkRequest`
5. trigger processing


What I like about your approach is, that we don’t need to create rather expensive String objects in step 3. E.g. decodeInt would just decode the int from a byte buffer - waaayy better ;-) This part is nice :-)
What I dislike about your approach is, that we need that transient structure we decode into. It could be more efficient than [(String,String)], e.g. [(Data,Data)], or maybe even something slice based on the base-buffer, but it still is a structure which captures the raw setup of the full request head.

I also have the feeling that Decodable is too heavyweight for the task (based on Strings etc), but that may not be justified and be demonstrated.


As mentioned before I’m actually a little against using Strings here, and my personal preference would actually be something like this:

1. grab a base-buffer, we receive data into it
2. trigger the parser, passes back pointers into base-buffer,
   we copy the data into a decoding-buffer until a token
   is complete
3. keep the data, add it to a structure of essentially the
   form [ ( DataSlice, DataSlice ) ] - lets call it ‘HTTPRawRequestHead’
4. if the head is complete, we trigger processing
5. potentially convert `HTTPRawRequestHead ` into `FrameworkRequest`
   (but maybe just wrap or use as-is!) - you could use Decoding here
   if you want, or not.

Or in other words: Even w/ Decodable we first need to decode into some complete header structure. What I would prefer over the other approaches is that we directly vend this lower level structure to the framework (it could still have stuff like `subscript(String)->String`!). Which can then decide what to do with it.
(I kinda envision that HTTPRawRequestHead as a single malloc buffer, and the header keys/values as just pointers into that)


Well, I think to go forward, I would suggest that you provide a test setup to measure the performance implications.
One should produce the current API's `HTTPRequestHead` using the current means (directly hooked up to the parser, incremental buildup of Strings), and one should produce the same structure using a Decoder based approach.
We could then compare the performance implications, and if they turn out to be negligible, I think it may be worthwhile to consider your approach for the added extra flexibility.


But this is just my opinion of course as an interested by-stander ;-)
  hh


> P.S. Sorry for the double-mail but I thought this was different enough from my handlers-as-structs proposal that I wanted to separate the responses into two threads.

Absolutely fine IMO, those are distinct things.