[swift-server-dev] Prototype of the discussed HTTP API Spec

Tue May 30 09:58:40 CDT 2017

Hi,

> On 30 May 2017, at 3:47 pm, Paulo Faria <paulo at zewo.io> wrote:
> 
> I guess we need two storages after all.. but instead of lowercasing in the dictionary storage we could keep Field.
> 
> public struct HTTPHeaders {
>     public var original: [(String, String)]
>     public var headers: [Field: [String]]
> }
> 
> This way we have:
> 
> - We don't have to do lowercase on all headers even if they're not needed later.
> - The case insensitive comparison from Field is done when the user actually needs the header.
> - We maintain the original order of the http message in cases of proxying by providing original.
> - Retrieval is done in O(1) from headers.

fine by me but tbh that can be implementation defined, no? We should document that accessing header fields must be in constant time and everyone decides how they implement that.

> But we don't have single source of truth.. which might not be that bad and we also might get a penalty by adding to both storages.

it's constant overhead is both time & space (we just duplicate storage), I'd argue that's fine.

-- Johannes

> On 30 May 2017 at 11:41, Paulo Faria <paulo at zewo.io> wrote:
> oh.. scratch that.. the ordered dict won't help with proxying at all.. ouch.
> 
> On 30 May 2017 at 11:40, Paulo Faria <paulo at zewo.io> wrote:
> Hi, Johannes!
> 
> You're right. It helps performance by not having to go through all the headers and lowercase then when they might not be needed during request handling. This worked well in my own code because I used a dictionary as a backing storage, having O(1) retrieval. Ideally the best backing storage would be:
> 
> OrderedDictionary<Field, [String]>
> 
> - This way we have a single source of truth (another problem with the two storages approach)
> - We don't have to do lowercase on all headers even if they're not needed later.
> - The case insensitive comparison from Field is done when the user actually needs the header.
> - We maintain the original order of the http message in cases of proxying.
> - Retrieval is done in O(1) 
> 
> Cheers,
> Paulo
> 
> On 30 May 2017 at 11:29, Johannes Weiss <johannesweiss at apple.com> wrote:
> Hi,
> 
> > On 27 May 2017, at 2:18 pm, Paulo Faria via swift-server-dev <swift-server-dev at swift.org> wrote:
> >
> > Michael, I narrowed the scope of the proposal and focused the rationale. I answered your comments about switch for HTTPMethod and HTTPStatus. Basically, we can switch on them because I implemented the pattern matching operator ~=. I also removed the HTTPMessage as it is not *needed* and users can define it and make HTTPRequest and HTTPResponse conform to it in extensions in their own module.
> >
> > https://github.com/paulofaria/http-api-proposal#rationale
> >
> > Carl, Helge, Chris and Rien what do you think about the code? This is pretty close to what Carl suggested.
> >
> > The main differences are..
> >
> > HTTPVersion as a struct.
> > HTTPHeaders backed by a single array storage and case insensitive checking is done with the Field type to improve performance.
> 
> not opposed to having a 'Field', but how does that improve performance? It reduces memory usage but having a single list makes the lookup of a certain header field with .filter O(n) instead of O(1).
> 
> 
> > HTTPMethod as a struct instead of enum with pattern matching added allowing switch cases. (pointed by Helge)
> > HTTPStatus as a struct instead of enum with pattern matching added allowing switch cases. (pointed by Helge)
> 
> happy with those. Except the proposed hashing functions
> 
> return method.hashValue ^ uri.hashValue ^ version.hashValue ^ headers.hashValue
> 
> is really not appropriate but I guess it's more a proof of concept.
> 
> -- Johannes
> 
> 
> > Apart from that there's only naming differences. I added some tests to showcase what I mentioned.
> >
> > https://github.com/paulofaria/http-api-proposal/blob/master/Tests/HTTPTests/HTTPTests.swift
> >
> > On 27 May 2017 at 08:41, Michael Chiu <hatsuneyuji at icloud.com> wrote:
> > The thing is, as I have mentioned before if we move on that route we have to have a well defined structure before hand, for example, there’s no such protocol “Message” in what Johannes proposed.
> >
> > As Chris and Carl mentioned before, this is only a prototype of the discussed API, so at least we have something to test, experiment and branchmark on.
> >
> > Also things like WebApp in Johannes’ proposal can easily remove without huge changes, since WebApp is simply a typealias. I really don’t think that’s a big issue, every other parts of the prototype is in fact pretty http-specific.
> >
> > Yes the prototype might dependent on some other frameworks like blue socket, but that’s how we can at least test and branchmark.
> >
> > Sincerely
> > Michael
> >
> >
> >> On May 27, 2017, at 4:06 AM, Paulo Faria <paulo at zewo.io> wrote:
> >>
> >> Sorry if I'm being annoying, but I really feel what we lack is process. There's no formal proposal and proposal review. I really think we should move incrementally with well defined scopes and deadlines for every round. We didn't have that so far. Carl and others said that my suggestion is counter productive. I think the opposite, of course, as what I'm proposing is a well defined process where when we settle on a design for a particular set of base APIs then we move on to a higher absctraction. This way we won't be discussing the same things over and over again. I'll repeat, if we can't agree on the base types how can we move on? I'm saying let's first *settle* on Version, Headers, Message, Request and Response. Really *define* the API so then we move on, incrementally, from lower abstraction to higher abstraction.
> >>
> >>
> >>
> >> On May 27, 2017 07:53, "Paulo Faria" <paulo at zewo.io> wrote:
> >> I'm just proposing we move the code incrementally. The "most consentual" list I sent yesterday isn't radically different from the code Johannes proposed. I really don't want to discuss this over and over, ad infinitum. The most important in the messages I sent before is that we need a well defined process. We don't have it. Just taking the first implementation, with a lot of things that admitedly don't fit the scope, and adding that so we can rework doesn't feel right to me. I'm actually confused about the scope this project is taking. The code there explicitly mentions a WebApp, which is a higher responsibility than HTTP. When we started this project the scope as very clear. Crypto/TLS, Socket, HTTP. A well designed HTTP module shouldn't depend *at all* on the socket implementation. Providing an implementation would be just a matter of injecting a dependency. Moving that code as is to the org really doesn't feel right to me. All I'm saying is that we definitely should start having code on the org. But I say we move first Version, Headers, Message, Request, Response. And again, the "least controversial" I sent yesterday isn't radically strange. It's an evolution of Johaness original proposal, plus Carl's, plus Helge's suggestions, plus my suggestions. The only thing I added that wasn't discussed before is HTTPHeader.Field which does a case insensitive comparison in its equatable implementation, and the Message protocol which holds the properties common to request and response (version and headers). If we can't agree on that, which is the sum of everything that was discussed about these particular types. How can we agree on that full implementation?
> >>
> >> On May 27, 2017 05:13, "Michael Chiu" <hatsuneyuji at icloud.com> wrote:
> >> Hi Carl
> >>
> >>
> >> >       This email thread isn’t about an API proposal. It’s about a prototype implementation of an API that was already proposed and discussed a month and a half ago.  The prototype isn't a full-featured framework like Vapor or Kitura, but it does actually work and it even has XCTests with decent (>75%) code coverage.
> >>
> >> I see, I was confused by the email contents instead of reading the subject and thought we are finally implementing some code. TBH, I don’t see any reason why this should not move to swift-server on github, It sounds a good start to me.
> >> Thank you guys’ hard work for building it.
> >>
> >> >       Also, please note that I didn’t play any part in proposing this API back in March/April - it’s not “Carl’s proposal.”  I just took the existing API that the group had previously discussed and implemented enough of it so that we could measure the utility and performance of the API as proposed and so that we could have better informed discussions about potential alternatives.
> >>
> >> You’re right. it was Johannes’ proposal, I’m so sorry for that.
> >>
> >> Michael.
> >>
> >>
> >
> >
> > _______________________________________________
> > swift-server-dev mailing list
> > swift-server-dev at swift.org
> > https://lists.swift.org/mailman/listinfo/swift-server-dev
> 
> 
> 
> 
> 