[swift-server-dev] HTTP Parser

Cătălin Stan catalin.stan at me.com
Mon Nov 7 03:50:08 CST 2016 

Hi everyone,

Just wanted to add a couple of thoughts to the conversation.

We have set this ambitions deadline in mind for the first version of the API’s and, with that in sight, it is highly unlikely that we’ll be able to write a parser in Swift. Also it is a monumental task to rewrite the whole thing - with all it’s specs, edge cases, support for v1.1,  v2, byte-ranges, pipelining, multipart requests, a.s.o. - I know. I’ve done it. It sucks.

That being said, I cannot ignore the opportunity that we have at this point. Developing a parser from scratch, while working along side people that develop Foundation would allow us to also improve the language at the same time. Is there need for something similar to pointer arithmetic? Let’s make it. Does Foundation make too many data copy ops? Let’s make it so that it doesn’t. Is dispatch missing a kernel space component so that it can come close to Darwin in terms of speed? Let’s get that done as well. 

All in all, I think that it’s worth looking more towards the long term benefits, rather the somewhat immediate result of having it “now”. After all it’s not like we have to *convince* the swift community to adopt it. The need is already there.  Also, since everyone has already, to a larger or lesser extent, wrapped node’s parser around swift API’s, what’s the point of doing that again? Just to be able to say that this implementation “comes with swift”? I don’t believe it’s worth it. Again, this is just personal preference; I guess, I believe in doing things once, and not coming back to them over and over again. I digress. 

As as I said in the begging, I do understand the time constraints and the inherent value that comes with using a tried and tested solution thus allowing us to focus on the design of the developer facing APIs. All in all, I agree with Chris that the first thing should be to spec out what an “ideal world” version would contain, without all the constraints we have.

Apologies for the rant,

Catalin


> On 7 Nov 2016, at 02.59, Chris Bailey via swift-server-dev <swift-server-dev at swift.org> wrote:
> 
> To add some data into the debate... 
> 
> The C based http_parser used by Node.js (and original from NGINX) is approx 3K lines of code. If we think strictly in terms of effort to implement (and ignore maintenance costs or what's required to reach the same quality levels etc) then it shouldn't be an unreasonable amount of effort to do a Swift implementation - and in fact we could look at both approaches and see which is preferable (as both approaches have been taken by Swift frameworks already). 
> 
> The nghttp2 library however is approx 64K lines of code (looking only at the src directory). As such, its a much bigger task to implement, and to do so in a bug-free manner - note that the tests are another 17K lines of code. 
> 
> As an aside, wrapping the nghttp2 library is the approach being taken by Node.js for their HTTP/2 support. 
> 
> Chris
> 
> 
> 
> 
> From:        Tyler Cloutier via swift-server-dev <swift-server-dev at swift.org> 
> To:        Helge Heß <me at helgehess.eu> 
> Cc:        Swift Server Dev <swift-server-dev at swift.org> 
> Date:        06/11/2016 19:35 
> Subject:        Re: [swift-server-dev] HTTP Parser 
> Sent by:        swift-server-dev-bounces at swift.org 
> 
> 
> 
> I would agree that wrapping the nodejs http_parser is probably the way to go for now. There is a lot of places we could spend our time, but given that that is an extremely well tested program at this point, I don’t see the need to put any wood behind that arrow. Writing a Swift wrapper around it is not difficult and frees us up to work on other more pressing things. In fact many of us, have already done it <https://github.com/SwiftOnEdge/Edge/blob/master/Sources/HTTP/Parser.swift>. 
> 
> 
> Tyler 
> 
> 
> On Nov 4, 2016, at 2:18 PM, Helge Heß via swift-server-dev <swift-server-dev at swift.org <mailto:swift-server-dev at swift.org>> wrote: 
> 
> On 4 Nov 2016, at 21:20, Paulo Faria <paulo at zewo.io <mailto:paulo at zewo.io>> wrote: 
> Logan, would you be kind to explain why you think we shouldn’t touch C? 
> 
> Well, I think the rational is pretty clear. Staying within a single language has obvious benefits. Add to that safety features inherent to Swift designed solutions (buffer overruns and such) as well as the relative beauty of a modern language.
> 
> http_parser is a very C thing, it makes extensive use of pointers, goto’s and other C ‘tricks' to accomplish the performance it has. It is reasonably small and focused code but certainly not code which is straight forward to read&understand.
> 
> At the same time http_parser is a nice demo on how a high performance C core implementation is used in a high-level language environment (Node.js). The user of Node.js doesn’t have to deal with the C stuff at all.
> 
> Anyways I stick to my original opinion:
> —snip---
> Personally I’d suggest a small wrapper around this one: https://github.com/nodejs/http-parser <https://github.com/nodejs/http-parser>
> —snap—
> 
> But if someone has a great Swift parser providing the key features I’m interested in (push based, as zero copy as possible, reasonably fast), I’m quite interested too :->
> 
> hh
> 
> 
> My argument is that using a C library that we all know works well gives us time to work on more important things, like the user-facing API. You mentioned that we all like Swift because it’s safe, succinct and clean. I think we can all agree with that, but that statement doesn’t correlate to using a C lib. We wouldn’t be _implementing_ the parser in c, we would be _using_ a existing c parser. So the work would fall into what we’ll already have to do when dealing with C POSIX APIs, for example. Using a C lib would be one less thing to maintain. So again we can focus on creating safe, succinct and clean code where it really matters, the API level.
> 
> 
> On Nov 4, 2016, at 5:18 PM, Logan Wright via swift-server-dev <swift-server-dev at swift.org <mailto:swift-server-dev at swift.org>> wrote:
> 
> Helge, if I had my way, we wouldn't touch C in any way whatsoever for any of the server side libraries except as an absolute last resort, but this life is full of compromise, and I'm trying to be amicable to come up with solutions that we can agree on.
> 
> For HTTP2 as well, I'd like plans to eventually move to pure swift native implementations rethought for the language. 
> 
> On Fri, Nov 4, 2016 at 3:17 PM Helge Heß via swift-server-dev <swift-server-dev at swift.org <mailto:swift-server-dev at swift.org>> wrote:
> On 4 Nov 2016, at 20:11, Logan Wright via swift-server-dev <swift-server-dev at swift.org <mailto:swift-server-dev at swift.org>> wrote: 
> That said, even if we do end up doing that short term, I'd like to include plans or at least intentions to do things in Swift. I (and I'm sure many of us) generally prefer working in Swift because it's safe, succinct, and clean. We also have several instances of existing HTTP parsers we could pull from, it's not like we're starting from 0 as a group.
> 
> For HTTP2, I think everyone is in agreement that a c library is the way to go. I'd like to give HTTP a bit more time for opinions to come in. 
> 
> I’m not entirely sure why you have double standards here. Why would you come up with a safe, succinct and clean HTTP/1.x parser but not do the same for HTTP/2?
> 
> hh
> 
> _______________________________________________
> swift-server-dev mailing list
> swift-server-dev at swift.org <mailto:swift-server-dev at swift.org>
> https://lists.swift.org/mailman/listinfo/swift-server-dev <https://lists.swift.org/mailman/listinfo/swift-server-dev>
> _______________________________________________
> swift-server-dev mailing list
> swift-server-dev at swift.org
> https://lists.swift.org/mailman/listinfo/swift-server-dev <https://lists.swift.org/mailman/listinfo/swift-server-dev> 
> 
> 
> _______________________________________________
> swift-server-dev mailing list
> swift-server-dev at swift.org <mailto:swift-server-dev at swift.org>
> https://lists.swift.org/mailman/listinfo/swift-server-dev <https://lists.swift.org/mailman/listinfo/swift-server-dev> 
> _______________________________________________
> swift-server-dev mailing list
> swift-server-dev at swift.org
> https://lists.swift.org/mailman/listinfo/swift-server-dev <https://lists.swift.org/mailman/listinfo/swift-server-dev>
> 
> 
> _______________________________________________
> swift-server-dev mailing list
> swift-server-dev at swift.org
> https://lists.swift.org/mailman/listinfo/swift-server-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-server-dev/attachments/20161107/446eface/attachment.html>

More information about the swift-server-dev mailing list