[swift-server-dev] Prototype of the discussed HTTP API Spec

[Johannes Weiss]

Hi,

> On 31 May 2017, at 3:39 pm, Helge Heß via swift-server-dev <swift-server-dev at swift.org> wrote:
> 
> On 31 May 2017, at 16:07, Johannes Weiss <johannesweiss at apple.com> wrote:
>>> Sorry if this is a little off-topic, but I just wanted to mention that I think neither is really true.
>>> 
>>> I’m not completely sure, but I think the Node.js HTTP implementation actually runs on multiple threads, it is just the user level JavaScript stack that is single threaded (I think it sparks CPU-count threads). May be wrong on that ;-)
>> https://www.quora.com/Is-Node-js-single-threaded
> 
> Well, yes. This doesn’t discuss what I was talking about. What I think (but I may be wrong) is that Node.js (the kernel, not the JS API) may run multiple HTTP threads where sockets are accepted, headers parsed etc, before those are handed over to the JavaScript thread. Or in other words, where those parts run multi-threaded (accepting, HTTP header parsing).
> 
> This makes some sense because doing all those things also takes time.
> 
> I said, I may be completely wrong here, it would make sense though :-)

https://stackoverflow.com/questions/2387724/node-js-on-multi-core-machines/8685968#8685968

you spawn multiple Node.js processes to utilise more than one core. But not important at all. Just wanted to mention that async API doesn't require more than one thread.


>> well, the basic API is similar I think
> 
> … yeah sorry, but no, the basic API is nothing like it … You completely ignore that IncomingMessage and ServerResponse themselves are full streams and generic event emitters with everything attached.
> 
> Even IncomingMessage is nothing like our HTTPRequestHead - our head is just an inactive struct whereas the IncomingMessage is an actual object. Representing the read stream, and the headers, and even state whether the headers has been sent or not.
> I don’t think that this is necessarily a good design, but it is really different to what was proposed.
> 
>> and the IncomingMessage (HTTPRequestHead for us) has headers, http version and stuff but not the body.
> 
> IncomingMessage *is* the body stream. It is like
> 
>  class IncomingMessage : AsyncInputStream ( : ErrorEmitter : EventEmitter)
> 
>> Instead of returning a closure, you register a handler in the request à la
>> 
>> --- SNIP ---
>> request.on('data',function(message){
>>   response.write(message);
>> });
>> 
>> request.on('end',function(){
>>   response.end();
>> });
>> --- SNAP —
> 
> This is the old style ‘push’ stream API. And even that has more features than callbacks. In ‘modern’ (years old) Node you would do:
> 
>  request.on(‘readable’) {
>    let block = request.read();
>    … do stuff with it ...
>  }
> 
> but more usually you would pipe it somewhere,
> 
>  let file = fs.createWriteStream(“/tmp/filedump”)
>  request.pipe(file)
> 
> or request.pipe(transform(func() { … })).pipe(file)

this is really the bit that we don't support. I fully agree that the proposed API is less powerful than Node.js . All I was trying to say is that Node.js has APIs similar to what is proposed (amongst others). But as soon as we need to compose streams (as you do in the example just above), we're out of luck with the API proposed.

But again, we're then getting into the territory of the Networking/Transport group. It is an absolutely valid opinion to say we shouldn't spend time on HTTP alone without composable streams/channels/pipes.


> This is a great (and pretty quick) intro to streams:
> 
>  https://github.com/substack/stream-handbook
> 
> 
>> which is reasonably close to .. return .processBody ..
> 
> Not at all, it is a huge superset of that.
> 
>> don't you think? To me it looks quite straightforward to port a Node HTTP app to the proposal we're discussing.
> 
> Since it provides no streams nor piping I have a hard time seeing you port anything but the most trivial demo.

I was probably overstating that, you're right.


> But yes, it can be used as a *basis* to layer a Node-like framework on top. I.e. I could write a Noze.io stream which reads from the Swift HTTP API.
> 
> This is what we should focus on: Make sure that the stuff can be used as a basis for more advanced stuff.

Sounds good to me. I was mainly trying to propose something that's not totally out of scope for the HTTP group and for which we have an actual implementation. But eventually we will hopefully achieve a more composable general IO/networking library. But as you (I think) mentioned in one of the previous mails, it'll be even harder to design something to make everybody happy (enough) as that is more general & abstract...


>> But I do agree that we don't have streams. I thought about something like that but I thought that's too much just for HTTP. If we want to have a more general streaming mechanism, we should probably discuss that in the Transport/Networking meetings, no?
> 
> Well, HTTP sits on top of two byte streams. Which can be plain TCP sockets, or TLS, or a local domain socket, even UDP, why not read from files for testing purposes? In short, HTTP should technically sit on top of arbitrary streams.
> It’s not utterly important (nor necessarily better for performance), but it would make a nicer architecture.

agreed


>>> That is why I said since the beginning that I’m more interested in how the API for the body stream will look like (as this affects sockets as well as HTTP). Now we essentially get separate APIs for the same thing ...
>> 
>> I do agree with you here. But the networking group basically hasn't happened yet so I was proposing something here as we happened to have an implementation that works.
> 
> Sure. In a way that really is something for Foundation anyways, IMO.

I thought that is what the Networking/Transport group is for. But as you say it's generally IO really not necessarily only networking.


-- Johannes

>>> While I’d love to have a nice streaming framework like the one Node has built right into Swift, maybe just some extension of GCD channels would do (if you don’t want to go the full stream-everything approach Noze.io does), but I understand that this is a little out of scope and probably impossible to agree on ;-) Doesn’t matter too much as you can put adapters on top of the stuff.
>> 
>> 👍
> 
> hh
> 
> _______________________________________________
> swift-server-dev mailing list
> swift-server-dev at swift.org
> https://lists.swift.org/mailman/listinfo/swift-server-dev