[swift-server-dev] HTTP API v0.1.0

Thu Nov 2 17:24:30 CDT 2017

Hi,

> On 2 Nov 2017, at 2:02 pm, Helge Heß via swift-server-dev <swift-server-dev at swift.org> wrote:
> 
> On 2. Nov 2017, at 21:23, Johannes Weiß <johannesweiss at apple.com> wrote:
>>>> On 2. Nov 2017, at 18:31, Johannes Weiß <johannesweiss at apple.com> wrote:
>>>>> I think if we really want async, we need a few API adjustments to make async efficient enough. E.g. maybe pass queues around (probably not a straight DispatchQueue if we don’t want to tie it to GCD, but a context which ensures synchronization - that would be efficient for sync too).
>>>> 
>>>> do you have suggestions how that could look?
>>> 
>>> Not really. I guess it would be sufficient if the handler gets it, like so:
>>> 
>>> func echo(request: .., response: .., queue: …)
>>> 
>>> Though I was also wondering whether there should be a more general `WOContext` (ha) object which carries more details. Like a logging function to use, or other global (or HTTP transaction local) information.
>>> 
>>> But maybe that belongs into a higher level (and can be captured to the handler function).
>>> 
>>> What I would like to avoid is to make `queue:` a `queue: DispatchQueue`, but rather something like a simple
>>> 
>>> protocol SyncContext { func sync(_ cb: () -> ()) }
>>> 
>>> extension DispatchQueue {
>>>  func sync(_ cb: () -> ()) { async(execute: cb) }
>>> }
>>> 
>>> Synchronous servers would immediately callback to the caller.
>> 
>> interesting. In our internal implementation we have an abstraction which has an API really similar to DispatchIO and two implementations of that. One is synchronous and one is DispatchIO. And at some point I had one which was DispatchSources.
> 
> Öhm, and why can’t we just use your stuff? 😬
> 
>> And on these I do in fact have sync/async/notify (for DispatchGroup) methods. So basically the HTTPServer is generic over the IO mechanism it uses. And the IO mechanism has sync/async/notify methods that do the 'right' thing depending on if it's a sync or an async implementation.
> 
> Along your lines, I also have an ‘Express’ like framework for Noze streams (completely asynchronous) and for Apache (completely synchronous). I initially copied the Noze one and then modified it to work with Apache which was rather trivial. BUT: For async code you very often have to do a lot of extra hops which make them slower (but more scalable). Doing this very often makes zero sense for sync variants (they just make them slower). So I ended up with pretty different implementations, though the surface API still *looks* very similar:
> 
> Compare those two ‘route evals’:
> - async: https://github.com/NozeIO/Noze.io/blob/master/Sources/express/Route.swift#L75
> - sync: https://github.com/modswift/ExExpress/blob/develop/Sources/ExExpress/express/Route.swift#L261
> 
> The first one needs those ‘next’ escaping closures which share-capture iterator state and all that. The second is just a plain loop and ‘next’ a simple flag.
> 
> 
> I mentioned that a while back, but something I also like is this:
> 
>  protocol AsyncAPI {
>    func doIt(_ cb: @escaping blah)
>  }
>  protocol SyncAPI {
>    func doIt(_ cb: blah)
>  }
> And then depending on what you want:
>  typealias API = SyncAPI // = AsyncAPI
> 
> The removed @escaping has two significant benefits for the sync variant:
> - it is waaaaayyyyy faster
> - it makes it explicit that the callback _cannot_ escape,
>  it is guaranteed by the compiler (you cannot accidentally
>  use it in an async way)
> 
> 
> Summary: While I agree that it is possible to share an async API and a sync one, I think it quite often doesn’t make sense performance wise.

Agreed but whilst I would like to make a synchronous API possible (for legacy and 'special use' systems) I don't think we should try to optimise for it. Most of the Swift/Cocoa ecosystem on macOS/iOS is already asynchronous.

>>>> In our internal implementation I have bits of that but never got to the point to actually profiling stuff and I didn't go all the way.
>>> Channels vs source and then doing manual read/write? Well, my basic assumption on this is that even if channels are slower today, they should be made as fast. Conceptually that should work.
>> 
>> the only problems with the DispatchIO channels (you mean https://developer.apple.com/documentation/dispatch/dispatchio), right?
> 
> Yes
> 
>> is that they don't support back pressure directly.
> 
> Well, API-wise they have suspend/resume? Which is quite something if that actually works now.
> 
> Say if you pipe, you could suspend the read channel and in the write channel you wait for the done and resume the read channel. I think last time I tried this, it didn’t work on Linux or something.

I _think_ I've tried suspend/resume on the DispatchIO channels but I don't think it actually stopped the underlying abstraction to read bytes off the file descriptor. It merely stops delivering bytes to you. But I'm not 100% sure anymore.

> (Back-pressure is one of things I need to finish up in my async-imp)
> 
>>> I don’t remember what uv does, I think they are more like sources, but I’m not sure.
>> yes, DispatchSources are just an eventing mechanism really. Libuv and friends are quite similar there.
> 
> I once started to rework Noze to use libuv, but then didn’t have time or some Lego was more interesting ;-)
> But it is another reason to maybe not tie the `Queue` to a `DispatchQueue` (could be uv queue something).

👍, maybe we could make it generic over some 'context'?

>>> As mentioned, dispatch source has the little advantage (or not? I’m not convinced it is a good idea) that you can pass in those arbitrary buffer based objects. (And retain that original object).
>> 
>> yes, DispatchSources are way more versatile, I just went for DispatchIO because I was lazy ;). You do the read(v)/write(v)/... yourself with DispatchSources so there's a lot that you can do that DispatchIO doesn’t.
> 
> There are more reasons to use channels. For example a `channel.write(DispatchData)` hopefully does a `writev`.

it does not today (AFAIK) but it _really_ should. But it's open-source ;)

> We have no API to do this w/ sources.
> I wouldn’t write off channels so easily. They are like tiny little Noze streams :->
> 
> 
>>>> the base queues will end up on different (kernel) threads and the request queues will be round-robin scheduled onto the base queues. That way we make sure we don't randomly spawn new threads which isn't good.
>>> 
>>> I’m not quite sure what 'randomly spawn new threads’ means. To be honest I expect GCD to do the work you describe above. That is, assign new queues to the optimal number of hosting threads.
>> 
>> it tries but it really can't do it well and on Linux it's pretty terrible.
> 
> Just make it better then ;-)

well unfortunately certain things in GCD need kernel support to be implemented as efficiently as on Darwin. I don't know what the way forward on that front really is :(

>> The problem is that you need application knowledge to decide if it's better to spawn a new thread or not.
> 
> I’m not entirely convinced of that. If an app really needs an own thread, it can still create it and communicate with it.

right, that's not how GCD has been 'marketed' so far. If you want this behaviour I think the best way is to create those base queues which isn't too hard. I reckon in real-world software there's just too many places where something blocks even if that's discouraged. And kicking off another thread for all those places is probably not the programming model anyone really wants. I'm sure the GCD team would make certain decisions quite a bit differently today if they could but I can't speak for them at all.

> BTW: is there a way to assign a queue to an existing thread?

no

>> GCD does (on macOS not Linux) have an upper thread level but it's 64 / 512 depending on your setup by default:
>> 
>> $ sysctl kern.wq_max_threads kern.wq_max_constrained_threads
>> kern.wq_max_threads: 512
>> kern.wq_max_constrained_threads: 64
>> 
>> but let's assume you have 4 cores, so for many high-performance networking needs, it'd be useful if GCD never spawned more than 4 threads (or whatever the best value would be for your workload). However that depends on the application. If your application might sometimes block a thread (sure, not great but real world etc) then it will be good if GCD spawned a few more threads. And that's exactly what it does. It tries to spawn new threads when it thinks it would be good for your app. But entirely without actual knowledge what's good for you.
>> 
>> With the base queues you can provide that knowledge to GCD and it will do the right thing. You can totally have 100k queues if they all target a very small number of base queues. You really shouldn't have 100k base queues, especially on Linux.
> 
> I’m not entirely convinced by all that, but I admit it makes my head hurt :->
> 
> My takeaway is that base queues make sense at least today. Fair enough ;-)

that is my understanding but I'm not on the GCD team. And the importance of the queue hierarchy (ie. have very few base queues that other queues/sources target) has been stressed a lot in this year's WWDC, see https://developer.apple.com/videos/play/wwdc2017/706/

>>>>> c) Something like a), but with multiple worker queues. Kinda like the Node resolution, but w/o the different processes. This needs an API change, all the callbacks need get passed ‘their’ main queue (because it is not a global anymore).
>>>> 
>>>> Sorry, should've read the whole email before writing above. That sounds pretty much like what I wrote above, right? If you agree that sounds like the best model on GCD to me.
>>> 
>>> Yes. But unlike a) and b), this requires that the handler gets the queue it is running on, so that it can do:
>>> 
>>> func handler(req, res, queue httpQueue:…) {
>>>   bgQueue.async {
>>>     // very very very expensive work, like doing an Animoji
>>>     // done with it,
>>>     httpQueue.async(doneCallback)
>>>   }
>>> }
>>> 
>>> If you get the point.
>> 
>> yes, agreed.
> 
> Just to be clear, the above is no different to
> 
>  doneCallback = {
>    internalQueue.async {
>      // do the done work
>    }
>  }
> 
> (i.e. the server could pass in a `done` which synchronises itself, which is what I plan to do in my demo free threaded imp).
> 
> The gain is the common case where you don’t have that, but just call `done` straight from the same queue and do not dispatch somewhere else.
> 
> hh
> 
> _______________________________________________
> swift-server-dev mailing list
> swift-server-dev at swift.org
> https://lists.swift.org/mailman/listinfo/swift-server-dev