[swift-corelibs-dev] libdispatch roadmap and api addition proposal

[Pierre Habouzit]

-Pierre

> On Dec 10, 2015, at 12:42 AM, Joakim Hassila via swift-corelibs-dev <swift-corelibs-dev at swift.org> wrote:
> 
> Hi,
> 
>> On 8 dec. 2015, at 16:56, Pierre Habouzit <pierre at habouzit.net> wrote:
>> 
>> FWIW, this is my personal, let’s call it enlightened, opinion, based on my knowledge of dispatch and my past extensive system programming experience with Linux before I joined Apple.
>> 
>> I think that long term, the best way to maintain a Linux libdispatch port is to go away from the libkqueue that tries to emulate kqueue fully, where dispatch only needs a small subset of the surface of kqueue. Given how source.c is written today, this is not a very small undertaking, but eventually dispatch source map to epoll_ctl(EPOLLONESHOT) very very well.
> 
> That makes sense, could simplify the implementation (and keep thing cleaner). Then the follow up question is of course how to split/manage source.c (as Daniel pointed out there is the merging issue).

we can decide when/if someone tries to tackle it. I humbly recognize that I have no great idea of how to do so.

> 
>> Given our experience with the work queue subsystem in Darwin, I think that it would make sense to integrate both projects together, as work queue are not that useful if you don’t have dispatch with it, and having it separate gives you all the woes of a stable interface, which you don’t really care for in the first place. It’s probably much better to integrate it and not care about backward and forward compatibility and make it a private library of dispatch on linux. And to not be tied to a given interface at all.
> 
> Agree, I don’t see much use for pwq except in this support role, so there would be a large degree of freedom.
> 
>> I also think that having a minimal kernel support for thread pool management isn’t that hard to write as a kernel module, I had started to work on this a very long time ago, using the KVM scheduling hooks that let you know when a thread blocks and/or becomes runnable[1]. Threads would declare to that interface that they are work queue threads, and get load information that the thread pool can use to regulate. It’s old code, maybe (probably?) not the right way to do it, but that’s an example of things you can do if you move away from the contrived interface from what libpthread_workqueue exposes. My idea required a linux adjustment that I posted to the LKML at the time (http://lkml.iu.edu/hypermail/linux/kernel/1112.2/00235.html) not sure if it ever made it to mainline (looks like it didn’t).
>> 
>> 
>> [1] http://git.madism.org/?p=~madcoder/pwqr.git;a=blob;f=kernel/pwqr.c;h=6d822ea6bca40a2ba8de3965526f33b4a98b8649;hb=HEAD
> 
> That would actually be very nice to be able to regulate on a system level just as on Darwin.
> 
> On a conceptual level, it would probably make sense to consider also non-work queue threads for regulation purposes (it is what is being done on the user level right now in pwq: simply doing thread introspection of /proc when needed) - then it plays more nicely in a mixed environment where there are legacy threads doing work.
> 
> Joakim
> 
> 
> ________________________________
> 
> This e-mail is confidential and may contain legally privileged information. It is intended only for the addressees. If you have received this e-mail in error, kindly notify us immediately by telephone or e-mail and delete the message from your system.
> _______________________________________________
> swift-corelibs-dev mailing list
> swift-corelibs-dev at swift.org
> https://lists.swift.org/mailman/listinfo/swift-corelibs-dev