[swift-server-dev] swift-server-dev Digest, Vol 2, Issue 1

[Logan Wright]

Chris and others,

I'm excited that we're starting to break out into teams with respective
focus. I'd like to throw my hat in the ring for HTTP and WebSockets. I've
built Swift implementations of both protocols off of RFC from scratch and I
think I have much more context there vs the other core groups.

Looking forward to continuing working with everyone.

- Logan

On Tue, Nov 1, 2016 at 12:58 PM <swift-server-dev-request at swift.org> wrote:

> Send swift-server-dev mailing list submissions to
>         swift-server-dev at swift.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://lists.swift.org/mailman/listinfo/swift-server-dev
> or, via email, send a message with subject or body 'help' to
>         swift-server-dev-request at swift.org
>
> You can reach the person managing the list at
>         swift-server-dev-owner at swift.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of swift-server-dev digest..."
>
>
> Today's Topics:
>
>    1. Working on the new Internet Application Protocol? (Muse M)
>    2. Re: Working on the new Internet Application Protocol?
>       (Alex Blewitt)
>    3. Re: Dispatch + Blocking Calls (Jean-Daniel)
>    4. Re: Dispatch + Blocking Calls (Darren Mo)
>    5. Re: Posix Module (Helge Heß)
>    6. Re: Sockets API (Helge Heß)
>    7. Server APIs Project - next steps (Chris Bailey)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 1 Nov 2016 01:10:19 +0800
> From: Muse M <james.lei65 at gmail.com>
> To: swift-server-dev at swift.org
> Subject: [swift-server-dev] Working on the new Internet Application
>         Protocol?
> Message-ID:
>         <
> CAB-Y3Cjs0LK-Eq_86hK1u+W0mwJymJFGCNuqmxjWgBS6G6XVog at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Are there any plans to create new standard of IAP that design to reduce
> complexity and improve performance?
>
>
> http://tutorials.jenkov.com/iap/why-http2-and-websockets-are-not-enough.html#websockets-and-http2
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> https://lists.swift.org/pipermail/swift-server-dev/attachments/20161101/54192d76/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 2
> Date: Mon, 31 Oct 2016 17:21:48 +0000
> From: Alex Blewitt <alblue at apple.com>
> To: Muse M <james.lei65 at gmail.com>
> Cc: swift-server-dev <swift-server-dev at swift.org>
> Subject: Re: [swift-server-dev] Working on the new Internet
>         Application Protocol?
> Message-ID: <1EEF4EBC-5139-4F16-9FB0-B136C8F73B04 at apple.com>
> Content-Type: text/plain; charset="utf-8"
>
> On 31 Oct 2016, at 17:10, Muse M via swift-server-dev <
> swift-server-dev at swift.org> wrote:
> >
> > Are there any plans to create new standard of IAP that design to reduce
> complexity and improve performance?
> >
> >
> http://tutorials.jenkov.com/iap/why-http2-and-websockets-are-not-enough.html#websockets-and-http2
> <
> http://tutorials.jenkov.com/iap/why-http2-and-websockets-are-not-enough.html#websockets-and-http2
> >
>
> I think the initial focus is likely to be on those protocols that are
> currently being used by other languages and environments, to bring parity
> in the implementation. These will build upon low level sockets and/or TCP
> and UDP streams, which would be necessary to contemplate implementing
> higher level protocols that aren't derivatives of HTTP.
>
> Alex
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> https://lists.swift.org/pipermail/swift-server-dev/attachments/20161031/94eb6242/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 3
> Date: Mon, 31 Oct 2016 22:25:17 +0100
> From: Jean-Daniel <dev at xenonium.com>
> To: Darren Mo <darren.mo at me.com>
> Cc: swift-server-dev at swift.org
> Subject: Re: [swift-server-dev] Dispatch + Blocking Calls
> Message-ID: <66CD9AE0-B37A-43B6-9DF6-1C3EDB3FF994 at xenonium.com>
> Content-Type: text/plain; charset="utf-8"
>
>
> > Le 31 oct. 2016 à 09:21, Darren Mo via swift-server-dev <
> swift-server-dev at swift.org> a écrit :
> >
> > Something to keep in mind… once SR-2905 <
> https://bugs.swift.org/browse/SR-2905> is resolved, it will be feasible
> to write Linux server applications in a synchronous manner.
> >
> > How does this work? Dispatch work items are scheduled onto a pool of
> kernel threads. If a work item makes a blocking system call, the kernel
> thread will be blocked. Ideally, the work item scheduler will start a new
> kernel thread to take the place of the blocked thread. This is only
> possible if the scheduler can be triggered at the moment the blocking
> system call is made. A Linux kernel module is required for this to work in
> Swift.
> >
> > This ideal threading model is the main selling point of Go. Go
> developers can write their code in a synchronous manner without any
> performance tradeoff. (You might be wondering why Go does not need a kernel
> module. The reason is all the system calls go through the Go runtime before
> reaching the kernel.)
>
> The main benefit of GCD and worker threads is that as the threads are
> managed by the kernel, it can accommodate thread allocation and io
> scheduling by considering the system load as a whole, and not only the
> process load.
> I’m not familiar with go enough to know how the runtime cope with this,
> but from the description it look like it don't care about other processes
> when it managed goroutines.
>
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> https://lists.swift.org/pipermail/swift-server-dev/attachments/20161031/9091c27e/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 4
> Date: Mon, 31 Oct 2016 21:55:32 -0700
> From: Darren Mo <darren.mo at me.com>
> To: Jean-Daniel <dev at xenonium.com>
> Cc: swift-server-dev at swift.org
> Subject: Re: [swift-server-dev] Dispatch + Blocking Calls
> Message-ID: <0DEADE51-DE48-4728-8767-7BF5E57209F3 at me.com>
> Content-Type: text/plain; charset="utf-8"
>
> You are right that the Go runtime does not care about system load. I
> suppose it was designed for server applications running in Docker
> containers or virtual machines. In this world, system load does not matter
> because a chunk of system resources will be allocated exclusively to the
> application.
>
> > On Oct 31, 2016, at 2:25 PM, Jean-Daniel <dev at xenonium.com> wrote:
> >
> >
> >> Le 31 oct. 2016 à 09:21, Darren Mo via swift-server-dev <
> swift-server-dev at swift.org> a écrit :
> >>
> >> Something to keep in mind… once SR-2905 is resolved, it will be
> feasible to write Linux server applications in a synchronous manner.
> >>
> >> How does this work? Dispatch work items are scheduled onto a pool of
> kernel threads. If a work item makes a blocking system call, the kernel
> thread will be blocked. Ideally, the work item scheduler will start a new
> kernel thread to take the place of the blocked thread. This is only
> possible if the scheduler can be triggered at the moment the blocking
> system call is made. A Linux kernel module is required for this to work in
> Swift.
> >>
> >> This ideal threading model is the main selling point of Go. Go
> developers can write their code in a synchronous manner without any
> performance tradeoff. (You might be wondering why Go does not need a kernel
> module. The reason is all the system calls go through the Go runtime before
> reaching the kernel.)
> >
> > The main benefit of GCD and worker threads is that as the threads are
> managed by the kernel, it can accommodate thread allocation and io
> scheduling by considering the system load as a whole, and not only the
> process load.
> > I’m not familiar with go enough to know how the runtime cope with this,
> but from the description it look like it don't care about other processes
> when it managed goroutines.
> >
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> https://lists.swift.org/pipermail/swift-server-dev/attachments/20161031/c8887906/attachment-0001.html
> >
>
> ------------------------------
>
> Message: 5
> Date: Tue, 1 Nov 2016 11:41:31 +0100
> From: Helge Heß <me at helgehess.eu>
> To: Swift Server Dev <swift-server-dev at swift.org>
> Subject: Re: [swift-server-dev] Posix Module
> Message-ID: <1B21A3EA-AC6A-4B77-A312-6209D13AE5FC at helgehess.eu>
> Content-Type: text/plain; charset="utf-8"
>
> Hi Johannes,
>
> On 31 Oct 2016, at 10:36, Johannes Weiß <johannesweiss at apple.com> wrote:
> >> All I :-) ask for is that those Posix or C99 funds are exposed under a
> common name whether on Linux, Darwin, Windoze or BeOS.
> > Understood but that topic is already covered on swift-evolution.
>
> OK, great.
>
> > I'm more interested in making those functions usable correctly in Swift.
> Right now, "hard-to-use function" is an understatement IMHO
>
> IMHO you are exaggerating quite a bit, but OK :-)
>
> >> It probably is, so are sockets, encryption and database access :-) I
> think you need to start somewhere, and Swift Server is the one place which
> has to deal with this specific issue right now.
> >
> > Not sure if I fully agree. I see two areas here:
> >
> > 1) making the basic OS functionality available to Swift. IMHO that
> covers being able to call the OS's syscalls and being able to deal with the
> errors in a robust way. That for me belongs to swift-evolution but this
> work group could for example come up with proposals.
>
> OK.
>
> > 2) building libraries on top of these basic primitives. There's a bit
> more opinion involved here on how to do things. For example obvious
> questions about the programming model depending if based on Dispatch
> (DisptchIO or DispatchSource), one of the CSP libraries (eg. lib{dill,
> mill, venice}), blocking IO ;), or something else. But we should try to not
> bake too much opinion in these basic building blocks because that might
> make them incompatible with layers further up (like Zewo, Vapor, Kitura).
>
> I think I completely agree with this. As mentioned I also think that this
> means it would be so low level, it essentially is little more than 'making
> the basic OS functionality available to Swift’. Similar to what has been
> done to CoreGraphics or GCD APIs.
>
> Presumably something everyone is fighting with is TLS, primarily due to
> the ‘nice’ API of the relevant library :-) Having a nice Swift TLS API
> which doesn’t tie into specific transports would be really great (actually
> having one for C would be great too :-).
>
> > Also we have to consider here that right now DispatchIO/DispatchSource
> is probably a reasonable place to start as Dispatch comes with Swift and a
> Swift API. But when (in maybe 1.5+ years) Swift gains a memory and
> concurrency model, Dispatch might not be the best way of doing things
> anymore.
>
> I was wondering about this as well. It may not make sense to invest too
> much into modelling something bigger when such language changes are on the
> horizon?
> Don’t know, interested to see what people come up with :-)
>
> hh
>
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: signature.asc
> Type: application/pgp-signature
> Size: 842 bytes
> Desc: Message signed with OpenPGP using GPGMail
> URL: <
> https://lists.swift.org/pipermail/swift-server-dev/attachments/20161101/88d36fcc/attachment-0001.sig
> >
>
> ------------------------------
>
> Message: 6
> Date: Tue, 1 Nov 2016 11:48:58 +0100
> From: Helge Heß <me at helgehess.eu>
> To: Swift Server Dev <swift-server-dev at swift.org>
> Subject: Re: [swift-server-dev] Sockets API
> Message-ID: <F094D362-ECC4-40E4-A9ED-5593A17FFFE6 at helgehess.eu>
> Content-Type: text/plain; charset="utf-8"
>
> Hi Daniel,
>
> On 31 Oct 2016, at 02:55, Daniel A. Steffen <dsteffen at apple.com> wrote:
> > 2) dispatch_io: this is a asynchronous IO facility built on top of
> dispatch sources (and advisory reads for disk IO)
> >
> > I can easily believe that 2) might be too high level/abstracted and
> missing functionality for a server networking framework esp. around
> encryption (that is not what it was designed for)
>
> you are the second one mentioning issues with ‘encryption’. I think I may
> be missing something here :-) What is the issue with layering TLS on top of
> channels?
>
> Is it an actual issue with TLS/encryption itself or is all that ‘just'
> because integration with OpenSSL is harder?
>
> Thanks,
>   hh
>
> -------------- next part --------------
> A non-text attachment was scrubbed...
> Name: signature.asc
> Type: application/pgp-signature
> Size: 842 bytes
> Desc: Message signed with OpenPGP using GPGMail
> URL: <
> https://lists.swift.org/pipermail/swift-server-dev/attachments/20161101/b9c49d55/attachment-0001.sig
> >
>
> ------------------------------
>
> Message: 7
> Date: Tue, 1 Nov 2016 16:11:03 +0000
> From: Chris Bailey <BAILEYC at uk.ibm.com>
> To: swift-server-dev at swift.org
> Cc: swift-corelibs-dev at swift.org
> Subject: [swift-server-dev] Server APIs Project - next steps
> Message-ID:
>         <
> OFC54DB5E9.98F986FF-ON8025805E.00581636-8025805E.0058E6F0 at notes.na.collabserv.com
> >
>
> Content-Type: text/plain; charset="utf-8"
>
> Since announcing the Server APIs project and work group only one week ago
> we seen a considerable amount of early interest. There are now over 30
> people signed up in addition to the steering team, and some health
> discussion has started on the mailing list.
>
> One question that's been raised more than once is: Ok, there's a project
> page and a work group, but what happens next?
>
> The Server APIs Project pages outlined three initial focus areas: Base
> Networking, Security and Encryption, and HTTP and WebSockets. For each of
> these areas we'll be forming a sub-team whose initial task is to building
> a Swift Evolution "Pitch". This means collating requirements from the
> community around function, capability, and usage models, alongside
> evaluating available technologies like C libraries that could be used to
> help implement low level function, or existing Swift packages that ideas
> could be borrowed from, to form a high level sketch of the capabilities
> that the APIs would implement and an overview of how they would be
> implemented. Based on feedback from the community, the iterative process
> of prototyping and development can then begin. The Development Process is
> documented in more detail on the Server APIs Project page on Swift.org.
>
> Over the next few days we'll start to schedule kick-off meetings for each
> of the sub-teams. We'll be using Doodle polls to try to schedule a time
> that works for most participants. Obviously what ever day/time it ends up
> being, there will be some people that cannot make it - don't worry though,
> we'll be publishing the agenda (along with a request for agenda items) and
> a full set of minutes on GitHub.
>
>
> Chris
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> https://lists.swift.org/pipermail/swift-server-dev/attachments/20161101/6498680f/attachment-0001.html
> >
>
> ------------------------------
>
> _______________________________________________
> swift-server-dev mailing list
> swift-server-dev at swift.org
> https://lists.swift.org/mailman/listinfo/swift-server-dev
>
>
> End of swift-server-dev Digest, Vol 2, Issue 1
> **********************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-server-dev/attachments/20161102/10cf54cb/attachment.html>