[swift-server-dev] Proposal for a standard Darwin/Glibc module

[Alex Blewitt]

In the near future it's likely that this project will need to have types that use C based functionality for representing sockets or other low-layer functionality. This has typically been implemented using a conditional include based on whether the code is on Linux or not to import Darwin or Glibc:

  #if os(Linux)
  import Glibc
  #else
  import Darwin
  #endif

This pattern exists in a number of different open source projects already, and it's a problem as new operating systems are added.

To that end, I've drafted a proposal that I intend to post to swift-evolution this weekend. If anyone would like to comment on this before I do, or add their names to the proposal as well, please let me know. I would like to send the draft off on Saturday, to give time for people to look over it before SwiftSummit convenes on Monday.

Arguably this is out of scope for Swift 3 phase 1, but I think it would be of benefit to this community to have a module that performs this functionality and we may be able to drive the requirements to include it in Swift 3 phase 2.

Comments welcome.

--- 8< ---

# Libc module for Swift

* Proposal: [SE-NNNN](NNNN-filename.md)
* Authors: [Alex Blewitt](https://github.com/alblue)
* Review Manager: TBD
* Status: **Under discussion**

## Introduction

When running on Darwin, the base module is called `Darwin`. When running
on Linux or other operating systems, it's called `GlibC`. 

This repeatedly leads to code such as:

    ````
    #if os(Linux)
      import Glibc
    #else
      import Darwin
    #endif
    ```

As the set of operating systems evolve, one of these conditional imports
needs to be updated. Instead of repeating this, make it available via a
standard `Libc` module in the base Swift library.

Swift-evolution thread: [Discussion thread topic for that proposal](https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161003/027621.html <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161003/027621.html>)

## Motivation

The [set of platforms](https://github.com/apple/swift/blob/fdf6ee20e4ca1fd32482f4b7b88a97ebdda52cd2/lib/Basic/LangOptions.cpp#L26-L36 <https://github.com/apple/swift/blob/fdf6ee20e4ca1fd32482f4b7b88a97ebdda52cd2/lib/Basic/LangOptions.cpp#L26-L36>)
that Swift currently runs on can be divided into two; Darwin and XNU based systems
(macOS, iOS, watchOS, tvOS), Windows, and Unix based systems
(Linux, FreeBSD, Android, PS4). 

The base module on Darwin is called `Darwin`, while on Linux and
other Unix systems the base module is called `Glibc`. The base
module is typically conditionally included when working at a lower layer
than Foundation (which has the same detail involved in importing the
base module).

As a result, conditionally importing the right version typically uses
a conditional test based on the operating system, and the same code is
seen in a number of different modules, both internal to Swift and external:

* [Test for mmap in stdlib](https://github.com/apple/swift/blob/07b196d2f9a5facc490b35e3649e18937796239b/test/stdlib/mmap.swift#L4-L9 <https://github.com/apple/swift/blob/07b196d2f9a5facc490b35e3649e18937796239b/test/stdlib/mmap.swift#L4-L9>)
* [Validation test for PassIfChildCrashedDuringTestExecution](https://github.com/apple/swift/blob/c3b7709a7c4789f1ad7249d357f69509fb8be731/validation-test/StdlibUnittest/ChildProcessShutdown/PassIfChildCrashedDuringTestExecution.swift#L4-L9 <https://github.com/apple/swift/blob/c3b7709a7c4789f1ad7249d357f69509fb8be731/validation-test/StdlibUnittest/ChildProcessShutdown/PassIfChildCrashedDuringTestExecution.swift#L4-L9>)
* [Kitura's Socket definitions](https://github.com/IBM-Swift/BlueSocket/blob/49c5af8b6953cecc8674a7fcf746fa27a72c056a/Sources/Socket.swift#L21-L25 <https://github.com/IBM-Swift/BlueSocket/blob/49c5af8b6953cecc8674a7fcf746fa27a72c056a/Sources/Socket.swift#L21-L25>)
* [Vapor's HTTP Server](https://github.com/vapor/engine/blob/1f95094ee470408309e98dd56b2251210d6a2a3d/Sources/HTTP/Models/Server/HTTP%2BServer.swift#L1-L5 <https://github.com/vapor/engine/blob/1f95094ee470408309e98dd56b2251210d6a2a3d/Sources/HTTP/Models/Server/HTTP+Server.swift#L1-L5>)

Some have already created a `Libc` module that effectively does what this
proposal suggests, such as [Vapor's Core Libc](https://github.com/vapor/core/blob/master/Sources/libc/libc.swift <https://github.com/vapor/core/blob/master/Sources/libc/libc.swift>)

    ```
    #if os(Linux)
      @_exported import Glibc
    #else
      @_exported import Darwin.C
    #endif
    ```

Each of these examples has subtly different behaviour; for example,
whether or not the os tests only include Linux (and then fail over to
Darwin), or whether they contain other Unices such as FreeBSD and Android.

## Proposed solution

The solution is to formalise these patterns in the base Swift library
and present a `Libc` module that conditionally imports `Glibc` or `Darwin`
based on the correct platform. Additional operating systems can be added
and kept up to date with the list of supported operating system conditionals
and including a failure message when an unknown operating system is detected.

## Detailed design

This will add a `Libc` module for the standard library that re-exports
the correct import depending on the operating system:

    ```
    #if os(macOS) || os(iOS) || os(tvOS) || os(watchOS)
      @_exported import Darwin
    #elseif os(Linux) || os(FreeBSD) || os(Android) || os(PS4)
      @_exported import Glibc
    #else
      fatalError("Libc not supported on operating system")
    #endif
    ```

As new operating systems are added or become supported (such as Windows)
the standard imports can be added appropriately to this module.

## Source compatibility

There is no impact to source compatibility, since this proposal is additive.
Existing source code will work regardless of if this module is used or not.
However it improves source compatibility going forwards, since as new
operating systems are added this file will be updated, instead of the change
having to be made in multiple open-source projects.

## Effect on ABI stability

There is no impact to ABI compatibility, since this proposal is additive.
Existing source code will work regardless of if this module is used or not.

## Effect on API resilience

There is no impact to ABI resilience, since this proposal is additive.
Existing source code will work regardless of if this module is used or not.

## Alternatives considered

The first alternative is to do nothing. Existing Swift projects already
conditionally import these modules, or import a higher-level module (such
as `Foundation`) that performs the conditional import.

The second alternative is to export sub-modules of the modules. Clang
permits imports of sub-modules, so it could be possible to import only
`Darwin.POSIX` and `GlibC.POSIX`. However, in Swift, importing a sub-module
makes the whole module available anyway, so the difference between importing
a whole module versus a submodule is irrelevant.

The third alternative is to explore creating standard functions (in Swift)
corresponding to POSIX functionality, but where the format of the return
results are known. This would require a per-operating system binding to
expose operating-system details such as the byte ordering of structures
as used in the various `getaddrinfo` calls. These may evolve out of future
evolution proposals and this does not conflict with those goals at this
stage. There are additional clean-ups that this could address, such as the
use of the (thread-local) `errno` which may not be reliably read from within
Swift. However, the (swift-evolution thread)[https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161003/027602.html]
calls this "the perfect being the enemy of the good". Instead of trying to
solve all of these problems, they should be handled by subsequent
proposals (such as (Johannes' proposal)[https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161031/028627.html]
regarding errno handling sent to swift-evolution previously).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-server-dev/attachments/20161103/7c04e429/attachment.html>