[swift-evolution] [Out of scope] Discussion on general Darwin/GlibC module

Alexis abeingessner at apple.com
Fri Nov 11 11:09:31 CST 2016 

I agree that trying to completely unify low-level platforms is usually a mess. That said, I also don’t think accessing platform specific behaviour needs to involve completely throwing away the nice abstractions in Foundation. Wherever possible, we should provide platform-specific extensions to the types in Foundation. For instance, we could expose methods/inits that operate in terms of file descriptors on unix-y systems, and handle_t on windows.

But I also think there should be some opt-in to doing this, so that Foundation users can be confident they’re writing portable software by default. I don’t think imports should be the mechanism for this because this necessarily forces awkward divisions. I’m cautiously optimistic the feature flag system we need to build out for language evolution purposes will provide a good fit here. Opting into platform-specific behaviour is fairly similar to opting into experimental APIs. 

(Note: I haven’t actually used Foundation much, so this may be inconsistent with its overarching design)

> On Nov 10, 2016, at 10:48 PM, Drew Crawford via swift-evolution <swift-evolution at swift.org> wrote:
> 
>     grep -R "import Glibc" ~/Code --include "*.swift" | wc -l
>     297
> 
> As someone who might be characterized as suffering from the problem this proposal purports to solve, I am not convinced.
> 
> The primary problem here is that "libc" is a misnomer.  Did you mean musl, dietlibc, or glibc?  Did you mean "whatever libc my distro likes?"  Swift in practice only supports one per platform, but that is a bug not a feature, and that bug should not be standardized.  We could try to invent some syntax to specify one but now we are back with the current system again.
> 
> The other problem is that in all my usages, "import Glibc" is not a real problem I face.  The real problems are that "the libcs plural" are *just different*.  Darwin has timeval64, glibc does not, and you'd better check your arch and pick the right one, only on one platform.  SO_REUSEADDR has one type in Brand X and another type in Brand Y.  Don't even get me *started* on poll, EREs, or half a dozen other behavioral variations.  
> 
> Taking two different libraries and pretending they are the same is not the solution, it's the disease.  The way out of this swamp for most developers is to use a real Swift library, the same damn Swift library, on all platforms (sadly, Foundation today does not meet this requirement).  The way out of this swamp for crazy people like me who must write to the metal is to actually write to the metal, to the particular libc being targeted, not to a hypothetical platonic ideal libc which does not exist.  
> 
> I realize that four lines at the top of my files is a *visible* annoyance, but fixing it just promotes it to an invisible one. 
> 
> Drew
> 
> --
>   Drew Crawford
>   drew at sealedabstract.com
> 
> 
> 
> On Wed, Nov 9, 2016, at 12:58 PM, Alex Blewitt via swift-evolution wrote:
>> Although out of scope for phase 1, something that keeps cropping up in a variety of Linux/Darwin Swift scripts is the conditional inclusion of Darwin or GlibC per platform. The last point was an observation that creating a 'nice' wrapper for LibC or a cleaned up POSIX API is a non-goal:
>> 
>> https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161003/027621.html <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161003/027621.html>
>> 
>>> I think it makes sense to have a cross platform “libc” which is an alias for darwin, glibc, or whatever, and just leave it at that.
>>> 
>>> Other proposals for a “POSIX” module have gotten bogged down because inevitably the idea comes up to make the resultant API nicer in various ways: rename creat, handle errno more nicely, make use of multiple return values, … etc.  The problem with this approach is that we don’t *want* people using these layer of APIs, we want higher level Foundation-like APIs to be used.
>>> 
>>> ...
>>> 
>>> I think we should formally decide that a “nice” wrapper for libc is a non-goal.  There is too much that doesn’t make sense to wrap at this level - the only Swift code that should be using this is the implementation of higher level API, and such extremely narrow cases that we can live with them having to handle the problems of dealing with the raw APIs directly.
>>> 
>>> -Chris
>> 
>> I have created a draft for a proposal to create such a module. Comments are welcome.
>> 
>> Alex
>> 
>> ---
>> 
>> # Libc module for Swift
>> 
>> * Proposal: [SE-NNNN](NNNN-filename.md)
>> * Authors: [Alex Blewitt](https://github.com/alblue <https://github.com/alblue>)
>> * Review Manager: TBD
>> * Status: **Under discussion**
>> 
>> ## Introduction
>> 
>> When running on Darwin, the base module is called `Darwin`. When running
>> on Linux or other operating systems, it's called `GlibC`. 
>> 
>> This repeatedly leads to code such as:
>> 
>>     ````
>>     #if os(Linux)
>>       import Glibc
>>     #else
>>       import Darwin
>>     #endif
>>     ```
>> 
>> As the set of operating systems evolve, one of these conditional imports
>> needs to be updated. Instead of repeating this, make it available via a
>> standard `Libc` module in the base Swift library.
>> 
>> Swift-evolution thread: [Discussion thread topic for that proposal](https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161003/027621.html <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161003/027621.html>)
>> 
>> ## Motivation
>> 
>> The [set of platforms](https://github.com/apple/swift/blob/fdf6ee20e4ca1fd32482f4b7b88a97ebdda52cd2/lib/Basic/LangOptions.cpp#L26-L36 <https://github.com/apple/swift/blob/fdf6ee20e4ca1fd32482f4b7b88a97ebdda52cd2/lib/Basic/LangOptions.cpp#L26-L36>)
>> that Swift currently runs on can be divided into two; Darwin and XNU based systems
>> (macOS, iOS, watchOS, tvOS), Windows, and Unix based systems
>> (Linux, FreeBSD, Android, PS4). 
>> 
>> The base module on Darwin is called `Darwin`, while on Linux and
>> other Unix systems the base module is called `Glibc`. The base
>> module is typically conditionally included when working at a lower layer
>> than Foundation (which has the same detail involved in importing the
>> base module).
>> 
>> As a result, conditionally importing the right version typically uses
>> a conditional test based on the operating system, and the same code is
>> seen in a number of different modules, both internal to Swift and external:
>> 
>> * [Test for mmap in stdlib](https://github.com/apple/swift/blob/07b196d2f9a5facc490b35e3649e18937796239b/test/stdlib/mmap.swift#L4-L9 <https://github.com/apple/swift/blob/07b196d2f9a5facc490b35e3649e18937796239b/test/stdlib/mmap.swift#L4-L9>)
>> * [Validation test for PassIfChildCrashedDuringTestExecution](https://github.com/apple/swift/blob/c3b7709a7c4789f1ad7249d357f69509fb8be731/validation-test/StdlibUnittest/ChildProcessShutdown/PassIfChildCrashedDuringTestExecution.swift#L4-L9 <https://github.com/apple/swift/blob/c3b7709a7c4789f1ad7249d357f69509fb8be731/validation-test/StdlibUnittest/ChildProcessShutdown/PassIfChildCrashedDuringTestExecution.swift#L4-L9>)
>> * [Kitura's Socket definitions](https://github.com/IBM-Swift/BlueSocket/blob/49c5af8b6953cecc8674a7fcf746fa27a72c056a/Sources/Socket.swift#L21-L25 <https://github.com/IBM-Swift/BlueSocket/blob/49c5af8b6953cecc8674a7fcf746fa27a72c056a/Sources/Socket.swift#L21-L25>)
>> * [Vapor's HTTP Server](https://github.com/vapor/engine/blob/1f95094ee470408309e98dd56b2251210d6a2a3d/Sources/HTTP/Models/Server/HTTP%2BServer.swift#L1-L5 <https://github.com/vapor/engine/blob/1f95094ee470408309e98dd56b2251210d6a2a3d/Sources/HTTP/Models/Server/HTTP%2BServer.swift#L1-L5>)
>> 
>> Some have already created a `Libc` module that effectively does what this
>> proposal suggests, such as [Vapor's Core Libc](https://github.com/vapor/core/blob/master/Sources/libc/libc.swift <https://github.com/vapor/core/blob/master/Sources/libc/libc.swift>)
>> 
>>     ```
>>     #if os(Linux)
>>       @_exported import Glibc
>>     #else
>>       @_exported import Darwin.C
>>     #endif
>>     ```
>> 
>> Each of these examples has subtly different behaviour; for example,
>> whether or not the os tests only include Linux (and then fail over to
>> Darwin), or whether they contain other Unices such as FreeBSD and Android.
>> 
>> ## Proposed solution
>> 
>> The solution is to formalise these patterns in the base Swift library
>> and present a `Libc` module that conditionally imports `Glibc` or `Darwin`
>> based on the correct platform. Additional operating systems can be added
>> and kept up to date with the list of supported operating system conditionals
>> and including a failure message when an unknown operating system is detected.
>> 
>> ## Detailed design
>> 
>> This will add a `Libc` module for the standard library that re-exports
>> the correct import depending on the operating system:
>> 
>>     ```
>>     #if os(macOS) || os(iOS) || os(tvOS) || os(watchOS)
>>       @_exported import Darwin
>>     #elseif os(Linux) || os(FreeBSD) || os(Android) || os(PS4)
>>       @_exported import Glibc
>>     #else
>>       fatalError("Libc not supported on operating system")
>>     #endif
>>     ```
>> 
>> As new operating systems are added or become supported (such as Windows)
>> the standard imports can be added appropriately to this module.
>> 
>> ## Source compatibility
>> 
>> There is no impact to source compatibility, since this proposal is additive.
>> Existing source code will work regardless of if this module is used or not.
>> However it improves source compatibility going forwards, since as new
>> operating systems are added this file will be updated, instead of the change
>> having to be made in multiple open-source projects.
>> 
>> ## Effect on ABI stability
>> 
>> There is no impact to ABI compatibility, since this proposal is additive.
>> Existing source code will work regardless of if this module is used or not.
>> 
>> ## Effect on API resilience
>> 
>> There is no impact to ABI resilience, since this proposal is additive.
>> Existing source code will work regardless of if this module is used or not.
>> 
>> ## Alternatives considered
>> 
>> The first alternative is to do nothing. Existing Swift projects already
>> conditionally import these modules, or import a higher-level module (such
>> as `Foundation`) that performs the conditional import.
>> 
>> The second alternative is to export sub-modules of the modules. Clang
>> permits imports of sub-modules, so it could be possible to import only
>> `Darwin.POSIX` and `GlibC.POSIX`. However, in Swift, importing a sub-module
>> makes the whole module available anyway, so the difference between importing
>> a whole module versus a submodule is irrelevant.
>> 
>> The third alternative is to explore creating standard functions (in Swift)
>> corresponding to POSIX functionality, but where the format of the return
>> results are known. This would require a per-operating system binding to
>> expose operating-system details such as the byte ordering of structures
>> as used in the various `getaddrinfo` calls. These may evolve out of future
>> evolution proposals and this does not conflict with those goals at this
>> stage. There are additional clean-ups that this could address, such as the
>> use of the (thread-local) `errno` which may not be reliably read from within
>> Swift. However, the (swift-evolution thread)[https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161003/027602.html <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161003/027602.html>]
>> calls this "the perfect being the enemy of the good". Instead of trying to
>> solve all of these problems, they should be handled by subsequent
>> proposals (such as (Johannes' proposal)[https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161031/028627.html <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161031/028627.html>]
>> regarding errno handling sent to swift-evolution previously).
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
> 
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20161111/7d26f705/attachment.html>

More information about the swift-evolution mailing list