[swift-users] porting to musl (was: building static binaries / reducing library dependencies?)

[Joe Groff]

I can believe LLVM and Clang only build against glibc, but that's not necessarily a problem for compiled executables. Does target codegen for Linux have dependencies on glibc semantics? LLVM supposedly targets Darwin, BSD, glibc, and Windows, among other platforms, and as you noted people have gotten Rust and other LLVM-based languages to target musl already.

At the source level, it would be nice to standardize a portable POSIX module, though that of course is a fairly involved project.

-Joe

> On Dec 28, 2015, at 5:11 PM, Drew Crawford <drew at sealedabstract.com> wrote:
> 
> https://lists.swift.org/pipermail/swift-users/2015-December/000020.html <https://lists.swift.org/pipermail/swift-users/2015-December/000020.html>
> 
> (Sorry I don't have the reply link handy)
> 
>> Joe Groff wrote:
>>  Porting to musl libc might be interesting too, but I'm not sure how dependent the core libs are/will be on glibc stuff.
> 
> I'm going to do a braindump on this.  I spent several hours researching a musl port.  I've decided I won't actually do one, but my research may be useful to the next person who picks up the torch.
> 
> The real problem with musl is that the swift ecosystem presumes linux == glibc, and it presumes this very, very, very deeply and very, very, broadly.  Divorcing those two concepts is going to be very very radical at this stage, and this comes from a person who makes radical changes for fun.
> 
> # Ecosystem woes
> 
> Probably the biggest problem is the ecosystem.  What people are actually doing with their Swift modules they write is
> 
> #if os(Linux)
> import Glibc
> #else  
> import Darwin
> #endif
> 
> For example in this tutorial here <http://blog.krzyzanowskim.com/2015/12/04/swift-package-manager-and-linux-compatible/>.  So unless you want to prosecute this "linux is not glibc" patch across every project in the Swift ecosystem you're gonna have a bad time.  IMO there should be some better way to handle "pick a libc" in the package manager or something, but I'm not completely sure what it is.
> 
> # The problem started in LLVM
> 
> The module authors aren't stupid arbitrarily.  They have learned this "linux == glibc" philosophy from the swift package itself, and ultimately it even goes all the way to llvm:
> 
>  // On linux we have a weird situation. The stderr/out/in symbols are both
>  // macros and global variables because of standards requirements. So, we
>  // boldly use the EXPLICIT_SYMBOL macro without checking for a #define first.
> #if defined(__linux__) and !defined(__ANDROID__)
> 
> Whereas what we wanted was:
> 
> #if defined(__GLIBC__)
> 
> And the comment is not quite accurate either for musl, but I'm not enough of a language lawyer to be able to fix it.
> 
> Anyway, I have written this patch (attached) which does build llvm with musl, although it breaks glibc builds, so obviously it's not mergeable.  There are similar patches floating around, but they're out of date AFAIK.  This one is current.
> 
> # One does not simply
> 
> Research suggests that I am not the first to be vaguely interested in getting musl into the llvm family tree of projects/languages, based on the patches floating around for various components.  And some downstream people have actually managed to build various musl-hacked llvms and llvm-backed languages (Alpine's llvm, rust-musl).  So why can't we get it done upstream where it belongs?
> 
> Well, there is a requirements conflict:
> 
> llvm/clang (and I assume Swift) want some kind of strongish guarantee against borking the behavior on other non-musl systems that work just fine already, thank you
> musl refuses to implement #if __MUSL__ and calls the feature a bug <http://wiki.musl-libc.org/wiki/FAQ#Q:_why_is_there_no_MUSL_macro_.3F>.
> 
> These both seem reasonable independently but when you put them together it's kind of a heavy rock / immovable object situation.
> 
> # tl;dr
> 
> * swift at every level from third-party packages to low-level llvm assumes linux == glibc
> * a requirements conflict prevents those concepts from getting properly divorced in relevant upstream projects
> * solving this requires a tower of hacks even a crazy person like me can't stomach
> 
> As a result of all this, I have resigned myself to maintaining the glibc dependency for the forseeable future.
> 
> 
> <swift-llvm.patch>
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-users/attachments/20151228/30f126b4/attachment.html>