[swift-build-dev] [swiftpm] Add proposal for C language support

Sun Jan 3 02:17:43 CST 2016

> You keep referring to "'real' projects" as a proxy for the individual project you want support for; while there's a lot to be said for real-world use cases, I don't think this proposal's direction should be dictated by just libsodium.

Nothing about this is reductive or specific to libsodium.

* libdispatch has a build manual <https://github.com/apple/swift-corelibs-libdispatch/blob/master/INSTALL> that runs over 100 lines and involves checking out *7* other repos.
* Foundation (incl CoreFoundation) has a python-based build <https://github.com/apple/swift-corelibs-foundation/tree/master/lib> system that runs to 1900 lines
* libjpeg is "the" example of a C dependency in our documentation, and its build system includes such goodies as choosing a memory manager <http://libjpeg.cvs.sourceforge.net/viewvc/libjpeg/libjpeg/configure.ac?view=markup> or configuring libpng.

And these are just the projects that *we* are associated with!

> There are C projects that would benefit from modularization, header auditing, and cleanups that Swift and swiftpm would bring to it. 

Swift and swiftpm don't do anything of the kind.  C developers do.  All we can do is try to impose new requirements on C developers.  And who is volunteering to implement those requirements?

It seems to me that if our new requirements are so amazing, it should be easy to convince a few projects to sign on to repackage.  libdispatch and Foundation are *our* projects; the bar is so low we're practically cheating.  Are they going to switch to this as their build system?  I don't know who makes this decision, but it seems like an important question to ask.

> No Swift project could be built with swiftpm when it was introduced without repackaging. I don't see why C support should be held to a different standard.

Because we're designing a package manager for the Swift language, not the C language.  C has had build systems for decades.  We're not going to just waltz in with a new standard for a 44-year-old language and everybody switches the next day.  This is https://xkcd.com/927/ <https://xkcd.com/927/>.

> On Jan 2, 2016, at 8:11 PM, Zach Waldowski via swift-build-dev <swift-build-dev at swift.org> wrote:
> 
>> I think an important feature of any C target proposal is that there will actually exist C targets which can be built under the proposal.  Until there are C people coming out of the woodwork saying "sure, I will repackage my software this way" I think the entire value is debatable.
> 
>  
> I almost couldn't disagree more. No Swift project could be built with swiftpm when it was introduced without repackaging. I don't see why C support should be held to a different standard.
>  
>> And I do not see realistically how we are ever going to support a project like libsodium, except calling out to automake.
>  
> A potential solution (one of many possible) would look a lot like how people generate Xcode projects for C build systems today; hand-tuning config.h headers and such. I know many people who will go to ungodly lengths to avoid the inevitable nightmare automake causes in a source-distributed dependency.
>  
>> IMO something like that is a much, much better direction in the short-term, and once we have done the first step of "packaging" those software via automake we will have "real" C projects in our package manager and we can design our C support around the concerns of real projects instead of imaginary ones.
>  
> There are C projects that would benefit from modularization, header auditing, and cleanups that Swift and swiftpm would bring to it. C projects are massively disorganized because build systems are a ridiculous hodgepodge; we didn't be subject to that long tail of good and bad decisions.
>  
> I don't think automake support would be a silver bullet at all, and contradict with many goals of swiftpm and llbuild to boot. Even targeting a really small subset of automake projects what liberties would unnecessarily complicate the project, and then there'd be the projects it doesn't support.  (Oh? Wait? What version of the tools? Oh, from trunk? Oh, does the project take any liberties with its own organization? God help us when we start talking about C++…) 
>  
> "imaginary" is a reductive way of phrasing the problem space. You keep referring to "'real' projects" as a proxy for the individual project you want support for; while there's a lot to be said for real-world use cases, I don't think this proposal's direction should be dictated by just libsodium.
>  
> Zachary
>  
> On Sat, Jan 2, 2016, at 04:57 PM, Drew Crawford via swift-build-dev wrote:
>> Thanks for directing me to this, I missed it.
>>  
>>> Most projects will not conform to these conventions.
>>  
>> Giggle.  Kind of an understatement, no?
>>  
>> Like, okay.  Here is a project I'd like to package.  (Read: I do package it, with features not in mainline swiftPM.)  https://github.com/jedisct1/libsodium <https://github.com/jedisct1/libsodium>
>>  
>> Let's take a look at how this package realistically builds:
>>  
>> * It has tests ("make check")
>> * It has various --enable-foo flags
>> * It swaps in special implementations depending on if you have AMD64 <https://github.com/jedisct1/libsodium/blob/master/src/libsodium/Makefile.am#L162> or AVX instructions <https://github.com/jedisct1/libsodium/blob/master/src/libsodium/Makefile.am#L145> or SSE2 <https://github.com/jedisct1/libsodium/blob/master/src/libsodium/Makefile.am#L229> etc.
>> * The optimization level is tuned on a per-architecture basis <https://github.com/jedisct1/libsodium/blob/master/dist-build/android-armv7-a.sh#L3>
>> * They build (also) on Windows.  They're not changing how they're packaged for "SwiftPM, the Mac/Linux build system".
>> * Oh and this is cryptography code.  Do you *really* want to touch it?
>>  
>> I think an important feature of any C target proposal is that there will actually exist C targets which can be built under the proposal.  Until there are C people coming out of the woodwork saying "sure, I will repackage my software this way" I think the entire value is debatable.
>>  
>> Getting signoff from libdispatch/CoreFoundation is necessary but not sufficient to clear that hurdle.  I would think getting the other C deps in our own project family to repackage would be "table stakes" for any new C build system.  The real test are projects that are third-party and less friendly.
>>  
>> And I do not see realistically how we are ever going to support a project like libsodium, except calling out to automake.  An automake solution coincidentally supports both libdispatch and CoreFoundation right now.  IMO something like that is a much, much better direction in the short-term, and once we have done the first step of "packaging" those software via automake we will have "real" C projects in our package manager and we can design our C support around the concerns of real projects instead of imaginary ones.
>>  
>>> On Jan 2, 2016, at 11:00 AM, Daniel Dunbar via swift-build-dev <swift-build-dev at swift.org <mailto:swift-build-dev at swift.org>> wrote:
>>>  
>>> Happy 2016!
>>>  
>>> I am working on an initial proposal for adding support for C language targets to the Swift package manager, and am interested in feedback:
>>> https://github.com/ddunbar/swift-evolution/blob/master/proposals/NNNN-swiftpm-c-language-targets.md <https://github.com/ddunbar/swift-evolution/blob/master/proposals/NNNN-swiftpm-c-language-targets.md>
>>>  
>>> Some TL;DR:
>>> - The proposal defines a basic convention for pure C language targets (no Swift/C mix and match, but other Swift targets can use the C targets).
>>> - This is just intended to be the minimal initial feature, there will be a lot of add on work which I expect should be tackled in follow on PRs/proposals.
>>> - The proposal doesn't try and outline every single nitty detail (e.g., exactly what C++ standard we will compile with). My intention is to pick a sensible default at implementation time and refine incrementally.
>>>  
>>> Unless there are serious objections, I am hoping to hope to land this proposal soon and start work on the feature shortly after.
>>>  
>>> Cheers,
>>> - Daniel
>>>  
>>>  
>>> _______________________________________________
>>> swift-build-dev mailing list
>>> swift-build-dev at swift.org <mailto:swift-build-dev at swift.org>
>>> https://lists.swift.org/mailman/listinfo/swift-build-dev
>> 
>> 
>> _______________________________________________
>> swift-build-dev mailing list
>> swift-build-dev at swift.org <mailto:swift-build-dev at swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-build-dev <https://lists.swift.org/mailman/listinfo/swift-build-dev>
>  
> 
> _______________________________________________
> swift-build-dev mailing list
> swift-build-dev at swift.org
> https://lists.swift.org/mailman/listinfo/swift-build-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-build-dev/attachments/20160103/09167323/attachment.html>