[swift-build-dev] [Discussion Starter] Package post-clone script

[Daniel Dunbar]

My $0.02 here is that I do want to see us have some support for plugging in arbitrary build processes. However, I want it to be done in a very controlled manner, so that we are guaranteed we can evolve the package manager in sensible ways. I definitely would not want this to take the form of an arbitrary "post clone" script that we just run, with no other imposed structure.

I have a concept in mind, but have not had time to write it up.

I think Max and I agree that we don't want to support "scripts" just as a way to build things which we should have well-defined, proper ways to build in the first place. It would be nice to see how ffar we can get with our own features or adaption of existing technologies before adding this capability...

 - Daniel

> On Mar 21, 2016, at 11:38 AM, Honza Dvorsky via swift-build-dev <swift-build-dev at swift.org> wrote:
> 
> I personally don't like the idea of an arbitrary script either, it was just the first obvious solution to a problem I believe needs solving. The advantage of building from source even of these base dependencies is that the exact version of system dependencies
> - can be ensured to be the same version across platforms
> - their version is part of the explicit dependency graph
> 
> Currently, AFAIR, installing Redis with homebrew and apt-get each produce a different version, which IMO defeats the point of having a dependency manager in the first place. But I guess if your proposal could ensure that the same version is installed on all platforms, this particular problem would be solved.
> 
> > So the question becomes, is this sufficient? To have `brew install openssl` run, or do you really need to install from source?
> 
> I'm still not sure. Now I think your proposal will probably help with most cases (which is a very good way to approach things), but definitely not all. Imagine you yourself having a cross-platform C project which needs platform configuring before compilation. Even when you're completely in control, the fact that you still can't get this working is troubling. But maybe I'm just inflating extremely uncommon usecases.
> 
> One more thing - consider two Swift packages each needing a different version of OpenSSL. Does your proposal handle that case?
> 
> Anyway, I'd love to read your proposal and then re-evaluate whether I still feel like there's a large enough benefit in introducing such customization to SwiftPM :)
> 
> Honza
> 
> On Mon, Mar 21, 2016 at 6:19 PM Max Howell <max.howell at apple.com <mailto:max.howell at apple.com>> wrote:
> 
>> On Mar 19, 2016, at 10:55 AM, Honza Dvorsky via swift-build-dev <swift-build-dev at swift.org <mailto:swift-build-dev at swift.org>> wrote:
>> 
>> Hi All,
>> 
>> today I was trying to investigate how to build a proper cross-platform server-side Swift project and only use SwiftPM to do it. In particular, I was interested in getting OpenSSL/libcurl working as a dependency (built from source). The main reason is that currently the big Swift server frameworks all require manual steps during deployment (running apt-get/brew to install libcurl/openssl/others) and I am trying to get rid of these manual steps, eventually only requiring a `swift build` and run.
>> 
>> With the great initial work <https://github.com/apple/swift-package-manager/pull/183> to support building C-family packages in SwiftPM done by Ankit, I was able to have a simple C package as a dependency, as advertised. However, the reality of big cross-platform open source projects like OpenSSL and libcurl is that in order to actually build it from source, a configure (or similar) script has to be run to get the source code ready for compilation. And this is where I got stuck.
>> 
>> Thus I wanted to kick off a discussion of what approach of solving this problem with SwiftPM should be. The simple solution, which I'm tentatively proposing, is to have an optional "post-clone" script in the package's repository. Then, the Package.swift would optionally contain a field for the path to this script - and if present, it'd get run after SwiftPM clones this package. Note that this would be a "post-clone" script, not a "prebuild" script - I imagine it'd only run once when cloning and then only after each clean. 
>> 
>> In order to get users to stop having to run manual script to install all dependencies, I believe that we need to allow packages to declare what work needs to be done on its source before compilation can begin.
> 
> I have a proposal I’d like to push today that adds knowledge to SwiftPM about how to install system dependencies for System Module Packages.
> 
> We are not keen to add arbitrary script execution to SwiftPM, because: 1) Arbitrary scripts cannot be controlled and this leads to dependency hell and 2) arbitrary scripts mean your package graph may do anything, which nobody wants.
> 
> Now I’m fine with the “root package” ie. the package the user has control over doing more, but this would be a different discussion.
> 
> So the question becomes, is this sufficient? To have `brew install openssl` run, or do you really need to install from source?
> 
> Max
> 
> _______________________________________________
> swift-build-dev mailing list
> swift-build-dev at swift.org
> https://lists.swift.org/mailman/listinfo/swift-build-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-build-dev/attachments/20160322/a1190f37/attachment.html>