[swift-build-dev] [Discussion Starter] Package post-clone script

[Honza Dvorsky]

Thanks everyone, I agree with most things said here. Daniel, please let us
know when you have time to write up your proposed solution. As I'm trying
to get more serious with server-side Swift applications, the inability to
use external libraries like libCurl and OpenSSL without manual installation
are holding me back most at the moment.

I'll welcome any solution that gets us closer to solving that problem and
honestly, I'm not a fan of the arbitrary script solution either, so I'm
happy to see any realistic alternatives.

Honza
On Wed, Mar 23, 2016 at 12:05 AM Daniel Dunbar <daniel_dunbar at apple.com>
wrote:

> My $0.02 here is that I do want to see us have some support for plugging
> in arbitrary build processes. However, I want it to be done in a very
> controlled manner, so that we are guaranteed we can evolve the package
> manager in sensible ways. I definitely would not want this to take the form
> of an arbitrary "post clone" script that we just run, with no other imposed
> structure.
>
> I have a concept in mind, but have not had time to write it up.
>
> I think Max and I agree that we don't want to support "scripts" just as a
> way to build things which we should have well-defined, proper ways to build
> in the first place. It would be nice to see how ffar we can get with our
> own features or adaption of existing technologies before adding this
> capability...
>
>  - Daniel
>
> On Mar 21, 2016, at 11:38 AM, Honza Dvorsky via swift-build-dev <
> swift-build-dev at swift.org> wrote:
>
> I personally don't like the idea of an arbitrary script either, it was
> just the first obvious solution to a problem I believe needs solving. The
> advantage of building from source even of these base dependencies is that
> the exact version of system dependencies
> - can be ensured to be the same version across platforms
> - their version is part of the explicit dependency graph
>
> Currently, AFAIR, installing Redis with homebrew and apt-get each produce
> a different version, which IMO defeats the point of having a dependency
> manager in the first place. But I guess if your proposal could ensure that
> the same version is installed on all platforms, this particular problem
> would be solved.
>
> > So the question becomes, is this sufficient? To have `brew install
> openssl` run, or do you really need to install from source?
>
> I'm still not sure. Now I think your proposal will probably help with most
> cases (which is a very good way to approach things), but definitely not
> all. Imagine you yourself having a cross-platform C project which needs
> platform configuring before compilation. Even when you're completely in
> control, the fact that you still can't get this working is troubling. But
> maybe I'm just inflating extremely uncommon usecases.
>
> One more thing - consider two Swift packages each needing a different
> version of OpenSSL. Does your proposal handle that case?
>
> Anyway, I'd love to read your proposal and then re-evaluate whether I
> still feel like there's a large enough benefit in introducing such
> customization to SwiftPM :)
>
> Honza
>
> On Mon, Mar 21, 2016 at 6:19 PM Max Howell <max.howell at apple.com> wrote:
>
>>
>> On Mar 19, 2016, at 10:55 AM, Honza Dvorsky via swift-build-dev <
>> swift-build-dev at swift.org> wrote:
>>
>> Hi All,
>>
>> today I was trying to investigate how to build a proper cross-platform
>> server-side Swift project and only use SwiftPM to do it. In particular, I
>> was interested in getting OpenSSL/libcurl working as a dependency (built
>> from source). The main reason is that currently the big Swift server
>> frameworks all require manual steps during deployment (running apt-get/brew
>> to install libcurl/openssl/others) and I am trying to get rid of these
>> manual steps, eventually only requiring a `swift build` and run.
>>
>> With the great initial work
>> <https://github.com/apple/swift-package-manager/pull/183> to support
>> building C-family packages in SwiftPM done by Ankit, I was able to have a
>> simple C package as a dependency, as advertised. However, the reality of
>> big cross-platform open source projects like OpenSSL and libcurl is that in
>> order to actually build it from source, a configure (or similar) script has
>> to be run to get the source code ready for compilation. And this is where I
>> got stuck.
>>
>> Thus I wanted to kick off a discussion of what approach of solving this
>> problem with SwiftPM should be. The simple solution, which I'm tentatively
>> proposing, is to have an optional "post-clone" script in the package's
>> repository. Then, the Package.swift would optionally contain a field for
>> the path to this script - and if present, it'd get run after SwiftPM clones
>> this package. Note that this would be a "post-clone" script, not a
>> "prebuild" script - I imagine it'd only run once when cloning and then only
>> after each clean.
>>
>> In order to get users to stop having to run manual script to install all
>> dependencies, I believe that we need to allow packages to declare what work
>> needs to be done on its source before compilation can begin.
>>
>>
>> I have a proposal I’d like to push today that adds knowledge to SwiftPM
>> about how to install system dependencies for System Module Packages.
>>
>> We are not keen to add arbitrary script execution to SwiftPM, because: 1)
>> Arbitrary scripts cannot be controlled and this leads to dependency hell
>> and 2) arbitrary scripts mean your package graph may do anything, which
>> nobody wants.
>>
>> Now I’m fine with the “root package” ie. the package the user has control
>> over doing more, but this would be a different discussion.
>>
>> So the question becomes, is this sufficient? To have `brew install
>> openssl` run, or do you really need to install from source?
>>
>> Max
>>
>> _______________________________________________
> swift-build-dev mailing list
> swift-build-dev at swift.org
> https://lists.swift.org/mailman/listinfo/swift-build-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-build-dev/attachments/20160323/06956fe8/attachment.html>