[swift-build-dev] importing a *local* native dependency

[Drew Crawford]

> Our "Future Features" list does include "Support for Other Build Systems" (https://github.com/apple/swift-package-manager/blob/master/Documentation/PackageManagerCommunityProposal.md#support-for-other-build-systems <https://github.com/apple/swift-package-manager/blob/master/Documentation/PackageManagerCommunityProposal.md#support-for-other-build-systems>), so long-term I think I'd like to see us develop a proposal for that as the solution to your problem here.

If I'm not mistaken, one solution to this problem would be

* Add support for custom include paths
* Add a "preflight step" where a package can run a bash script as part of its build process
* Make the bash script run the external build system and point the include paths to a modulemap pointing to built products

I have an implementation + tests for all of these, plus I've used them to package libdispatch, in a private fork.  That solution exists today.

I'm not saying it's the best solution.  Would it make sense to sit down and spec out what is the "right way" to support an automake project like libdispatch?  I would be more than happy to reimplement this some other way, if someone can explain what way that is.

I'm trying to minimize the scope of what I'm going to maintain myself in my fork, but being able to link C and Rust dependencies is non-optional for me, so I just went ahead and did it.  I think it would be unfortunate if this feature is only available from me for awhile.

> Your second proposal, for custom include paths, seems really specific to supporting pre-built binaries

Based on the above, this seems plainly false.  I am using it to package libdispatch, which is a source package.  There may be good reasons not to do this, but I don't think this argument is one.

> On Jan 2, 2016, at 2:01 PM, Rick Ballard <rballard at apple.com> wrote:
> 
> Hi Drew,
> 
> This use case – facilitating packages that include pre-built binaries – is not one that we want to support right now. For the foreseeable future we'd really like to stick to source-based distribution, as discussed in https://github.com/apple/swift-package-manager/blob/master/Documentation/PackageManagerCommunityProposal.md#source-based-distribution <https://github.com/apple/swift-package-manager/blob/master/Documentation/PackageManagerCommunityProposal.md#source-based-distribution>.
> 
> Your proposal for dependencies on subpaths within a repo is one that we might consider at some point independent of this problem, but we'd need a better driving reason to do so. (We've had some discussions internally about whether to enforce that each package lives at the top level of a repo, vs supporting repos that contains multiple packages, and, at least for now, I think we'd like to have a repo per package). Your second proposal, for custom include paths, seems really specific to supporting pre-built binaries, so I don't think that aligns with our goals.
> 
> Our "Future Features" list does include "Support for Other Build Systems" (https://github.com/apple/swift-package-manager/blob/master/Documentation/PackageManagerCommunityProposal.md#support-for-other-build-systems <https://github.com/apple/swift-package-manager/blob/master/Documentation/PackageManagerCommunityProposal.md#support-for-other-build-systems>), so long-term I think I'd like to see us develop a proposal for that as the solution to your problem here.
> 
> So at the moment, I don't think I would want to accept a PR for either of these proposals. But I really appreciate your bringing this up.
> 
> Thank you,
> 
> 	- Rick
> 
>> On Dec 25, 2015, at 8:22 AM, Drew Crawford via swift-build-dev <swift-build-dev at swift.org <mailto:swift-build-dev at swift.org>> wrote:
>> 
>> Just to be clear...
>> 
>> If I PRed one or both of these, it would be rejected?
>> 
>> I'm going to cook up some solution in any case, it's a question of whether I should be gearing up to distribute and maintain a long term fork.  It's undesirable since it complicates upstreaming further work I do in my tree. 
>> 
>> But it will be unavoidable if I need a solution before you want to design it, which sounds like it may be the present situation. 
>> 
>> Sent from my iPhone
>> 
>> On Dec 25, 2015, at 9:36 AM, Max Howell <max.howell at apple.com <mailto:max.howell at apple.com>> wrote:
>> 
>>> This fulls under our "mixed targets" future plan.
>>> 
>>> For now I think you'll not be able to use swiftpm at all. Sorry :(
>>> 
>>> On Dec 25, 2015, at 3:19 AM, Drew Crawford via swift-build-dev <swift-build-dev at swift.org <mailto:swift-build-dev at swift.org>> wrote:
>>> 
>>>> I'm trying to package a Swift library.  This Swift library is tightly coupled to a native library.  By "tightly coupled" here I really mean that only half the library is written in Swift, and the other half, as an implementation detail, is in some other language.
>>>> 
>>>> You *probably* do not have a build environment for this other language just lying around, so my plan (actually, my current practice) is to distribute binaries of the non-Swift part, just checked into source control, and then link against them with a module map.  Yes, I realize this isn't portable, but neither is anything else in this project, so I'll save that particular surprise for another day.
>>>> 
>>>> The trouble is, swiftpm wants there to be some "other" repo called `CFoo` containing only a modulemap with paths that point to... /usr/local I guess? <https://github.com/apple/swift-package-manager/blob/master/Documentation/SystemModules.md>  But 
>>>> 
>>>> 1.  There is no other repo
>>>> 2.  You have not installed the non-Swift half of my library to /usr/local
>>>> 3.  You do not want to link against "some" version of the binary blob you have lying around from times past.  You want to link against exactly the one I give you, because the work is versioned as a whole.
>>>> 
>>>> I see two solutions to this problem.
>>>> 
>>>> # Non-URL dependencies
>>>> 
>>>> Behind this door, I can write a Package.swift like this:
>>>> 
>>>> let package = Package(
>>>>     name: "Foo",
>>>>     dependencies: [
>>>>         .Package(path: "CFoo"),
>>>>     ],
>>>> )
>>>> 
>>>> The semantic of this is that Foo depends on a package CFoo that is merely a folder in the current repository (e.g., we do not need to clone it from somewhere else).
>>>> 
>>>> In my case CFoo would contain a module.map and a binary blob, but in the general case it is any arbitrary kind of package.
>>>> 
>>>> This solves my three problems since 1) There's only one repo, 2) The binary is located with repo-based paths, 3) You're linking against the particular CFoo distributed in the repository.
>>>> 
>>>> # Custom include paths
>>>> 
>>>> Another possibility goes like this:
>>>> 
>>>> let package = Package(
>>>>     name: "Foo",
>>>>     extraIncludePaths: ["CFoo"]
>>>> )
>>>> 
>>>> The semantic of this is that we pass `-I CFoo` to swiftc (e.g. that is emitted into `other-args` of llbuild.yaml)
>>>> 
>>>> In my case CFoo contains a module.map and a binary blob, but in the general case it contains any kind of header path that a package may want to import.
>>>> 
>>>> There are certainly other possibilities besides these two, I'm just suggesting them because they seem straightforward based on my poking around in the sourcecode today.
>>>> 
>>>> I am happy to contrib either of these (or something else proposed) but would like to achieve consensus on a design instead of PRing by surprise.
>>>> 
>>>> _______________________________________________
>>>> swift-build-dev mailing list
>>>> swift-build-dev at swift.org <mailto:swift-build-dev at swift.org>
>>>> https://lists.swift.org/mailman/listinfo/swift-build-dev <https://lists.swift.org/mailman/listinfo/swift-build-dev>
>>  _______________________________________________
>> swift-build-dev mailing list
>> swift-build-dev at swift.org <mailto:swift-build-dev at swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-build-dev <https://lists.swift.org/mailman/listinfo/swift-build-dev>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-build-dev/attachments/20160102/fba134f5/attachment.html>