[swift-build-dev] Advice on using SwiftPM for developing large projects

Thu Nov 30 12:32:13 CST 2017

I posted earlier this week about tool I wrote for managing multiple repositories that could possibly help, at least for the short-term: https://github.com/JrGoodle/clowder <https://github.com/JrGoodle/clowder>

Kind of like Ankit suggested, you could create a root repo containing the Package config for the multiple products, and specify relative locations for all the various repos to be cloned in under the root repo. The example LLVM projects clowder.yaml config file has repo's nested in other ones (you'd just need to be careful with the clean commands because some options would delete the nested repo if you tell it to delete files ignored by git). https://github.com/JrGoodle/llvm-projects/blob/master/clowder.yaml <https://github.com/JrGoodle/llvm-projects/blob/master/clowder.yaml>

I didn't know about the top of tree development mode, but that probably seems like the way to go, and I think clowder would combine nicely with it as well. It would probably be useful for daily development, since you can have the repos all track branches and pull all the changes in one command. Or you can specify tags/commits to pin to as well.

I just got started with SwiftPM recently, so I haven't tried this out yet, but I'm gonna give it a go with some personal projects I've been working on.

P.S. Sorry for the blatant "marketing" of a personal project, but I think it's pretty useful and am dying for some outside feedback :)

> On Nov 30, 2017, at 11:10 AM, Ankit Aggarwal via swift-build-dev <swift-build-dev at swift.org> wrote:
> 
> 
> 
> On Thu, Nov 30, 2017 at 8:37 AM, Geordie J via swift-build-dev <swift-build-dev at swift.org <mailto:swift-build-dev at swift.org>> wrote:
> Hello Swift Build Devs!
> 
> I filed a bug to track progress of the proposed Multi-Package Repo enhancement to Swift Package Manager and was referred to the mailing list (thanks to Jordan Rose for the tip!)
> 
> We have a reasonably complex project consisting of five or so separate modules. For iOS we had been managing this with various git submodules containing Xcode projects. This worked great, but doesn’t gel with SwiftPM: making a commit (until relatively recently a tag as well), pushing to origin, waiting for SPM to download everything that is already on the local machine, seeing that the entire thing doesn’t even compile, and repeating, doesn’t work for our daily active development.
> 
> The only viable workaround for this has been to put all of the packages in editable mode. But this has proven to be very frustrating too: I have spent literally weeks fighting against changes made as new SPM versions are released with different quirks that each time break how or whether that workaround works. In that time I’m sure we could have gone a long way to developing and releasing a fix instead.
> 
> In Swift 4, we introduced a top of the tree development mode, which is basically edit mode but it uses the existing sources (if present). Have you tried it? If yes, I would love to hear about the issues you're having in using it.
>  
> 
> Multi-package repos certainly sound like the solution we’re looking for. It was proposed over a year ago here: https://github.com/ddunbar/swift-evolution/blob/multi-package-repos/proposals/NNNN-swiftpm-multi-package-repos.md <https://github.com/ddunbar/swift-evolution/blob/multi-package-repos/proposals/NNNN-swiftpm-multi-package-repos.md>. I guess the reason I didn’t consider working on this proposal six months ago was I'd understood that a) the proposal had been accepted in some way, and b) that active work was being done on it already.
> 
> I’m not sure if either of those are true today. So as an aside/meta-question, does anybody know of a resource that tracks progress of evolution proposals?
> 
> We're actively working on a proposal for multi-package repositories and we will email it to this list once a draft is ready!
> 
> 
> I did find this commit by Ankit Aggarwal: https://github.com/apple/swift-package-manager/commit/1ff987c64c00e9dc60e040f6e960602c84eede5e <https://github.com/apple/swift-package-manager/commit/1ff987c64c00e9dc60e040f6e960602c84eede5e> which seems to tantalisingly point in the right direction, via the isLocal property. I couldn’t see any way of using it though.
> 
> The isLocal property is an implementation detail and is not expected to be modified by the user.
>  
> 
> So I’m looking for input as to whether we as individuals or as a community can do anything to push progress on this proposal: whether there is another way around it, or what else we could do? 
> 
> 
> Once a draft is out, we can discuss it on this list and then put it up for review on swift-evolution. In general, we would love to see more proposals from the community but in this case, we already have something that we want to propose.
>  
> At this point I’d even be overjoyed with a solution that just spits out the clang / swiftc / linker calls made by `swift build` so we could write our own script and avoid re-working-around changes every release or so. We really don’t need any management of dependencies at all, just building.
> 
> 
> One option could be consolidating all the submodules into one repository and then declaring multiple products in it. It would be similar to mono repository but then you can't use individual packages as a dependency. Another option could be creating a package structure and adding symlinks to the different repositories (which could be checked out relative to the package). 
> 
> We also have a slack channel <https://lists.swift.org/pipermail/swift-build-dev/Week-of-Mon-20160530/000497.html> if you want to chat about some more workarounds.
>  
> Cheers and thank you in advance,
> Geordie
> 
> 
> _______________________________________________
> swift-build-dev mailing list
> swift-build-dev at swift.org <mailto:swift-build-dev at swift.org>
> https://lists.swift.org/mailman/listinfo/swift-build-dev <https://lists.swift.org/mailman/listinfo/swift-build-dev>
> 
> 
> _______________________________________________
> swift-build-dev mailing list
> swift-build-dev at swift.org <mailto:swift-build-dev at swift.org>
> https://lists.swift.org/mailman/listinfo/swift-build-dev <https://lists.swift.org/mailman/listinfo/swift-build-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-build-dev/attachments/20171130/a25c1fdc/attachment.html>