[swift-build-dev] Draft SwiftPM proposal: Multi-package repositories

[Daniel Dunbar]

> On Nov 12, 2016, at 9:43 PM, Russ Bishop <xenadu at gmail.com> wrote:
> 
> 
>> On Nov 12, 2016, at 1:02 PM, Daniel Dunbar via swift-build-dev <swift-build-dev at swift.org> wrote:
>> 
>> Hi all,
>> 
>> I'm reposting a request for feedback on my proposal for extending SwiftPM to support multiple packages inside one repository (i.e. "monorepo" support, although it is a misnomer in this use case).
>> https://github.com/ddunbar/swift-evolution/blob/multi-package-repos/proposals/NNNN-swiftpm-multi-package-repos.md
>> 
>> I would like to move this proposal forward so we can start on an implementation, even if we need to refine it over time, but I was hoping to get at least some concrete feedback first.
>> 
>> Thanks,
>> - Daniel
> 
> 
> It seems like you’re going through contortions to deal with arbitrary directory layouts and some odd consequences fall out of that decision. Not being able to deterministically detect non-unique sub-packages is one. 
> 
> Why not just require a top-level Package.swift that explicitly specifies the sub-packages? The name for the sub-package should be in the main package manifest. You’d gain the ability to import all the sub-packages in one go; importing the root package without any sub-packages specified automatically imports all sub-packages. This also allows library authors to organize a library into sub-packages later without breakage. Come up with a convention, e.g. a sub-package is in “/subpackageName” but allow overriding that default. That allows reorganization if needed but the convention should work for most libraries.

Mostly because I am concerned this doesn't scale well to *very* large repositories, in which commits to that file would be "contentious" (in the lock contention sense, not subject to debate sense). Of course, this argument is a little bogus as the current proposal doesn't scale that great either since we have to discover the packages (although I believe we can probably do a good job of caching this information).

It certainly would simplify the implementation & proposal to have this.

The other reason is it is yet another thing for people to maintain (and remember the syntax for). Most repos are small enough that I think the current proposal would perform fine and have a tendency to do what people might naively expect (even if they didn't really think about why). On the other hand, this file is likely to be quite static, so I'm not sure that is a very important issue.

I was already on the fence on this, but I hadn't considered the benefits you mention of allowing import of the package w/ no sub package specifier to mean import of all sub-packages. That tips me a little more towards thinking maybe a better proposal is to KISS and require this in some root file (whether or not that root file is itself a package manifest or a different kind of file is another question, you assume it would be the regular package manifest but I don't think it *need* be, and there is some value in not having any nesting relationship amongst packages).

 - Daniel

> A top-level Package.swift would also allow immediate detection of non-unique sub-packages, etc. Also if you are using things like git submodules, subtree, or some other mechanism that ends up putting package files in your source tree you don’t automatically re-export that package unless you take explicit action.
> 
> 
> I like the idea in general.
> 
> 
> Russ