[swift-build-dev] Dependency consistency within package major version

[Simon Pilkington]

> On 8 Dec 2015, at 11:03 AM, Max Howell <max.howell at apple.com> wrote:
> 
> 
>> My question actually related to dependency (sub-dependency) conflict resolution which is listed as future feature.
>> 
>> For example, considering the initial state
>> 
>> Module A -> depends on Module B v.1 (1.0.0) -> depends on Module D v.1
>> Module A -> depends on Module E v.1 (1.0.0) -> depends on Module D v.1
>> 
>> In this case you would get a consistent dependency graph and everything would correctly compile statically into Module A. But if Module E makes a minor version update so that
>> 
>> Module A -> depends on Module B v.1 (1.0.0) -> depends on Module D v.1
>> Module A -> depends on Module E v.1 (1.1.0) -> depends on Module D v.2
>> 
>> Without changing any code, Module A suddenly doesn’t have a consistent dependency graph, even if the changes to the public API of Module D don’t happen to impact the public APIs of Module E or Module A. Is this a concern or am I missing something here?
> 
> Thank you, I understand now.
> 
> Indeed, this is yet another possible dependency hell, so I’ll add it to our list [1]
> 
> We have at least one strategy that we hope we will be able to implement, i.e.: pure modules. We would like people to make their libraries small, focused and “pure”—in that they do not have any global state (even things like file system access). If they then declare their purity we can load both version 1 and version 2 of the same module into the same process.
> 
> This would work even if the modules are exposed publicly via API to another module lower in the graph, though in such cases it may be confusing to consumers of that library. So we may insist this sort of thing can only apply if the modules don’t get exported.
> 
> Another avenue we have explored is being more forceful with what package authors can do. That is, prohibiting such actions altogether. This has the benefit that dependency hell is much less likely. Ultimately we decided that sometimes you have to make such changes and it would be against our greater goals of providing flexible tools to have it operate this way.
> 
> Instead we plan to add a good deal of warnings to the tool that eventually will lint/publish packages. In the above case we would warn in big colorful letters to the user: “WARNING increasing the dependency version of Foo from v1 to v2 is irresponsible! Be sure you have good rationale before pushing!”
> 
> I would be interested to hear any more thoughts you have and if you think our approach will be successful. Thanks,
> 
> Max
> 
> [1]: https://github.com/apple/swift-package-manager/blob/master/Documentation/DependencyHells.md <https://github.com/apple/swift-package-manager/blob/master/Documentation/DependencyHells.md>
As you said, versioning is hard and these are the reasons why.

I am guessing that to be able to load two versions of the same module into the same process the package builder re-namespace one or both to avoid a collision and change the referencing modules. Something like this should work, the only concern is making sure you limit the amount of ‘black magic’ that is going on - make sure its clearly explained what the package builder is doing otherwise there might be side effects - such as the size of a binary increasing because there are now two versions of a sub-module present - that suddenly appear without explanation.

I think the approach of warning on these issues is a good one - at most having them as errors that fail the build but then can be explicitly ignored (as its very easy for devs to take the ‘well it compiled, must be good’ approach and completely ignore warnings). In terms 'make doing the correct/common thing easy and the rest possible’ its rare that you’d actually want to outright prohibit something so your approach seems solid with respect to that.

I’m pretty excited by what I have seen so far with the package builder and it looks like its starting off with a good foundation. Let me know if there is anything I can contribute.

-Simon



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-build-dev/attachments/20151214/1631791f/attachment.html>