[swift-evolution] [Review] SE-0145: Package Manager Version Pinning

Daniel Dunbar daniel_dunbar at apple.com
Thu Nov 17 11:46:44 CST 2016 

I have implemented the revisions to SE-0145 to support automatic pinning.

The approach we took was to leave much of the existing behaviors, but to support an "automatic pinning" mode which is then on by default.

If anyone wants to provide early feedback, my draft is here:
  https://github.com/ddunbar/swift-evolution/blob/automatic-pinning/proposals/0145-package-manager-version-pinning.md

Otherwise I intend to take this to review again as soon as I hear from other team members.

 - Daniel

> On Nov 11, 2016, at 11:46 AM, Daniel Dunbar <daniel_dunbar at apple.com> wrote:
> 
>> On Nov 11, 2016, at 9:28 AM, Paul Cantrell <cantrell at pobox.com <mailto:cantrell at pobox.com>> wrote:
>> 
>> Thanks for being open to feedback, Daniel, even when it’s difficult feedback. :)
>> 
>> Pondering #2: is there a behavioral difference between
>> 
>> 1. explicit “do not pin” option in SwiftPM and
>> 2. Package.pins added to .gitignore?
>> 
>> (In #2, individual ill-behaved packages would be version-pinned in Package.swift.)
>> 
>> I imagine that the effect of #1 moves version pinning from Package.pins to some local hidden/internal location that isn’t shared across machines. That sounds to me like “there is always a pinfile, it always works the same way, but sometimes teams choose not to share it across machines” — which sounds a lot like #2. But maybe it’s more subtle than that?
> 
> I think the behavior difference comes in in how individual commands like `swift package update` behave and what their diagnostics are.
> 
> I want it to be possible for developers to, for example, maintain individual pins but not auto-pin everything else (and even potentially check in that configuration).
> 
> Hoping to have a concrete proposal written up shortly...
> 
>  - Daniel
> 
>> 
>> Cheers, P
>> 
>> 
>>> On Nov 10, 2016, at 11:41 AM, Daniel Dunbar <daniel_dunbar at apple.com <mailto:daniel_dunbar at apple.com>> wrote:
>>> 
>>> Thanks to everyone who participated in this review!
>>> 
>>> Based on the pretty universal negative feedback, we are going to reject this proposal as is, and take it back for another round of revisions.
>>> 
>>> Our revised plan is:
>>> 1. To introduce an "autopin" behavior to cover the problem Paul outlined where `pin --all` effectively needs to be "sticky" for any new dependencies which come into play.
>>> 2. To make auto pinning on by default, with an explicit mechanism for projects to opt out.
>>> 
>>> I hope to have this written up for review next week.
>>> 
>>> Thanks!
>>>  - Daniel
>>> 
>>>> On Nov 4, 2016, at 9:06 AM, Paul Cantrell via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>>> 
>>>> What is your evaluation of the proposal?
>>>> 
>>>> General +1, with reservations. Novel elements of the proposed behavior will need careful evaluation and refinement as we see how they plays out in practice, with open-mindedness from both users and the core team.
>>>> 
>>>> Breaking it down:
>>>> 
>>>> +1 on having this feature in some form. It’s essential.
>>>> 
>>>> +1 on making user choice about .gitignore the thing that controls whether and how pinning is shared within a team. That’s simple, clear, accommodates a wide range of needs, and is consistent with other package managers.
>>>> 
>>>> +1 that pinfiles have no effect whatsoever on dependent projects. That’s the only sensible way for it to work, but since there was some debate about that, I’ll just reiterate support.
>>>> 
>>>> -1 on making dependencies unpinned by default. Trying to induce unexpected behavior to encourage testing can be a good technique — in contexts where testing is the goal. My gut tells me that doing this when building is the goal will cause a lot of confusion and kvetching. I follow the proposal’s argument that unexpected breakages are a nice way to make strict semver a community norm … and I just do not buy it.
>>>> 
>>>> However, given that we hashed this out at great length and the core team is still enamored of the idea, I’m willing to give it a try! I’d love to be proved wrong.
>>>> 
>>>> +1 on the proposed command structure given that I’ve lost the aforementioned “always pin” argument. Living in a sometimes-pinned-sometimes-not world is going to be confusing, but the proposed commands help as best they can.
>>>> 
>>>> ¿-1? This is a big one. If I do:
>>>> 
>>>>     swift package pin --all
>>>> 
>>>> …and then add a new dependency, is the new dependency also pinned? It should be. To pin or not to pin is typically a project- and team-wide policy decision. I do see the use case for pinning just one ill-behaved dependency, but more typically pinning is something that is built in to a team’s testing process and their assumptions about a whole build’s behavior.
>>>> 
>>>> The proposal is vague on this point, but could be interpreted to mean that --all does not pin new dependencies: “Dependencies are never automatically pinned, pinning is only ever taken as a result of an explicit user action.” (Aside: there should be a semicolon instead of a comma in that sentence.) I assume — hope — that this is not the case! If --all does not affect new dependencies, then I’m -1 on the proposal.
>>>> 
>>>> ¿-1? The proposal mentions that SwiftPM already effectively performs local pinning, but is ambiguous about whether this behavior remains separate from the new pinfile. I’m dubious about having two separate pinning mechanisms, one visible and one invisible.
>>>> 
>>>> +1 to the proposal’s repeated mentions of clear output and helpful diagnostics. Since this proposal introduces behavior that’s somewhat off the beaten path for package managers, this will be essential.
>>>> 
>>>> On the pin/lock controversy
>>>> 
>>>> I don’t care. Computer science is full of heavily overloaded terms where a loose underlying concept takes on radically different meanings (bridge, channel, dispatch, edge, graph, header, key, model, module, node, open, parameter, port, process, protocol, query, return, row, source, union). We do just fine disambiguating all these in context, thank you very much. Renaming “lock” to “pin” solves a problem that doesn’t exist.
>>>> 
>>>> However, we programmers are _also_ used to dealing with synonyms or partially overlapping near-synonyms (nil / null; closure / lambda / block; field / instance variable; tagged union / associated type enum; etc) and we also do just fine with those too. I’m sure we’ll learn to deal with lock / pin, and nobody will care after 6 months.
>>>> 
>>>> In short, “pin” is an unobjectionable solution to a non-problem. Core team is excited about “pin?” Grand. It’s a fine term. Do it and move on.
>>>> 
>>>> 
>>>> Is the problem being addressed significant enough to warrant a change to Swift?
>>>> 
>>>> It’s essential. SwiftPM will be impractical in many real-world situations until this is sorted out.
>>>> 
>>>> 
>>>> Does this proposal fit well with the feel and direction of Swift?
>>>> 
>>>> Yes, it’s consistent with the general approach of SwiftPM.
>>>> 
>>>> 
>>>> If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
>>>> 
>>>> I’ve used bundler, Carthage, and CocoaPods extensively. All of them always generate a lockfile (Gemfile.lock, Cartfile.resolved, and Podfile.lock). All of them use these files as the unique mechanism for version locking, and all use version control of that file as the unique mechanism for controlling whether to locked versions are shared across teams.
>>>> 
>>>> We have many years of evidence that this model works well.
>>>> 
>>>> Note that all of these package managers also work in in environments that do not support using multiple versions of a dependency in the same artifact at the same time. Therefore this statement from the proposal:
>>>> 
>>>>> Overconstraint is much more of a risk in Swift than in other languages using this style of package management.
>>>> 
>>>> 
>>>> …is incorrect.
>>>> 
>>>> In particular, note that Ruby does not support using multiple versions of a lib simultaneously, and that fact alone — even in the presence of _ubiquitous_ version pinning — has been sufficient to encourage widespread mindfulness about semver compliance. All of the concerns expressed in the “Pin by default” section of the proposal also apply to Ruby, and have failed to materialize there.
>>>> 
>>>> —
>>>> 
>>>> I’ve also used npm and bower, which either do not have version locking or only provide it via add-ons. It’s a nightmare. Lack of locking has caused headaches and lost hours — not hypothetic headaches, but real ones on actual projects — in two scenarios:
>>>> 
>>>> 1. onboarding new developers who get fresh, incompatible dependency versions on initial checkout; and
>>>> 2. picking projects back up for a new round of development.
>>>> 
>>>> Are these two situations really the right time for people to accidentally test whether their dependencies have properly followed semantic versioning? No. There are better ways, and better times. I am troubled by the insistence on ignoring experience here. However, as I said above, I’m willing to give it a try. I will keep an open mind in the name of bold experimentation, and would be happy to have my concerns proven wrong.
>>>> 
>>>> Please do keep in mind, however, that this is an experiment. Be ready for all that careful theorizing to be falsified by experience. You may have to murder this darling: http://www.slate.com/blogs/browbeat/2013/10/18/_kill_your_darlings_writing_advice_what_writer_really_said_to_murder_your.html <http://www.slate.com/blogs/browbeat/2013/10/18/_kill_your_darlings_writing_advice_what_writer_really_said_to_murder_your.html>
>>>> 
>>>> 
>>>> How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
>>>> 
>>>> In depth, though I only read some of the discussion thread.
>>>> 
>>>> Cheers,
>>>> 
>>>> Paul
>>>> 
>>>> 
>>>>> On Oct 31, 2016, at 4:23 PM, Anders Bertelrud via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>>>> 
>>>>> Hello Swift community,
>>>>> 
>>>>> The review of SE-0145 "Package Manager Version Pinning" begins now and runs through November 4. The proposal is available here:
>>>>> 
>>>>> 	https://github.com/apple/swift-evolution/blob/master/proposals/0145-package-manager-version-pinning.md <https://github.com/apple/swift-evolution/blob/master/proposals/0145-package-manager-version-pinning.md>
>>>>> 
>>>>> Reviews are an important part of the Swift evolution process. All reviews should be sent to the swift-evolution mailing list at
>>>>> 
>>>>> 	https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
>>>>> 
>>>>> or, if you would like to keep your feedback private, directly to the review manager.
>>>>> 
>>>>> What goes into a review?
>>>>> 
>>>>> The goal of the review process is to improve the proposal under review through constructive criticism and contribute to the direction of Swift. When writing your review, here are some questions you might want to answer in your review:
>>>>> 
>>>>> 	* What is your evaluation of the proposal?
>>>>> 	* Is the problem being addressed significant enough to warrant a change to Swift?
>>>>> 	* Does this proposal fit well with the feel and direction of Swift?
>>>>> 	* If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
>>>>> 	* How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
>>>>> 
>>>>> More information about the Swift evolution process is available at
>>>>> 
>>>>> 	https://github.com/apple/swift-evolution/blob/master/process.md <https://github.com/apple/swift-evolution/blob/master/process.md>
>>>>> 
>>>>> Thank you,
>>>>> 
>>>>> Anders Bertelrud
>>>>> Review Manager
>>>>> _______________________________________________
>>>>> swift-evolution mailing list
>>>>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>>>>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
>>>> 
>>>> _______________________________________________
>>>> swift-evolution mailing list
>>>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>>>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
>>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20161117/51c220e3/attachment.html>

More information about the swift-evolution mailing list