[swift-build-dev] [swift-evolution] [Review] SE-0145: Package Manager Version Pinning (Revised)

[Martin Waitz]

> The review of "SE-0145: Package Manager Version Pinning" begins again after revisions, starting now and running through November 28th. The proposal is available here:
> 
> 	https://github.com/apple/swift-evolution/blob/master/proposals/0145-package-manager-version-pinning.md <https://github.com/apple/swift-evolution/blob/master/proposals/0145-package-manager-version-pinning.md>
> 


> What is your evaluation of the proposal?

+1
I think this is an important change for the Swift package ecosystem.
Having reproducible builds by default is a big step forward.
Storing the version of all dependencies in version control and having an easy way to update them helps to reduce maintenance overhead of packages.

I have two comments / requests for improvement:
 * Having two completely separate modes (—enable-autopin vs. —disable-autopin) with a different semantic (e.g. for `swift update`) unnecessarily complicates the design. I understand that there are different use cases where packages have to be handled differently, however I think we should invest some more time on understanding these use cases (see below).
 * For me the term ‚pin‘ (just like ‚lock‘) implies that the version of pinned dependencies should be fixed and should therefore not be changed/updated. (But I’m not a native speaker, so you may have other associations here.) We should make clear that we *manage* dependencies and that dependencies should be easy to update even while they can be reproduced exactly.

Let’s look at the features of the proposal and how they can be used:
 * Defines two groups of dependencies (pinned vs. unpinned) where one group can be updated independently;
 * unpinned dependencies are automatically updated on clone, not just on update;
 * updates of unpinned dependencies are not considered a change in the package’s repository.
This can be used to easily update internal dependencies while keeping the same version of external dependencies.
Or to keep some difficult-to-update dependencies while constantly updating all the other dependencies.
Or to not create git changes for some selected dependencies.
You name it.

Basically it boils down to better control when and how dependencies are updated.
However, I’m not yet convinced that the proposed pinning system is the best way to do that.
Why only support two groups? Why these two modes of operation? Why not support a specific set of dependencies which are unmanaged while still tracking all new dependencies?

When we want to create groups of dependencies and handle them differently, then we should do that explicitly, by introducing groups which can be defined by the package maintainer.

E.g.:

	swift package dependency <dep-name> —group <group-name>

We could store each group in a file of its own (e.g. `Package-<group-name>.versions` or whatever) and the maintainer could put each one under version control or not as he likes.

A whole group could be updated in one go:

	swift package update —group internal

This could simulate the pinning as described in the proposal:

	swift package dependency A —group unpinned
	swift package dependency B —group pinned
	echo Package-unpinned.versions > .gitignore
	git add .gitignore Package-pinned.versions

Each group can be updated individually and the package maintainer can put this group under version control or not.
One group would have to be special to automatically include all new dependencies.
Other than that we would not need any special handling ala —repin/—enable-autopin. Groups could be defined to make updating related packages easier.
When needed, we could even set some groups to update automatically on build (not only on clone/update).
This could be useful for closely related packages. Why should clone be special anyways?


>  Is the problem being addressed significant enough to warrant a change to Swift?

Yes.
Not managing the package dependencies would unnecessarily complicate package maintenance in the long term.


> Does this proposal fit well with the feel and direction of Swift?

Yes.
Reproducible builds fit well into the safe-by-default approach of Swift.


> If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

I always hated `npm` and all those node packages which had overly restrictive dependency requirements.
I guess these were used in order to have more control about which version to use, but this resulted
in both packages which I could not build out of the box (because some dependencies changed),
and packages I could not update because of conflicting requirements.
Using loose version requirements + semver in `Package.swift` files together with exact versions
of all dependencies stored in the top-level package solves everything: you get reproducible builds
(based on `Package*.versions`) together with easy updates (based on all packages’ `Package.swift`).


> How much effort did you put into your review? A glance, a quick reading, or an in-depth study?


Read both versions of the proposal, took part in the discussion.

— Martin Waitz
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-build-dev/attachments/20161124/a13427a0/attachment.html>