[swift-build-dev] Proposal: Git Branch and ref support for dependencies in Swift Package Manager

[Ankit Agarwal]

Considering all the suggestions so far I am thinking of the following
workflow :

.lock file will contain all the resolved dependencies in some simple format
containing resolved version number or commit hash in case of untagged
package.

1. Running initial `$ swift build` fetches all the dependencies specified
in manifest file and generates a .lock file if not present and if present,
updates .lock only for a new dependency introduced or for some dependency
removed (or maybe not generate .lock at all?) but `$ swift build` always
uses local state of Packages dir

2. To update the .lock file run something like `$ swift build --lock` which
will always update the .lock file with local state of Packages dir.

3. Commit and push the .lock file so others can reproduce the same
environment using some command like `$ swift build --bootstrap` (like
carthage)

4. If some dependency depends on branch/commit hash (ie non-tagged commit)
the author mentions that in their readme and the end user and maybe other
parallel dependencies will have to use only that tag in order to avoid
dependency hell, this should probably be tackled with the #1 problem Daniel
mentioned (alternate namespaces)

5. This proposal will address only .lock file problem and pointing to a
different namespace will be taken care in a separate proposal.

6. One issue that'll arrive with solving #2 outside manifest file is user
wanting to use untagged dependency will have to enter some "fake" version
in the manifest file.

Does this sound consistent with what you guys have in mind?


On Wed, Dec 16, 2015 at 3:11 AM, Drew Crawford via swift-build-dev <
swift-build-dev at swift.org> wrote:

> +1 for lockfiles, although I would implement them slightly differently.
>
> Again, this is a place where looking to Cargo's history is useful.
> Cargo.toml is roughly analagous to Packages.swift:
>
> [dependencies]
> openssl-sys = { version = "0.5.0", git = "
> https://github.com/wycats/openssl-sys" }
>
>
> Then when you run `cargo update` it (in addition to building) generates a
> lock file:
>
> [[package]]
> name = "openssl-sys"
> version = "0.5.0"
> source = "registry+https://github.com/rust-lang/crates.io-index"
>
>
> This lock file contains the exact software versions used to build.
> Subsequent invocation of `cargo build` use the version specified in the
> lockfile.
>
> So to summarize:
>
> 1.  `build` uses exact version specified in lockfile
> 2.  `update` updates the lockfile to point to the newest versions
> compatible with the requirements in Packages.swift
> 3.  lockfile always points to exact versions used
>
>
>
>
>
>
>
> On Dec 15, 2015, at 2:30 PM, Max Howell via swift-build-dev <
> swift-build-dev at swift.org> wrote:
>
> 2. Max had a great suggestion for how "locking" should work:
>  - "swift build --lock" (or however it is spelled) would simply create the
> .lock file based on the current state of your Packages tree.
>
> This simple idea gives a great workflow for developing packages: you start
> by declaring your dependencies in your Package.swift, you use `swift build`
> to pull them down for you, then you start hacking on them (in your local
> checkout) to get the changes you need, iterating using `swift build --lock
> && swift build`, and then once you have something that works you can move
> to pushing those changes back and you already have the right .lock file
>
>
> To add more detail here, I propose that, initially you `swift build`:
>
>     $ swift build
>     $ cd Packages
>     $ ls
>     FooBar-1.2.3
>
> Then you can step into Packages/FooBar-1.2.3 and change the branch:
>
>     $ git checkout mybranch
>
> Then build against that branch
>
>     $ swift build
>
> Now, to ensure consumers of your terminal-package or other members of your
> team are using the same packages as you, you need to commit the lock file:
>
>     $ git add ../../Packages.lock
>     $ git commit -m Lock against devbranch for FooBar
>     $ git push
>
> Any .lock files that are cloned into dependencies are *ignored*. If a
> dependency depends on a branch of a package they should specify to the
> end-user that they will need to lock against that branch in their README.
>
> The justification here is: dependency graphs are fragile, and introducing
> additional fragility should be avoided directly as part of our design going
> forward.
>
> Now, we can add a convenience command line to do the above steps all in
> one, but exposing and documenting the mechanism transparently in the above
> manner offers consumers of swiftpm power and flexibility.
>
> In addition the above workflow exposes a simple method for fixing issues
> in your dependencies directly without the need for convoluted tooling.
> Simply step into your Packages directory, make your changes and change the
> remote to your own fork, then lock.
>
> Part of the proposal should thus make swiftpm emit warnings for modified
> packages without changes that are committed or pushed remotely. We don’t
> want people to accidentally push non-atomic changes to their dependency
> graph resulting in their teams not being able to build their projects.
>
> Max
> _______________________________________________
> swift-build-dev mailing list
> swift-build-dev at swift.org
> https://lists.swift.org/mailman/listinfo/swift-build-dev
>
>
>
> _______________________________________________
> swift-build-dev mailing list
> swift-build-dev at swift.org
> https://lists.swift.org/mailman/listinfo/swift-build-dev
>
>


-- 
Ankit
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-build-dev/attachments/20151219/14e75116/attachment.html>