[swift-dev] proposed change for master-next merges

Fri Feb 10 11:39:41 CST 2017

> On Feb 9, 2017, at 5:49 PM, Bob Wilson via swift-dev <swift-dev at swift.org> wrote:
> 
> Since the feedback on this was all positive, we’re moving ahead with this change. It seems like a good time for a status update.
> 
> Some changes are already done:
> 
> * We test up a test bot to build against the upstream-with-swift branches of LLVM/Clang (https://ci.swift.org/view/swift-master-next/job/oss-swift-incremental-RA-osx-master-next-upstream-with-swift/ <https://ci.swift.org/view/swift-master-next/job/oss-swift-incremental-RA-osx-master-next-upstream-with-swift/>). This has been working well.

I think we can remove this test bot now, so don’t be surprised if it goes away soon.

> 
> * Mishal set up an automerger to continuously update the Swift master-next branch with changes from master.
> 
> * I audited the master-next branches in the other repos (e.g., swiftpm, etc.) and found that none of them have any changes from master. For now, we will simply stop using those branches, and once we’re done with this transition and things are settled, we can remove them.
> 
> * We’ve had some discussions about how LLDB will fit into this plan. See the recent thread on “Changes to LLDB Branch Management” for details on that.
> 
> Here is what is left to do:
> 
> * Update all of our master-next bots to build against the “new” branches (e.g., upstream-with-swift instead of stable-next).

New update: Mishal switched all the master-next bots to build against Clang/LLVM upstream-with-swift, so this step is done now.

The bad news is that things are in kind of bad shape. Beginning on Wednesday, the build stopped working — it is getting killed by timeouts, so maybe something is hanging.

> 
> * Update documentation.
> 
> * Remove unused branches.
> 
>> On Dec 8, 2016, at 10:54 AM, Michael Gottesman <mgottesman at apple.com <mailto:mgottesman at apple.com>> wrote:
>> 
>>> 
>>> On Dec 8, 2016, at 7:43 AM, Saleem Abdulrasool via swift-dev <swift-dev at swift.org <mailto:swift-dev at swift.org>> wrote:
>>> 
>>> Having been involved in the update process for the next branches, I'm really excited to see this type of change.
>>> 
>>> I think that the "simple" approach is both better to work and collaborate in as well as closer to the llvm development model which makes it easier to cross pollinate.
>>> 
>>> The one thing that I think could be more strongly called out is that normal PR CI shouldn't be gated on master next passing.
>>> 
>>> Beyond that, I think that this proposal should ease collaboration and maintenance pains in the current process.
>> 
>> I agree with everything said here. The collaboration issue is the high order bit that needs to be fixed. Any other issues that come up can be fixed incrementally if necessary based on experience (for instance using the "back up" alternative). This is even more true since it is clear that that infrastructure would take time to develop and an incremental solution now that improves the collaboration will not make it more difficult to develop such a solution.
>> 
>> My only strong feeling here is that I think we need documentation of this process in ./docs in addition to this proposal.
>> 
>> Michael
>> 
>>> 
>>> On Wed, Dec 7, 2016 at 7:30 PM Bob Wilson via swift-dev <swift-dev at swift.org <mailto:swift-dev at swift.org>> wrote:
>>> I would like to make a change in the way we handle the master-next branch.
>>> 
>>> Summary: I’d like to switch to a model where we continuously test against the latest upstream LLVM changes. The goal is to simplify the process and make it easier to collaborate on maintaining master-next.
>>> 
>>> Background: We develop Swift against “stable” branches of LLVM (which I am using here to refer to the llvm, clang, and compiler-rt repositories) that are typically rebranched from trunk once for each release, with other commits individually cherry-picked for specific bug fixes and other changes. This insulates Swift development from the churn of changes in LLVM. At the same time, we maintain the “master-next” branches of Swift repos to keep up to date with trunk LLVM. For Swift, our “trunk” comes from the “upstream-with-swift” branches in our GitHub LLVM repos. We have existing automation to continuously merge changes from llvm.org <http://llvm.org/> into those upstream-with-swift branches.
>>> 
>>> We currently use a manual process to update master-next. Someone on the Swift team is designated as the "merge czar" and is responsible for this. This merge typically happens once every few weeks. Michael Gottesman developed some internal tools to help automate the process, but someone still needs to drive those tools manually. The process involves merging “master” to “master-next” for all the Swift repos and updating the “stable-next” branches of the GitHub LLVM repos for Swift. The “stable-next” branches are basically snapshots of the LLVM upstream-with-swift branches at the point where master-next was most recently merged.
>>> 
>>> Swift CI includes a set of Jenkins bots to test master-next building with the stable-next branches of LLVM (https://ci.swift.org/view/swift-master-next <https://ci.swift.org/view/swift-master-next>). The merge czar can use these bots to confirm that everything is working after a merge.
>>> 
>>> Reasons to change: The current process has the advantage that the merge czar can choose when to do a merge and can schedule that around other work, but it has some significant problems.
>>> 
>>> - It is difficult for multiple people to collaborate on updating master-next. The changes involved are often rev-locked between Swift and the LLVM repos, so there is no good way for someone to fix a problem without doing the whole merge process.
>>> 
>>> - The current system is hard to understand. I’ve been serving as the merge czar for the last few months, and it took me a while to figure out how to do it well.
>>> 
>>> - It requires extra “stable-next” branches in our GitHub LLVM repos, further adding to the complexity.
>>> 
>>> - The tools we have to help automate the process are currently internal to Apple and require ongoing maintenance. They could be cleaned up to release publicly but that would take more work.
>>> 
>>> Proposal: We already have Jenkins bots testing master-next. I would like to add a job to continuously merge master to master-next and change the existing bots to build against the “upstream-with-swift” branches in our GitHub LLVM repos. The bots would then detect any new problems soon after they are introduced. Anyone could fix those problems, whether they are merge conflicts, build failures, or test issues. A partial fix could be applied directly without needing to resolve all of the outstanding issues.
>>> 
>>> This would avoid the need for our current internal merging tools. We already have automatic merging bots, so adding another one would not be difficult.
>>> 
>>> The biggest advantage is that it provides a straightforward model that anyone can understand: master-next becomes just another branch that anyone can modify, build and test in the usual way. Collaboration is no more difficult than for other branches.
>>> 
>>> The cost of this simpler approach is that we would need to be willing to let the master-next branch break occasionally. An LLVM change might break things in a way that takes some time to fix, and the master-next bots would continue to fail during that time. Someone might want to apply a partial fix that does not resolve all the issues, and we would want to allow that even if the bots still fail. That would mean we would have to relax (or override) the requirement for PR testing for commits in that kind of situation. In the worst case, if new problems are introduced more quickly than we can fix them, this approach could fall apart. My experience as merge czar over the last few months suggests that is unlikely. Usually there are no more than a few problems per week and most of them are easy to fix.
>>> 
>>> I propose to roll this out in steps. First, we can add a new Jenkins bot that tests master-next building against upstream-with-swift. If that goes well, and if there are no objections to this proposal, we can add the automerger to merge master into master-next. At the same time, we would update the other master-next bots to use upstream-with-swift instead of stable-next for the LLVM repos.
>>> 
>>> Alternative: We can achieve some of the same goals at a considerable increase in complexity by introducing an automatic gated merge solution. We would have automation perform the merge and commit it as long as everything works. If there were any problems, the automation would create a pull request that would need to be manually updated to resolve the problems. People could still collaborate by working together on the pull request branch. Until the problems were resolved, no further merging would take place. I would like to try the simple approach before considering this more complex solution, since I don’t think it will be necessary, at least in the near future.
>>> 
>>> Any objections to this? Comments or suggestions?
>>> _______________________________________________
>>> 
>>> swift-dev mailing list
>>> 
>>> swift-dev at swift.org <mailto:swift-dev at swift.org>
>>> 
>>> https://lists.swift.org/mailman/listinfo/swift-dev <https://lists.swift.org/mailman/listinfo/swift-dev>
>>> 
>>> _______________________________________________
>>> swift-dev mailing list
>>> swift-dev at swift.org <mailto:swift-dev at swift.org>
>>> https://lists.swift.org/mailman/listinfo/swift-dev <https://lists.swift.org/mailman/listinfo/swift-dev>
> _______________________________________________
> swift-dev mailing list
> swift-dev at swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-dev/attachments/20170210/c7975f25/attachment.html>