[swift-dev] History of build-script, build-script-impl, and CMake?

[Jordan Rose]

Heh. Let's see:

> On Sep 8, 2017, at 08:14, Brian Gesiak via swift-dev <swift-dev at swift.org> wrote:
> 
> Hello all,
> 
> I'm hoping a long-time Swift contributor can help shed some light on
> how the apple/swift repository ended up with a utils/build-script, a
> utils/build-script-impl, and CMake files.
> 
> Based on the Git history, I've pieced together the following:
> 
> - July 17, 2010: Chris Lattner first adds a Makefile to apple/swift in
> his initial checkin. Until its eventual removal in 2014, the Makefile
> only ever built apple/swift, and it required users to have built LLVM
> in advance. [1].

1. Makefiles are easy, but…

> - July 14, 2011: Doug Gregor adds CMake files to build apple/swift [2].

2. …LLVM standardized on CMake. (And this gave us both Ninja and Xcode support.)

> - February 6, 2013: Joe Groff adds utils/buildbot-script.sh. This
> appears very similar to the modern-day uitls/build-script-impl. It
> configures and builds LLVM, Clang, and Swift [3].

3. This one really was for buildbots to have a standard configuration, but it's possible people were using it locally as well. There were also a lot of homegrown scripts at this time, and then people like me just used CMake directly.

> - February 25, 2014: Ted Kremenek removes the original Makefile Chris
> added in 2010. I believe only CMake-based builds became possible as of
> this point [4].

4. Yeah, nobody cared by this point.

> - December 23, 2014: Dmitri Gribenko (now no longer working on the
> project at Apple) rewrites a large portion of the CMake files, and
> deletes utils/buildbot-script.sh [5].
> - October 30, 2015: Ashley Gardland "adds back" a bunch of build
> files, including utils/build-script and utils/build-script-impl [6].


5/6. This is actually a misdirection! [5] is when build-script appeared, as part of a big push to get everyone building using the same tool. All the buildbots had their presets recorded in the build-presets file, which was the biggest win: if your own configuration didn't reproduce something, you didn't have to crawl int Jenkins to figure out what was different. And the Python was, in theory, much more maintainable than the increasingly-esoteric shell code used in build-script-impl (formerly buildbot-script.sh). That's why we had the task to migrate everything into Python, even though we never quite managed it.

The trouble was that we also had a bunch of Apple-internal configuration mixed in here. Rather than try to separate out the Apple and non-Apple parts in the commit log, we just removed it from the SVN history, took out the Apple parts, and then stuck in back in wholesale. That's [6].

(also, "Garland" ;-) )


So your guesses are all pretty much on the nose. The other thing that's cemented build-script and build-script-impl, though, is that they handle building multiple projects, even those with non-LLVMish build systems. (There's long been an advantage for Xcode users to build LLVM/Clang and Swift separately, but for things like swiftpm and swift-corelibs-foundation we'd never have a chance. And even LLDB is only just recently building like the other LLVM projects, if I remember correctly.)

Jordan

> 
> Could someone familiar with the apple/swift project during the years
> 2010 until 2015 shed some light on how the project was built during
> this time? Specifically:
> 
> - Did most developers use the Makefile? What replaced the Makefile?
> Did developers configure and build LLVM and Clang themselves,
> manually, without using a script?
> - Was the utils/buildbot-script.sh the precursor for
> utils/build-script and utils/build-script-impl?
> - What were the files that were "added back" in [6]? Did they exist in
> some form internally to Apple? Were they used by project developers at
> the time, or were they written solely for the benefit of external,
> open-source contributors?
> - Why do both utils/build-script and utils/build-script-impl exist?
> The fact that they were "added back" makes me think that they grew
> over time, and so were not written for the first time as part of the
> commit in [6].
> 
> Thanks in advance to anyone who can indulge my curiosity here! :)
> 
> - Brian Gesiak
> 
> [1] https://github.com/apple/swift/commit/afc81c1855bf711315b8e5de02db138d3d487eeb
> [2] https://github.com/apple/swift/commit/86ab76d62468241e03029e3e0790e4a0c8fbe3c1
> [3] https://github.com/apple/swift/commit/482826925812c8aa6d7ef861adbe36b26a4dedda
> [4] https://github.com/apple/swift/commit/f383eb90abc0afd6f78c7589005b61916e314870
> [5] https://github.com/apple/swift/commit/6670bb76ecc71a35ac5fa8f54cb81de6f555072f
> [6] https://github.com/apple/swift/commit/4ac9c80809450e4566946f40bbdb639b2ba745d8
> _______________________________________________
> swift-dev mailing list
> swift-dev at swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-dev/attachments/20170908/634a0790/attachment.html>