[swift-evolution] [swift-build-dev] [Review] SE-0019 Swift Testing (Package Manager)

[David Owens II]

> One thing I'll add here: Unless I misunderstand, I don't think that this is a design/architectural limitation, but rather just a limitation of what functionality is currently exposed. In your example with xcodebuild, architecturally we do have the ability to test without building, and that functionality is even exposed in the Xcode.app Product menu, we've just never implemented a flag to xcodebuild for it. But we haven't boxed ourselves into an architectural corner there; there's no improper coupling at a technical level. Likewise, while this initial proposal doesn't specify the ability to test without building, I think that would be easy to add later on when we want it. Does that address your concern here?
> 
> 	- Rick

No, it doesn’t. While on the Office for Mac team, we repeatedly asked for many of the items in Xcode and xcodebuild to be made less coupled with each of the steps in the process. We were told no in most cases and given hacky workarounds in others that were prone to break with each incremental update to Xcode. Most of these breaks were found out because we picked up the latest beta build and attempted to run our automation tools only to find that something subtle had changed in our workaround or the basis of the workaround is no longer there.

Sure, SPM is “open source”, however, it’s unrealistic for people/teams to maintain forks of the tools to support features they need. Maybe the changes get accepted into a pull request, maybe not.

Honestly, I’m a bit shocked that the separation of building and executing tests is such a controversial thing. There should be three high-level tasks:

1. build
2. run-tests
3. build + run-tests

What I’m saying is that by not supporting #2, you’re creating an adoption blocker for any team that cannot or does not want to build every time it runs their tests. There are many reasons for this. I’ve already given several, however, another reason is instrumentation for things like code-coverage or product code markers, often happen as an intermediate step outside of the normal build operation. This is especially true in the cases of bug investigation. Saying that I cannot trigger the running of these tests without a build step baffling to me.

-David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20160107/d624402a/attachment.html>