[swift-evolution] [Review] SE-0019 Swift Testing (Package Manager)

[Paul Cantrell]

This proposal is a nice improvement over the previous iteration, and I support it.

The new command line makes much more sense: it’s worth the trouble to make “swift test” work right of the gate, and the hierarchical Module.TestGroup.individualTest structure makes sense and addresses several concerns.

It’s unclear to me how this plays out:

    swift test --kind=performance

…but it doesn’t seem to constrain what’s actually proposed here, so I’ll wait for the future proposal on that one.

I continue to think this is a bad decision:

> Additionally, we propose that building a module also builds that module's corresponding tests. Although this would result in slightly increased build times, we believe that tests are important enough to justify this (one might even consider slow building tests to be a code smell).

I expounded on my this thoroughly (perhaps too) in my previous review, so I won’t repeat those concerns; I’ll just note that they still stand.

This is important:

> This proposal also does not cover the need for utility code, ie. a module that is built for tests to consume that is provided as part of a package and is not desired to be an external package. This is something we would like to add as part of a future proposal.


…and it makes me wonder whether multiple test files can share helper code within a single test module (crucial!), or whether every test file is compiled in isolation.

I’ll be interested in the future proposals that work out how test module options are specified in Package.swift.

I assume that all test modules are run by default with “swift test”? Again, I won’t repeat the more details concerns about that from the previous review, but just note that it should be possible for Package.swift to specify that some test modules don’t get run by default. I imagine this might be part of the proposal that establishes what “--kind=performance” means.

Looking forward to seeing this in action!

Cheers,

Paul


> On Jan 15, 2016, at 4:34 PM, Rick Ballard via swift-evolution <swift-evolution at swift.org> wrote:
> 
> Hello Swift community,
> 
> A 2nd review of “Swift Testing” for the Swift Package Manager begins now and runs through Tuesday, January 19th. The proposal is available here:
> 
> 	https://github.com/apple/swift-evolution/blob/master/proposals/0019-package-manager-testing.md
> 
> This is the 2nd review of the proposal, after revisions based on the first review. A summary of issues raised in the first review can be found at:
> 
> 	https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160111/006466.html
> 
> Reviews are an important part of the Swift evolution process. All reviews should be sent to the swift-evolution mailing list at
> 
> 	https://lists.swift.org/mailman/listinfo/swift-evolution
> 
> or, if you would like to keep your feedback private, directly to the review manager.
> 
> What goes into a review?
> 
> The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:
> 
> 	* What is your evaluation of the proposal?
> 	* Is the problem being addressed significant enough to warrant a change to the Swift Package Manager?
> 	* Does this proposal fit well with the feel and direction of the Swift Package Manager?
> 	* If you have you used other package managers with a similar feature, how do you feel that this proposal compares to those?
> 	* How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
> 
> More information about the Swift evolution process is available at
> 
> 	https://github.com/apple/swift-evolution/blob/master/process.md
> 
> Thank you,
> 
> 	- Rick
> 	  Review Manager
> 
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution