[swift-build-dev] [swift-evolution] [Review] SE-0019 Swift Testing (Package Manager)

[John Joyce]

> On Jan 6, 2016, at 4:06 AM, Rick Ballard via swift-evolution <swift-evolution at swift.org> wrote:
> 
> 	* What is your evaluation of the proposal?
Overall this is great direction, but needs more deliberation and sifting about the details. This kind of infrastructure will live a while, or if it ends up stinky, will go unused.

> 	* Is the problem being addressed significant enough to warrant a change to Swift?
Not really a language change as far as I can tell.

> 	* Does this proposal fit well with the feel and direction of Swift?
Definitely.

> 	* If you have you used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
This is an excellent addition that keeps the whole life cycle view in.

> 	* How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
> 



On Build Configuration/Testability
This seems to be a fairly narrow view, but maybe it's enough.
I think release and debug are not the only configurations, though likely the most common set.
Perhaps some sensible defaults should be implicit (written automatically in a PM file) but customizable? 
(apologies if this exists and I missed it.)

Additionally, this takes a narrow view on the nature of testing. 
Unit tests are not the only game in town and I would like to see this left open to other kinds of automated testing.
You might have UI testing among others. XCUITest is not the only idea here.
You may need to test interactions and flows between systems.

For the Command Line Interface section
I would suggest the subcommand test makes a lot of sense.

Is there a proper place here somewhere for ensuring a home for test resources? 
(a standardized test/ subdirectory for static/standard test data )

Lovely open ended callouts on the test reporting section.
Important for that to be malleable and replaceable. Might be JSON would be an obvious addition there...?

Lastly, should there be any built-in facility for bug/issue tracking relationships?
Perhaps too much minutia for this proposal, but seems a good thing to include.
In particular for reporting, any ticket/tracking numbers referring to regression tests are always good callouts in manager friendly reporting.


Lastly as an addition, would it make sense to include some kind of test/code-coverage element here?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-build-dev/attachments/20160106/e3be749f/attachment.html>