[swift-corelibs-dev] [swift-evolution] Proposal: XCTest Support for Swift Error Handling

[Mike Ferris]

Honestly, we’re still trying to figure out how best to handle the balance between work that we are planning and carrying out in the Xcode version of XCTest under our normal development process and engagement of the open source community that is developing around the corelibs version of XCTest.

We would like to do as much as we can in the open. There are several competing tensions we’re trying to balance though.

- We need to be cognizant of our own release schedules and targets for the work we’re doing to extend the features and IDE integration of XCTest. Unlike the Swift language itself, the things we may be adding to corelibs XCTest are not the whole story. We are adding these things to Xcode’s XCTest as well and that work also must be accommodated. The fact that the corelibs XCTest is not the same code base as Xcode’s XCTest is going to have implications for how development on this project goes that I think we’re all still trying to figure out.

- Especially when the changes we’re making are specific to Swift or even highly relevant to Swift, we want to be able to allow the corelibs XCTest to run a little ahead of Xcode’s XCTest, but only where we’re committed to closing the gap in a known and relatively short time frame. So, for the moment, we’ve actually added this API to corelibs XCTest even though the corresponding API has not yet showed up in Xcode’s XCTest (at least not in a shipping Xcode yet… since this particular change is implemented in the overlay for Xcode’s XCTest, it should be available ahead of official release to those using toolchains from the swift project.)

- And, at times there will be cases where we’re unable to do certain XCTest work in the open (not in this case, but it will come up in the future) and there are also times when we will decide to do work in Xcode’s XCTest that is not simultaneously brought to corelibs XCTest by Apple.


In this particular case, I’d say the main issue that caused us to take a more abbreviated path to getting this stuff in was scheduling (which is an area that I cannot go into specifics about). As well there’s an aspect of us still trying to get used to the idea of how this will all work. So, we decided to dip our toes in, at least, by pushing the proposal out and getting some feedback, but we did not treat it fully as a formal proposal and review process.


Some of this is perhaps a bit vague, I realize, but hopefully it gives at least a bit of insight into the decision…

Mike


> On Jan 13, 2016, at 5:20 PM, Brian Gesiak via swift-corelibs-dev <swift-corelibs-dev at swift.org> wrote:
> 
> Loving those commits! Especially the corresponding tests :)
> 
> Can you explain the decision to not go through the review process? I think it's a prudent decision and I'm supportive of it, but I was under the impression that additional APIs should be proposed and reviewed. Is the rationale that swift-corelibs-xctest is too immature to justify going through proposals?
> 
> - Brian Gesiak
> 
> 
> On Wed, Jan 13, 2016 at 4:52 PM, Chris Hanson via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
> Thanks, everyone, for all of the feedback.
> 
> I sent this proposal out as more of a “heads up” that we were planning to make these changes and get feedback in advance of that, so we won’t be taking this through the full proposal process. That said, here’s what we’re going to be doing:
> 
> 1. We’re going to let test methods throw, and treat those as unexpected failures, as planned.
> 
> 2. We’re going to let test assertion expressions throw, and treat those as unexpected failures, as planned. We recognize that this will result in slightly different style in tests versus in regular code, but the convenience and reduction in boilerplate makes this worthwhile.
> 
> 3. We’re going to add the XCTAssertThrowsError assertion, incorporating Kevin Ballard’s suggestion for how to avoid the “_ = blah()” in its expression.
> 
> 4. We’re not going to add the ability for XCTAssertThrowsError to check that a specific error was thrown. This could definitely be useful but would require the function to take an ErrorType instance that also conforms to Equatable. We need to think about that some more before making such a change. Fortunately this sort of addition will be straightforward to put in via an overload or default argument, so if we choose to do it later, it shouldn’t break any code that starts to adopt XCTAssertThrowsError.
> 
> We’re going to start landing these changes today, so thanks again for all of your attention and feedback!
> 
>   -- Chris
> 
> 
> 
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
> 
> 
>  _______________________________________________
> swift-corelibs-dev mailing list
> swift-corelibs-dev at swift.org
> https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-corelibs-dev/attachments/20160114/34d971fe/attachment.html>