[swift-evolution] Proposal: XCTest Support for Swift Error Handling

[Brian Gesiak]

Chris: Thank you for also sending this to the swift-corelibs-dev mailing list!

I am in favor of the proposed changes; I think they are a great step
forward for testing throwing functions in Swift. That being said, I've
rarely used Swift error handling in my own applications--until now I
have preferred Result<T, U> (see:
http://www.sunsetlakesoftware.com/2015/06/12/swift-2-error-handling-practice).

I agree with Kevin Ballard's proposed amendment to the
`XCTAssertThrowsError` parameter type. As-is, I imagine many users of
the proposed API would be confused as to why they need to type
`XCTAssertThrowsError(_ = try vendingMachine.vend(row: 1, column:
1))`.

> Because Swift defines a closure that can throw an error to be a proper supertype of a closure that does not

Chris: How fortuitous! Glad to hear that we won't have to append a
`throws` to all of our test functions.

> I'm considering migrating away from XCTest, although not over this issue.

Drew: I'd be very interested to hear why. If it's something you feel
can be addressed in swift-corelibs-xctest, please email the
swift-corelibs-dev mailing list with your concerns. Otherwise I would
be glad if you could send me an email.

- Brian Gesiak


On Sun, Jan 10, 2016 at 6:08 PM, Kevin Ballard via swift-evolution
<swift-evolution at swift.org> wrote:
> On Sat, Jan 9, 2016, at 06:58 PM, Chris Hanson via swift-evolution wrote:
>
> We’d like feedback on a proposed design for adding support for Swift error
> handling to XCTest, attached below. I’ll mostly let the proposal speak for
> itself, but there are three components to it: Allowing test methods to throw
> errors, allowing the expressions evaluated by assertions to throw errors,
> and adding an assertion for checking error handling.
>
> We’d love to hear your feedback. We’re particularly interested in some
> feedback on the idea of allowing the expressions evaluated by assertions to
> throw errors; it’s generated some debate because it results in writing test
> code slightly differently than other code that makes use of Swift error
> handling, so any thoughts on it would be particularly appreciated.
>
>
> Very strong +1 to the entire proposal. I've been meaning to submit something
> like this for a while, though not as comprehensive as what you have here. In
> particular, making all the @autoclosure-accepting functions take a throwing
> closure and reporting errors as failures solves my biggest annoyance with
> XCTest. Making the test functions themselves also be able to throw is a neat
> idea too. And the trailing closure argument to XCTAssertThrowsError is a
> good idea.
>
> The only change I'd like to make to this proposal is to
> XCTAssertThrowsError. Instead of having it take an autoclosure that returns
> Void, it should just be generic so it can take a closure that returns any
> type. That gets rid of the need for `_ =` in the argument, and XCTest could
> even report the actual return value in the event that the closure does not,
> in fact, throw an error.
>
> I also wonder if maybe there's some utility in adding an overload to
> XCTAssertThrowsError that accepts any Equatable ErrorType as the second
> parameter, to simplify the cases where you do want to test against some
> specific error that happens to be Equatable. Although I'm not sure how
> likely it is for any given ErrorType to also be Equatable. The overload
> might look like
>
> public func XCTAssertThrowsError<T, E: ErrorType where E: Equatable>(
>     @autoclosure expression: () throws -> T,
>     _ message: String = "",
>     file: StaticString = __FILE__,
>     line: UInt = __LINE__,
>     _ expectedError: E
> ) {
>     XCTAssertThrowsError(expression, message, file: file, line: line) {
> error in
>         if let error = error as? E {
>             if error != expectedError {
>                 XCTFail("Expected error \(expectedError), found error
> \(error)")
>             }
>         } else {
>             XCTFail("Unexpected type of error thrown: \(error)")
>         }
>     }
> }
>
> -Kevin Ballard
>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>