[swift-evolution] Proposal: XCTest Support for Swift Error Handling

[Kevin Ballard]

On Sat, Jan 9, 2016, at 06:58 PM, Chris Hanson via swift-evolution wrote:
> We’d like feedback on a proposed design for adding support for Swift error handling to XCTest, attached below. I’ll mostly let the proposal speak for itself, but there are three components to it: Allowing test methods to throw errors, allowing the expressions evaluated by assertions to throw errors, and adding an assertion for checking error handling.
>  
> We’d love to hear your feedback. We’re particularly interested in some feedback on the idea of allowing the expressions evaluated by assertions to throw errors; it’s generated some debate because it results in writing test code slightly differently than other code that makes use of Swift error handling, so any thoughts on it would be particularly appreciated.
 
Very strong +1 to the entire proposal. I've been meaning to submit something like this for a while, though not as comprehensive as what you have here. In particular, making all the @autoclosure-accepting functions take a throwing closure and reporting errors as failures solves my biggest annoyance with XCTest. Making the test functions themselves also be able to throw is a neat idea too. And the trailing closure argument to XCTAssertThrowsError is a good idea.
 
The only change I'd like to make to this proposal is to XCTAssertThrowsError. Instead of having it take an autoclosure that returns Void, it should just be generic so it can take a closure that returns any type. That gets rid of the need for `_ =` in the argument, and XCTest could even report the actual return value in the event that the closure does not, in fact, throw an error.
 
I also wonder if maybe there's some utility in adding an overload to XCTAssertThrowsError that accepts any Equatable ErrorType as the second parameter, to simplify the cases where you do want to test against some specific error that happens to be Equatable. Although I'm not sure how likely it is for any given ErrorType to also be Equatable. The overload might look like
 
public func XCTAssertThrowsError<T, E: ErrorType where E: Equatable>(
    @autoclosure expression: () throws -> T,
    _ message: String = "",
    file: StaticString = __FILE__,
    line: UInt = __LINE__,
    _ expectedError: E
) {
    XCTAssertThrowsError(expression, message, file: file, line: line) { error in
        if let error = error as? E {
            if error != expectedError {
                XCTFail("Expected error \(expectedError), found error \(error)")
            }
        } else {
            XCTFail("Unexpected type of error thrown: \(error)")
        }
    }
}
 
-Kevin Ballard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20160110/7916b9d1/attachment.html>