[swift-evolution] [Pitch] Typed throws

Tue Feb 28 12:36:17 CST 2017

> On Feb 28, 2017, at 11:55 AM, Karl Wagner <razielim at gmail.com> wrote:
> 
> 
>> On 28 Feb 2017, at 18:00, Matthew Johnson <matthew at anandabits.com <mailto:matthew at anandabits.com>> wrote:
>> 
>> 
>>> On Feb 28, 2017, at 10:44 AM, Daniel Leping <daniel at crossroadlabs.xyz <mailto:daniel at crossroadlabs.xyz>> wrote:
>>> 
>>> Mathew, I totally understand what you're talking about. And this is exactly what I want to avoid.
>>> 
>>> If a library developer knows how to handle certain case (i.e. recover or throw a special error) I'm ok with it. Though I'm not ok with completely hiding errors from the underlying implementation. The library developer might not handle EVERYTHING. I want to know what happened and where. Otherwise I will have to explicitly debug his library to find out what the issue is. In most cases I will not have time to wait developer fixing the library. I will fix it myself and make PR. So I consider tranparent errors rather a benefit.
>>> 
>>> Hiding errors under library errors might have worked for proprietary middleware. Today the world changed. It's open source now.
>> 
>> I agree with you that having *access* to the original error is important and that a library should not be expected to handle every case.  I hope I have not said anything that suggests otherwise.
>> 
>> What I’m suggesting is that if the error originated in a dependency that might change it should be wrapped in a stable abstraction that the users of the library can depend on.  The underlying error should still be available via a property or associated value exposed with an existential type.  If you expect users to catch errors thrown by a dependency that might change you have a fragile API contract in the area of errors.  
>> 
>> This can easily lead to unreliable software - when you change the dependency nothing alerts users to the fact that their error handling code is probably broken.  Even if the compiler told them it was broken, they now have the burden of learning about the errors that might come from your new dependency.  An API that could lead to this consequence is badly designed IMO.
>> 
>> I even suggested adding a new requirement to `Error` that includes a default implementation to standardize how we access wrapped errors, including the originating error underneath all wrapper layers.
>> 
> 
> I don’t see that it’s such a big problem. There are lots of ways to expose the information once we have it, and it could easily be limited to public errors from the current module (substituting private/unbound errors with an erasing “Error”).

If you restrict it in this way then it will require wrapping if the caller has any meaningful way to handle errors thrown by your dependencies that is not “something went wrong (but I have no idea what it is or how to fix it)”.

> There are lots of small improvements we could make: such as having functions which re-throw errors from other, manually documented functions inherit their documentation (or at least appear to, when viewed). That would not be totally compiler-generated documentation, but would mean you can document your errors in one place and make doing so much more palatable. :
> 
> // Manual documentation in one place:
> 
> /// - throws: X, Y, Z
> func doSomething() throws { … }
> 
> // Inherited by functions which call it and rethrow its errors:
> 
> /// [generated]: - throws: X, Y, X
> func doSomethingExtended() throws {
>     try doSomething()
> }
> 
> /// [generated]: - throws: X, Y, Z
> func doSomethingMapped() throws {
>     // (call to rethrowing function ‘map’) -> (throwing closure expression) -> (call to throwing function “doSomething”, maybe others, too...) -> X, Y, Z
>     try [1, 2, 3].map { try doSomething() }
> }
> 
> Personally, I don’t like wrapping.

I don’t like the boilerplate wrapping can lead to but I think the semantics are often the right semantics.  If you don’t do this you either prevent callers from handling some errors with a more specific error path or you create unwanted coupling between your callers and your dependencies.

The right way to solve this (IMO) is to find ways to eliminate the boilerplate.

> I like error-handling to be flat at each level because it’s much easier to reason about.

I like wrapping because it is often easier to reason about high-level groupings of underlying errors that have been thoughtfully designed than it is to reason about everything that can go wrong.  Often this will still result in flat error handling because the grouping is sufficient and the underlying error is irrelevant to how the error is handled.

> You would basically need catch on "error as? MyError” and “error.underlyingError as? MyError”, duplicating code or forwarding them to a refactored handler function.

Or we could introduce some syntax to allow you to ignore the wrapper when you really need to and catch the underlying error directly.  Wrapping and unwrapping is a problem that can be solved by a language feature.

> Also, consider that an Error might be any type at all (including a class). If any instance of a thing which conforms to Error could potentially be holding a reference-counted pointer, that could limit optimisations. If it’s flat, we can potentially prove that it has an exact type (or at least is never reference-counted), and that could be valuable.

I think performance is the last thing we should be concerned about in the vast majority of error handling use cases.  Providing a good experience to users which includes robust and helpful recovery and / or reporting is much more important.  In cases where there is a performance reason, it is easy enough to avoid wrapping (and typed errors) if necessary.

> 
> Error-contracts in libraries are only really fixed in the case of an implementation which is never going to change substantially. In that case, the function is a possible candidate for @inlineable and can be treated as though part of the client’s module, with full error-flow intelligence.

I’m not suggesting they be fixed at all.  Resilient enums will often be a good way to provide a wrapper.  In fact, they allow users to handle errors in a stable manner *despite* underlying errors changing over time as the implementation changes.  

What I am saying is that thoughtful categorization of underlying errors is an important API design technique.  It is true that this technique is currently burdened by unfortunate boilerplate.  This is a problem that can be solved if we decide we want to solve it.

I think we should be encouraging thoughtful design of the error contracts our APIs have rather than encouraging mindless propagation of errors thrown by our dependencies.

> 
> - Karl
> 
>>> 
>>> My point is that I'm completely against typed throws. I'm sure there will be a lot of developers who would try to use the feature to hide underlying errors under their own. I can't consider it a good practice (reasons above). This is how good intentions (caring about API and library users) can lead to bad consequences. Sounds good in theory - works bad in practice (in real, non ideal world).
>> 
>> What is your opinion about the problem I describe above?  Do you think this is unimportant?
>> 
>>> 
>>> On Tue, 28 Feb 2017 at 18:18 Matthew Johnson <matthew at anandabits.com <mailto:matthew at anandabits.com>> wrote:
>>> 
>>>> On Feb 28, 2017, at 5:09 AM, Daniel Leping <daniel at crossroadlabs.xyz <mailto:daniel at crossroadlabs.xyz>> wrote:
>>>> 
>>>> When you're going to present a user with specific error message you don't do it for all errors. First of all at the end of a call chain you will have a couple of dozens of possible errors and only 2 or 3 will do really matter for the end user. Would you list all the dozen in params? Probably not. You'll just catch what matters and put the rest to the log with a default "sorry" message.
>>>> 
>>>> What I mean is it's OK to be "unaware" why everything failed. What really does matter is what you can deal with and what you can't (the rest). "The rest" is usually a magnitude bigger, though.
>>>> 
>>>> I might agree, that listing some errors in the throws clause is useful feature for documentation (even so, I think the docs should be in the comments), but only and only if it can throw the listed errors + a bunch of other stuff. In Java it's resolved with a terrible solution of RuntimeError interface which I definitely suggest to avoid.
>>>> 
>>>> As a library developer I can say that even for a library it's a nightmare and libraries will start wrapping system errors with their own if the trend would become to list errors in middleware. My preference though is to have all errors transparently passed in both roles, being a library developer and a library user.
>>> 
>>> Whether to wrap or not is a judgment call.  
>>> 
>>> If the purpose of your library is to simply provide a cleaner experience for a system API that is a stable dependency that will not change.  Users might expect to have the errors from the system API forwarded directly and that would be perfectly appropriate.  
>>> 
>>> In other cases you might have dependencies that are considered implementation details.  In that case if you don’t wrap the errors you’re doing users a disservice by giving them an unstable interface for error handling.  Changing out your dependency that is supposed to be an implementation detail becomes a breaking change.
>>> 
>>>> 
>>> 
>>>> On Tue, 28 Feb 2017 at 8:29 Karl Wagner <razielim at gmail.com <mailto:razielim at gmail.com>> wrote:
>>> 
>>>>> On 28 Feb 2017, at 02:43, Matthew Johnson via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>>>> 
>>>>> 
>>>>> 
>>>>> Sent from my iPad
>>>>> 
>>>>> On Feb 27, 2017, at 7:19 PM, Daniel Leping <daniel at crossroadlabs.xyz <mailto:daniel at crossroadlabs.xyz>> wrote:
>>>>> 
>>>>>> Well, as Dave pointed, you can very rarely recover from an error, which IMO is absolutely true.
>>>>>> 
>>>>>> If your operation fails you don't really care unless you can recover. And you know your cases, which you can recover from (in reality one usually does it in optimization phase, though).
>>>>>> 
>>>>>> What else do you need the type of error for at the very end of your call stack? In 90% you will tell the user "ah, sorry, something happened. Come back later" and log the error (if you haven't forgot it).
>>>>>> 
>>>>>> In most cases the errors are not for recovering. They neither are to be presented to users. They are for developers to read the log/crash report/whatever else and analyze it. Most of the errors are for debugging purposes.
>>>>>> 
>>>>>> I don't want to deal with cumbersome code the purpose of which is to just "obey the language rules". Unless I know how to recover I would rethrow it. Than catch at the top of the stack and log + show the user a nice "sorry" message without getting techy.
>>>>> 
>>>>> In order to provide a helpful experience to end users an app needs to know about what might have caused the error and therefore what might (or might not) resolve it, allowing the operation to succeed on a subsequent attempt.  This can influence the strategy an app uses to avoid bothering the user if it might be resolvable without user intervention and can also influence the content of the message presented to users if that is necessary.  
>>>>> 
>>>> 
>>>> Errors are certainly useful, but the key distinction (IMO) is that callers should never rely on an error happening or not. They are typically highly implementation and hence version-specific. I think that means specific errors shouldn’t be part of the function signature.
>>>> 
>>>> That doesn’t mean you can’t handle the error. A function that throws is one that reserves the right to fail; maybe you can resolve the problem, but maybe you can’t. That means your enclosing operation also needs to be prepared to fail for reasons beyond _its_ control, and so on. At some point your entire UI-level sub-operation (like opening the file, downloading the data from the network) just fails, and you decide whether or not the user needs to be told of that and how to do so.
>>> 
>>>>> Indeed, many apps don't bother with this kind of detail and just treat all errors the same.  Personally, I find that to be an unfortunate state of affairs both as a user and as a developer.
>>>>> 
>>>>> Types can be useful in conveying this kind of information and abstracting low level details that are not helpful at higher levels in the stack.  Of course types are not the only way to convey this information.  But my experience is that untyped errors often result in libraries with poor documentation of error cases and not enough consideration of the error handling experience of users of the library in general.   That makes it very difficult to handle errors well.  I have experiences like this in several languages and on several platforms, including Apple's.  
>>>> 
>>> 
>>>> I’m not sure that types are really the best abstraction for “the list of errors that can be thrown by this function”. They’re fine for error themselves, but not for per-function error-lists IMO. Most of our types are way too rigid for this to be easy to live with.
>>>> 
>>>> If I understand what you’re saying, the core problem can be summarised as: "I don’t know/can’t easily communicate which errors this function throws”. Really, it is a documentation problem; it's too onerous to document every individual throwing function, even with our very pretty markdown-like syntax. I’m just not sure that the relatively rigid type-system is the solution.
>>>> 
>>>> Now, if we look at this from a documentation perspective: obviously, the compiler _typically_ can't generate documentation for you, because it doesn’t know what your types/functions are meant for. Error documentation, though, is different: the compiler *can* often know the specific errors which get thrown (or rethrown from a call to another function from whom it can get that information); and even when it can’t know that, often it can at least know the specific type of error. We could enhance the compiler libraries to track that; I spent half a day modifying the existing error-checker in Sema to prove the concept and it’s close.
>>>> 
>>>> Swift libraries don’t have header files, so third-parties will only ever see your documentation as generated by some tool which integrates the compiler. That means the compiler can “improve” the documentation when it is lacking. For yourself, if you care about explicitly writing out the errors which get thrown and you’re not working with an IDE, you could enable compiler-checking of your documentation comments. We can invent new shorthands to make it easier to list all of your various errors (individual cases, entire types, and maybe some private “other errors”), isolated from the rest of the language. For example:
>>>> 
>>>> /// - throws: MyError.{errorOne, errorThree}, AnotherError.*, *
>>>> func doSomething() throws
>>>> 
>>>> Maybe you can think of other ways we could help the documentation problem from the compiler side?
>>> 
>>>>> Typed errors are certainly no panacea but I believe they can be an important tool.  Experience in other languages has shown that to be the case for many people writing many kinds of software.
>>>>> 
>>>>> If you don't believe typed errors will improve your code or if you just don't want to deal with typed errors, just don't use them!  You will be able to continue using untyped throws just as you do today.  You can even do this if you use libraries that throw typed errors.
>>>>> 
>>> 
>>>> If it’s the wrong solution to the problem, it does make the language worse overall.
>>> 
>>>>>> On Tue, 28 Feb 2017 at 1:12 Matthew Johnson via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>>>>> 
>>>>>> > On Feb 27, 2017, at 5:01 PM, Dave Abrahams <dabrahams at apple.com <mailto:dabrahams at apple.com>> wrote:
>>>>>> >
>>>>>> >
>>>>>> > on Mon Feb 27 2017, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:
>>>>>> >
>>>>>> >>> On Feb 27, 2017, at 4:20 PM, Dave Abrahams <dabrahams at apple.com <mailto:dabrahams at apple.com>> wrote:
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> I'm sorry, I don't see any substantive difference, based on what you've
>>>>>> >>> written here, between this feature and const.
>>>>>> >>
>>>>>> >> Let me give it one more shot and then I’ll drop it.  :)
>>>>>> >>
>>>>>> >> Const is viral because if an API does not declare its arguments const
>>>>>> >> it cannot be used by a caller who has a const argument.
>>>>>> >
>>>>>> > Unless the caller casts away const, thus erasing information that was
>>>>>> > previously encoded in the type system.
>>>>>> 
>>>>>> Yes, of course.
>>>>>> 
>>>>>> >
>>>>>> >> It is required in order to make an API as generally useful as
>>>>>> >> possible.
>>>>>> >>
>>>>>> >> Typed errors are not viral in this way because no callers are
>>>>>> >> prevented from calling an API regardless of whether it declares error
>>>>>> >> types or just throws Error like we have today.
>>>>>> >
>>>>>> > Unless the caller can recover (which is *very* rare) or it catches and
>>>>>> > rethrows one of the errors *it* declares, thus erasing information that
>>>>>> > was previously encoded in the type system.
>>>>>> 
>>>>>> I view this as being fundamentally different than casting away const.  Casting away const says “I know better than the types”.
>>>>>> 
>>>>>> Converting an error to a different type is extremely different.  It much more similar to any other kind of value wrapper a library might create in order to shield its users from being coupled to its implementation details / dependencies.  This is not a way *around* the type system in the sense that casting away const is.  It is a way of *using* the type system (hopefully) to your advantage.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> >
>>>>>> >> Pressure to declare error types in your signature in order to make
>>>>>> >> your function as generally useful as possible does not exist.  Each
>>>>>> >> function is free to declare error types or not according to the
>>>>>> >> contract it wishes to expose.
>>>>>> >>
>>>>>> >> An argument can be made that community expectations might develop that
>>>>>> >> good APIs should declare error types and they could be considered
>>>>>> >> viral in this sense because any API that is simply declared `throws`
>>>>>> >> is dropping type information.  But I think this overstates the case.
>>>>>> >> The community appears to be very sensitive to the problems that can
>>>>>> >> arise when error types are too concrete, especially across module
>>>>>> >> boundaries.  I think we can learn to use the tool where it works well
>>>>>> >> and to avoid it where it causes problems.
>>>>>> >>
>>>>>> >>>
>>>>>> >>> --
>>>>>> >>> -Dave
>>>>>> >>
>>>>>> >
>>>>>> > --
>>>>>> > -Dave
>>>>>> 
>>>>>> _______________________________________________
>>>>>> swift-evolution mailing list
>>>>>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>>>>>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
>>> 
>>>>> _______________________________________________
>>>>> swift-evolution mailing list
>>>>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>>>>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170228/4937e140/attachment.html>