[swift-evolution] [Pitch] Revamp Optional and Throws

[Rod Brown]

> On 1 May 2017, at 8:16 pm, Gor Gyolchanyan <gor at gyolchanyan.com> wrote:
> 
> Yeah, you’re absolutely right. the “value-or-nil” and “value-or-reason-why-not-value” are two different things and the former is used too liberally in place of the latter because of lack of support.
> In that case, the Result type should not replace the error handling, but augment it. The error handling mechanism is extremely well though-out and convenient for its purpose, but it makes it difficult to capture and store a union of (value; flying error).

I agree that the key problem with the current architecture that you're alluding to is it can't be easily stored and transferred. Swift errors are great for live action but holding and passing after the throwing event is problematic, and this is an elegant solution. The storage issue is when holding it as a property, and the transferring issue is when passing it to a closure as a results of an asynchronous operation etc. These are both definitely cases where storage of the type-or-error makes perfect sense.

I think the key problem getting this accepted by the Swift Team will be that it doesn't currently have any specific use in the standard library. As a low level set of types, errors are generated by the lower levels but rarely stored, so the Standard library doesn't need the storage. Generally the only place we have to do that is in end user code. And currently the standard library doesn't have to support asynchronous operations natively, so there's nothing inside the kit that would require it to do completion handlers with errors.

This would therefore be an element in the standard library purely so we don't have 50,000 different libraries with 50,000 different result types. I'd love to see this standardised so frameworks were more compatible. I'm just not sure whether the Core Team would see it as pressing to try and officiate a certain type that they themselves don't use.


> A built-in Failable enum with syntactic support to losslessly catch it from a throwing expression and unpack it into a throwing scope would be a very useful feature.
> Optionals are extremely convenient, but in cases where the Optional is used as “value-or-error” rather then “value-or-nil” it falls a bit short and the programmer has to choose between extreme convenience of Optionals with the downside of lack of error information or the expressive power of throwing an error with the downside of a lot of boilerpate and poor integration with generics.
> Here’s an example pseudo-swift that illustrates this:
> 
> enum Failable<Wrapped> {
> 	
> 	case success(Wrapped)
> 
> 	case failure(Error)
> 
> }
> 
> func foo() throws -> Int {
> 	guard myCondition else {
> 		throw EmbarressingError.oops
> 	}
> 	return 42
> }
> 
> let failable = catch foo() // Failable<Int>
> 
> func bar() throws -> Int {
> 	throw failable
> 
> 
> 
>> On May 1, 2017, at 11:17 AM, Rod Brown <rodney.brown6 at icloud.com> wrote:
>> 
>> The problem I see with your argument is that the core reason why the optional cast failed is actually there: It was an optional value, and you forced it to unwrap without checking. This is a correct description of the error.
>> 
>> If we plumbed our code with a tonne of errors saying “why this is optional, and why it is null” then we are practically making every optional an error in the case of nil, which is completely illogical considering that nil could be a completely legitimate case (especially in the case of not-implicitly-unwrapped optionals).
>> 
>> Optional is a wrapper for "value-or-null", not "value-or-reason-not-value".
>> 
>> The type you are talking about is a result/sum type as has been mentioned, which is fine, and is completely valid (I use them a lot too) but they are definitely not the same thing as an optional, and I think you’re conflating the two ideas.
>> 
>> - Rod
>> 
>> 
>>> On 1 May 2017, at 5:58 pm, Gor Gyolchanyan via swift-evolution <swift-evolution at swift.org> wrote:
>>> 
>>> I have read those documents before, but It’s worth re-reading them to see if I missed something, but I’l still explain my motivation and seek arguments against the postulated problem (rather then a specific solution).
>>> 
>>> (a) There are different types of error.
>>> 
>>> Yes, there are different types of error in Swift, which require different reactions from the programmer.
>>> If I’m not missing something, the three main types of error in Swift are:
>>>  - Simple encapsulatable errors that are expected to be treated as normal values until the time comes for someone to take care of them by unpacking the content.
>>>  - Automatic propagatable errors that require the programmer to either handle the error immediately or propagate it by delegating to its own caller.
>>> - Fatal errors, which represent logic errors and broken invariants and preconditions, which are purely a programmer error and should not be dealt with dynamically, hence the terminated process with a message.
>>> 
>>> (b) The programmer is expected to react differently to different types of error.
>>> 
>>> Yes, and the three main ways a programmer is expected to react to the an error are:
>>>  - If it’s an optional, they’re encouraged to store and pass it around freely until someone down the line decides to unpack it and deal with the possibility that it isn’t there.
>>>  - If it’s an error, they’re encouraged to either handle it on the spot or declare themselves throwing and delegate the responsibility to the caller.
>>>  - Look at the standard output and figure out why the fatal error occurred, perhaps with the help of the debugger.
>>> 
>>> (c) The language is a tool to help the programmer react.
>>> 
>>> Yes, that comes in the form of three language constructs:
>>>  - Optionals, which allow storing a union of a value and its absence (for an undefined and hopefully obvious reason).
>>>  - Throwing functions, which allow making sure that the error will be handled as soon as possible.
>>>  - Fatal errors, which allow the programmer to mark points in code which should never be reached in a correct system in order to keep the logic from going AWOL in case the programmer screwed up somewhere.
>>> 
>>> (d) Optionals and errors are not unified, and unification is a non-goal, because they are designed to help the programmer react differently to different types of error.
>>> 
>>> Yes, and those different types of error with different reactions are all valid and shouldn’t be unified.
>>> My point is that the language should make it easy for a programmer to transition from one type of error to another, because the same error has different severity in different contexts.
>>> For instance, a “file not found” error when trying to open a file handler is not critical at all in the context of the file opening function, because it’s a perfectly expected outcome of the operation.
>>> However, for a module that handles loading critical data from disk (like encryption keys needed to decrypt the manipulated content) it is a critical error that cannot be dealt with.
>>> In this case it deserves to be a fatal error, because the programmer didn’t bother to implement a logic for creating the missing file or properly notifying the user of the lacking permissions to do so.
>>> Conversely, some errors start as being urgent (like a JSON parser that throws an error when it encounters invalid syntax), but become less urgent for the client (a JSON editor that simply displays the error message).
>>> 
>>> As for my use case:
>>> 
>>> I have a JSON parser that may throw, and I have a a JSON Editor class that allows editing JSON files as well as displaying the parsing errors.
>>> I have a malformed JSON file that I open in the editor. The JSON parser throws an error, which should be caught and stored somewhere for the editor to display.
>>> I have file reader that reads a file in some encoding and returns an optional string with the file contents (nil means file couldn’t be read or the encoding is wrong).
>>> 
>>> For the JSON parser, a malformed JSON file is an obvious error, but for the editor, it’s a perfectly valid and expected condition, which doesn’t deserve to be an error.
>>> Therefore, the thrown error of the JSON parse has to be caught and encapsulated indiscriminately to demote it from an error to a return value.
>>> Conversely, the returned nil form the file reader is perfectly valid and expected condition, but for the editor, it’s an error.
>>> Therefore, the returned nil should be checked and converted to an error that will be thrown to promote it to a full-fledged error.
>>> 
>>> I would want to have a way to easily promote/demote different types of errors to accommodate the changing perception of their urgency.
>>> For instance, by being able to throw an optional, thus introducing a new way of unpacking it (by promoting it to an error). Currently, it is by manually unpacking the optional, deciding what error to throw and throwing it manually.
>>> Or, being able to catch an error into an optional, thus introducing a new way of handling it (by demoting it to an optional). There is a way to do that currently in the form of `try?` and `try!`, but their downside is that they are lossy (losing the error itself).
>>> 
>>> All I want is for the language to help me losslessly catch errors into something like an optional, except with the error intact with the possibility of easily re-throwing it in the future.
>>> This would also solve the problem of multiple throwing calls having different urgency to them and being forced to write a lot of boilerplate to catch their errors individually and deal with them separetely.
>>> 
>>>> On May 1, 2017, at 1:44 AM, Xiaodi Wu <xiaodi.wu at gmail.com> wrote:
>>>> 
>>>>> On Sun, Apr 30, 2017 at 5:05 PM, Gor Gyolchanyan <gor at gyolchanyan.com> wrote:
>>>>> 
>>>>>> On May 1, 2017, at 12:10 AM, Xiaodi Wu <xiaodi.wu at gmail.com> wrote:
>>>>>> 
>>>>>> You may wish to read the rationale behind the current error handling design:
>>>>>> 
>>>>>> https://github.com/apple/swift/blob/master/docs/ErrorHandlingRationale.rst
>>>>>> 
>>>>>> A Result type like you suggest has been considered and rejected in favor of the current design. Briefly, optionals and throwing errors are distinct because they are considered superior ways for handling distinct types of error.
>>>>>> 
>>>>>> In the case of a simple domain error, there is only one way to fail; therefore, optional return values are considered the best way to model that error.
>>>>>> 
>>>>>> In the case of a recoverable error, the document above describes why marked propagation (the current implementation in Swift) is considered superior to typed propagation (your suggestion).
>>>>> 
>>>>> 
>>>>> My proposal is not about replacing Optionals and throwing functions with a Result type, it’s about separating the representation of an error from its propagation.
>>>>> Optionals and throwing functions solve two different problems, but they are not dealing with two different types of error.
>>>> 
>>>> The basic premise of Swift error handling design is that there exist different types of error. From the document:
>>>> 
>>>> > What is an error? There may be many different possible error conditions in a program, but they can be categorized into several kinds based on how programmers should be expected to react to them. Since the programmer is expected to react differently, and since the language is the tool of the programmer's reaction, it makes sense for each group to be treated differently in the language.
>>>> 
>>>>> Optionals are for storing and representing a value that might not exist (most commonly due to an unambiguous error).
>>>>> Error handling is for propagating an error.
>>>>> Returning an optional is essentially the same as returning a non-optional and throwing a dedicated “something went wrong” error, because due to the optional unwrapping mechanic, you cannot avoid dealing with the fact that there might have been an error. Optionals only allow you to delay the inevitable error handling, not avoid it. The use cases where the exact reason for an error is no important have nothing to do with whether or not that error should be available. The optional chaining, if-let statements and all other ways one might try to handle an optional value do not fundamentally require lack of error information.
>>>>> The error handling mechanism, on the other hand, does not concern itself with representing the error, but only propagating it. Even an optional value with a general-purpose .none case has different levels of importance in different cases. More often than not, when propagating an optional value to a non-optional target, you’ll be stuck with dealing with the error immediately, which is exactly what throwing functions force you to do.
>>>>> I suggest we enhance the current error representation and propagation mechanisms to be able to seamlessly handle cases where an erroneous value need to be stored as-is (along with its error) or unpacked and propagated (by throwing the error), not just representing a general “error”.
>>>>> The general use case is to be able to catch a throwing call into an enum that stores the value or the error and then being able to unpack it in a throwing context (unpack it or throw the error).
>>>>> This use case is a strict superset of the current Optional mechanic (catching a potentially missing value) and error handling (always manually throwing an error after manual checks).
>>>>> The aforementioned suggestion about how to do that is indeed faulty, as pointed out by Robert Widmann, but the problem is still valid, in my opinion.
>>>> 
>>>> I'd highly recommend taking some time to digest the existing rationale. You're basing your argument on contradicting the fundamental premise of the existing design, which begins with this: (a) there are different types of error; (b) the programmer is expected to react differently to different types of error; (c) the language is a tool to help the programmer react; (d) optionals and errors are not unified, and unification is a non-goal, because they are designed to help the programmer react differently to different types of error.
>>>> 
>>>> Do you have a specific use case in mind that is not well accommodated by optionals or by throwing functions? What is it? Into what category does that use case fall, in terms of the types of error enumerated in the error handling rationale document?
>>>> 
>>>>>>> On Sun, Apr 30, 2017 at 13:51 Gor Gyolchanyan via swift-evolution <swift-evolution at swift.org> wrote:
>>>>>>> 
>>>>>>> > On Apr 30, 2017, at 9:29 PM, Robert Widmann <devteam.codafi at gmail.com> wrote:
>>>>>>> >
>>>>>>> >
>>>>>>> >> On Apr 30, 2017, at 1:43 PM, Gor Gyolchanyan <gor at gyolchanyan.com> wrote:
>>>>>>> >>
>>>>>>> >> It doesn’t have to be a massive source-break, since this pitch is supposed to be a strict superset of what Optional and throwing is currently.
>>>>>>> >> The only thing that I can think of at this moment that would break is this syntax:
>>>>>>> >>
>>>>>>> >> let foo: Int? = .none // Error: Can’t convert (Error) -> Int? to Int?
>>>>>>> >>
>>>>>>> >
>>>>>>> > Except it’s not a strict superset if you break every use of this case as an RValue.  Especially when so much of Swift’s syntax and major patterns revolve around the manipulation of optionals.
>>>>>>> >
>>>>>>> >> The ExpressibleByNilLiteral, the try/throw syntax, all of those things would work as they are right now.
>>>>>>> >> Error handling as it is currently, is essentially a hidden `error` out parameter and a whole bunch of codegen.
>>>>>>> >> Even the semantical changes described earlier would be purely additive.
>>>>>>> >
>>>>>>> > Don’t get me wrong, I think you’ve identified the problem space well, I just disagree with the solution.
>>>>>>> 
>>>>>>> Yeah, you’re right. It would take some next-level fixits to deal with the consequences of changing the most fundamental data type of Swift I can think of.
>>>>>>> I’d really appreciate it if you’d offer an alternative solution to this problem.
>>>>>>> The problem, as I understand it, is as follows:
>>>>>>> 
>>>>>>> A lot of Swift’s logic revolves around the notion that some values might be missing for whatever reason and some functions might fail for whatever reason.
>>>>>>> Any function’s effect can be summed up as the union of its return value and the global state that it changes (that includes captured closure scopes).
>>>>>>> This could be boiled down to the statement that “Values that a function sets and returns completely express the purpose of the function”.
>>>>>>> The optional gives an extremely convenient way of representing values that might not exist (which, when returned from a function often means “failed for an unknown reason”).
>>>>>>> The fact that Optional is a type, rather then a function attribute allows us to store and imperatively manipulate the outcome of logically failable functions, but unfortunately, it doesn’t allow us to reason about the cause of the failure.
>>>>>>> On the other hand, throwing functions captures the logic of dealing with specific failures very well, but does not allow us to store and manipulate them easily, leaving us with workarounds like wrapping errors in enums with values and re-throwing the errors on their way out of the generic pipeline.
>>>>>>> I’d like to come up with a solution that would unify the optionals and the throwing functions into a single mechanism for dealing with the concept of failure, taking the best of both worlds and getting the benefits of the new synergies.
>>>>>>> This pitch was a first rough idea about the direction in which we could go in trying to find a solution.
>>>>>>> I chose to enhance Optional instead of introducing a new type like Failable, so that we could make do with minimal language changes and migration procedures.
>>>>>>> 
>>>>>>> This problem is kinda similar to the variadic parameter problem, which makes it impossible to forward calls to variadic functions simply because that feature is too magical and does not provide a way to store and propagate its logic.
>>>>>>> 
>>>>>>> Another way I could think of solving it would be to allow overloading the postfix `!` and `?` operators (which would currently only be defined for Optionals), which would allow us to define the Failable enum type with some error handling syntax integration and make it feel more at home in the midst of Optionals.
>>>>>>> 
>>>>>>> Or better yet, make an OptionalProtocol and move the current magical logic to it, leaving the existing Optional perfectly intact and allowing userspace implementations.
>>>>>>> This would also greatly benefit numerous use cases of “invalidatable” types (like file handlers that can be closed) that would no longer have to either fatalError or use unwieldy wrappers that operate on Optionals.
>>>>>>> 
>>>>>>> > ~Robert Widmann
>>>>>>> >
>>>>>>> >>
>>>>>>> >>> On Apr 30, 2017, at 8:35 PM, Robert Widmann <devteam.codafi at gmail.com> wrote:
>>>>>>> >>>
>>>>>>> >>> This "revamp" is isomorphic to adding a Sum type to stdlib and plumbing error handling syntax through.  I'd much rather see that than the massive source-break this would entail.
>>>>>>> >>>
>>>>>>> >>> ~Robert Widmann
>>>>>>> >>>
>>>>>>> >>> 2017/04/30 13:11、Gor Gyolchanyan via swift-evolution <swift-evolution at swift.org> のメッセージ:
>>>>>>> >>>
>>>>>>> >>>> I’d like to suggest a bit of redesigning the Optional type and throwing functions to provide a single powerful and flexible mechanism for dealing with unexpected situations.
>>>>>>> >>>>
>>>>>>> >>>> In short, The Optional would have an associated value of type Error added to its `none` case, which would describe the reason why the wrapped value is missing.
>>>>>>> >>>>
>>>>>>> >>>> public enum Optional<Wrapped> {
>>>>>>> >>>>
>>>>>>> >>>> case .some(Wrapped)
>>>>>>> >>>>
>>>>>>> >>>> case .none(Error)
>>>>>>> >>>>
>>>>>>> >>>> }
>>>>>>> >>>>
>>>>>>> >>>> The Optional's ExpressibleByNilLiteral would initialize it with an error that corresponds to what is currently fatalError-ed as "unexpectedly found nil while unwrapping an Optional value".
>>>>>>> >>>>
>>>>>>> >>>> The forced unwrapping operator (postfix `!`) would behave the same way as it does now, except in case of a fatal error it would print out the underlying error, instead of the aforementioned hard-coded string.
>>>>>>> >>>>
>>>>>>> >>>> The optional chaining operator (postfix `?`) would behave the same way as it does now, except when it stops evaluating and returns the Optional, it would contain the error, returned by the sub-expression that failed to evaluate.
>>>>>>> >>>>
>>>>>>> >>>> Any throwing function would be equivalent to a function that returns an Optional. If the function is declared as throwing and returning an Optional at the same time, it would be equivalent to a function returning an Optional Optional.
>>>>>>> >>>>
>>>>>>> >>>> The if-let statement would bind the `let` variable to the wrapped value inside the "then" block and would bind it to the error in the "else" block. Chained else-if blocks would all be considered part of the overarching "else" block, so all of them would be able to access the error bound to the if-let name.
>>>>>>> >>>>
>>>>>>> >>>> The guard-let and case-let statements are essentially just rewrites of if-let with some added logic.
>>>>>>> >>>>
>>>>>>> >>>> The `try` keyword, applied to an optional would behave like this:
>>>>>>> >>>>
>>>>>>> >>>> public func try<T>(_ optional: T?) throws -> T {
>>>>>>> >>>> guard let wrapped = optional else {
>>>>>>> >>>>     throw wrapped // Remember, if-let, guard-let and case-let statements bind the let name to the error in case of a failure.
>>>>>>> >>>> }
>>>>>>> >>>> return wrapped
>>>>>>> >>>> }
>>>>>>> >>>>
>>>>>>> >>>> Multiple let bindings in a single if-let statement are essentially rewrites of a nested chain of if-let statements.
>>>>>>> >>>>
>>>>>>> >>>> The `try` keyword applied to an optional would unwrap the value or throw the error.
>>>>>>> >>>> The `try?` keyword applied to a throwing function call would cause any thrown errors to be caught and put into the returned Optional, instead of simply ignored.
>>>>>>> >>>> The `try!` keyword applied to a throwing function call would behave as you'd expect: just like `try?` except immediately force-unwrapped.
>>>>>>> >>>>
>>>>>>> >>>> A throwing function would be convertible to a non-throwing optional-returning function and vice versa.
>>>>>>> >>>> This would allow making use of throwing functions when dealing with generics or protocols that allow arbitrary return types, without having to sacrifice the convenience of error-handling logic. Conversely, it would allow to write generic code that deals with any type of function without having to implement special cases for throwing functions. This means that the two function types would be interchangeable and one would be able to satisfy protocol requirements of the other. The `rethrows` idiom would then become a natural consequence of writing generic functions that may return optional and non-optional results just as well.
>>>>>>> >>>>
>>>>>>> >>>> _______________________________________________
>>>>>>> >>>> swift-evolution mailing list
>>>>>>> >>>> swift-evolution at swift.org
>>>>>>> >>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>>>>>> >>
>>>>>>> >
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> swift-evolution mailing list
>>>>>>> swift-evolution at swift.org
>>>>>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>> 
>>> _______________________________________________
>>> swift-evolution mailing list
>>> swift-evolution at swift.org
>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170501/edd2b321/attachment.html>