[swift-evolution] [Pitch] Revamp Optional and Throws

[Gor Gyolchanyan]

The real beauty of this added mechanism is when you consider generics. It would be easy to write a generic function that operates on both throwing and non-throwing parameters, while preserving their throwing-ness. Think `rethrows` except not limited to immediate nonescaping calls.
This part needs some serious thought, but having throwing functions play nice with generics would be spectacular.

> On May 1, 2017, at 1:53 PM, Gor Gyolchanyan <gor at gyolchanyan.com> wrote:
> 
> I guess I failed to communicate my thoughts properly. My bad.
> You’re right, this is a completely orthogonal issue and you're right: this does take nothing more then a new enum type and syntax sugar on top of it.
> 
> My first guess is to allow this:
> 
> func foo() throws -> Int {
> 	guard myCondition else {
> 		throw EmbarrassingError.oops
> 	}
> 	return 42
> }
> 
> let failable: Failable<Int> = catch foo()
> 
> func bar() throws -> String {
> 	let int = try failable
> 	return “\(int)"
> }
> 
> This doesn’t intersect with any existing syntax, so it should be additive.
> 
>> On May 1, 2017, at 1:15 PM, Xiaodi Wu <xiaodi.wu at gmail.com> wrote:
>> 
>>> On Mon, May 1, 2017 at 2:58 AM, Gor Gyolchanyan <gor at gyolchanyan.com> wrote:
>>> I have read those documents before, but It’s worth re-reading them to see if I missed something, but I’l still explain my motivation and seek arguments against the postulated problem (rather then a specific solution).
>>> 
>>> (a) There are different types of error.
>>> 
>>> Yes, there are different types of error in Swift, which require different reactions from the programmer.
>>> If I’m not missing something, the three main types of error in Swift are:
>>>  - Simple encapsulatable errors that are expected to be treated as normal values until the time comes for someone to take care of them by unpacking the content.
>>>  - Automatic propagatable errors that require the programmer to either handle the error immediately or propagate it by delegating to its own caller.
>>> - Fatal errors, which represent logic errors and broken invariants and preconditions, which are purely a programmer error and should not be dealt with dynamically, hence the terminated process with a message.
>>> 
>>> (b) The programmer is expected to react differently to different types of error.
>>> 
>>> Yes, and the three main ways a programmer is expected to react to the an error are:
>>>  - If it’s an optional, they’re encouraged to store and pass it around freely until someone down the line decides to unpack it and deal with the possibility that it isn’t there.
>>>  - If it’s an error, they’re encouraged to either handle it on the spot or declare themselves throwing and delegate the responsibility to the caller.
>>>  - Look at the standard output and figure out why the fatal error occurred, perhaps with the help of the debugger.
>>> 
>>> (c) The language is a tool to help the programmer react.
>>> 
>>> Yes, that comes in the form of three language constructs:
>>>  - Optionals, which allow storing a union of a value and its absence (for an undefined and hopefully obvious reason).
>>>  - Throwing functions, which allow making sure that the error will be handled as soon as possible.
>>>  - Fatal errors, which allow the programmer to mark points in code which should never be reached in a correct system in order to keep the logic from going AWOL in case the programmer screwed up somewhere.
>>> 
>>> (d) Optionals and errors are not unified, and unification is a non-goal, because they are designed to help the programmer react differently to different types of error.
>>> 
>>> Yes, and those different types of error with different reactions are all valid and shouldn’t be unified.
>>> My point is that the language should make it easy for a programmer to transition from one type of error to another, because the same error has different severity in different contexts.
>>> For instance, a “file not found” error when trying to open a file handler is not critical at all in the context of the file opening function, because it’s a perfectly expected outcome of the operation.
>>> However, for a module that handles loading critical data from disk (like encryption keys needed to decrypt the manipulated content) it is a critical error that cannot be dealt with.
>>> In this case it deserves to be a fatal error, because the programmer didn’t bother to implement a logic for creating the missing file or properly notifying the user of the lacking permissions to do so.
>>> Conversely, some errors start as being urgent (like a JSON parser that throws an error when it encounters invalid syntax), but become less urgent for the client (a JSON editor that simply displays the error message).
>> 
>> Again, that's not my understanding for the rationale behind having both Optional return values and errors. It's not that one is more "urgent" or "important" than the other. An Optional is used when something can only fail in one obvious way; an error is thrown when it can fail in multiple, but recoverable, ways. You can care a lot about a nil value and not at all about an error, or vice versa.
>> 
>> 
>>> As for my use case:
>>> 
>>> I have a JSON parser that may throw, and I have a a JSON Editor class that allows editing JSON files as well as displaying the parsing errors.
>>> I have a malformed JSON file that I open in the editor. The JSON parser throws an error, which should be caught and stored somewhere for the editor to display.
>>> I have file reader that reads a file in some encoding and returns an optional string with the file contents (nil means file couldn’t be read or the encoding is wrong).
>>> 
>>> For the JSON parser, a malformed JSON file is an obvious error, but for the editor, it’s a perfectly valid and expected condition, which doesn’t deserve to be an error.
>>> Therefore, the thrown error of the JSON parse has to be caught and encapsulated indiscriminately to demote it from an error to a return value.
>>> Conversely, the returned nil form the file reader is perfectly valid and expected condition, but for the editor, it’s an error.
>>> Therefore, the returned nil should be checked and converted to an error that will be thrown to promote it to a full-fledged error.
>>> 
>>> I would want to have a way to easily promote/demote different types of errors to accommodate the changing perception of their urgency.
>>> For instance, by being able to throw an optional, thus introducing a new way of unpacking it (by promoting it to an error). Currently, it is by manually unpacking the optional, deciding what error to throw and throwing it manually.
>>> Or, being able to catch an error into an optional, thus introducing a new way of handling it (by demoting it to an optional). There is a way to do that currently in the form of `try?` and `try!`, but their downside is that they are lossy (losing the error itself).
>>> 
>>> All I want is for the language to help me losslessly catch errors into something like an optional, except with the error intact with the possibility of easily re-throwing it in the future.
>> 
>> This is an orthogonal concern to whether to model an error with an Optional. Optional is used when there's only one obvious (and recoverable) way of failing; throwing an error is used when there are multiple recoverable ways of failing. As Rod mentions, the _point_ of an Optional is that it doesn't come with an associated Error type. Every Optional.none compares equal to every other Optional.none, even when the wrapped type differs. Obviously, this means that there's no error to propagate manually or automatically, but whether or not an error propagates automatically is not the reason to use or not to use Optional to model your error. Likewise, the _point_ of using `try!` (not a downside) is to _lose_ the error, not just to stop the propagation of it.
>> 
>> By contrast, your use case is about _storing_ an error, which is totally the opposite of what Optional does. It sounds like you don't like how errors are designed to automatically propagate (albeit with marking at the origination site), and you want to manually propagate errors using a Result<T, Error> type instead. I guess this is what you mean by errors being "urgent" and wanting to "demote" it. I wouldn't disagree that it's worth thinking about a syntax to offer some additional control over that. Suppose, for instance, we had a new type:
>> 
>> ```
>> enum Result<Wrapped, Error> {
>>   case some(Wrapped)
>>   case none(Error)
>> }
>> ```
>> 
>> We could invent a new operator `try*`, which returns a `Result` instead of throwing. And then we could invent new sugar; perhaps, `result?` would be sugar for `result.some` and `result*` would be sugar for `result.none`. I'm not proposing this, but I can see how something along these lines would give you more options.
>> 
>> 
>>> This would also solve the problem of multiple throwing calls having different urgency to them and being forced to write a lot of boilerplate to catch their errors individually and deal with them separetely.
>>> 
>>>> On May 1, 2017, at 1:44 AM, Xiaodi Wu <xiaodi.wu at gmail.com> wrote:
>>>> 
>>>>> On Sun, Apr 30, 2017 at 5:05 PM, Gor Gyolchanyan <gor at gyolchanyan.com> wrote:
>>>>> 
>>>>>> On May 1, 2017, at 12:10 AM, Xiaodi Wu <xiaodi.wu at gmail.com> wrote:
>>>>>> 
>>>>>> You may wish to read the rationale behind the current error handling design:
>>>>>> 
>>>>>> https://github.com/apple/swift/blob/master/docs/ErrorHandlingRationale.rst
>>>>>> 
>>>>>> A Result type like you suggest has been considered and rejected in favor of the current design. Briefly, optionals and throwing errors are distinct because they are considered superior ways for handling distinct types of error.
>>>>>> 
>>>>>> In the case of a simple domain error, there is only one way to fail; therefore, optional return values are considered the best way to model that error.
>>>>>> 
>>>>>> In the case of a recoverable error, the document above describes why marked propagation (the current implementation in Swift) is considered superior to typed propagation (your suggestion).
>>>>> 
>>>>> 
>>>>> My proposal is not about replacing Optionals and throwing functions with a Result type, it’s about separating the representation of an error from its propagation.
>>>>> Optionals and throwing functions solve two different problems, but they are not dealing with two different types of error.
>>>> 
>>>> The basic premise of Swift error handling design is that there exist different types of error. From the document:
>>>> 
>>>> > What is an error? There may be many different possible error conditions in a program, but they can be categorized into several kinds based on how programmers should be expected to react to them. Since the programmer is expected to react differently, and since the language is the tool of the programmer's reaction, it makes sense for each group to be treated differently in the language.
>>>> 
>>>>> Optionals are for storing and representing a value that might not exist (most commonly due to an unambiguous error).
>>>>> Error handling is for propagating an error.
>>>>> Returning an optional is essentially the same as returning a non-optional and throwing a dedicated “something went wrong” error, because due to the optional unwrapping mechanic, you cannot avoid dealing with the fact that there might have been an error. Optionals only allow you to delay the inevitable error handling, not avoid it. The use cases where the exact reason for an error is no important have nothing to do with whether or not that error should be available. The optional chaining, if-let statements and all other ways one might try to handle an optional value do not fundamentally require lack of error information.
>>>>> The error handling mechanism, on the other hand, does not concern itself with representing the error, but only propagating it. Even an optional value with a general-purpose .none case has different levels of importance in different cases. More often than not, when propagating an optional value to a non-optional target, you’ll be stuck with dealing with the error immediately, which is exactly what throwing functions force you to do.
>>>>> I suggest we enhance the current error representation and propagation mechanisms to be able to seamlessly handle cases where an erroneous value need to be stored as-is (along with its error) or unpacked and propagated (by throwing the error), not just representing a general “error”.
>>>>> The general use case is to be able to catch a throwing call into an enum that stores the value or the error and then being able to unpack it in a throwing context (unpack it or throw the error).
>>>>> This use case is a strict superset of the current Optional mechanic (catching a potentially missing value) and error handling (always manually throwing an error after manual checks).
>>>>> The aforementioned suggestion about how to do that is indeed faulty, as pointed out by Robert Widmann, but the problem is still valid, in my opinion.
>>>> 
>>>> I'd highly recommend taking some time to digest the existing rationale. You're basing your argument on contradicting the fundamental premise of the existing design, which begins with this: (a) there are different types of error; (b) the programmer is expected to react differently to different types of error; (c) the language is a tool to help the programmer react; (d) optionals and errors are not unified, and unification is a non-goal, because they are designed to help the programmer react differently to different types of error.
>>>> 
>>>> Do you have a specific use case in mind that is not well accommodated by optionals or by throwing functions? What is it? Into what category does that use case fall, in terms of the types of error enumerated in the error handling rationale document?
>>>> 
>>>>>>> On Sun, Apr 30, 2017 at 13:51 Gor Gyolchanyan via swift-evolution <swift-evolution at swift.org> wrote:
>>>>>>> 
>>>>>>> > On Apr 30, 2017, at 9:29 PM, Robert Widmann <devteam.codafi at gmail.com> wrote:
>>>>>>> >
>>>>>>> >
>>>>>>> >> On Apr 30, 2017, at 1:43 PM, Gor Gyolchanyan <gor at gyolchanyan.com> wrote:
>>>>>>> >>
>>>>>>> >> It doesn’t have to be a massive source-break, since this pitch is supposed to be a strict superset of what Optional and throwing is currently.
>>>>>>> >> The only thing that I can think of at this moment that would break is this syntax:
>>>>>>> >>
>>>>>>> >> let foo: Int? = .none // Error: Can’t convert (Error) -> Int? to Int?
>>>>>>> >>
>>>>>>> >
>>>>>>> > Except it’s not a strict superset if you break every use of this case as an RValue.  Especially when so much of Swift’s syntax and major patterns revolve around the manipulation of optionals.
>>>>>>> >
>>>>>>> >> The ExpressibleByNilLiteral, the try/throw syntax, all of those things would work as they are right now.
>>>>>>> >> Error handling as it is currently, is essentially a hidden `error` out parameter and a whole bunch of codegen.
>>>>>>> >> Even the semantical changes described earlier would be purely additive.
>>>>>>> >
>>>>>>> > Don’t get me wrong, I think you’ve identified the problem space well, I just disagree with the solution.
>>>>>>> 
>>>>>>> Yeah, you’re right. It would take some next-level fixits to deal with the consequences of changing the most fundamental data type of Swift I can think of.
>>>>>>> I’d really appreciate it if you’d offer an alternative solution to this problem.
>>>>>>> The problem, as I understand it, is as follows:
>>>>>>> 
>>>>>>> A lot of Swift’s logic revolves around the notion that some values might be missing for whatever reason and some functions might fail for whatever reason.
>>>>>>> Any function’s effect can be summed up as the union of its return value and the global state that it changes (that includes captured closure scopes).
>>>>>>> This could be boiled down to the statement that “Values that a function sets and returns completely express the purpose of the function”.
>>>>>>> The optional gives an extremely convenient way of representing values that might not exist (which, when returned from a function often means “failed for an unknown reason”).
>>>>>>> The fact that Optional is a type, rather then a function attribute allows us to store and imperatively manipulate the outcome of logically failable functions, but unfortunately, it doesn’t allow us to reason about the cause of the failure.
>>>>>>> On the other hand, throwing functions captures the logic of dealing with specific failures very well, but does not allow us to store and manipulate them easily, leaving us with workarounds like wrapping errors in enums with values and re-throwing the errors on their way out of the generic pipeline.
>>>>>>> I’d like to come up with a solution that would unify the optionals and the throwing functions into a single mechanism for dealing with the concept of failure, taking the best of both worlds and getting the benefits of the new synergies.
>>>>>>> This pitch was a first rough idea about the direction in which we could go in trying to find a solution.
>>>>>>> I chose to enhance Optional instead of introducing a new type like Failable, so that we could make do with minimal language changes and migration procedures.
>>>>>>> 
>>>>>>> This problem is kinda similar to the variadic parameter problem, which makes it impossible to forward calls to variadic functions simply because that feature is too magical and does not provide a way to store and propagate its logic.
>>>>>>> 
>>>>>>> Another way I could think of solving it would be to allow overloading the postfix `!` and `?` operators (which would currently only be defined for Optionals), which would allow us to define the Failable enum type with some error handling syntax integration and make it feel more at home in the midst of Optionals.
>>>>>>> 
>>>>>>> Or better yet, make an OptionalProtocol and move the current magical logic to it, leaving the existing Optional perfectly intact and allowing userspace implementations.
>>>>>>> This would also greatly benefit numerous use cases of “invalidatable” types (like file handlers that can be closed) that would no longer have to either fatalError or use unwieldy wrappers that operate on Optionals.
>>>>>>> 
>>>>>>> > ~Robert Widmann
>>>>>>> >
>>>>>>> >>
>>>>>>> >>> On Apr 30, 2017, at 8:35 PM, Robert Widmann <devteam.codafi at gmail.com> wrote:
>>>>>>> >>>
>>>>>>> >>> This "revamp" is isomorphic to adding a Sum type to stdlib and plumbing error handling syntax through.  I'd much rather see that than the massive source-break this would entail.
>>>>>>> >>>
>>>>>>> >>> ~Robert Widmann
>>>>>>> >>>
>>>>>>> >>> 2017/04/30 13:11、Gor Gyolchanyan via swift-evolution <swift-evolution at swift.org> のメッセージ:
>>>>>>> >>>
>>>>>>> >>>> I’d like to suggest a bit of redesigning the Optional type and throwing functions to provide a single powerful and flexible mechanism for dealing with unexpected situations.
>>>>>>> >>>>
>>>>>>> >>>> In short, The Optional would have an associated value of type Error added to its `none` case, which would describe the reason why the wrapped value is missing.
>>>>>>> >>>>
>>>>>>> >>>> public enum Optional<Wrapped> {
>>>>>>> >>>>
>>>>>>> >>>> case .some(Wrapped)
>>>>>>> >>>>
>>>>>>> >>>> case .none(Error)
>>>>>>> >>>>
>>>>>>> >>>> }
>>>>>>> >>>>
>>>>>>> >>>> The Optional's ExpressibleByNilLiteral would initialize it with an error that corresponds to what is currently fatalError-ed as "unexpectedly found nil while unwrapping an Optional value".
>>>>>>> >>>>
>>>>>>> >>>> The forced unwrapping operator (postfix `!`) would behave the same way as it does now, except in case of a fatal error it would print out the underlying error, instead of the aforementioned hard-coded string.
>>>>>>> >>>>
>>>>>>> >>>> The optional chaining operator (postfix `?`) would behave the same way as it does now, except when it stops evaluating and returns the Optional, it would contain the error, returned by the sub-expression that failed to evaluate.
>>>>>>> >>>>
>>>>>>> >>>> Any throwing function would be equivalent to a function that returns an Optional. If the function is declared as throwing and returning an Optional at the same time, it would be equivalent to a function returning an Optional Optional.
>>>>>>> >>>>
>>>>>>> >>>> The if-let statement would bind the `let` variable to the wrapped value inside the "then" block and would bind it to the error in the "else" block. Chained else-if blocks would all be considered part of the overarching "else" block, so all of them would be able to access the error bound to the if-let name.
>>>>>>> >>>>
>>>>>>> >>>> The guard-let and case-let statements are essentially just rewrites of if-let with some added logic.
>>>>>>> >>>>
>>>>>>> >>>> The `try` keyword, applied to an optional would behave like this:
>>>>>>> >>>>
>>>>>>> >>>> public func try<T>(_ optional: T?) throws -> T {
>>>>>>> >>>> guard let wrapped = optional else {
>>>>>>> >>>>     throw wrapped // Remember, if-let, guard-let and case-let statements bind the let name to the error in case of a failure.
>>>>>>> >>>> }
>>>>>>> >>>> return wrapped
>>>>>>> >>>> }
>>>>>>> >>>>
>>>>>>> >>>> Multiple let bindings in a single if-let statement are essentially rewrites of a nested chain of if-let statements.
>>>>>>> >>>>
>>>>>>> >>>> The `try` keyword applied to an optional would unwrap the value or throw the error.
>>>>>>> >>>> The `try?` keyword applied to a throwing function call would cause any thrown errors to be caught and put into the returned Optional, instead of simply ignored.
>>>>>>> >>>> The `try!` keyword applied to a throwing function call would behave as you'd expect: just like `try?` except immediately force-unwrapped.
>>>>>>> >>>>
>>>>>>> >>>> A throwing function would be convertible to a non-throwing optional-returning function and vice versa.
>>>>>>> >>>> This would allow making use of throwing functions when dealing with generics or protocols that allow arbitrary return types, without having to sacrifice the convenience of error-handling logic. Conversely, it would allow to write generic code that deals with any type of function without having to implement special cases for throwing functions. This means that the two function types would be interchangeable and one would be able to satisfy protocol requirements of the other. The `rethrows` idiom would then become a natural consequence of writing generic functions that may return optional and non-optional results just as well.
>>>>>>> >>>>
>>>>>>> >>>> _______________________________________________
>>>>>>> >>>> swift-evolution mailing list
>>>>>>> >>>> swift-evolution at swift.org
>>>>>>> >>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>>>>>> >>
>>>>>>> >
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> swift-evolution mailing list
>>>>>>> swift-evolution at swift.org
>>>>>>> https://lists.swift.org/mailman/listinfo/swift-evolution
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170501/1c1d77c5/attachment.html>