[swift-evolution] [Pitch] Revamp Optional and Throws

[Xiaodi Wu]

On Mon, May 1, 2017 at 2:58 AM, Gor Gyolchanyan <gor at gyolchanyan.com> wrote:

> I have read those documents before, but It’s worth re-reading them to see
> if I missed something, but I’l still explain my motivation and seek
> arguments against the postulated problem (rather then a specific solution).
>
> (a) There are different types of error.
>
> Yes, there are different types of error in Swift, which require different
> reactions from the programmer.
> If I’m not missing something, the three main types of error in Swift are:
>  - Simple encapsulatable errors that are expected to be treated as normal
> values until the time comes for someone to take care of them by unpacking
> the content.
>  - Automatic propagatable errors that require the programmer to either
> handle the error immediately or propagate it by delegating to its own
> caller.
> - Fatal errors, which represent logic errors and broken invariants and
> preconditions, which are purely a programmer error and should not be dealt
> with dynamically, hence the terminated process with a message.
>
> (b) The programmer is expected to react differently to different types of
> error.
>
> Yes, and the three main ways a programmer is expected to react to the an
> error are:
>  - If it’s an optional, they’re encouraged to store and pass it around
> freely until someone down the line decides to unpack it and deal with the
> possibility that it isn’t there.
>  - If it’s an error, they’re encouraged to either handle it on the spot or
> declare themselves throwing and delegate the responsibility to the caller.
>  - Look at the standard output and figure out why the fatal error
> occurred, perhaps with the help of the debugger.
>
> (c) The language is a tool to help the programmer react.
>
> Yes, that comes in the form of three language constructs:
>  - Optionals, which allow storing a union of a value and its absence (for
> an undefined and hopefully obvious reason).
>  - Throwing functions, which allow making sure that the error will be
> handled as soon as possible.
>  - Fatal errors, which allow the programmer to mark points in code which
> should never be reached in a correct system in order to keep the logic from
> going AWOL in case the programmer screwed up somewhere.
>
> (d) Optionals and errors are not unified, and unification is a non-goal,
> because they are designed to help the programmer react differently to
> different types of error.
>
> Yes, and those different types of error with different reactions are all
> valid and shouldn’t be unified.
> My point is that the language should make it easy for a programmer to
> transition from one type of error to another, because the same error has
> different severity in different contexts.
> For instance, a “file not found” error when trying to open a file handler
> is not critical at all in the context of the file opening function, because
> it’s a perfectly expected outcome of the operation.
> However, for a module that handles loading critical data from disk (like
> encryption keys needed to decrypt the manipulated content) it is a critical
> error that cannot be dealt with.
> In this case it deserves to be a fatal error, because the programmer
> didn’t bother to implement a logic for creating the missing file or
> properly notifying the user of the lacking permissions to do so.
> Conversely, some errors start as being urgent (like a JSON parser that
> throws an error when it encounters invalid syntax), but become less urgent
> for the client (a JSON editor that simply displays the error message).
>

Again, that's not my understanding for the rationale behind having both
Optional return values and errors. It's not that one is more "urgent" or
"important" than the other. An Optional is used when something can only
fail in one obvious way; an error is thrown when it can fail in multiple,
but recoverable, ways. You can care a lot about a nil value and not at all
about an error, or vice versa.


As for my use case:
>
> I have a JSON parser that may throw, and I have a a JSON Editor class that
> allows editing JSON files as well as displaying the parsing errors.
> I have a malformed JSON file that I open in the editor. The JSON parser
> throws an error, which should be caught and stored somewhere for the editor
> to display.
> I have file reader that reads a file in some encoding and returns an
> optional string with the file contents (nil means file couldn’t be read or
> the encoding is wrong).
>
> For the JSON parser, a malformed JSON file is an obvious error, but for
> the editor, it’s a perfectly valid and expected condition, which doesn’t
> deserve to be an error.
> Therefore, the thrown error of the JSON parse has to be caught and
> encapsulated indiscriminately to demote it from an error to a return value.
> Conversely, the returned nil form the file reader is perfectly valid and
> expected condition, but for the editor, it’s an error.
> Therefore, the returned nil should be checked and converted to an error
> that will be thrown to promote it to a full-fledged error.
>
> I would want to have a way to easily promote/demote different types of
> errors to accommodate the changing perception of their urgency.
> For instance, by being able to throw an optional, thus introducing a new
> way of unpacking it (by promoting it to an error). Currently, it is by
> manually unpacking the optional, deciding what error to throw and throwing
> it manually.
> Or, being able to catch an error into an optional, thus introducing a new
> way of handling it (by demoting it to an optional). There is a way to do
> that currently in the form of `try?` and `try!`, but their downside is that
> they are lossy (losing the error itself).
>
> All I want is for the language to help me losslessly catch errors into
> something like an optional, except with the error intact with the
> possibility of easily re-throwing it in the future.
>

This is an orthogonal concern to whether to model an error with an
Optional. Optional is used when there's only one obvious (and recoverable)
way of failing; throwing an error is used when there are multiple
recoverable ways of failing. As Rod mentions, the _point_ of an Optional is
that it doesn't come with an associated Error type. Every Optional.none
compares equal to every other Optional.none, even when the wrapped type
differs. Obviously, this means that there's no error to propagate manually
or automatically, but whether or not an error propagates automatically is
not the reason to use or not to use Optional to model your error. Likewise,
the _point_ of using `try!` (not a downside) is to _lose_ the error, not
just to stop the propagation of it.

By contrast, your use case is about _storing_ an error, which is totally
the opposite of what Optional does. It sounds like you don't like how
errors are designed to automatically propagate (albeit with marking at the
origination site), and you want to manually propagate errors using a
Result<T, Error> type instead. I guess this is what you mean by errors
being "urgent" and wanting to "demote" it. I wouldn't disagree that it's
worth thinking about a syntax to offer some additional control over that.
Suppose, for instance, we had a new type:

```
enum Result<Wrapped, Error> {
  case some(Wrapped)
  case none(Error)
}
```

We could invent a new operator `try*`, which returns a `Result` instead of
throwing. And then we could invent new sugar; perhaps, `result?` would be
sugar for `result.some` and `result*` would be sugar for `result.none`. I'm
not proposing this, but I can see how something along these lines would
give you more options.


This would also solve the problem of multiple throwing calls having
> different urgency to them and being forced to write a lot of boilerplate to
> catch their errors individually and deal with them separetely.
>
> On May 1, 2017, at 1:44 AM, Xiaodi Wu <xiaodi.wu at gmail.com> wrote:
>
> On Sun, Apr 30, 2017 at 5:05 PM, Gor Gyolchanyan <gor at gyolchanyan.com>
> wrote:
>
>>
>> On May 1, 2017, at 12:10 AM, Xiaodi Wu <xiaodi.wu at gmail.com> wrote:
>>
>> You may wish to read the rationale behind the current error handling
>> design:
>>
>> https://github.com/apple/swift/blob/master/docs/ErrorHandlin
>> gRationale.rst
>>
>> A Result type like you suggest has been considered and rejected in favor
>> of the current design. Briefly, optionals and throwing errors are distinct
>> because they are considered superior ways for handling distinct types of
>> error.
>>
>> In the case of a simple domain error, there is only one way to fail;
>> therefore, optional return values are considered the best way to model that
>> error.
>>
>> In the case of a recoverable error, the document above describes why
>> marked propagation (the current implementation in Swift) is considered
>> superior to typed propagation (your suggestion).
>>
>>
>> My proposal is not about replacing Optionals and throwing functions with
>> a Result type, it’s about separating the representation of an error from
>> its propagation.
>> Optionals and throwing functions solve two different problems, but they
>> are not dealing with two different types of error.
>>
>
> The basic premise of Swift error handling design is that there exist
> different types of error. From the document:
>
> > What is an error? There may be many different possible error conditions
> in a program, but they can be categorized into several kinds based on how
> programmers should be expected to react to them. Since the programmer is
> expected to react differently, and since the language is the tool of the
> programmer's reaction, it makes sense for each group to be treated
> differently in the language.
>
> Optionals are for storing and representing a value that might not exist
>> (most commonly due to an unambiguous error).
>> Error handling is for propagating an error.
>> Returning an optional is essentially the same as returning a non-optional
>> and throwing a dedicated “something went wrong” error, because due to the
>> optional unwrapping mechanic, you cannot avoid dealing with the fact that
>> there might have been an error. Optionals only allow you to delay the
>> inevitable error handling, not avoid it. The use cases where the exact
>> reason for an error is no important have nothing to do with whether or not
>> that error should be available. The optional chaining, if-let statements
>> and all other ways one might try to handle an optional value do not
>> fundamentally require lack of error information.
>> The error handling mechanism, on the other hand, does not concern itself
>> with representing the error, but only propagating it. Even an optional
>> value with a general-purpose .none case has different levels of importance
>> in different cases. More often than not, when propagating an optional value
>> to a non-optional target, you’ll be stuck with dealing with the error
>> immediately, which is exactly what throwing functions force you to do.
>> I suggest we enhance the current error representation and propagation
>> mechanisms to be able to seamlessly handle cases where an erroneous value
>> need to be stored as-is (along with its error) or unpacked and propagated
>> (by throwing the error), not just representing a general “error”.
>> The general use case is to be able to catch a throwing call into an enum
>> that stores the value or the error and then being able to unpack it in a
>> throwing context (unpack it or throw the error).
>> This use case is a strict superset of the current Optional mechanic
>> (catching a potentially missing value) and error handling (always manually
>> throwing an error after manual checks).
>> The aforementioned suggestion about how to do that is indeed faulty, as
>> pointed out by Robert Widmann, but the problem is still valid, in my
>> opinion.
>>
>
> I'd highly recommend taking some time to digest the existing rationale.
> You're basing your argument on contradicting the fundamental premise of the
> existing design, which begins with this: (a) there are different types of
> error; (b) the programmer is expected to react differently to different
> types of error; (c) the language is a tool to help the programmer react;
> (d) optionals and errors are not unified, and unification is a non-goal,
> because they are designed to help the programmer react differently to
> different types of error.
>
> Do you have a specific use case in mind that is not well accommodated by
> optionals or by throwing functions? What is it? Into what category does
> that use case fall, in terms of the types of error enumerated in the error
> handling rationale document?
>
> On Sun, Apr 30, 2017 at 13:51 Gor Gyolchanyan via swift-evolution <
>> swift-evolution at swift.org> wrote:
>>
>>>
>>> > On Apr 30, 2017, at 9:29 PM, Robert Widmann <devteam.codafi at gmail.com>
>>> wrote:
>>> >
>>> >
>>> >> On Apr 30, 2017, at 1:43 PM, Gor Gyolchanyan <gor at gyolchanyan.com>
>>> wrote:
>>> >>
>>> >> It doesn’t have to be a massive source-break, since this pitch is
>>> supposed to be a strict superset of what Optional and throwing is currently.
>>> >> The only thing that I can think of at this moment that would break is
>>> this syntax:
>>> >>
>>> >> let foo: Int? = .none // Error: Can’t convert (Error) -> Int? to Int?
>>> >>
>>> >
>>> > Except it’s not a strict superset if you break every use of this case
>>> as an RValue.  Especially when so much of Swift’s syntax and major patterns
>>> revolve around the manipulation of optionals.
>>> >
>>> >> The ExpressibleByNilLiteral, the try/throw syntax, all of those
>>> things would work as they are right now.
>>> >> Error handling as it is currently, is essentially a hidden `error`
>>> out parameter and a whole bunch of codegen.
>>> >> Even the semantical changes described earlier would be purely
>>> additive.
>>> >
>>> > Don’t get me wrong, I think you’ve identified the problem space well,
>>> I just disagree with the solution.
>>>
>>> Yeah, you’re right. It would take some next-level fixits to deal with
>>> the consequences of changing the most fundamental data type of Swift I can
>>> think of.
>>> I’d really appreciate it if you’d offer an alternative solution to this
>>> problem.
>>> The problem, as I understand it, is as follows:
>>>
>>> A lot of Swift’s logic revolves around the notion that some values might
>>> be missing for whatever reason and some functions might fail for whatever
>>> reason.
>>> Any function’s effect can be summed up as the union of its return value
>>> and the global state that it changes (that includes captured closure
>>> scopes).
>>> This could be boiled down to the statement that “Values that a function
>>> sets and returns completely express the purpose of the function”.
>>> The optional gives an extremely convenient way of representing values
>>> that might not exist (which, when returned from a function often means
>>> “failed for an unknown reason”).
>>> The fact that Optional is a type, rather then a function attribute
>>> allows us to store and imperatively manipulate the outcome of logically
>>> failable functions, but unfortunately, it doesn’t allow us to reason about
>>> the cause of the failure.
>>> On the other hand, throwing functions captures the logic of dealing with
>>> specific failures very well, but does not allow us to store and manipulate
>>> them easily, leaving us with workarounds like wrapping errors in enums with
>>> values and re-throwing the errors on their way out of the generic pipeline.
>>> I’d like to come up with a solution that would unify the optionals and
>>> the throwing functions into a single mechanism for dealing with the concept
>>> of failure, taking the best of both worlds and getting the benefits of the
>>> new synergies.
>>> This pitch was a first rough idea about the direction in which we could
>>> go in trying to find a solution.
>>> I chose to enhance Optional instead of introducing a new type like
>>> Failable, so that we could make do with minimal language changes and
>>> migration procedures.
>>>
>>> This problem is kinda similar to the variadic parameter problem, which
>>> makes it impossible to forward calls to variadic functions simply because
>>> that feature is too magical and does not provide a way to store and
>>> propagate its logic.
>>>
>>> Another way I could think of solving it would be to allow overloading
>>> the postfix `!` and `?` operators (which would currently only be defined
>>> for Optionals), which would allow us to define the Failable enum type with
>>> some error handling syntax integration and make it feel more at home in the
>>> midst of Optionals.
>>>
>>> Or better yet, make an OptionalProtocol and move the current magical
>>> logic to it, leaving the existing Optional perfectly intact and allowing
>>> userspace implementations.
>>> This would also greatly benefit numerous use cases of “invalidatable”
>>> types (like file handlers that can be closed) that would no longer have to
>>> either fatalError or use unwieldy wrappers that operate on Optionals.
>>>
>>> > ~Robert Widmann
>>> >
>>> >>
>>> >>> On Apr 30, 2017, at 8:35 PM, Robert Widmann <
>>> devteam.codafi at gmail.com> wrote:
>>> >>>
>>> >>> This "revamp" is isomorphic to adding a Sum type to stdlib and
>>> plumbing error handling syntax through.  I'd much rather see that than the
>>> massive source-break this would entail.
>>> >>>
>>> >>> ~Robert Widmann
>>> >>>
>>> >>> 2017/04/30 13:11、Gor Gyolchanyan via swift-evolution <
>>> swift-evolution at swift.org> のメッセージ:
>>> >>>
>>> >>>> I’d like to suggest a bit of redesigning the Optional type and
>>> throwing functions to provide a single powerful and flexible mechanism for
>>> dealing with unexpected situations.
>>> >>>>
>>> >>>> In short, The Optional would have an associated value of type Error
>>> added to its `none` case, which would describe the reason why the wrapped
>>> value is missing.
>>> >>>>
>>> >>>> public enum Optional<Wrapped> {
>>> >>>>
>>> >>>> case .some(Wrapped)
>>> >>>>
>>> >>>> case .none(Error)
>>> >>>>
>>> >>>> }
>>> >>>>
>>> >>>> The Optional's ExpressibleByNilLiteral would initialize it with an
>>> error that corresponds to what is currently fatalError-ed as "unexpectedly
>>> found nil while unwrapping an Optional value".
>>> >>>>
>>> >>>> The forced unwrapping operator (postfix `!`) would behave the same
>>> way as it does now, except in case of a fatal error it would print out the
>>> underlying error, instead of the aforementioned hard-coded string.
>>> >>>>
>>> >>>> The optional chaining operator (postfix `?`) would behave the same
>>> way as it does now, except when it stops evaluating and returns the
>>> Optional, it would contain the error, returned by the sub-expression that
>>> failed to evaluate.
>>> >>>>
>>> >>>> Any throwing function would be equivalent to a function that
>>> returns an Optional. If the function is declared as throwing and returning
>>> an Optional at the same time, it would be equivalent to a function
>>> returning an Optional Optional.
>>> >>>>
>>> >>>> The if-let statement would bind the `let` variable to the wrapped
>>> value inside the "then" block and would bind it to the error in the "else"
>>> block. Chained else-if blocks would all be considered part of the
>>> overarching "else" block, so all of them would be able to access the error
>>> bound to the if-let name.
>>> >>>>
>>> >>>> The guard-let and case-let statements are essentially just rewrites
>>> of if-let with some added logic.
>>> >>>>
>>> >>>> The `try` keyword, applied to an optional would behave like this:
>>> >>>>
>>> >>>> public func try<T>(_ optional: T?) throws -> T {
>>> >>>> guard let wrapped = optional else {
>>> >>>>     throw wrapped // Remember, if-let, guard-let and case-let
>>> statements bind the let name to the error in case of a failure.
>>> >>>> }
>>> >>>> return wrapped
>>> >>>> }
>>> >>>>
>>> >>>> Multiple let bindings in a single if-let statement are essentially
>>> rewrites of a nested chain of if-let statements.
>>> >>>>
>>> >>>> The `try` keyword applied to an optional would unwrap the value or
>>> throw the error.
>>> >>>> The `try?` keyword applied to a throwing function call would cause
>>> any thrown errors to be caught and put into the returned Optional, instead
>>> of simply ignored.
>>> >>>> The `try!` keyword applied to a throwing function call would behave
>>> as you'd expect: just like `try?` except immediately force-unwrapped.
>>> >>>>
>>> >>>> A throwing function would be convertible to a non-throwing
>>> optional-returning function and vice versa.
>>> >>>> This would allow making use of throwing functions when dealing with
>>> generics or protocols that allow arbitrary return types, without having to
>>> sacrifice the convenience of error-handling logic. Conversely, it would
>>> allow to write generic code that deals with any type of function without
>>> having to implement special cases for throwing functions. This means that
>>> the two function types would be interchangeable and one would be able to
>>> satisfy protocol requirements of the other. The `rethrows` idiom would then
>>> become a natural consequence of writing generic functions that may return
>>> optional and non-optional results just as well.
>>> >>>>
>>> >>>> _______________________________________________
>>> >>>> swift-evolution mailing list
>>> >>>> swift-evolution at swift.org
>>> >>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>> >>
>>> >
>>>
>>> _______________________________________________
>>> swift-evolution mailing list
>>> swift-evolution at swift.org
>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170501/ac2dfc71/attachment.html>