[swift-evolution] [Review] SE 0192 - Non-Exhaustive Enums

[Karl Wagner]

> On 2. Jan 2018, at 16:38, Matthew Johnson via swift-evolution <swift-evolution at swift.org> wrote:
> 
> 
> 
> Sent from my iPad
> 
> On Jan 1, 2018, at 11:47 PM, Chris Lattner <clattner at nondot.org <mailto:clattner at nondot.org>> wrote:
> 
>>> On Dec 31, 2017, at 12:14 PM, Matthew Johnson via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>> 
>>> I agree that we need a solution to the problem described.  I also agree that non-exhaustive is most in keeping with the overall design of Swift at module boundaries.  However, I believe this proposal should be modified before being accepted
>> 
>> Thanks for writing this up - you’ve explained a common concern in an interesting way:
>> 
>>> This is likely to be a relatively rare need mostly encountered by 3rd party libraries but it will happen.  When it does happen it would be really unfortunate to be forced to use a `default` clause rather than something like a `future` clause which will produce an error when compiled against an SDK where the enum includes cases that are not covered.  I can imagine cases where this catch-all case would need to do something other than abort the program so I do not like the `switch!` suggestion that has been discussed.  The programmer should still be responsible for determining the behavior of unknown cases.
>> ..
>>> While library authors have a legitimate need to reserve the right to introduce new cases for some enums this need can be met without taking away a useful tool for generating static compiler errors when code does not align with intent (in this case, the intent being to cover all known cases).  Switch statements working with these kinds of enums should be required to cover unknown cases but should be able to do so while still being statically checked with regards to known cases.  
>> 
>> I think that this could be the crux of some major confusion, the root of which is the difference between source packages and binary packages that are updated outside your control (e.g. the OS, or a dynamic library that is updated independently of your app like a 3rd party plugin).  Consider:
>> 
>> 1) When dealing with independently updated binary packages, your code *has* to implement some behavior for unexpected cases if the enum is non-exhaustive.  It isn’t acceptable to not handle that case, and it isn’t acceptable to abort because then your app will start crashing when a new OS comes out. You have to build some sort of fallback into your app.
>> 
>> 2) When dealing with a source package that contributes to your app (e.g. through SwiftPM), *YOU* control when you update that package, and therefore it is entirely reasonable to exhaustively handle enums even if that package owner didn’t “intend” for them to be exhaustive.  When *you* chose to update the package, you get the “unhandled case” error, and you have maximal “knowability” about the package’s behavior.
>> 
>> 
>> It seems that your concern stems from the fact that the feature as proposed is aligned around module boundaries, and therefore overly punishes source packages like #2.  I hope you agree that in case #1, that the feature as proposed is the right and only thing we can do: you really do have to handle unknown future cases somehow.
>> 
>> If I’m getting this right, then maybe there is a variant of the proposal that ties the error/warning behavior to whether or not a module is a source module vs a binary module.  The problem with that right now is that we have no infrastructure in the language to know this…
> 
> Hi Chris, thanks for your reply.
> 
> The concern you describe isn’t exactly what I was describing but it is related.  John McCall recently posted a sketch of a solution to the concern you describe which looked great to me.  I don’t have time to look up the link this morning but I think it was in this review thread.
> 
> The actual concern I am describing is where a 3rd party library (or app) wants to switch over a non-exhaustive enum provided by a module that is a binary (not source) dependency.  The author of the 3rd party library may have a legitimate reason to switch over an enum despite the author of the binary module reserving the right to add additional cases.  
> 
> When this circumstance arises they will do it using the tools provided by the language.  Regardless of the final language solution they obviously need to cover unknown cases - their library could be shipping on a device which receives an update to the binary dependency that contains a new case.  I agree with you that a language-defined crash is not appropriate.  The author of the switch must take responsibility for the behavior of unknown cases.  
> 
> I am arguing that these “pseudo-exhaustive” switch statements will exist in the wild.  The crucial point of contention is whether or not the language provides assistance to the author of the 3rd party library in updating their library when the enum provided by the binary dependency changes.  Is the author forced to use a `default` case which turns of exhaustiveness checking?  Or are they able to use an alternative mechanism for handling unknown cases which does not turn off exhaustiveness checking - all statically known cases must be covered.  The most common example of such a mechanism is the `future` (or perhaps `unknown`) case which would only be used for cases that are not statically known.

> 
> This facility will of course help authors of these switch statements make the necessary updates as the enum vended by the binary dependency changes.  It will also help alert authors of apps that depend on that 3rd party library (which will usually be a source dependency).  If the author of the app attempts to rebuild the dependency against a new SDK with added cases the library will fail to build, alerting the user that they should update the 3rd party library.
> 
> My position is that if there are reasonable use cases for these kinds of “pseudo-exhaustive” switches then the language should provide exhaustiveness checking of statically known cases via some mechanism that authors can opt-in to using.  It’s ok with me if this is a relatively esoteric feature.  It won’t be commonly needed, but when it is necessary it will provide significant value.
> 
> IIRC there were some reasonable examples of these kinds of switches posted in earlier threads on this topic.  I don’t have time to look those up right now either but it would be good for the core team to be aware of them before making a final decision.
> 
> I am only aware of two arguments against this kind of “pseudo-exhaustive” switch.  One is that users should not attempt to switch over an enum that a library author does not intend to be exhaustive (i.e. it is an “input-only” enum).  The other is that a `future` or `unknown` case is not testable.
> 
> The first argument is a moral one which I believe should not carry much weight relative to concrete, pragmatic counter-examples such as those that (IIRC) were provided on this list in the past.
> 
> The second argument doesn’t make sense to me: as I noted in my review post, the code path is equally untestable when a `default` case is used, but all statically known cases are matched in earlier patterns.  The testability problem of non-exhaustive enums is orthogonal to the issue of language support for “pseudo-exhaustive” switches.
> 
> This is the line of reasoning which leads me to conclude that we should dig up the concrete examples which have been provided and evaluate them for merit.  If we can’t discard them as a bad coding practice for which a better solution is available then we should strongly consider providing language support for these use cases.  The existence of such use cases should also motivate a solution to the testability problem.
> 
> - Matthew
> 
>> 
>> -Chris
>> 
>> 
>> 
>> 
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>

So if I can summarise the problem (for myself), what you’re saying is there is an enum in some OS library...

enum OSButtonState {
    case normal, selected
}

And in my App, I switch its values. As far as this proposal goes, there is only one way to write it (with a “default”):

switch buttonState {
    case .selected: …
    case .normal: ...
    default: // reset state, do not react… I suppose?
}

Then, an update happens. Some new functionality is added, and the enum gains a new case to expose that:

enum OSButtonState {
    case normal, selected
    @available(1.1)
    case focussed
}

Now, on the one hand: yes, my original “switch” statement is still valid Swift code (from the perspective of “it will compile”), but it may no longer be semantically correct.
Those new enum cases may be reflecting important state transitions which my App needs to be aware of to maintain its own internal state. There are a couple of points that I take from this:

1) OK, fair point. “Default” is not expressive enough, and I’m starting to see the point in a “future” case. I think it’s useful to have a way to write a switch which tracks the latest version of the enum that the compiler can see. It would behave the same as a “default” in practice, but wouldn’t short-circuit the compiler checking that all currently-known cases are handled. Of course, it’s not the same as the strict guarantee given by an @exhaustive enum, and I have some concerns in general with our “catch-all” approach to dealing with enum evolution, but if we go that way I see value in annotating a best-attempt to stay in sync with the library.

2) As for apps supporting "pseudo-exhaustive” switching against multiple versions of the library, I assume something like this would be possible:

if @available(OSKit 1.1) {
    switch buttonState {
        case .selected: …
        case .focussed: ...
        case .normal: …
        future: ...
    }
} else {
    switch buttonState {
        case .selected: …
        case .normal: …
        future: ...
    }
}

This could be useful if you have complex state-machines which would be awkward when some cases were @available and others not. So you could hoist the availability check over the entire switch.


3) This proposal is predicated on the assumption that App developers are able to write non-fatal “default” execution paths to keep their internal state consistent, even if they can’t predict which new features or options or state-transitions the enum will later expose. I’m not sure that’s entirely realistic. Even if the compiler was going to help us out with exhaustiveness in switching, App developers would still require excellent documentation, API design, support and testing from library authors in order to make this work in practice. Perhaps in the end it won’t actually make anybody’s life easier, just less certain?

I see an analogue to protocols. Whenever people propose new protocols (or additions to existing protocols), we ask: "which useful generic algorithms can you write with this?”. Similarly, for this new control flow event (an unknown value in an enum), we need to ask: “how can you reasonably handle this event?”. Is it really the correct approach to punt all future enum values to some generic “catch-all” event in the client?

The App has absolutely no idea what value the enum has, which semantic implications that might have (if any), or if the library has any unique requirements when these events are triggered. These are all easy-to-miss details that lie behind simply which cases are available; typically they will require that your library author has carefully planned their API to avoid such unique handling requirements and has documented to you exactly how you should handle these future cases.

I think that’s what Dave DeLong has been saying; that we should be focussing on tools to help library authors make backwards-compatible changes, rather than forcing app developers to make their logic forwards-compatible to account for changes they cannot influence or foresee.

Maybe we are focussing too much on the language impact, and ensuring code continues to compile. The devil is in the details, as they say. In an updated-library situation, it’s the library which knows everything, and the App has absolutely no idea what changed. I’m not sure it makes sense to insist that the App handle this situation.

I don’t have a concrete counter-proposal in that respect, but then, I don’t have the experience dealing with these issues that the former/current Apple folks have. I would naively guess libraries would need a runtime-defined compatibility version, and would be able to inspect it in a way similar to @availability checking, returning only values that are consistent with the semantics the client expects (and allowing them to define their own “unknown” cases with documented handling behaviour, if appropriate for that enum). I could see how something like that could be more code-heavy and burdensome for library authors, though.


- Karl



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20180102/dacf0c1e/attachment.html>