[swift-evolution] [swift-evolution-announce] [Review] SE 0192 - Non-Exhaustive Enums

[Jordan Rose]

Hi, Dave. You're right, all these points are worth addressing. I'm going to go in sections.

> This whole “unexpected case” thing is only a problem when you’re linking libraries that are external to/shipped independently of your app. Right now, the *only* case where this might exist is Swift on the server. We *might* run in to this in the future once the ABI stabilizes and we have the Swift libraries shipping as part of iOS/macOS/Linux. Other than this, unexpected enum cases won’t really be a problem developers have to deal with.


I wish this were the case, but it is not. Regardless of what we do for Swift enums, we are in dire need of a fix for C enums. Today, if a C enum doesn't have one of the expected values, the behavior is undefined in the C sense (as in, type-unsafe, memory-unsafe, may invoke functions that shouldn't be invoked, may not invoke functions that should be invoked, etc).

Obviously that's an unacceptable state of affairs; even without this proposal we would fix it so that the program will deterministically trap instead. This isn't perfect because it results in a (tiny) performance and code size hit compared to C, but it's better than leaving such a massive hole in Swift's safety story.

The trouble is that many enums—maybe even most enums—in the Apple SDK really are expected to grow new cases, and the Apple API authors rely on this. Many of those—probably most of them—are the ones that Brent Royal-Gordon described as "opaque inputs", like UIViewAnimationTransition, which you're unlikely to switch over but which the compiler should handle correctly if you do. Then there are the murkier ones like SKPaymentTransactionState.

I'm going to come dangerously close to criticizing Apple and say I have a lot of sympathy for third-party developers in the SKPaymentTransactionState case. As Karl Wagner said, there wasn't really any way an existing app could handle that case well, even if they had written an 'unknown case' handler. So what could the StoreKit folks have done instead? They can't tell themselves whether your app supports the new case, other than the heavy-handed "check what SDK they compiled against" that ignores the possibility of embedded binary frameworks. So maybe they should have added a property "supportsDeferredState" or something that would have to be set before the new state was returned.

(I'll pause to say I don't know what consideration went into this API and I'm going to avoid looking it up to avoid perjury. This is all hypothetical, for the next API that needs to add a case.)

Let's say we go with that, a property that controls whether the new case is ever passed to third-party code. Now the new case exists, and new code needs to switch over it. At the same time, old code needs to continue working. The new enum case exists, and so even if it shouldn't escape into old code that doesn't know how to handle it, the behavior needs to be defined if it does. Furthermore, the old code needs to continue working without source changes, because updating to a new SDK must not break existing code. (It can introduce new warnings, but even that is something that should be considered carefully.)

So: this proposal is designed to handle the use cases both for Swift library authors to come and for C APIs today, and in particular Apple's Objective-C SDKs and how they've evolved historically.


There's another really interesting point in your message, which Karl, Drew Crawford, and others also touched on.

> Teaching the compiler/checker/whatever about the linking semantics of modules. For modules that are packaged inside the final built product, there is no need to deal with any unexpected cases, because we already have the exhaustiveness check appropriate for that scenario (regardless of whether the module is shipped as a binary or compiled from source). The app author decides when to update their dependencies, and updating those dependencies will produce new warnings/errors as the compiler notices new or deprecated cases. This is the current state of things and is completely orthogonal to the entire discussion.

This keeps sneaking into discussions and I hope to have it formalized in a proposal soon. On the library side, we do want to make a distinction between "needs binary compatibility" and "does not need binary compatibility". Why? Because we can get much better performance if we know a library is never going to change. A class will not acquire new dynamic-dispatch members; a stored property will not turn into a computed property; a struct will not gain new stored properties. None of those things affect how client code is written, but they do affect what happens at run-time.

Okay, so should we use this as an indicator of whether an enum can grow new cases? (I'm going to ignore C libraries in this section, both because they don't have this distinction and because they can always lie anyway.)

- If a library really is shipped separately from the app, enums can grow new cases, except for the ones that can't. So we need some kind of annotation here. This is your "B" in the original email, so we're all agreed here.

- If a library is shipped with the app, there's no chance of the enum growing a new case at run time. Does that mean we don't need a default case? (Or "unknown case" now.)

The answer here is most easily understood in terms of semantic versioning <https://semver.org/>. If adding a new enum case is a source-breaking change, then it's a source-breaking change, requiring a major version update. The app author decides when to update their dependencies, and might hold off on getting a newer version of a library because it's not compatible with what they have.

If adding a new enum case is not a source-breaking change, then it can be done in a minor version release of a library. Like deprecations, this can produce new warnings, but not new errors, and it should not (if done carefully) break existing code. This isn't a critical feature for a language to have, but I would argue (and have argued) that it's a useful one for library developers. Major releases still exist; this just makes one particular kind of change valid for minor releases as well.

(It also feels very subtle to me that 'switch' behaves differently based on where the enum came from. I know this whole proposal adds complexity to the language, and I'd like to keep it as consistent as possible.)

Okay, so what if we did this based on the 'import' rather than on how the module was compiled—Karl's `@static import`? That feels a little better to me because you can see it in your code. (Let's ignore re-exported modules for now.) But now we have two types of 'import', only one of which can be used with system libraries. That also makes me uncomfortable. (And to be fair, it's also something that can be added after the fact without disturbing the rest of the language.)

Finally, it's very important that whatever you do in your code doesn't necessarily apply to your dependencies. We've seen in practice that people are not willing to edit their dependencies, even to handle simple SDK changes or language syntax changes (of which there are hopefully no more). That's why I'm pushing the source compatibility aspect so hard, even for libraries that won't be shipped separately from an app.


Overall, I think we're really trying to keep from breaking Swift into different dialects, and making this feature dependent on whether or not the library is embedded in the app would work at cross-purposes to that. Everyone would still be forced to learn about the feature if they used C enums anyway, so we're not even helping out average developers. Instead, it's better that we have one, good model for dealing with other people's enums, which in practice can and do grow new cases regardless of how they are linked.

Jordan



> On Jan 3, 2018, at 09:07, Dave DeLong <swift at davedelong.com> wrote:
> 
> IMO this is still too large of a hammer for this problem.
> 
> This whole “unexpected case” thing is only a problem when you’re linking libraries that are external to/shipped independently of your app. Right now, the *only* case where this might exist is Swift on the server. We *might* run in to this in the future once the ABI stabilizes and we have the Swift libraries shipping as part of iOS/macOS/Linux. Other than this, unexpected enum cases won’t really be a problem developers have to deal with.
> 
> Because this will be such a relatively rare problem, I feel like a syntax change like what’s being proposed is a too-massive hammer for such a small nail.
> 
> What feels far more appropriate is:
> 
> 🅰️ Teaching the compiler/checker/whatever about the linking semantics of modules. For modules that are packaged inside the final built product, there is no need to deal with any unexpected cases, because we already have the exhaustiveness check appropriate for that scenario (regardless of whether the module is shipped as a binary or compiled from source). The app author decides when to update their dependencies, and updating those dependencies will produce new warnings/errors as the compiler notices new or deprecated cases. This is the current state of things and is completely orthogonal to the entire discussion.
> 
> and
> 
> 🅱️ Adding an attribute (@frozen, @tangled, @moana, @whatever) that can be used to decorate an enum declaration. This attribute would only need to be consulted on enums where the compiler can determine that the module will *not* be part of the final built product. (Ie, it’s an “external” module, in my nomenclature). This, then, is a module that can update independently of the final app, and therefore there are two possible cases:
> 
> 	1️⃣ If the enum is decorated with @frozen, then I, as an app author, have the assurance that the enum case will not change in future releases of the library, and I can safely switch on all known cases and not have to provide a default case. 
> 
> 	2️⃣ If the enum is NOT decorated with @frozen, then I, as an app author, have to account for the possibility that the module may update from underneath my app, and I have to handle an unknown case. This is simple: the compiler should require me to add a “default:” case to my switch statement. This warning is produced IFF: the enum is coming from an external module, and the enum is not decorated with @frozen.
> 
> 
> ==========
> 
> With this proposal, we only have one thing to consider: the spelling of @frozen/@moana/@whatever that we decorate enums in external modules with. Other than that, the existing behavior we currently have is completely capable of covering the possibilities: we just keep using a “default:” case whenever the compiler can’t guarantee that we can be exhaustive in our switching.
> 
> Where the real work would be is teaching the compiler about internally-vs-externally linked modules.
> 
> Dave
> 
>> On Jan 2, 2018, at 7:07 PM, Jordan Rose via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>> 
>> [Proposal: https://github.com/apple/swift-evolution/blob/master/proposals/0192-non-exhaustive-enums.md <https://github.com/apple/swift-evolution/blob/master/proposals/0192-non-exhaustive-enums.md>]
>> 
>> Whew! Thanks for your feedback, everyone. On the lighter side of feedback—naming things—it seems that most people seem to like '@frozen', and that does in fact have the connotations we want it to have. I like it too.
>> 
>> More seriously, this discussion has convinced me that it's worth including what the proposal discusses as a 'future' case. The key point that swayed me is that this can produce a warning when the switch is missing a case rather than an error, which both provides the necessary compiler feedback to update your code and allows your dependencies to continue compiling when you update to a newer SDK. I know people on both sides won't be 100% satisfied with this, but does it seem like a reasonable compromise?
>> 
>> The next question is how to spell it. I'm leaning towards `unexpected case:`, which (a) is backwards-compatible, and (b) also handles "private cases", either the fake kind that you can do in C (as described in the proposal), or some real feature we might add to Swift some day. `unknown case:` isn't bad either.
>> 
>> I too would like to just do `unknown:` or `unexpected:` but that's technically a source-breaking change:
>> 
>> switch foo {
>> case bar:
>>   unknown:
>>   while baz() {
>>     while garply() {
>>       if quux() {
>>         break unknown
>>       }
>>     }
>>   }
>> }
>> 
>> Another downside of the `unexpected case:` spelling is that it doesn't work as part of a larger pattern. I don't have a good answer for that one, but perhaps it's acceptable for now.
>> 
>> I'll write up a revision of the proposal soon and make sure the core team gets my recommendation when they discuss the results of the review.
>> 
>> ---
>> 
>> I'll respond to a few of the more intricate discussions tomorrow, including the syntax of putting a new declaration inside the enum rather than outside. Thank you again, everyone, and happy new year!
>> 
>> Jordan
>> 
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-evolution
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20180104/8eea3816/attachment-0001.html>