[swift-dev] What can you change in a non-exhaustive enum?

[Xiaodi Wu]

On Sat, Sep 30, 2017 at 11:58 AM, <swift-dev-request at swift.org> wrote:

> Message: 2
> Date: Fri, 29 Sep 2017 18:21:44 -0700
> From: Jordan Rose <jordan_rose at apple.com>
> To: swift-dev <swift-dev at swift.org>
> Subject: [swift-dev] What can you change in a non-exhaustive enum?
> Message-ID: <31DF689E-1AD3-47CB-9FCE-6CBC7E34BC43 at apple.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hello again, swift-dev! This is a sort of follow-up to "What can you
> change in a fixed-contents struct" from a few weeks ago, but this time
> concerning enums. Worryingly, this seems to be an important consideration
> even for non-exhaustive enums, which are normally the ones where we'd want
> to allow a developer to do anything and everything that doesn't break
> source compatibility.
>
> [This only affects libraries with binary compatibility concerns. Libraries
> distributed with an app can always allow the app to access the enum's
> representation directly. That makes this an Apple-centric problem in the
> near term.]
>
> So, what's the issue? We want to make it efficient to switch over a
> non-exhaustive enum, even from a client library that doesn't have access to
> the enum's guts. We do this by asking for the enum's tag separately from
> its payload (pseudocode):
>
> switch getMyOpaqueEnumTag(&myOpaqueEnum) {
> case 0:
>   var payload: Int
>   getMyOpaqueEnumPayload(&myOpaqueEnum, 0, &payload)
>   doSomething(payload)
> case 1:
>   var payload: String
>   getMyOpaqueEnumPayload(&myOpaqueEnum, 1, &payload)
>   doSomethingElse(payload)
> default:
>   print("unknown case")
> }
>
> The tricky part is those constant values "0" and "1". We'd really like
> them to be constants so that the calling code can actually emit a jump
> table rather than a series of chained conditionals, but that means case
> tags are part of the ABI, even for non-exhaustive enums.
>
> Like with struct layout, this means we need a stable ordering for cases.
> Since non-exhaustive enums can have new cases added at any time, we can't
> do a simple alphabetical sort, nor can we do some kind of ordering on the
> payload types. The naive answer, then, is that enum cases cannot be
> reordered, even in non-exhaustive enums. This isn't great, because people
> like being able to move deprecated enum cases to the end of the list, where
> they're out of the way, but it's at least explainable, and consistent with
> the idea of enums some day having a 'cases' property that includes all
> cases.
>
> Slava and I aren't happy with this, but we haven't thought of another
> solution yet. The rest of this email will describe our previous idea, which
> has a fatal flaw.
>
>
> Availability Ordering
>
> In a library with binary compatibility concerns, any new API that gets
> added should always be explicitly annotated with an availability attribute.
> Today that looks like this:
>
> @available(macOS 10.13, iOS 11, tvOS 11, watchOS 4, *)
>
> It's a model we only support for Apple platforms, but in theory it's
> extendable to arbitrary "deployments". You ought to be able to say
> `@available(MagicKit 5)` and have the compiler actually check that.
>
> Let's say we had this model, and we were using it like this:
>
> public enum SpellKind {
>   case hex
>   case charm
>   case curse
>   @available(MagicKit 5)
>   case blight
>   @available(MagicKit 5.1)
>   case jinx
> }
>
> "Availability ordering" says that we can derive a canonical ordering from
> the names of cases (which are API) combined with their versions. Since we
> "know" that newly-added cases will always have a newer version than
> existing cases, we can just put the older cases first. In this case, that
> would give us a canonical ordering of [charm, curse, hex, blight, jinx].
>
>
> The Fatal Flaw
>
> It's time for MagicKit 6 to come out, and we're going to add a new
> SpellKind:
>
> @available(MagicKit 6)
> case summoning
> // [charm, curse, hex, blight, jinx, summoning]
>
> We ship out a beta to our biggest clients, but realize we forgot a vital
> feature. Beta 2 comes with another new SpellKind:
>
> @available(MagicKit 6)
> case banishing
> // [charm, curse, hex, blight, jinx, banishing, summoning]
>
> And now we're in trouble: anything built against the first beta expects
> 'summoning' to have tag 5, not 6. Our clients have to recompile everything
> before they can even try out the new version of the library.
>
> Can this be fixed? Sure. We could add support for beta versions to
> `@available`, or fake it somehow with the existing version syntax. But in
> both of these cases, it means you have to know what constitutes a
> "release", so that you can be sure to use a higher number than the previous
> "release". This could be made to work for a single library, but falls down
> for an entire Apple OS. If the Foundation team wants to add a second new
> enum case while macOS is still in beta, they're not going to stop and
> recompile all of /System/Library/Frameworks just to try out their change.
>
> So, availability ordering is great when you have easily divisible
> "releases", but falls down when you want to make a change "during a
> release".
>
>
> Salvaging Availability Ordering?
>
> - We could still sort by availability, so that you can reorder the
> sections but not the individual cases in them. That doesn't seem very
> useful, though.
>
> - We could say "this is probably rare", and state that anything added "in
> the same release" needs to get an explicit annotation for ordering
> purposes. (This is equivalent to the `@abi(2)` Dave Zarzycki mentioned in
> the previous thread—it's not the default but it's there if you need it.)
>
> - We could actually require libraries to annotate all of their "releases",
> but in order to apply that within Apple we'd need some translation from
> library versions (like "Foundation 1258") to OS versions ("macOS 10.11.4"),
> and then we'd still need to figure out what to do about betas. (And there's
> a twist, at least at Apple, where a release's version number isn't decided
> until the new source code is submitted.)
>
> - There might be something clever that I haven't thought of yet.
>
>
> This kind of known ordering isn't just good for enum cases; it could also
> be applied to protocol witnesses, so that those could be directly
> dispatched like C++ vtables. (I don't think we want to restrict reordering
> of protocol requirements, as much as it would make our lives easier.) So if
> anyone has any brilliant ideas, Slava and I would love to hear them!
>
> Jordan


Kind of a hybrid idea, but hopefully one that circumvents the internal
issues that you outline here and simplifies the mental model. I'll
introduce it stepwise:

Currently, the @available annotation is supported for platforms and for
Swift versions; you propose extending support to arbitrary deployments
(e.g., MagicKit). Instead, suppose you extended support to arbitrary
versioned types (open, public, and @_versioned internal):

@_versioned(abi: 2)
public enum SpellKind {
  case hex, charm, curse
  @available(abi: SpellKind 1) case blight
  @available(abi: SpellKind 1.1) case jinx
}

Now, clearly, the value of "abi" is arbitrary inside the @_versioned
annotation; it's not really necessary for our limited purposes, and if
there's a typo and it's lower than the highest ABI version referenced in an
@available annotation, things get wonky. So, drop it:

public enum SpellKind {
  case hex, charm, curse
  @available(abi: 1) case blight
  @available(abi: 1.1) case jinx
}

This is looking like the original @abi(2) proposal that Dave Zarzycki
brought up. However, a key difference here: multiple cases can have the
same ABI "version" and would be ordered relative to each other by name;
that is, a user would only be annotating when new cases are added and
doesn't have to think about memory layout; Swift takes care of the rest.

This can be simplified further; these ABI version numbers are entirely
arbitrary. Suppose we instead extended the "@available(introduced:)" syntax
to allow dates or timestamps:

public enum SpellKind {
  case hex, charm, curse
  @available(*, introduced: 2017-09-30): case blight
  @available(*, introduced: 2017-10-12): case jinx
}

I suspect there are wrinkles to this scheme, but the overall idea here is
to salvage availability ordering but have some way to version a type in a
low-mental-overhead way instead of resorting to a syntax for manually
ordering cases.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-dev/attachments/20170930/c4f8c8e7/attachment.html>