[swift-evolution] [swift-dev] Re-pitch: Deriving collections of enum cases

[Xiaodi Wu]

On Tue, Nov 14, 2017 at 11:06 PM, Brent Royal-Gordon <brent at architechies.com
> wrote:

> On Nov 14, 2017, at 5:21 PM, Xiaodi Wu <xiaodi.wu at gmail.com> wrote:
>
> 1. It must be possible to easily access the count of values, and to access
>> any particular value using contiguous `Int` indices. This could be achieved
>> either by directly accessing elements in the list of values through an Int
>> subscript, or by constructing an Array from the list of values.
>>
>> 2. It must be possible to control the order of values in the list of
>> values, either by using source order or through some other simple,
>> straightforward mechanism.
>>
>
> OK, first of all, nowhere in the proposal text are these requirements
> stated as part of the use case. You're free to put forward new use cases,
> but here I am trying to design the most elegant way to fulfill a stated
> need and you're telling me that it's something other than what's written.
>
>
> Honestly, re-reading the proposal, it never cites a fully-formed use case.
> Instead, it cites several blog posts, Stack Overflow questions, and small
> code samples without digging in to the underlying reasons why developers
> are doing what they're doing. Most of the people discussing it so far seem
> to have had a tacit understanding that we wanted roughly Array-like access,
> but we haven't explicitly dug into which properties of an Array are
> important.
>
> (If anyone involved feels like they had a different understanding of the
> use case, please speak up.)
>
> I think this is a place where the proposal can be improved, and I'm
> willing to do some writing to improve it.
>
> You say that:
>>
>>  Essentially all other uses for enumeration of enum cases can be
>> trivially recreated based on just that.
>>
>>
>> But with this use case in mind, we can see that it is "trivial" in the
>> sense that the annoying boilerplate you need to bridge the significant
>> impedance mismatch is easy to come up with. Yes, you could construct an
>> array using the magic `for` loop, but that would be a serious pain. (And
>> there would be no ergonomic, mistake-resistant way to hide that pain behind
>> a function/initializer call, because there's no way to say that a parameter
>> must be a metatype for an enum.) What you really want is a way to access or
>> construct an `Array` or array-like type containing the type's values.
>>
>
> You cannot truly believe that
>
> ```
> var cases = [BugStatus]()
> for c in BugStatus.self { cases.append(c) }
> ```
>
> is "serious pain." Yes, part of being an incomplete implementation is that
> it lacks the ergonomics of a fleshed-out conformance to `Collection`. But
> "serious pain"?
>
>
> Yes, I'll stand by "serious pain".
>

If only all of life's challenges were such "serious pain."


> This is fundamentally a convenience feature,
>

I greatly disagree with this characterization of the proposed feature. If
it were "fundamentally a convenience," then it would be best deferred from
Swift 5 entirely.

No, this feature is important to discuss--and hopefully implement--in the
Swift 5 timeframe because it is related to resilience. It is currently
_impossible_ to get all the cases of an enum. Even when you manually write
boilerplate, it can only cover the existing cases of an enum; if the enum
is vended by a library that later adds a case, your code may break. And if
you peer into the memory at runtime to do clever things, the runtime layout
of enums is not guaranteed at present. Put simply, you cannot retrieve all
the cases of an enum in today's Swift in a future-proof way. Making it
possible to do so is not about convenience--not at all. Convenience can
(and should, I'd argue) be deferred while more urgent solutions need to be
shipped first.

so it needs to actually *be* convenient—more convenient than writing out
> the enum cases in an array literal. Forcing users to write imperative-style
> code that can't be encapsulated is not convenient.
>
> The point here is that even a minimal step towards what we agree is the
> ideal design would make _possible_ what today is _impossible_: namely,
> future-proof enumeration of all the cases of an enum type without
> associated values.
>
>>
> We can take a minimal step towards having the feature…or we can just have
> the feature.
>

The minimal step I propose makes _possible_ the core use case in its
entirety. Again, what makes this proposal so timely is that it's about
making _possible_ certain uses of resilient enums that are currently
_impossible_. The "feature" here to be achieved is the _possibility_ of
enumerating all the cases of an enum.

*Actually* conforming the metatype to `Sequence` or `Collection` would be a
>> different story. There, you could construct `Array`s or access elements
>> using ordinary APIs and type system features. And you could write generic
>> algorithms which used the set of all types: they would require conformance
>> to `Sequence` or `Collection`, and users would specify `Foo.Type` as the
>> generic parameter.
>>
>
> Indeed. The point here is that we don't need a name for this protocol.
> You'd be able to write useful generic algorithms by using functions such as
> `map` on `T.self` with intuitive constraints such as `T where T.Type :
> Collection`. Isn't that a sublime way of expressing exactly what we mean?
>
>
> It is! As I said, I love the idea of conforming the metatype to
> `Collection`—it's very elegant. But the only advantage I can identify over
> this proposal is a slight gain in elegance, while its
> *disadvantages*—requiring several nontrivial enhancements to the language,
> and therefore deferring large amounts of very desirable functionality to an
> unspecified future release—are significant.
>

Here, I'd agree with you that what would be deferred _are_ purely
conveniences. You can call it very desirable functionality, but such
conveniences can wait. There are so many deep features in Swift 5 that will
also have the effect of making Swift more enjoyable and ergonomic to write.
Given two options to which we can devote time and effort going forward,
nontrivial long-term enhancements versus quick-fix pure conveniences, it is
an *advantage* and not a *disadvantage* when a design falls into the first
category.

In the meantime, we have an urgent issue here related to that very deep
feature of library resilience about how to retrieve all the cases of an
enum.

Basically, I don't think it's worth waiting for the "sublime way".
>
> (There's also the disadvantage that the meaning of `Foo.self[i]` is not
> immediately obvious in the way `Foo.allValues[i]` is. As it is, I'm not
> totally convinced that `ValueEnumerable` is an obvious enough name;
> `Foo.self` would be much more problematic in that sense.)
>
> (And there's the opt-in question. Public types may not *want* to
> participate in this feature, but you seem to suggest they should have to.)
>

That is orthogonally debatable. We could, of course (though I would not
advocate for it), require users always to explicitly conform their
metatypes (like so: `extension Foo.Type : Collection { }`) and then offer
synthesis of requirements. But yes, I do think it should be opt-out rather
than opt-in for enums.

But I suspect that would require deeper compiler changes than we can be
>> certain to get in Swift 5 or really at any specific point on the roadmap,
>> and I don't think we should delay this feature indefinitely to get a design
>> whose only real benefit is elegance.
>>
>
> We may (almost certainly) need more time to realize the full design. But
> we don't need much to take the first steps towards it, which--as I write
> above--would already make possible the currently impossible. It seems you'd
> rather ship a complete but inelegant design forever than an incomplete but
> useful part of an elegant design now. I wouldn't make that trade-off.
>
>
> This is a feature people have wanted—asked for constantly—since Swift was
> released. SR-30 is the oldest bug labeled "LanguageFeatureRequest" in the
> Swift.org bug tracker. We've had several different threads, posted by
> several different people, independently proposing it. Outside the evolution
> process, questions and answers about how to do this (and related questions,
> like "how do I get the number of cases?", which is usually a backdoor
> version of it) are extremely common.
>
> Apple deferred it beyond Swift 2, and Swift open source deferred it beyond
> Swift 3 and 4, in part because we saw that future associated type features
> and finalized type metadata formats would make the implementation
> significantly better if we waited. The type features are now here, and the
> metadata format will be frozen in the next release. We have everything we
> need right now to make a very good version of this feature. You propose
> deferring it again, not because the functionality would be appreciably
> improved by the delay, but because it could be done more cleverly.
>
> So let me turn the question around: Why *should* we wait?
>

We shouldn't wait. We should make it possible to enumerate all the cases of
an enum; this is important for resilient enums.

What appreciable advantages, *besides* elegance, do conforming the metatype
> to `Collection` offer over a static constant?
>

It avoids the introduction of a protocol to the standard library that has
scant semantic significance. Recall that our bar for introducing a protocol
is to ask what semantic guarantees are made possible, and what useful
generic algorithms can be written that could not be written otherwise. What
do all `ValueEnumerable` types [as opposed to their corresponding
metatypes] have in common semantically? [Answer: What they have in common
is that we can retrieve the set of possible values of such a type; but
wait, one pretty darn good definition of a _type_ is the set of possible
values that a variable can have.] And what can you do in generic code with
`ValueEnumerable` other than access its one static property, after which
you end up relying on the semantics guaranteed by `Collection` and not that
of `ValueEnumerable`?

It results in a design that has a smaller API surface area, which is (all
other things being equal) superior to a design that has a larger API
surface area.

It permits each metatype to choose their own most suitable conformance in
the `Sequence : Collection : BidirectionalCollection :
RandomAccessCollection` hierarchy, should we choose to include non-enum
metatypes, and it permits users to use appropriate generic algorithms that
operate on sequences or collections with compatible metatypes. Suppose a
type has an infinite set of possible values, but they're well ordered and
there's a clear starting value (maybe an unsigned BigInt, say): the
corresponding metatype would be able to conform to Sequence instead of
Collection. Again, I don't really think of enabling such features on
integer types as a positive, but nonetheless it bears mentioning.

It correctly fits our mental model that enums themselves, not some static
property of enums, are enumerable--and, moreover, that the set of possible
values that a variable can have _is_ what a type represents, not what a
type's `allValues` property represents.

If someone buttonholes you at a conference and asks, "Why did you choose a
> magic `for` loop, instead of just giving us an array of cases?", what is
> your answer, and is it one that will satisfy them?
>

Firstly, it's not going to be an array of cases; we've already decided that
it's got to be some special collection.

Secondly, the reason that you should be able to use a `for` loop with the
metatype is that enum types are supposed to be, well, enumerable--not
merely some static property of enums. In the future, we'll permit
conformance to protocols that add more conveniences, just like we'll add
more conveniences for tuple equality, or for String-Substring conversion,
or any variety of APIs. In each case, we offer the best possible design
given the expressiveness of today's Swift, then wait for additional
language features rather than trying to design workaround hacks that become
permanent parts of the standard library.

Personally, I don't think I could give the answer "We thought of a design
> that offered no additional functionality, but was much cooler" with a
> straight face.
>

No, we thought of a superior design. It allows users to maximally reuse
their existing knowledge of standard library protocols and apply it to a
new context, and it adds impetus to expanding what metatypes can do, laying
the groundwork for a more expressive language.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20171115/eed6ce83/attachment.html>