[swift-evolution] [Review] SE-0194: Derived Collection of Enum Cases

[Xiaodi Wu]

On Wed, Jan 10, 2018 at 5:06 AM, Brent Royal-Gordon <brent at architechies.com>
wrote:

> On Jan 9, 2018, at 10:26 PM, Xiaodi Wu via swift-evolution <
> swift-evolution at swift.org> wrote:
>
> I continue to have concerns about this proposal, and I'm gravely and very
> bitterly disappointed that the concerns have not even been acknowledged in
> the Alternatives section, which is in my view the minimum action that an
> author should take when points are raised during a pitch phase, even (and
> especially) when the author disagrees, along with a cogent write-up of why
> the proposed design is superior in the author's view to the alternative. In
> this case, the proposal authors write:
>
>   "The community has not raised any solutions whose APIs differ
> significantly from this proposal, except for solutions which provide
> strictly more functionality."
>
> This is false, as I have offered a solution in the past whose API differs
> entirely from this proposal, and which provides strictly a subset of the
> functionality which goes to the core of the issue at stake.
>
>
> I can't speak for the other co-authors, but for my part, this was an
> oversight and I apologize for it. I think we should have discussed your
> `MyType.self` alternative.
>
> I won't rehash the entire discussion in previous threads, but to summarize
> my objections:
>

Again, I do appreciate the detailed reply. Time is never enough, so I'll
try to be both concise and complete in my reply. Hopefully, it'll live up
to your example in terms of thoroughness.


> 1. `MyType.self` is harder to understand for someone who's never seen it
> than `MyType.allValues`. For instance, the expression
> `AccountStatus.allValues[0]` is completely self-explanatory, while
> `AccountStatus.self[0]` is more obscure and would require a trip to the
> documentation. (And since `self` here is a language keyword, not a real
> member, the most obvious route to the documentation is not available.) In
> my opinion, we should not prefer the `MyType.self` syntax.
>
> 2. Even if the community disagrees with me and thinks `MyType.self` is a
> better syntax than `MyType.allValues`, it is not better *enough* to
> outweigh the costs:
>
> • The metatype solution provides no additional functionality; it is merely
> a matter of which syntax we choose to support, and how much effort this
> support requires.
>
> • Conforming the metatype to `Collection` requires a lot of type system
> features we do not currently have. Currently, structural types cannot
> conform to protocols, and metatypes are a structural type. Metatypes also
> cannot have subscripts currently. Your proposed design has a lot of
> prerequisites. That is not in and of itself disqualifying, but it should be
> weighed against it.
>
> • You suggest that we should add bits and pieces of "`Collection`
> functionality" incrementally as engineering resources become available. The
> problem is that the most valuable part of the "`Collection` functionality"
> is conformance to `Collection` or at least `Sequence`, not the existence of
> any particular members of `Collection`, and this is the part that would
> require the most engineering resources.
>
> • A large part of the work we would do before supporting this conformance
> would be disposed of immediately once we could get it. For instance, all of
> the work to support having a metatype instead of a sequence in a `for-in`
> statement would be thrown away as soon as we could conform to `Sequence`.
> So this looks less like slowly building up pieces of the feature until we
> have all of them in place, and more like creating temporary hacks to
> emulate the feature until we have the time to do it right.
>
> • While this feature is not a showstopper, it is a highly desirable
> convenience. The proposal documents the high demand for this feature, so I
> won't elaborate on this point further.
>
> • Therefore, adopting this design would add a lot of engineering
> complexity before we could fully support a highly desirable feature, merely
> to get a syntax that we *might* prefer.
>
> To summarize the summary: The primary advantage of `MyType.self` is that
> it's elegant. To get that elegance, we must trade away getting a
> fully-functional implementation sooner, spending a lot of engineering
> resources (much of which would be wasted in the end), and—most crucially in
> my opinion—clarity at the point of use. It's not worth it.
>
>
Let me take two steps back, then address your points (1) and (2). One
overarching critique of mine is that you are conflating two (related)
features into one. The stated motivation is the desire to enumerate all the
cases of an enum (let's call this problem A). The presented solution
enlarges the problem to be solved to that of enumerating all the values of
a type (let's call this problem B, which is kind of (*) a superset of
problem A).

(*) More on the "kind of" part later.

It does not follow that simply because a solution to A is often requested,
therefore a solution to B needs to be urgently implemented. This is a
straight-up error in logic. If, instead, problem A were a superset of
problem B and not the other way around, then the argument would follow
logically. But that is not the case here. Let's return to this issue a
little later. I bring it up here because points (1) and (2) are about
solutions to problem B.

So, on to your points (1) and (2). To restate my original objection, it is
that conceptually, "all the possible values of a variable" is a definition
of a type. Whether it's "elegant" or not, "obscure" or not, if all the
values representable by a type can be enumerated, then the metatype meets
the semantics requirements of conformance to Collection. This is simply
true (or it's not, and I'm mistaken), but in any case it's not a matter of
opinion or taste but of fact.

Now, if one day we try to solve problem B and you say, "I think
`MyType.self` is obscure, and I'd like to have a synonym called
`MyType.allValues` that's more discoverable," then the wisdom of adding a
member named `allValues` could be evaluated on its own merits. But if
`MyType.allValues != MyType.self` because of missing features in Swift,
then quite simply Swift is incapable of fully supporting a solution to
problem B.

But that's still OK: what you've documented is that there is high demand
for a solution to problem A, a kind of (*) subset of problem B. If problem
B cannot be solved in an appropriate manner for Swift 5, it doesn't have to
be solved at all for Swift 5. We should focus on problem A.

Earlier in this thread (or was it in the companion one?), another community
> member suggested that if `allValues` were to conform to `Sequence` instead
> of `Collection`, then even types that have an infinite number of possible
> values could conform to `ValueEnumerable`. Here's the rub: the definition
> of a type, or at least one of them, _is_ precisely the set of all possible
> values of a variable. If unconstrained by finiteness, then *all types*
> would meet the semantic requirements of `ValueEnumerable`.
>
>
> Not quite. There are types whose valid values are unknowable; for
> instance, a type representing "the ID of a record on the server" cannot
> know which IDs actually exist on the server, merely which ones *could*
> exist, and so an implementation of `allValues` would return invalid
> instances.
>

Careful. You're conflating different definitions of "validity" here. If I
store the ID of an existing record in a variable of type T, then another
process deletes that record, the ID I have stored is plainly still of type
T. So then the question is, valid for *what*? (Certainly, for example, it's
still valid as far as the type checker is concerned.)

Or to rephrase, what kind of "possible" are we talking about in terms of
"possible values"? Are the only "possible" ID values those that exist in
the database (i.e., "actual" ID values)? Or are they the full range of
"possible" ID values present and future according to the table schema? This
is not merely word play here: it demonstrates fuzziness in the semantic
requirements of your protocol.

But that's beside the point. What I think the "`allValues` should be
> allowed to be infinite" suggestion misses is that one of
> `ValueEnumerable`'s semantics is that it's not only theoretically
> *possible* to enumerate all the values, but actually *reasonable* to do so.
> This is more slippery and subjective than most protocol semantics, but I
> don't think that should disqualify it. There are plenty of places in the
> standard library where we make judgment calls like this. For instance, the
> decision that `Optional` should not conform to `Collection` is a similar
> judgment call: `Optional` could easily meet all of the formal requirements
> of a `Collection`, but we chose not to do it because we decided it didn't
> make *subjective* sense.
>
> Some simple enums could not be reasonably conformed to `ValueEnumerable`;
> for instance, there's little sense in conforming an `Error` enum, because
> you're unlikely to need to discover all the possible errors expressed by a
> type at runtime. Some non-simple enums could be reasonably conformed to
> `ValueEnumerable`; `Bool` is an obvious example.
>
> Some types, of course, fall into a gray area. `Int8` is fairly reasonable,
> but larger integer types get increasingly unreasonable until, by `Int64`,
> we reach types that would take decades to enumerate. Where the line should
> be drawn is a matter of opinion. (My opinion, to be clear, is that we
> shouldn't conform any of them; if someone really wants to do it, they can
> add a retroactive conformance.)
>

Whether or not to *conform* a type to a protocol can certainly be a
judgment call, and there are many places where that happens in the standard
library; I suppose it's accepted in part because someone who disagrees can
add a retroactive conformance where there isn't one, as you say.

But that is not solely the task before us, nor the criticism here.

My problem is that the semantics of your proposed protocol are slippery and
subjective, as you acknowledge, and (as you'll see here and below) that is
to be assiduously avoided in the standard library. As to your own example,
there is no question whether `Optional` could fulfill the semantic
requirements of conformance to `Collection`: the semantics defined on that
protocol are such that there can be no disagreement that `Optional` does in
fact meet the requirements. The judgment call is whether it's wise to
conform or not, but the protocol's semantic requirements are anything but
"slippery."

> As proposed, "`ValueEnumerable`...indicate[s] that a type has a finite,
> enumerable set of values"; now, we have the constraint that the type merely
> must have an upper bound in terms of memory referenced. Brent just wrote
> that he might later propose to extend `ValueEnumerable` to `Bool` and
> `Optional`, but theoretically and practically speaking it appears that it
> can correctly be extended to any type in the standard library that is not a
> `Sequence`, and with minimal effort even `Collection` types of fixed size
> (e.g., CollectionOfOne<T> with the right constraints as to the generic type
> T).
>
>
> Sure, but theoretically speaking, we could synthesize a `Comparable`
> implementation for all classes which compared them by address. This
> implementation would be totally correct, would fulfill all of the
> requirements of the protocol it was conforming to, and would usually be
> meaningless. So we don't.
>

No, actually, you are incorrect about this example. We don't have such an
implementation not because of some judgment that it's "meaningless" but
because the documented semantics of `Equatable` unambiguously prohibit it:
that is, it is not theoretically correct to do so. I refer you to the
section in the `Equatable` documentation titled "Equality is Separate From
Identity," which begins: "The identity of a class instance is not part of
an instance’s value." Since `Comparable` refines `Equatable`, comparison by
address would violate the clearly stated semantic requirements of the
protocol.

Again, my criticism of your proposed design is precisely that it lacks
semantic requirements such as this. Both this example, and the `Optional`
example you give above in trying to demonstrate that it's all right to have
ambiguity in protocol semantics, actually show that the standard library
goes to great lengths not to leave that ambiguity. We have an opportunity
to do so by sticking closely to what's desired in terms of solving only
problem A, not trying to generalize a solution to encompass all types for
which values are "reasonably enumerable"--which is very, very flimsy as to
semantics.

The fact that some types could be given a useless conformance to a protocol
> does not imply that the protocol shouldn't exist.
>
> First, "ValueEnumerable"-ness is neither universal to all enums (as those
> with associated types, indirect cases (think of all those tutorials about
> linked lists implemented as Swift enums), etc., are clearly not enumerable)
> nor unique as a property of enums,
>
>
> It's absolutely true that not all enums should be enumerable, and also
> true that many non-enums should be enumerable. That's precisely why the
> protocol is not `CaseEnumerable` and the property is not `allCases`.
>

Rather, I think that's precisely why problem A is only *kind of* a subset
of problem B and why we should be solving only problem A, which is strongly
motivated, not problem B. The example of an enum with cases having an
associated type of another enum only reinforces the idea that "all values"
and "all cases" are actually sometimes not the same thing for an enum.


>
> yet this proposal first makes a large generalization of the proposed
> protocol's semantics when the stated motivation is only about enums,
>
>
> Do you disagree that there are many types which are not enums, but
> which—like the enums we are trying to address with this proposal—it is also
> reasonable to want to retrieve all values of? Or do you think we have
> missed important aspects of these types by not deeply analyzing them? Or
> are you simply criticizing how this part of the proposal was drafted,
> despite believing that its conclusion is correct?
>

I think that many important issues related to the general question of types
being enumerable (problem B) have not been deeply analyzed, and that you
have proposed a solution to problem B where the motivation presented is
only for problem A, and that problem A is only a kind of subset of problem
B which does not engage the same panoply of difficult issues and may well
have a different ideal solution than that for problem B.

In other words, I think your proposal correctly states a problem (A) with
strong motivation, attempts to solve a related problem (B) with no good
motivation for the switch (from A to B), and proposes a solution (for B)
which is expedient rather than deeply correct with the attempted
justification that the original problem (A) needs to be solved right away.


>
> then proceeds only to implement protocol conformance for enums.
>
>
> I think it's more accurate to say that it "only synthesizes a default
> implementation for simple enums". This is for three reasons:
>
> 1. Simple enums will probably be the most common conforming types, even if
> they aren't the only ones.
>
> 2. We can very easily synthesize an implementation which uses private APIs
> to return an `Int`-indexed and zero-based `RandomAccessCollection`, is
> highly optimized, and is forward-compatible. That is, for this subset of
> types and no others, we can implement a no-compromises, ideal
> implementation using special knowledge.
>
> 3. We are only confident that we know enough about the type to synthesize
> its implementation when it's a simple enum.
>
> More on that last point: Enums explicitly list all of the valid values in
> the source code. By contrast, a struct definition often permits values
> which are not actually valid because Swift's type system is not rich enough
> to conveniently express the constraints on their properties. For example,
> in these types:
>
> struct PlayingCard: ValueEnumerable {
> enum Suit: ValueEnumerable {
> case hearts, spades, diamonds, clubs
> }
> var suit: Suit
> var rank: Int
> }
>
> The compiler can correctly synthesize `PlayingCard.Suit.allValues` because
> all of its constraints are specified in code. By contrast, the compiler
> cannot know that `rank` must be between 1 and 13, so if it tried to
> synthesize `PlayingCard.allValues`, it would contain invalid values.
>
> These limitations are quite common in the types of code synthesis we've
> introduced so far. For example, we only synthesize an `Equatable`
> conformance if all the types involved are themselves `Equatable`; that's
> not because `Equatable` is only applicable to those types, it's just that
> we can't be reasonably sure of the desired semantics. The limitation of
> `ValueEnumerable` synthesis to simple enums is similar.
>
> Second, the collection of all values is properly just the type and not a
> property of the type, for the reasons outlined above.
>
>
> Okay, let's say that's true. Is that the only or best way to express it?
>

No, in need not be the only or the best way, but for the reasons given
above it should be one of the ways.

In other words, suppose you believe that `allValues` is the best spelling.
Fine. But something is wrong if `type(of: MyType.allValues) != type(of:
MyType.self) || MyType.allValues != MyType.self`.

So far, the justification I have heard for ignoring this objection is that
> (a) lots of people want the specific use case of knowing all the cases of
> an enum; and (b) a complete design which makes metatypes conform to
> `Collection` is not feasible for Swift 5. But that, in my view, cannot
> justify the _permanent_ inclusion (with ABI stability) of a protocol whose
> semantics apply to all non-`Sequence` types, littering the standard library
> and untold many other libraries with this conformance for the sake of
> having something done for Swift 5.
>
>
> My suggestion was, and is: if the motivation is to enumerate all the cases
> of an enum, deliver the best possible design for the specifically motivated
> use case rather than trying to deliver the most easy-to-implement design
> for the most general use case. In other words, give properly conformed
> enums (e.g. `enum MyEnum : Enumerable`--and I do suggest shortening the
> name, since what's enumerable is "the set of all values" == "the type") the
> synthesized ability to have all cases enumerated by iterating over the
> metatype: `for case in MyEnum.self { ... }`. Add as much other `Collection`
> functionality as can be implemented in the Swift 5 timeframe, starting with
> the most pressing based on the motivating use case. Then deliver more and
> more of the best possible design with each version of Swift as engineering
> resources permit.
>
>
> There is nothing wrong with the proposed design. It's a good design, and
> depending on one's priorities, it's arguably the *best* design. That it's
> feasible to deploy today is just icing on the cake.
>

It may be the most feasible design for solving problem B, and maybe even
the best design, but in my view it's not correct if `MyType.allValues !=
MyType.self`. But crucially, even if you don't buy that argument, solving
problem B is not appropriately motivated, and the best design for solving
problem B may not be that for solving problem A, and I believe that we
ought to focus on the latter.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20180113/407ec6b0/attachment.html>