[swift-evolution] Enhanced existential types proposal discussion

[Austin Zheng]

Thank you for your feedback, Doug! I appreciate you taking the time to read
through everything and write up your thoughts. I'll revise the proposal and
plan for a 2017 time frame.

I would also be happy to see someone from the core team write up a proposal
when the time is right, should they feel this one isn't in the spirit of
the desired feature.

Austin

On Sun, Jun 5, 2016 at 4:20 PM, Douglas Gregor <dgregor at apple.com> wrote:

>
> On May 18, 2016, at 12:35 AM, Austin Zheng <austinzheng at gmail.com> wrote:
>
> I've put together a considerably more detailed draft proposal, taking into
> account as much of Matthew's feedback as I could. You can find it below:
>
>
> https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md
>
> Since there is no chance this will come up for review anytime soon, I
> expect to make significant revisions to it over the next month or so. Any
> feedback would be greatly appreciated.
>
>
> This is very much Swift 4 territory, but I can’t help myself… so…
>
> The actual feature description is spread out through this very long
> document, with user-facing ideas (e.g., using “anonymous associated types”)
> intermixed with deeper technical details (existential type equivalence), so
> it’s very daunting to read. Please bring the user-facing features to the
> front (“Proposed Solution”) with examples, and save the deeper technical
> details for “Detailed Design”. You want more readers to make it through the
> part that affects them.
>
> Shortcut 'dot' notation: If there is only one protocol with associated
> types specified in the requirements, and there are no nested Any<...> requirements
> with where clauses of their own, that protocol's name can be omitted from
> the whereclause constraints:
>
> // Okay// Would otherwise be Any< ~ where Collection.Element == Int>let a : Any<class, Collection, Any<Streamable, CustomStringConvertible> where .Element == Int>
> // NOT ALLOWED// Both Collection and OptionSetType have associated types.let b : Any<Collection, OptionSetType where .Element == Int>
>
> FWIW, I think “.Element == Int” should be the only syntax. In generic
> signatures, if you have two different protocols with same-named associated
> types, and a given type parameter (or associated type) conforms to both
> protocols, the associated types are (implicitly) made equivalent via an
> inferred same-type constraint. So there’s no reason to introduce the
> “Collection.Element == Int” syntax, because the “Collection” part is
> basically irrelevant.
>
> Once existentials have been suitably enhanced, there is a strong analogy
> between an existential and a generic signature with a single type parameter
> that you can’t name. An existential Any<Collection where .Element :
> Equatable> has most of the same characteristics as a generic something with
> the signature <T : Collection where T.Element : Equatable>. Specifically,
> the sections on “Existential type equivalence”, “Ordering”, “Real types to
> anonymous associated types”, “Anonymous associated types to real types”.
> could be reduced to a few small, simple examples and a mention of the
> analogous behavior of generics. It will be far easier to explain this way,
> and readers don’t need to get immersed in the details. Where there are
> differences vs. generics, that’s important to point out.
>
> “Associated typealias rewriting”: this also falls out of the equivalence
> with generics + SE-0092.
>
> “Associated types and member exposure”: you don’t make the point that it
> only makes sense to refer to the associated types of a *let* constant; a
> *var* could change its type dynamically, which would invalidate the
> typing rules. Did you consider just using “x.dynamicType” in the type
> grammar for this? It’s more general, in that you can refer to associated
> types but also talk about the dynamic type of “x” itself, e.g.,
>
> let x: Equatable = …
> let y: Equatable = …
> if let yAsX = y as? x.dynamicType { … x == yAsX … }
>
> which is (almost?) as powerful as a general “open” expression.
>
> I’m not a fan of the “anonymous associated types” terminology: these are
> associated types of a type of some runtime-defined value. The only thing
> “anonymous” about them is that it’s harder to spell the base type;
> otherwise, they’re just like associated types of a generic type parameter.
> Again, the generics analogy is strong here.
>
> FWIW, I don’t think we’ll ever need “opening existentials” with what
> you’ve described here. Also, remember that a method of a protocol extension
> essentially opens “Self”, so we already have one way to open an existential
> (and that’s probably enough).
>
> I was a little surprised you didn’t point out that AnyObject could become
>
> typealias AnyObject = Any<class>
>
> or give the nice “AnyCollection” syntax:
>
> typealias AnyCollection<T> = Any<Collection where .Element == T>
>
> the latter of which is fairly important, because it gives nice syntactic
> sure to one of the most highly-requested features [*]. I’d suggest having
> that example very, very early.
>
> - Doug
>
> [*] That generally comes in as “Swift should have parameterized protocols…”
>
>
> Austin
>
> On Tue, May 17, 2016 at 9:52 PM, Austin Zheng <austinzheng at gmail.com>
> wrote:
>
>>
>>
>> On Tue, May 17, 2016 at 1:25 PM, Matthew Johnson <matthew at anandabits.com>
>> wrote:
>>>
>>>
>>>
>>>>
>>>>
>>>> Within the angle brackets are zero or more 'clauses'. Clauses are
>>>> separated by semicolons. (This is so commas can be used in where
>>>> constraints, below. Better ideas are welcome. Maybe it's not necessary; we
>>>> can use commas exclusively.)
>>>>
>>>>
>>>> I’m not a fan of the semicolon idea.  I don’t see any reason for this.
>>>> The `where` keyword separates the protocol list from the constraints just
>>>> fine.  The list on either side should be able to use commas with no problem
>>>> (or line breaks if that proposal goes through).
>>>>
>>>>
>>> I'm leaning towards getting rid of the commas, but would like to write
>>> out a few 'dummy' examples to see if there are any readability issues that
>>> arise.
>>>
>>>
>>> Replaced with what?  Whitespace separation?  I suppose that might work
>>> for the protocol list but it feels inconsistent with the rest of Swift.
>>> Commas plus (hopefully) the alternative of newline seem like the right
>>> direction to me.
>>>
>>
>> Sorry, I completely misspoke (mistyped?). I meant I want to get rid of
>> the semicolons and use commas. I've come to the conclusion that there are
>> no readability issues, protocol<> already uses commas, and semicolons used
>> in this manner don't have a precedent anywhere else in the language.
>>
>>
>>>
>>>
>>>> There are five different possible clauses:
>>>>
>>>>
>>>>    - 'class'. Must be the first clause, if present. Places a
>>>>    constraint on the existential to be any class type. (Implies: Only one can
>>>>    exist. Mutually exclusive with class name clause.)
>>>>
>>>>
>>>> (In the future a follow-up proposal should add in 'struct' or 'value'
>>>> as a counterpart.)
>>>>
>>>>
>>>> If we’re going to allow `struct` we should also allow `enum`.  `value`
>>>> would allow either of those.
>>>>
>>>>
>>> Of course. A future proposal can allow list members to discuss the exact
>>> details as to how struct, value, or enum specifiers should work.
>>>
>>>
>>> Yep, agree.  Just mentioning that if we’re going to reference it we
>>> should not leave obvious holes in what would be considered.  :)
>>>
>>
>> Absolutely.
>>
>>
>>>
>>>
>>>>
>>>>    - Class name. Must be the first clause, if present. (Implies: Only
>>>>    one can exist. Mutually exclusive with 'class'.) Places a constraint on the
>>>>    existential (not really an existential anymore) to be an instance of the
>>>>    class, or one of its subclasses.
>>>>
>>>> It is still be an existential if it includes protocol requirements that
>>>> the class does not fulfill.  For example, you might have Any<UIView,
>>>> SomeProtocol> where UIView does not conform to SomeProtocol, but various
>>>> subclasses do.
>>>>
>>>>
>>> Fair enough. (I don't think the way things work would be affected.)
>>>
>>>
>>>> Your proposal doesn’t discuss composing Any in the way that Adrian’s
>>>> did like this:
>>>>
>>>> typealias Foo = Any<SomeClass, SomeProtocol, OtherProtocol>
>>>> Any<AnotherProtocol, Foo>
>>>>
>>>
>>> I didn't think it needed to be discussed. An Any<...> existential type
>>> is a type 'expression' just like any other, and should be allowed to
>>> participate in other Any<...>s.
>>>
>>>
>>>
>>>>
>>>> I like the idea of composition as it allows us to factor out
>>>> constraints.  If we are going to do that we should allow a class to be
>>>> specified in the composition as long is it is a subclass of all class
>>>> requirements of Any types it composes.  For example, this should be allowed:
>>>>
>>>> typealias Bar = Any<SubclassOfSomeClass, Foo, AnotherProtocol>
>>>>
>>>> This is still one class requirement for Bar, it just refines the class
>>>> requirement of Foo to be SubclassOfSomeClass rather than just SomeClass.
>>>>
>>>
>>> This is a good point. There should be clarification as to how special
>>> cases of Any<...> used in another Any<...> behave. For example, like you
>>> said Any<MyClass, Any<SomeSubclassOfMyClass, Protocol>> should be valid.
>>> This will go into any proposal that emerges from the discussion.
>>>
>>>
>>> Yes, this is why we need to discuss Any composition.  There are also
>>> cases of incompatible associated type constraints which need to be rejected
>>> (such as composing two Any’s where one has Element == String and another
>>> has Element == Int).
>>>
>>
>>>
>>>
>>>>
>>>> Example: Any<UIViewController; UITableViewDataSource;
>>>> UITableViewDelegate>
>>>> "Any UIViewController or subclass which also satisfies the table view
>>>> data source and delegate protocols"
>>>>
>>>>    - Dynamic protocol. This is entirely composed of the name of a
>>>>    protocol which has no associated types or Self requirement.
>>>>
>>>> Example: Any<CustomStringConvertible; BooleanType>
>>>> "Any type which conforms to both the CustomStringConvertible and
>>>> BooleanType protocols"
>>>>
>>>> I'm going to use 'static protocol' to refer to a protocol with
>>>> associated types or self requirements. Feel free to propose a more sound
>>>> name.
>>>>
>>>>
>>>>    - Self-contained static protocol, simple. This is composed of the
>>>>    name of a static protocol, optionally followed by a 'where' clause in which
>>>>    the associated types can be constrained (with any of the three basic
>>>>    conformance types: subclassing, protocol conformance, or type equality).
>>>>    Associated types are referred to with a leading dot.
>>>>
>>>> Please do not introduce terms “dynamic protocol” and “static
>>>> protocol”.  We want to support existentials of protocols that have self or
>>>> associated type requirements.  The dynamic vs static distinction is a
>>>> limitation of the current implementation of Swift and doesn’t make sense
>>>> for the long term vision.
>>>>
>>>
>>> I'm not trying to introduce new terms, these are just placeholders. At
>>> the same time "protocols with self or associated type requirements" is
>>> cumbersome to work with and it would be nice for someone to come up with a
>>> descriptive term of art for referring to them.
>>>
>>>
>>> I agree that a better term would be useful.  In the meantime, I would
>>> prefer something like “trivial” and “nontrivial” protocols.
>>>
>>
>> I've decided to just use the full name until the community comes up with
>> better names. Clarity is preferable to brevity in this case.
>>
>>
>>>
>>>
>>>
>>>>
>>>>
>>>> Example: Any<Collection where .Generator.Element : NSObject,
>>>> .Generator.Element : SomeProtocol>
>>>> "Any type that is a Collection, whose elements are NSObjects or their
>>>> subclasses conforming to SomeProtocol.”
>>>>
>>>>
>>>> Swift does not allow disjunction of requirements.  Only conjunctions
>>>> are supported.  That means the correct reading is:
>>>>
>>>> "Any type that is a Collection, whose elements are NSObjects *and* their
>>>> subclasses conforming to SomeProtocol.”
>>>>
>>>>
>>> Yes, that is what I meant. "whose elements are (NSObjects or their
>>> subclasses) conforming to SomeProtocol”.
>>>
>>>
>>> Ok, good.  Wasn’t quite clear to me.
>>>
>>
>> Yes, the verbiage will need to be clearer in the future. That sentence
>> could be ambiguously parsed.
>>
>>
>>>
>>>
>>>>
>>>>    - Bound static protocol. This is the same as a self-contained
>>>>    static protocol, but with a leading "<name> as " which binds the protocol
>>>>    to a generic typealias. The name can be then be used in subsequent clauses
>>>>    to build constraints.
>>>>
>>>>
>>>> Example: Any<T as Collection; IntegerLiteralConvertible where
>>>> .IntegerLiteralType == T.Element>.
>>>> "Any type that is a Collection, and also can be built from an integer
>>>> literal, in which the collection elements are the same type as the type of
>>>> the integer used for the integer literal conformance.”
>>>>
>>>>
>>>> I’m not sure about this, but if we’re going to do it it should be the
>>>> other way around: `Collection as T` with the alias *after* the name of
>>>> the protocol.
>>>>
>>>>
>>> I like this, it flows better. "Protocol as T where Protocol.Foo == Int,
>>> Protocol.Bar : Baz”.
>>>
>>>
>>> Why did you introduce an alias here and then not use it?  Did you mean
>>> "Protocol as T where T.Foo == Int, T.Bar : Baz"
>>>
>>
>> Another result of rushing to compose an email. Sorry!
>>
>>
>>>
>>>
>>>
>>>> You are also using “dot shorthand” here to refer to an associated type
>>>> of IntegerLiteralConvertible.  I think “dot shorthand” should be limited to
>>>> cases where there is only one protocol that is getting constrained.  In
>>>> other cases, we need to be clear about which protocol we are referring to.
>>>>
>>>
>>> I borrowed dot shorthand from the generics manifesto. But you are right,
>>> it should only be allowed if there is one protocol with associated types or
>>> self requirements clause in the Any<...> construction.
>>>
>>>
>>> I would actually go further and limit it to one protocol period, and
>>> possibly even to one protocol and no type names (as types can have nested
>>> types and typealiases).  When we allow shorthand it should be immediately
>>> unambiguous what the shorthand references with no need to look at type or
>>> protocol declarations.
>>>
>>
>> It might be desirable to propose the proposal with no allowance for
>> shorthand, and have the dot shorthand be a smaller follow-up proposal.
>>
>>
>>>
>>>
>>>
>>>>
>>>>
>>>> There will be rules to prevent recursive nesting. For example, if
>>>> generic typealiases are allowed, they cannot refer to each other in a
>>>> circular manner (like how structs can't contain themeselves, and you can't
>>>> create a cyclic graph of enums containing themselves).
>>>>
>>>> How an existential can be used depends on what guarantees are provided
>>>> by the clauses. For example, 'Any<Equatable>' can't be used for much; if
>>>> there were any methods on Equatable that did not use the associated types
>>>> at all you'd be able to call them, but that's about it. However,
>>>> 'Any<Equatable where .Self == String>' would allow for == to be called on
>>>> instances. (This is a stupid example, since Any<Equatable where .Self ==
>>>> String> is equivalent to 'String', but there are almost certainly useful
>>>> examples one could come up with.)
>>>>
>>>> In order of increasing 'power':
>>>>
>>>>    - Don't constrain any associated types. You can pass around
>>>>    Any<Equatable>s, but that's about it.
>>>>    - Constrain associated types to conform to protocols.
>>>>    - Fully constrain associated types.
>>>>
>>>> I think we need to spell out pretty clearly what members we expect to
>>>> be available or not available.  This section probably needs the most design
>>>> and elaboration.
>>>>
>>>> For example, we probably can’t access a member who uses an associated
>>>> type as an input unless it is constrained to a specific type.  On the other
>>>> hand output types probably don’t need to limit access to a member.
>>>> However, if the output type is Self or an associated type the visible
>>>> signature would have an output type which has the relevant constraints of
>>>> the existential applied, but no more.  In some cases this means the output
>>>> type would simply be Any.
>>>>
>>>
>>> Absolutely. This is vaguely what I had in mind but I wanted to get
>>> something down first. Thanks for thinking through some of the implications
>>> :).
>>>
>>>
>>> That’s what I thought.  Just wanted to start the process of elaborating
>>> expectations.
>>>
>>>
>>>
>>>>
>>>> Where this really gets tricky is for compound types like functions,
>>>> generic types, etc.  Working out the details in these cases is pretty
>>>> complex.  I will defer to Doug on whether it is best to just defer those
>>>> cases to the future, leave them up to the implementer, or try to work out
>>>> all of the relevant details in the proposal (in which case we probably need
>>>> a type system expert to help!).
>>>>
>>>
>>> Yes, exactly! For example, can Any<...> existentials involving protocols
>>> with associated types or self requirements be used within generic function
>>> or type definitions? Maybe there's an argument that existential types of
>>> this nature are redundant if you have access to generics (e.g. defining a
>>> property on a generic type that is a Collection containing Ints; you should
>>> be able to do that today). On the other hand, maybe there are use cases I
>>> haven't thought of…
>>>
>>>
>>> I see no reason they shouldn’t be.  They are not redundant at all.  For
>>> example, you may want to store instances in a heterogeneous collection.
>>> You need existentials to do that.
>>>
>>> A simple example of what I was referring to there is something like this:
>>>
>>> protocol P {
>>>     associatedtype Foo
>>>
>>>     func bar(callback: (Foo) -> ())
>>> }
>>>
>>> In other words, types in the signature of a protocol member are complex
>>> types that reference Self or associated types.  I think you really need a
>>> formal understanding of the type system to understand how to expose these
>>> members through a constrained existential.  We can probably understand the
>>> expected behavior in some of the simpler cases on a case by case basis, but
>>> that approach doesn’t scale at all and is arbitrary.  If they’re going to
>>> be supported an expert is going to need to be involved in the design.
>>>
>>
>> Yes. I have some ideas regarding this topic.
>>
>>
>>>
>>>
>>>
>>>>
>>>> One area you didn’t touch on is “opening” the existential?  Is that out
>>>> of scope for this proposal?  That would be fine with me as this proposal is
>>>> already taking on a lot.  But if so, you should mention something about
>>>> future directions as it is pretty closely related to this proposal.
>>>>
>>>
>>> Yes, existential opening is explicitly separate from this (although I
>>> wanted to mention it in the section where I talk about how Any<Equatable>
>>> is not very useful). But you are absolutely right, this proposal should
>>> discuss how it wants to interact with possible future directions.
>>>
>>>
>>>
>>>>
>>>> Another area you didn’t touch on is whether Any constructs (and
>>>> typealiases referring to them) should be usable as generic constraints.  I
>>>> would expect this to be possible but I think we need to spell it out.
>>>>
>>>
>>> I'm hoping for community input. This is a tricky subject, and at some
>>> point we'll bump into implementation limitations.
>>>
>>>
>>> I don’t think it’s too tricky.  You can just unpack the constraints of
>>> the Any into the list of generic constraints.  Maybe I’m missing something,
>>> but I don’t think so.
>>>
>>>
>>>
>>>>
>>>> -Matthew
>>>>
>>>
>>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20160605/d23a7a5d/attachment.html>