[swift-evolution] [swift-evolution-announce] [Review] SE-0089: Replace protocol<P1, P2> syntax with Any<P1, P2>

Dave Abrahams dabrahams at apple.com
Fri Jun 10 19:46:59 CDT 2016

on Thu Jun 09 2016, Matthew Johnson <swift-evolution-m3FHrko0VLzYtjvyW6yDsg at public.gmane.org> wrote:

>> On Jun 9, 2016, at 3:05 PM, Dave Abrahams <dabrahams-2kanFRK1NckAvxtiuMwx3w at public.gmane.org> wrote:
>> on Thu Jun 09 2016, Matthew Johnson <matthew-AT-anandabits.com> wrote:
>>>> On Jun 9, 2016, at 11:42 AM, Dave Abrahams <dabrahams-2kanFRK1NckAvxtiuMwx3w at public.gmane.org> wrote:
>>>> on Thu Jun 09 2016, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:
>>>>>> On Jun 9, 2016, at 9:55 AM, Dave Abrahams
>>>>>> <dabrahams at apple.com
>>>>>> <mailto:dabrahams at apple.com>>
>>>>>> wrote:
>>>>>> on Wed Jun 08 2016, Matthew Johnson <matthew-AT-anandabits.com
>>>>>> <http://matthew-at-anandabits.com/>
>>>>>> <http://matthew-at-anandabits.com/
>>>>>> <http://matthew-at-anandabits.com/>>> wrote:
>>>>>>>> On Jun 8, 2016, at 1:33 PM, Dave Abrahams
>>>>>>>> <dabrahams at apple.com
>>>>>>>> <mailto:dabrahams at apple.com>>
>>>>>>>> wrote:
>>>>>>>> on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:
>>>>>>>>>> On Jun 7, 2016, at 9:15 PM, Dave Abrahams
>>>>>>>>>> <dabrahams at apple.com
>>>>>>>>>> <mailto:dabrahams at apple.com>>
>>>>>>>>>> wrote:
>>>>>>>>>> on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com
>>>>>>>>>> <http://matthew-at-anandabits.com/>
>>>>>>>>>> <http://matthew-at-anandabits.com/
>>>>>>>>>> <http://matthew-at-anandabits.com/>>> wrote:
>>>>>>>>>>>> On Jun 7, 2016, at 4:13 PM, Dave Abrahams via
>>>>>>>>>>>> swift-evolution <swift-evolution-m3FHrko0VLzYtjvyW6yDsg at public.gmane.org
>>>>>>>>>>>> <mailto:swift-evolution-m3FHrko0VLzYtjvyW6yDsg at public.gmane.org>> wrote:
>>>>>>>>>>>> on Tue Jun 07 2016, Matthew Johnson
>>>>>>>>>>>> <swift-evolution at swift.org
>>>>>>>>>>>> <mailto:swift-evolution-m3FHrko0VLzYtjvyW6yDsg at public.gmane.org>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> , but haven't realized
>>>>>>>>>>>>>> that if you step around the type relationships encoded in Self
>>>>>>>>>>>>>> requirements and associated types you end up with types that appear to
>>>>>>>>>>>>>> interoperate but in fact trap at runtime unless used in exactly the
>>>>>>>>>>>>>> right way.
>>>>>>>>>>>>> Trap at runtime?  How so?  Generalized existentials should still be
>>>>>>>>>>>>> type-safe.  
>>>>>>>>>>>> There are two choices when you erase static type relationships:
>>>>>>>>>>>> 1. Acheive type-safety by trapping at runtime
>>>>>>>>>>>> FloatingPoint(3.0 as Float) + FloatingPoint(3.0 as Double) // trap
>>>>>>>>>>>> 2. Don't expose protocol requirements that involve these relationships,
>>>>>>>>>>>> which would prevent the code above from compiling and prevent
>>>>>>>>>>>> FloatingPoint from conforming to itself.
>>>>>>>>>>>>> Or are you talking about the hypothetical types / behaviors people
>>>>>>>>>>>>> think they want when they don’t fully understand what is happening...
>>>>>>>>>>>> I don't know what you mean here.  I think generalized existentials will
>>>>>>>>>>>> be nice to have, but I think most people will want them to do something
>>>>>>>>>>>> they can't possibly do.
>>>>>>>>>>> Exactly.  What I meant is that people think they want that expression
>>>>>>>>>>> to compile because they don’t understand that the only thing it can do
>>>>>>>>>>> is trap.  I said “hypothetical” because producing a compile time error
>>>>>>>>>>> rather than a runtime trap is the only sane thing to do.  Your comment
>>>>>>>>>>> surprised me because I can’t imagine we would move forward in Swift
>>>>>>>>>>> with the approach of trapping.
>>>>>>>>>> I would very much like to be able to create instances of “Collection
>>>>>>>>>> where Element == Int” so we can throw away the wrappers in the stdlib.
>>>>>>>>>> That will require some type mismatches to be caught at runtime via
>>>>>>>>>> trapping.
>>>>>>>>> For invalid index because the existential accepts a type erased index?
>>>>>>>> Exactly.
>>>>>>>>> How do you decide where to draw the line here?  It feels like a very
>>>>>>>>> slippery slope for a language where safety is a stated priority to
>>>>>>>>> start adopting a strategy of runtime trapping for something as
>>>>>>>>> fundamental as how you expose members on an existential.
>>>>>>>> If you don't do this, the alternative is that “Collection where Element
>>>>>>>> == Int” does not conform to Collection.  
>>>>>>> This isn’t directly related to having self or associated type
>>>>>>> requirements.  It is true of all existentials.  
>>>>>> That is just an implementation limitation today, IIUC.  What I'm talking
>>>>>> about here would make it impossible for some to do that.
>>>>> If it is just an implementation limitation I am happy to hear that.
>>>>>>> If that changes for simple existentials and generalized existentials
>>>>>>> expose all members (as in the latest draft of the proposal) maybe it
>>>>>>> will be possible for all existentials to conform to their protocol.
>>>>>> Not without introducing runtime traps.  See my “subscript function”
>>>>>> example.
>>>>>>>> That's weird and not very
>>>>>>>> useful.  You could expose all the methods that were on protocol
>>>>>>>> extensions of Collection on this existential, unless they used
>>>>>>>> associated types other than the element type.  But you couldn't pass the
>>>>>>>> existential to a generic function like
>>>>>>>> func scrambled<C: Collection>(_ c: C) -> [C.Element]
>>>>>>>>> IMO you should *have* to introduce unsafe behavior like that manually.
>>>>>>>> Collection where Element == Int & Index == *
>>>>>>>> ?
>>>>>>> I didn’t mean directly through the type of the existential.
>>>>>> My question is, why not?  That is still explicit.
>>>>> It’s not explicit in the sense that nobody wrote `fatalError` or
>>>>> similar in their code.  It’s too easy to write something like that
>>>>> without realizing that it introduces the possibility of a crash.  If
>>>>> we adopt syntax like that to introduce an existential that introduces
>>>>> traps we should at least require members that can be trap to be
>>>>> invoked using a `!` suffix or something like that to make it clear to
>>>>> users that a trap will happen if they are not extremely careful when
>>>>> using that member.
>>>>> More generally though, I don’t want the rules of the language to be
>>>>> written in a way that causes the compiler to synthesize traps in such
>>>>> a general way.
>>>>> The existential should not introduce a precondition that isn’t already
>>>>> present in the semantics of the protocol itself.  If the semantics of
>>>>> the protocol do not place preconditions on arguments beyond their type
>>>>> (such as “must be a valid index into this specific instance”) the
>>>>> compiler should not allow the existential to conform if a trap is
>>>>> required in some circumstances.  That is a new precondition and
>>>>> therefore the existential does not actually fulfill the requirements
>>>>> of the protocol.
>>>>> I could *maybe* live with a solution where protocol requirements are
>>>>> marked as trapping, etc depending on the specific argument received at
>>>>> runtime.  This is a total straw man syntax, but maybe `IndexableBase`
>>>>> would declare the subscript `@trapping` (probably something different
>>>>> but I hope this communicates the idea).  This alerts users to the fact
>>>>> that they need to be extra careful - not any value of `Self.Index` is
>>>>> valid and you can get a crash if you’re not careful.
>>>>> Having this semantic explicit in the definition of the protocol opens
>>>>> the door to maybe considering an existential synthesized by the
>>>>> compiler that traps because it doesn’t introduce a new precondition
>>>>> that wasn’t already present in the protocol.
>>>>> I would want to give consideration to specific details of a proposal
>>>>> along these lines before deciding how I feel about it, but I have a
>>>>> more open mind to this approach than introducing traps not present in
>>>>> the preconditions of the protocol.
>>>>> /// You can subscript a collection with any valid index other than the
>>>>> /// collection's end index. The end index refers to the position one past
>>>>> /// the last element of a collection, so it doesn't correspond with an
>>>>> /// element.
>>>>> ///
>>>>> /// - Parameter position: The position of the element to access. `position`
>>>>> ///   must be a valid index of the collection that is not equal to the
>>>>> ///   `endIndex` property.
>>>>> @trapping public subscript(position: Self.Index) -> Self._Element { get }
>>>>>>> One obvious mechanism for introducing unsafe behavior is to write
>>>>>>> manual type erasure wrappers like we do today.
>>>>>>> Another possibility would be to allow extending the existential type
>>>>>>> (not the protocol).  This would allow you to write overloads on the
>>>>>>> Collection existential that takes some kind of type erased index if
>>>>>>> that is what you want and either trap if you receive an invalid index
>>>>>>> or better (IMO) return an `Element?`.  I’m not sure how extensions on
>>>>>>> existentials might be implemented, but this is an example of the kind
>>>>>>> of operation you might want available on it that you wouldn’t want
>>>>>>> available on all Collection types.
>>>>>>>>> Collection indices are already something that isn’t fully statically
>>>>>>>>> safe so I understand why you might want to allow this.  
>>>>>>>> By the same measure, so are Ints :-)
>>>>>>>> The fact that a type's methods have preconditions does *not* make it
>>>>>>>> “statically unsafe.”
>>>>>>> That depends on what you mean by safe.  Sure, those methods aren’t
>>>>>>> going corrupt memory, but they *are* going to explicitly and
>>>>>>> intentionally crash for some inputs.  That doesn’t qualify as “fully
>>>>>>> safe” IMO.
>>>>>> Please pick a term other than “unsafe” here; it's not unsafe in the
>>>>>> sense we mean the word in Swift.  It's safe in exactly the same way that
>>>>>> array indexes and integers are.  When you violate a precondition, it
>>>>>> traps.
>>>>> I am happy to use any word you like here.
>>>>> Can you clarify what you mean by the word safe in Swift?  It doesn’t
>>>>> appear to be limited to memory safety in the public about page
>>>>> https://swift.org/about/ <https://swift.org/about/> <https://swift.org/about/ <https://swift.org/about/>>:
>>>> I mean memory- and type-safe.
>>>>> Safe. The most obvious way to write code should also behave in a safe
>>>>> manner. Undefined behavior is the enemy of safety, and developer
>>>>> mistakes should be caught before software is in production. Opting for
>>>>> safety sometimes means Swift will feel strict, but we believe that
>>>>> clarity saves time in the long run.
>>>>> Safety
>>>>> Swift was designed from the outset to be safer than C-based languages,
>>>>> and eliminates entire classes of unsafe code. Variables are always
>>>>> initialized before use, arrays and integers are checked for overflow,
>>>>> and memory is managed automatically. Syntax is tuned to make it easy
>>>>> to define your intent — for example, simple three-character keywords
>>>>> define a variable (var) or constant (let).
>>>>> Another safety feature is that by default Swift objects can never be
>>>>> nil, and trying to make or use a nil object will results in a
>>>>> compile-time error. This makes writing code much cleaner and safer,
>>>>> and prevents a common cause of runtime crashes. However, there are
>>>>> cases where nil is appropriate, and for these situations Swift has an
>>>>> innovative feature known as optionals. An optional may contain nil,
>>>>> but Swift syntax forces you to safely deal with it using ? to indicate
>>>>> to the compiler you understand the behavior and will handle it safely.
>>>>> This positioning statement makes it appear as if preventing common
>>>>> causes of crashes falls within the meaning of safe that Swift is
>>>>> using.  Having existentials introduce new preconditions and traps when
>>>>> they are not met does not seem aligned with that goal IMO.
>>>> Static typing “increases safety,” in the casual sense.  That doesn't
>>>> mean that an operation that traps on a failed precondition check is
>>>> “unsafe.”
>>>>>> The user doesn't do anything “manual” to introduce that trapping
>>>>>> behavior for integers.  Preconditions are a natural part of most types.
>>>>> The user doesn’t, but isn’t the overflow trap implemented in the
>>>>> standard library?  
>>>> Whether it is or is not is an implementation detail.
>>>>> Regardless, this is a specific case that has been given explicit
>>>>> design attention by humans.  The precondition is designed, not
>>>>> introduced by compiler rules that haven’t considered the specific case
>>>>> in question.
>>>>>>>>> But I don’t think having the language's existentials do this
>>>>>>>>> automatically is the right approach.  Maybe there is another
>>>>>>>>> approach that could be used in targeted use cases where the less
>>>>>>>>> safe behavior makes sense and is carefully designed.
>>>>>>>> Whether it makes sense or not really depends on the use-cases.  There's
>>>>>>>> little point in generalizing existentials if the result isn't very useful.
>>>>>>> Usefulness depends on your perspective.  
>>>>>> Of course.  As I've said, let's look at the use cases.
>>>>> Agree.  We can consider those in depth when the time comes to ramp up
>>>>> discussion of Austin’s proposal.
>>>>>>> I have run into several scenarios where they would be very useful
>>>>>>> without needing to be prone to crashes when used incorrectly.  One
>>>>>>> obvious basic use case is storing things in a heterogenous collection
>>>>>>> where you bind .
>>>>>> bind what?
>>>>> Sorry, I must have gotten distracted and not finished that paragraph.
>>>>> I meant to say bind the associated types that are necessary for your
>>>>> use case.  Sometimes you bind *all* of the associated types to
>>>>> concrete types and the protocol has no `Self` requirements.  In that
>>>>> case there is no trouble at all in conforming the type-erased
>>>>> “existential" to the protocol itself.  Austin’s proposal would
>>>>> eliminate the need to manually write these “existentials” manually.
>>>>>>>> The way to find out is to take a look at the examples we currently have
>>>>>>>> of protocols with associated types or Self requirements and consider
>>>>>>>> what you'd be able to do with their existentials if type relationships
>>>>>>>> couldn't be erased.  
>>>>>>>> We have known use-cases, currently emulated in the standard library, for
>>>>>>>> existentials with erased type relationships.  *If* these represent the
>>>>>>>> predominant use cases for something like generalized existentials, it
>>>>>>>> seems to me that the language feature should support that.  Note: I have
>>>>>>>> not seen anyone build an emulation of the other kind of generalized
>>>>>>>> existential.  My theory: there's a good reason for that :-).
>>>>>>> AFAIK (and I could be wrong) the only rules in the language that
>>>>>>> require the compiler to synthesize a trap except using a nil IUO, `!`
>>>>>>> on a nil Optional, and an invalid `as` cast .  These are all
>>>>>>> syntactically explicit unsafe / dangerous operations.  All other traps
>>>>>>> are in the standard library (array index, overflow, etc).  Most
>>>>>>> important about all of these cases is that they have received direct
>>>>>>> human consideration.
>>>>>> There is no distinction in the user model between what might be
>>>>>> synthesized by the language and what appears on standard library types.
>>>>> Maybe I shouldn’t have made that distinction.  
>>>>> The point I am trying to emphasize is that each of these are special
>>>>> cases that have received direct human consideration.  The potential
>>>>> for a trap is not introduced by language rules that apply to
>>>>> user-defined constructs in without consideration of the specific
>>>>> details of that construct.
>>>>>>> Introducing a language (not library) mechanism that exposes members on
>>>>>>> generalized existentials in a way that relies on runtime traps for
>>>>>>> type safety feels to me like a pretty dramatic turn agains the stated
>>>>>>> priority of safety.  It will mean you must understand exactly what is
>>>>>>> going on and be extremely careful to use generalized existentials
>>>>>>> without causing crashes.  This will either make Swift code much more
>>>>>>> crashy or will scare people away from using generalized existentials
>>>>>>> (and maybe both).  
>>>>>> I don't accept either of those statements without seeing some analysis
>>>>>> of the use-cases.  For example, I don't believe that AnyCollection et al
>>>>>> are particularly crash-prone.  The likelihood that you'll use the wrong
>>>>>> index type with a collection is very, very low.  I'm less certain of
>>>>>> what happens with Self requirements in real cases.
>>>>> But again, I believe this is an exceptional case as the precondition
>>>>> is explicitly stated in the semantics of the protocol.
>>>> IIUC, it has been cited by Doug as the exemplar of the
>>>> predominantly-requested case by a 10:1 ratio!
>>> In terms of forming the existential, storing it in variables,
>>> accepting arguments of that type, etc yes.  I don’t know how many of
>>> those requests expect it to conform to the protocol and expect to be
>>> able to use it in generic code constrained to the protocol.
>>>>> IMO the burden of proof should be on the side that proposes a
>>>>> mechanism to introduce traps, not the side that proposes avoiding
>>>>> them.
>>>> If you really want to make this about sides and burdens, the burden of
>>>> proof always rests with the side proposing to extend the language.  We
>>>> shouldn't be making changes without understanding how they will play out
>>>> in real use-cases.
>>> I agree with this.  But if we are discussing two different options for
>>> extending the language I think the option that doesn’t introduce
>>> crashes should be preferred without pretty compelling reasons to
>>> choose the option that can introduce crashes.
>>>>>>> Neither of those outcomes is good.
>>>>>>> Collection indices are a somewhat special case as there is already a
>>>>>>> strong precondition that people are familiar with because it would be
>>>>>>> too costly to performance and arguably too annoying to deal with an
>>>>>>> Optional result in every array lookup.  IMO that is why the library is
>>>>>>> able to get away with it in the current type erased AnyCollection.
>>>>>>> But this is not a good model for exposing any members on an
>>>>>>> existential that do not already have a strong precondition that causes
>>>>>>> a trap when violated.
>>>>>>> I think a big reason why you maybe haven’t seen a lot of examples of
>>>>>>> people writing type erased “existentials" is because it is a huge pain
>>>>>>> in the neck to write this stuff manually today.  People may be
>>>>>>> designing around the need for them.  I haven’t seen a huge sampling of
>>>>>>> type erased “existentials" other people are writing but I haven’t
>>>>>>> written any that introduce a trap like this.  The only traps are in
>>>>>>> the “abstract" base class whose methods will never be called (and
>>>>>>> wouldn’t even be implemented if they could be marked abstract).
>>>>>>> What specific things do you think we need to be able to do that rely
>>>>>>> on the compiler synthesizing a trap in the way it exposes the members
>>>>>>> of the existential?
>>>>>> I don't know.  I'm saying, I don't think we understand the use-cases
>>>>>> well enough to make a determination.
>>>>> That’s fair.  I agree that use cases should be carefully considered.  
>>>>>>> Here are a few examples from Austin’s proposal that safely use
>>>>>>> existential collections.  I don’t understand why you think this
>>>>>>> approach is insufficient.  Maybe you could supply a concrete example
>>>>>>> of a use case that can’t be written with the mechanism in Austin’s
>>>>>>> proposal.
>>>>>>> https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure <https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure> <https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure <https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure>> <https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure <https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure><https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure <https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure>>>
>>>>>>> let a : Any<Collection>
>>>>>>> // A variable whose type is the Index associated type of the underlying
>>>>>>> // concrete type of 'a'.
>>>>>>> let theIndex : a.Index = ...
>>>>>>> // A variable whose type is the Element associated type of the underlying
>>>>>>> // concrete type of 'a'.
>>>>>>> let theElement : a.Element = ...
>>>>>>> // Given a mutable collection, swap its first and last items.
>>>>>>> // Not a generic function. 
>>>>>>> func swapFirstAndLast(inout collection: Any<BidirectionalMutableCollection>) {
>>>>>>>  // firstIndex and lastIndex both have type "collection.Index"
>>>>>>>  guard let firstIndex = collection.startIndex,
>>>>>>>      lastIndex = collection.endIndex?.predecessor(collection) where lastIndex != firstIndex else {
>>>>>>>          print("Nothing to do")
>>>>>>>          return
>>>>>>>  }
>>>>>>>  // oldFirstItem has type "collection.Element"
>>>>>>>  let oldFirstItem = collection[firstIndex]
>>>>>>>  collection[firstIndex] = collection[lastIndex]
>>>>>>>  collection[lastIndex] = oldFirstItem
>>>>>>> }
>>>>>>> var a : Any<BidirectionalMutableCollection where .Element == String> = ...
>>>>>>> let input = "West Meoley"
>>>>>>> // Not actually necessary, since the compiler knows "a.Element" is String.
>>>>>>> // A fully constrained anonymous associated type is synonymous with the concrete
>>>>>>> // type it's forced to take on, and the two are interchangeable.
>>>>>>> // However, 'as' casting is still available if desired.
>>>>>>> let anonymousInput = input as a.Element
>>>>>>> a[a.startIndex] = anonymousInput
>>>>>>> // as mentioned, this also works:
>>>>>>> a[a.startIndex] = input
>>>>>>> // If the collection allows it, set the first element in the collection to a given string.
>>>>>>> func setFirstElementIn(inout collection: Any<Collection> toString string: String) {
>>>>>>>  if let element = string as? collection.Element {
>>>>>>>      // At this point, 'element' is of type "collection.Element"
>>>>>>>      collection[collection.startIndex] = element
>>>>>>>  }
>>>>>>> }
>>>>>> Neither of these look like they actually make *use* of the fact that
>>>>>> there's type erasure involved (and therefore should probably be written
>>>>>> as generics?).  The interesting cases with Any<Collection...>, for the
>>>>>> purposes of this discussion, arise when you have multiple instances of
>>>>>> the same existential type that wrap different concrete types.
>>>>> One use case I have found is to work around the lack of higher-kinder
>>>>> types.  
>>>> Really, now: a use-case for feature A that is a workaround for the lack
>>>> of feature B hardly justifies adding feature A!  We do want to add
>>>> higher-kinded types eventually.
>>> Good to know.  I thought higher-kinder types were on the “maybe if
>>> someone shows a compelling enough use case” list.  AFAIK this is the
>>> first time a member of the core team has stated the intent to add
>>> them.  
>> Well, please don't take this as a formal statement on behalf of the
>> team.  IIUC, the team is generally interested in having this feature.
> Of course not.  The only official plan of record for Swift is approved
> proposals.  But it’s always interesting to know the opinions of the
> core team about potential features.  Previously my impression had been
> that the general leaning was towards skepticism that the practical
> benefits of HKT would pay for the complexity.  

That's certainly a valid question.  I have been very vocally skeptical
that HKTs like Monad belong in the standard library, but I am not
opposed to having the language feature if it supports important use


More information about the swift-evolution mailing list