[swift-evolution] [swift-evolution-announce] [Review] SE-0089: Replace protocol<P1, P2> syntax with Any<P1, P2>
Matthew Johnson
matthew at anandabits.com
Thu Jun 9 12:26:36 CDT 2016
> On Jun 9, 2016, at 11:42 AM, Dave Abrahams <dabrahams at apple.com> wrote:
>
>
> on Thu Jun 09 2016, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:
>
>>> On Jun 9, 2016, at 9:55 AM, Dave Abrahams <dabrahams at apple.com <mailto:dabrahams at apple.com>> wrote:
>>>
>>>
>>> on Wed Jun 08 2016, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/> <http://matthew-at-anandabits.com/ <http://matthew-at-anandabits.com/>>> wrote:
>>>
>>
>>>>> On Jun 8, 2016, at 1:33 PM, Dave Abrahams <dabrahams at apple.com <mailto:dabrahams at apple.com>> wrote:
>>>>>
>>>>>
>>>>> on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/>> wrote:
>>>>>
>>>>
>>>>>>> On Jun 7, 2016, at 9:15 PM, Dave Abrahams <dabrahams at apple.com <mailto:dabrahams at apple.com>> wrote:
>>>>>>>
>>>>>>>
>>>>>>> on Tue Jun 07 2016, Matthew Johnson <matthew-AT-anandabits.com <http://matthew-at-anandabits.com/> <http://matthew-at-anandabits.com/ <http://matthew-at-anandabits.com/>>> wrote:
>>>>>>>
>>>>>>
>>>>>>>>> On Jun 7, 2016, at 4:13 PM, Dave Abrahams via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> on Tue Jun 07 2016, Matthew Johnson <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> , but haven't realized
>>>>>>>>>>> that if you step around the type relationships encoded in Self
>>>>>>>>>>> requirements and associated types you end up with types that appear to
>>>>>>>>>>> interoperate but in fact trap at runtime unless used in exactly the
>>>>>>>>>>> right way.
>>>>>>>>>>
>>>>>>>>>> Trap at runtime? How so? Generalized existentials should still be
>>>>>>>>>> type-safe.
>>>>>>>>>
>>>>>>>>> There are two choices when you erase static type relationships:
>>>>>>>>>
>>>>>>>>> 1. Acheive type-safety by trapping at runtime
>>>>>>>>>
>>>>>>>>> FloatingPoint(3.0 as Float) + FloatingPoint(3.0 as Double) // trap
>>>>>>>>>
>>>>>>>>> 2. Don't expose protocol requirements that involve these relationships,
>>>>>>>>> which would prevent the code above from compiling and prevent
>>>>>>>>> FloatingPoint from conforming to itself.
>>>>>>>>>
>>>>>>>>>> Or are you talking about the hypothetical types / behaviors people
>>>>>>>>>> think they want when they don’t fully understand what is happening...
>>>>>>>>>
>>>>>>>>> I don't know what you mean here. I think generalized existentials will
>>>>>>>>> be nice to have, but I think most people will want them to do something
>>>>>>>>> they can't possibly do.
>>>>>>>>
>>>>>>>> Exactly. What I meant is that people think they want that expression
>>>>>>>> to compile because they don’t understand that the only thing it can do
>>>>>>>> is trap. I said “hypothetical” because producing a compile time error
>>>>>>>> rather than a runtime trap is the only sane thing to do. Your comment
>>>>>>>> surprised me because I can’t imagine we would move forward in Swift
>>>>>>>> with the approach of trapping.
>>>>>>>
>>>>>>> I would very much like to be able to create instances of “Collection
>>>>>>> where Element == Int” so we can throw away the wrappers in the stdlib.
>>>>>>> That will require some type mismatches to be caught at runtime via
>>>>>>> trapping.
>>>>>>
>>>>>> For invalid index because the existential accepts a type erased index?
>>>>>
>>>>> Exactly.
>>>>>
>>>>>> How do you decide where to draw the line here? It feels like a very
>>>>>> slippery slope for a language where safety is a stated priority to
>>>>>> start adopting a strategy of runtime trapping for something as
>>>>>> fundamental as how you expose members on an existential.
>>>>>
>>>>> If you don't do this, the alternative is that “Collection where Element
>>>>> == Int” does not conform to Collection.
>>>>
>>>> This isn’t directly related to having self or associated type
>>>> requirements. It is true of all existentials.
>>>
>>> That is just an implementation limitation today, IIUC. What I'm talking
>>> about here would make it impossible for some to do that.
>>
>> If it is just an implementation limitation I am happy to hear that.
>>
>>>
>>>> If that changes for simple existentials and generalized existentials
>>>> expose all members (as in the latest draft of the proposal) maybe it
>>>> will be possible for all existentials to conform to their protocol.
>>>
>>> Not without introducing runtime traps. See my “subscript function”
>>> example.
>>
>>>
>>>>
>>>>> That's weird and not very
>>>>> useful. You could expose all the methods that were on protocol
>>>>> extensions of Collection on this existential, unless they used
>>>>> associated types other than the element type. But you couldn't pass the
>>>>> existential to a generic function like
>>>>>
>>>>> func scrambled<C: Collection>(_ c: C) -> [C.Element]
>>>>>
>>>>>> IMO you should *have* to introduce unsafe behavior like that manually.
>>>>>
>>>>> Collection where Element == Int & Index == *
>>>>>
>>>>> ?
>>>>
>>>> I didn’t mean directly through the type of the existential.
>>>
>>> My question is, why not? That is still explicit.
>>
>> It’s not explicit in the sense that nobody wrote `fatalError` or
>> similar in their code. It’s too easy to write something like that
>> without realizing that it introduces the possibility of a crash. If
>> we adopt syntax like that to introduce an existential that introduces
>> traps we should at least require members that can be trap to be
>> invoked using a `!` suffix or something like that to make it clear to
>> users that a trap will happen if they are not extremely careful when
>> using that member.
>>
>> More generally though, I don’t want the rules of the language to be
>> written in a way that causes the compiler to synthesize traps in such
>> a general way.
>>
>> The existential should not introduce a precondition that isn’t already
>> present in the semantics of the protocol itself. If the semantics of
>> the protocol do not place preconditions on arguments beyond their type
>> (such as “must be a valid index into this specific instance”) the
>> compiler should not allow the existential to conform if a trap is
>> required in some circumstances. That is a new precondition and
>> therefore the existential does not actually fulfill the requirements
>> of the protocol.
>>
>> I could *maybe* live with a solution where protocol requirements are
>> marked as trapping, etc depending on the specific argument received at
>> runtime. This is a total straw man syntax, but maybe `IndexableBase`
>> would declare the subscript `@trapping` (probably something different
>> but I hope this communicates the idea). This alerts users to the fact
>> that they need to be extra careful - not any value of `Self.Index` is
>> valid and you can get a crash if you’re not careful.
>>
>> Having this semantic explicit in the definition of the protocol opens
>> the door to maybe considering an existential synthesized by the
>> compiler that traps because it doesn’t introduce a new precondition
>> that wasn’t already present in the protocol.
>>
>> I would want to give consideration to specific details of a proposal
>> along these lines before deciding how I feel about it, but I have a
>> more open mind to this approach than introducing traps not present in
>> the preconditions of the protocol.
>>
>> /// You can subscript a collection with any valid index other than the
>> /// collection's end index. The end index refers to the position one past
>> /// the last element of a collection, so it doesn't correspond with an
>> /// element.
>> ///
>> /// - Parameter position: The position of the element to access. `position`
>> /// must be a valid index of the collection that is not equal to the
>> /// `endIndex` property.
>> @trapping public subscript(position: Self.Index) -> Self._Element { get }
>>
>>>
>>>> One obvious mechanism for introducing unsafe behavior is to write
>>>> manual type erasure wrappers like we do today.
>>>>
>>>> Another possibility would be to allow extending the existential type
>>>> (not the protocol). This would allow you to write overloads on the
>>>> Collection existential that takes some kind of type erased index if
>>>> that is what you want and either trap if you receive an invalid index
>>>> or better (IMO) return an `Element?`. I’m not sure how extensions on
>>>> existentials might be implemented, but this is an example of the kind
>>>> of operation you might want available on it that you wouldn’t want
>>>> available on all Collection types.
>>>>
>>>>>
>>>>>> Collection indices are already something that isn’t fully statically
>>>>>> safe so I understand why you might want to allow this.
>>>>>
>>>>> By the same measure, so are Ints :-)
>>>>>
>>>>> The fact that a type's methods have preconditions does *not* make it
>>>>> “statically unsafe.”
>>>>
>>>> That depends on what you mean by safe. Sure, those methods aren’t
>>>> going corrupt memory, but they *are* going to explicitly and
>>>> intentionally crash for some inputs. That doesn’t qualify as “fully
>>>> safe” IMO.
>>>
>>> Please pick a term other than “unsafe” here; it's not unsafe in the
>>> sense we mean the word in Swift. It's safe in exactly the same way that
>>> array indexes and integers are. When you violate a precondition, it
>>> traps.
>>
>> I am happy to use any word you like here.
>>
>> Can you clarify what you mean by the word safe in Swift? It doesn’t
>> appear to be limited to memory safety in the public about page
>> https://swift.org/about/ <https://swift.org/about/> <https://swift.org/about/ <https://swift.org/about/>>:
>
> I mean memory- and type-safe.
>
>> Safe. The most obvious way to write code should also behave in a safe
>> manner. Undefined behavior is the enemy of safety, and developer
>> mistakes should be caught before software is in production. Opting for
>> safety sometimes means Swift will feel strict, but we believe that
>> clarity saves time in the long run.
>>
>> Safety
>>
>> Swift was designed from the outset to be safer than C-based languages,
>> and eliminates entire classes of unsafe code. Variables are always
>> initialized before use, arrays and integers are checked for overflow,
>> and memory is managed automatically. Syntax is tuned to make it easy
>> to define your intent — for example, simple three-character keywords
>> define a variable (var) or constant (let).
>>
>> Another safety feature is that by default Swift objects can never be
>> nil, and trying to make or use a nil object will results in a
>> compile-time error. This makes writing code much cleaner and safer,
>> and prevents a common cause of runtime crashes. However, there are
>> cases where nil is appropriate, and for these situations Swift has an
>> innovative feature known as optionals. An optional may contain nil,
>> but Swift syntax forces you to safely deal with it using ? to indicate
>> to the compiler you understand the behavior and will handle it safely.
>>
>> This positioning statement makes it appear as if preventing common
>> causes of crashes falls within the meaning of safe that Swift is
>> using. Having existentials introduce new preconditions and traps when
>> they are not met does not seem aligned with that goal IMO.
>
> Static typing “increases safety,” in the casual sense. That doesn't
> mean that an operation that traps on a failed precondition check is
> “unsafe.”
>
>>> The user doesn't do anything “manual” to introduce that trapping
>>> behavior for integers. Preconditions are a natural part of most types.
>>
>> The user doesn’t, but isn’t the overflow trap implemented in the
>> standard library?
>
> Whether it is or is not is an implementation detail.
>
>> Regardless, this is a specific case that has been given explicit
>> design attention by humans. The precondition is designed, not
>> introduced by compiler rules that haven’t considered the specific case
>> in question.
>>
>>>
>>>>>> But I don’t think having the language's existentials do this
>>>>>> automatically is the right approach. Maybe there is another
>>>>>> approach that could be used in targeted use cases where the less
>>>>>> safe behavior makes sense and is carefully designed.
>>>>>
>>>>> Whether it makes sense or not really depends on the use-cases. There's
>>>>> little point in generalizing existentials if the result isn't very useful.
>>>>
>>>> Usefulness depends on your perspective.
>>>
>>> Of course. As I've said, let's look at the use cases.
>>
>> Agree. We can consider those in depth when the time comes to ramp up
>> discussion of Austin’s proposal.
>>
>>>
>>>> I have run into several scenarios where they would be very useful
>>>> without needing to be prone to crashes when used incorrectly. One
>>>> obvious basic use case is storing things in a heterogenous collection
>>>> where you bind .
>>>
>>> bind what?
>>
>> Sorry, I must have gotten distracted and not finished that paragraph.
>> I meant to say bind the associated types that are necessary for your
>> use case. Sometimes you bind *all* of the associated types to
>> concrete types and the protocol has no `Self` requirements. In that
>> case there is no trouble at all in conforming the type-erased
>> “existential" to the protocol itself. Austin’s proposal would
>> eliminate the need to manually write these “existentials” manually.
>>
>>>>
>>>>> The way to find out is to take a look at the examples we currently have
>>>>> of protocols with associated types or Self requirements and consider
>>>>> what you'd be able to do with their existentials if type relationships
>>>>> couldn't be erased.
>>>>>
>>>>> We have known use-cases, currently emulated in the standard library, for
>>>>> existentials with erased type relationships. *If* these represent the
>>>>> predominant use cases for something like generalized existentials, it
>>>>> seems to me that the language feature should support that. Note: I have
>>>>> not seen anyone build an emulation of the other kind of generalized
>>>>> existential. My theory: there's a good reason for that :-).
>>>>
>>>> AFAIK (and I could be wrong) the only rules in the language that
>>>> require the compiler to synthesize a trap except using a nil IUO, `!`
>>>> on a nil Optional, and an invalid `as` cast . These are all
>>>> syntactically explicit unsafe / dangerous operations. All other traps
>>>> are in the standard library (array index, overflow, etc). Most
>>>> important about all of these cases is that they have received direct
>>>> human consideration.
>>>
>>> There is no distinction in the user model between what might be
>>> synthesized by the language and what appears on standard library types.
>>
>> Maybe I shouldn’t have made that distinction.
>>
>> The point I am trying to emphasize is that each of these are special
>> cases that have received direct human consideration. The potential
>> for a trap is not introduced by language rules that apply to
>> user-defined constructs in without consideration of the specific
>> details of that construct.
>>
>>>
>>>> Introducing a language (not library) mechanism that exposes members on
>>>> generalized existentials in a way that relies on runtime traps for
>>>> type safety feels to me like a pretty dramatic turn agains the stated
>>>> priority of safety. It will mean you must understand exactly what is
>>>> going on and be extremely careful to use generalized existentials
>>>> without causing crashes. This will either make Swift code much more
>>>> crashy or will scare people away from using generalized existentials
>>>> (and maybe both).
>>>
>>> I don't accept either of those statements without seeing some analysis
>>> of the use-cases. For example, I don't believe that AnyCollection et al
>>> are particularly crash-prone. The likelihood that you'll use the wrong
>>> index type with a collection is very, very low. I'm less certain of
>>> what happens with Self requirements in real cases.
>>
>> But again, I believe this is an exceptional case as the precondition
>> is explicitly stated in the semantics of the protocol.
>
> IIUC, it has been cited by Doug as the exemplar of the
> predominantly-requested case by a 10:1 ratio!
In terms of forming the existential, storing it in variables, accepting arguments of that type, etc yes. I don’t know how many of those requests expect it to conform to the protocol and expect to be able to use it in generic code constrained to the protocol.
>
>> IMO the burden of proof should be on the side that proposes a
>> mechanism to introduce traps, not the side that proposes avoiding
>> them.
>
> If you really want to make this about sides and burdens, the burden of
> proof always rests with the side proposing to extend the language. We
> shouldn't be making changes without understanding how they will play out
> in real use-cases.
I agree with this. But if we are discussing two different options for extending the language I think the option that doesn’t introduce crashes should be preferred without pretty compelling reasons to choose the option that can introduce crashes.
>
>>>> Neither of those outcomes is good.
>>>>
>>>> Collection indices are a somewhat special case as there is already a
>>>> strong precondition that people are familiar with because it would be
>>>> too costly to performance and arguably too annoying to deal with an
>>>> Optional result in every array lookup. IMO that is why the library is
>>>> able to get away with it in the current type erased AnyCollection.
>>>> But this is not a good model for exposing any members on an
>>>> existential that do not already have a strong precondition that causes
>>>> a trap when violated.
>>>>
>>>> I think a big reason why you maybe haven’t seen a lot of examples of
>>>> people writing type erased “existentials" is because it is a huge pain
>>>> in the neck to write this stuff manually today. People may be
>>>> designing around the need for them. I haven’t seen a huge sampling of
>>>> type erased “existentials" other people are writing but I haven’t
>>>> written any that introduce a trap like this. The only traps are in
>>>> the “abstract" base class whose methods will never be called (and
>>>> wouldn’t even be implemented if they could be marked abstract).
>>>>
>>>> What specific things do you think we need to be able to do that rely
>>>> on the compiler synthesizing a trap in the way it exposes the members
>>>> of the existential?
>>>
>>> I don't know. I'm saying, I don't think we understand the use-cases
>>> well enough to make a determination.
>>
>> That’s fair. I agree that use cases should be carefully considered.
>>
>>>
>>>> Here are a few examples from Austin’s proposal that safely use
>>>> existential collections. I don’t understand why you think this
>>>> approach is insufficient. Maybe you could supply a concrete example
>>>> of a use case that can’t be written with the mechanism in Austin’s
>>>> proposal.
>>>>
>>>> https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure <https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure> <https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure <https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure>> <https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure <https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure><https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure <https://github.com/austinzheng/swift-evolution/blob/az-existentials/proposals/XXXX-enhanced-existentials.md#associated-types-and-member-exposure>>>
>>>>
>>>> let a : Any<Collection>
>>>>
>>>> // A variable whose type is the Index associated type of the underlying
>>>> // concrete type of 'a'.
>>>> let theIndex : a.Index = ...
>>>>
>>>> // A variable whose type is the Element associated type of the underlying
>>>> // concrete type of 'a'.
>>>> let theElement : a.Element = ...
>>>>
>>>> // Given a mutable collection, swap its first and last items.
>>>> // Not a generic function.
>>>> func swapFirstAndLast(inout collection: Any<BidirectionalMutableCollection>) {
>>>> // firstIndex and lastIndex both have type "collection.Index"
>>>> guard let firstIndex = collection.startIndex,
>>>> lastIndex = collection.endIndex?.predecessor(collection) where lastIndex != firstIndex else {
>>>> print("Nothing to do")
>>>> return
>>>> }
>>>>
>>>> // oldFirstItem has type "collection.Element"
>>>> let oldFirstItem = collection[firstIndex]
>>>>
>>>> collection[firstIndex] = collection[lastIndex]
>>>> collection[lastIndex] = oldFirstItem
>>>> }
>>>>
>>>> var a : Any<BidirectionalMutableCollection where .Element == String> = ...
>>>>
>>>> let input = "West Meoley"
>>>>
>>>> // Not actually necessary, since the compiler knows "a.Element" is String.
>>>> // A fully constrained anonymous associated type is synonymous with the concrete
>>>> // type it's forced to take on, and the two are interchangeable.
>>>> // However, 'as' casting is still available if desired.
>>>> let anonymousInput = input as a.Element
>>>>
>>>> a[a.startIndex] = anonymousInput
>>>>
>>>> // as mentioned, this also works:
>>>> a[a.startIndex] = input
>>>>
>>>> // If the collection allows it, set the first element in the collection to a given string.
>>>> func setFirstElementIn(inout collection: Any<Collection> toString string: String) {
>>>> if let element = string as? collection.Element {
>>>> // At this point, 'element' is of type "collection.Element"
>>>> collection[collection.startIndex] = element
>>>> }
>>>> }
>>>
>>> Neither of these look like they actually make *use* of the fact that
>>> there's type erasure involved (and therefore should probably be written
>>> as generics?). The interesting cases with Any<Collection...>, for the
>>> purposes of this discussion, arise when you have multiple instances of
>>> the same existential type that wrap different concrete types.
>>
>> One use case I have found is to work around the lack of higher-kinder
>> types.
>
> Really, now: a use-case for feature A that is a workaround for the lack
> of feature B hardly justifies adding feature A! We do want to add
> higher-kinded types eventually.
Good to know. I thought higher-kinder types were on the “maybe if someone shows a compelling enough use case” list. AFAIK this is the first time a member of the core team has stated the intent to add them. If that is the case I agree that this use case isn’t relevant. The workaround isn’t great because it loses type information that is critical to the optimizer (but it’s all we have available today).
>
>> If you have a protocol where specific implementations will return
>> different types, but all conform to a second protocol you can define
>> the protocol in terms of a generic type-erased wrapper which conforms
>> to the second protocol and accepts type arguments that match the
>> associated types (thus binding the associated types to concrete
>> types). I have found this to be a useful technique (granted it is a
>> workaround and I’m not sure how useful it would continue to be if
>> Swift eventually gets higher-kinder types).
>>
>>>
>>> Another problem I see: in this new world, what is the model for choosing
>>> whether to write a function as a protocol extension/generic, or as a
>>> regular function taking existential parameters? Given that either of
>>> the above could have been written either way, we need to be able to
>>> answer that question. When existentials don't conform to their
>>> protocols, it seems to me that the most general thing to do is use
>>> existentials whenever you can, and only resort to using generics when
>>> forced by the type system. This does not seem like a particularly good
>>> programming model to me, but I might be convinced otherwise.
>>
>> That doesn’t seem like a particularly good programming model to me either.
>>
>> The rule of thumb I am operating with for protocols with Self or
>> associated type requirements is to prefer generics and use type
>> erasure / existentials when that isn’t possible. For example, when
>> heterogeneity is required or when you can’t form the necessary type in
>> a protocol requirement (as in the preceding example).
>>
>> This heuristic has been working out pretty well for me thus far.
>
> I do worry a bit that people will choose the opposite heuristic.
>
> It would be somewhat reassuring to me if we could prove to ourselves
> that, using your heuristic, one is never forced to copy/paste a generic
> function implementation into a corresponding function that uses
> existentials.
>
>> The primary impact of introducing a language mechanism for generalized
>> existentials in my code would be to eliminate a lot of manual type
>> erasing boilerplate.
>
> If your code has many manual type erasing wrappers corresponding to
> protocols with associated types and/or Self requirements that also never
> have to trap type mismatches, that would certainly be instructive
> empirical data. Would you care to share the protocols and wrappers you
> are talking about?
I put together a sample implementation of a Cocoa-like responder chain in Swift a while ago when the “Swift dynamism” debate was raging.
It isn't intended to be a Swifty design. It is intended to be similar to Cocoa and show techniques that can be used to do things similar to Cocoa’s responder chain and targer-action in Swift. It uses a type erased wrapper for actions that binds `Sender` while hiding the concrete `Action` type and also the `Handler` associated type. It cannot and should not conform to the protocol it is derived from and could be replaced with the generalized existentials in Austin’s proposal.
https://gist.github.com/anandabits/ec26f67f682093cf18b170c21bcf433e <https://gist.github.com/anandabits/ec26f67f682093cf18b170c21bcf433e>
This is a good example to start with because it is related to a topic that has been hotly debated and is clearly something a lot of people want to be able to do.
>
>>> Anyway, my overall point is that this all seems like something we *can*
>>> do and that nicely fills gaps in the type system, but not necessarily
>>> something we *should* do until we better understand what it's actually
>>> *for* and how it affects the programming model.
>>
>> That’s a very fair position to take. :)
>>
>>>
>>> --
>>> Dave
>>
>
> --
> Dave
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20160609/ea04db9c/attachment-0001.html>
More information about the swift-evolution
mailing list