[swift-evolution] [Idea] How to eliminate 'optional' protocol requirements

[Dietmar Planitzer]

> On Apr 10, 2016, at 11:46, Dave Abrahams via swift-evolution <swift-evolution at swift.org> wrote:
> 
> 
> on Sun Apr 10 2016, Dietmar Planitzer <swift-evolution at swift.org> wrote:
> 
>> I’m not sure whether you’ve read the conclusion of my mail since
>> you’ve only commented on the introductory part. In the conclusion I
>> wrote that a possible approach for the replacement of ObjC-style
>> optional protocol methods would be:
>> 
>> 1) the default implementation of a protocol method must be defined in
>> the protocol (so just like in native Swift protocols today).
> 
> ? They can and must be defined in protocol extensions today.

I know.

>> 
>> 2) we add a way for a protocol provider to check whether the protocol
>> adopter has provided an “override” of the default method.
> 
> I object to this part.

You object why? I do understand why you object to the ObjC model since there is not necessarily an implementation of the protocol method and thus the protocol provider has to guard every call with an existence check. But in this model here we would be guaranteed that there would be an implementation of the protocol method and thus guarding the call wouldn’t be necessary.

>> 3) we improve the Xcode interface generator so that it clearly shows
>> whether a protocol method comes with a default or whether it doesn’t.
> 
> Obvious goodness, long overdue.
> 
>> (1) should address your main concern since it would guarantee that the
>> protocol provider is always able to call the protocol method without
>> having to do any checks. (2) would address the main concern of
>> protocol providers who need to guarantee that the protocol using type
>> achieves a certain minimum speed and does not use more than a certain
>> amount of memory for its internal book-keeping.
> 
> I don't how (2) can possibly help with that.

It helps because it allows the protocol provider to *understand* whether the protocol adopter is actually using a certain feature or isn’t. Here is the table view example again:

func useDelegate(delegate: NSTableViewDelegate) {

  if has_override(delegate, tableView(_:, heightForRow:)) {
     // call tableViewNumberOfRows() on the delegate
     // allocate the geometry cache (1 entry per row)
     // call tableView(_:, heightForRow:) for each row
  } else {
    // nothing to do here since here all rows have the same height
   }
}


Note that has_override() is just a placeholder syntax because I’ve not had a good idea yet of how to express this in a Swiftier way.

In this example the table view is able to check whether the protocol adopter has actually “overriden” the default implementation of tableView(_:, heightForRow:). If the adopter did, then the table view knows that the adopter wants variable row heights and thus the table view can now create a cache of row heights and it can enable the layouting code that knows how to lay out rows with different heights. If however the adopter did not provide its own implementation of this method then the table view does not need to create a geometry cache and it can switch over to the simpler fixed-row-height layout code. The reason why we want to cache the row heights in the table view is because computing those heights can be nontrivial and the layout code needs to access those height values in every layoutSubviews() call. And layoutSubviews() is invoked 60 times per second while the user is scrolling. Also keep in mind that, if we would not cache the row heights, then the row height computation would end up competing for CPU cycles with the code that properly configures the views for each row.

Without the ability to do this check on the protocol provider side, we are forced to increase the API surface so that the protocol adopter can explicitly tell us which layouting model he wants. But this also means that the protocol adopter now has to remember that he needs to configure the layouting option correctly in order to get a working and efficiently working table view. So the end result would be a table view that’s hard to use correctly.


Regards,

Dietmar Planitzer

>> 
>> (3) is important because it would fix one of the many aspects that
>> make Swift protocols confusing for people who are new to the language.
>> 
>> Finally, let me restate that the goal should really be that we
>> completely remove the syntactical differences between @objc and native
>> Swift protocols. There should be one concept of protocol in Swift and
>> we should be able to cover the use cases of formal and informal ObjC
>> Protocols with them. The use case of formal protocols is already
>> covered today. The use case of informal protocols could be covered
>> with the approach above.
>> 
>> So is this an approach that would be acceptable to you?
>> 
>> Regards,
>> 
>> Dietmar Planitzer
>> 
>>> 
>>> On Apr 10, 2016, at 10:29, Dave Abrahams via swift-evolution <swift-evolution at swift.org> wrote:
>>> 
>>> 
>>> on Fri Apr 08 2016, Dietmar Planitzer <swift-evolution at swift.org> wrote:
>>> 
>>>> The biggest missing part with this model is that we are still not able
>>>> to enable macro-level optimizations in the delegating type by checking
>>>> whether the delegate does provide his own implementation of an
>>>> optional method or doesn’t. However, this is an important advantage of
>>>> the ObjC model that we should not lose.
>>>> 
>>>> Maybe it’s time to take a big step back and ignore the question of how
>>>> to implement things for a moment and to instead focus on the question
>>>> of what the conceptual differences are between ObjC protocols with
>>>> optional methods and Swift protocols with default
>>>> implementations. There are two relevant viewpoints here:
>>>> 
>>>> 1) From the viewpoint of a protocol adaptor:
>>>> 
>>>> ObjC:
>>>> 
>>>> 1a) adopter may provide his own implementation of the protocol method,
>>>> but he is no required to.
>>>> 
>>>> 1b) adopter can see in the protocol declaration for which methods he
>>>> must provide an implementation. Those methods do not have the
>>>> “optional” keyword in front of them while optional methods do.
>>>> 
>>>> Swift:
>>>> 
>>>> 1c) same as (1a).
>>>> 
>>>> 1d) opening a binary-only Swift file in Xcode with a protocol
>>>> definition in it which contains methods with default implementations
>>>> will not give any indication of which method has a default
>>>> implementation and which doesn’t. It’s only possible to see a
>>>> difference on the syntax level if you have access to the sources.
>>> 
>>> This visibility problem is something we aim to correct in Swift, but
>>> that is a question of syntax, documentation, and “header” generation,
>>> and really orthogonal to what's fundamental about “optional
>>> requirements:” 
>>> 
>>> 1. The ability to “conform” to the protocol without a
>>>  default implementation of the requirement have been provided
>>>  anywhere.
>>> 
>>> 2. The ability to dynamically query whether a type actually provides the
>>>  requirement.
>>> 
>>> Both of these “features,” IMO, are actually bugs.
>>> 
>>>> So from the viewpoint of the protocol adopter, there isn’t much of a
>>>> difference. The only relevant difference is that its always possible
>>>> in ObjC to tell whether a protocol method must be implemented by the
>>>> adopter or whether a method already has a default behavior. We
>>>> shouldn’t actually have to change anything on the syntax-level in
>>>> Swift to fix this problem. It should be sufficient to improve the
>>>> Swift interface generator in Xcode so that it gives an indication
>>>> whether a protocol method has a default implementation or doesn’t. Eg
>>>> if we want to ensure that the generated interface is valid syntax then
>>>> we could do this:
>>>> 
>>>> protocol Foo {
>>>> 
>>>>  func void bar() -> Int /* has default */
>>>> 
>>>> }
>>>> 
>>>> or if we say that it is fine that the generated interface is not valid
>>>> syntax (I think it already shows "= default” for function arguments
>>>> with a default value which I don’t think is valid syntax), then we
>>>> could do this:
>>>> 
>>>> protocol Foo {
>>>> 
>>>>  func void bar() -> Int {…}
>>>> 
>>>> }
>>>> 
>>>> Now on to the other side of the equation.
>>>> 
>>>> 2) From the viewpoint of the protocol provider (the person who defines
>>>> the protocol and the type that will invoke the protocol methods):
>>>> 
>>>> ObjC:
>>>> 
>>>> 2a) provider has freedom in deciding where to put the default
>>>> implementation and he can put the default implementation in a single
>>>> place or spread it out if necessary over multiple places. So has the
>>>> freedom to choose whatever makes the most sense for the problem at
>>>> hand.
>>> 
>>> But freedom for protocol implementors reduces predictability for protocol
>>> clients and adopters.
>>> 
>>>> 2b) provider can detect whether the adopter provides his own protocol
>>>> method implementation without compromising the definition of the
>>>> protocol (compromising here means making return values optional when
>>>> they should not be optional based on the natural definition of the
>>>> API). This enables the provider to implement macro-level optimizations
>>>> (eg table view can understand whether fixed or variable row heights
>>>> are desired).
>>>> 
>>>> Swift:
>>>> 
>>>> 2c) provider is forced to put the default implementation in a specific
>>>> place.
>>>> 
>>>> 2d) provider has no way to detect whether the adopter has provided his
>>>> own implementation of the protocol method.
>>>> 
>>>> I do think that (2a) would be nice to have but we can probably do
>>>> without it if it helps us to make progress with this topic. However,
>>>> the ability to detect whether a protocol adopter provides his own
>>>> implementation of a protocol method which comes with a default is a
>>>> useful and important feature which helps us in optimizing the
>>>> implementation of types and which allows us to keep the API surface
>>>> smaller than it would be without this ability. Just go and compare eg
>>>> UITableView to the Android ListView / RecyclerView to see the
>>>> consequences of not having that ability and how it inflates the API
>>>> surface (and keep in mind that the Android equivalents provide a
>>>> fraction of the UITableView functionality).
>>>> 
>>>> The important point about (2b) is actually that we are able to detect
>>>> whether an “override” (I’ll just call this overriding for now) of the
>>>> default implementation exists or does not exist. 
>>> 
>>> IMO the important point about (2b) is that it leads to protocol designs
>>> that create work and complexity for clients of the protocol, and being
>>> constrained to make your protocol work so that clients don't have to do
>>> these kinds of checks is a Very Good Thing™.
>>> 
>>>> In ObjC we make this distinction by checking whether an implementation
>>>> of the method exists at all. But we don’t have to do it that way. An
>>>> alternative approach could be based on a check that sees whether the
>>>> dispatch table of the delegate contains a pointer to the default
>>>> implementation of the protocol method or to some other method. So
>>>> conceptually what we want is an operation like this:
>>>> 
>>>> func void useDelegate(delegate: NSTableViewDelegate) {
>>>> 
>>>>  if has_override(delegate, tableView(_:, heightOfRow:)) { // ask the
>>>> delegate how many rows it has // allocate the geometry cache // fill
>>>> in the geometry cache by calling tableView(_:, heightForRow:) for each
>>>> row } else { // nothing to do here } }
>>>> 
>>>> Which would get the job done but doesn’t look good. Maybe someone has
>>>> a better idea of how the syntax such an operator could look.
>>>> 
>>>> So my point here is that what we care about is the ability to detect
>>>> whether the adopter provides an implementation of a protocol method
>>>> which comes with a default implementation. The point is not that Swift
>>>> protocols should work the exact same way that ObjC protocols have been
>>>> working under the hood. But I do think that we want to eventually get
>>>> to a point where the @objc attribute disappears and that we get a
>>>> truly unified language on the syntactical level. An approach where:
>>>> 
>>>> I) we accept that the default behavior of a protocol method has to be
>>>> provided by the protocol itself
>>>> 
>>>> II) the language is extended with a mechanism that makes it possible
>>>> for a protocol provider to detect whether the adopter has “overridden”
>>>> the default implementation
>>>> 
>>>> III) we improve the Xcode Swift interface generator so that it gives a
>>>> clear indication whether a protocol method does come with a default
>>>> implementation
>>>> 
>>>> would give us all the relevant advantages of ObjC-style optional
>>>> protocol methods and it should allow us to create a unified syntax
>>>> where there is no longer a visible difference between an optional
>>>> protocol method that was imported from ObjC and a native Swift
>>>> protocol with default implementations.
>>>> 
>>>> Regards,
>>>> 
>>>> Dietmar Planitzer
>>>> 
>>>>> On Apr 7, 2016, at 17:12, Douglas Gregor via swift-evolution
>>>>> <swift-evolution at swift.org> wrote:
>>>>> 
>>>>> Hi all,
>>>>> 
>>>>> Optional protocol requirements in Swift have the restriction that
>>>>> they only work in @objc protocols, a topic that’s come up a number
>>>>> of times. The start of these threads imply that optional
>>>>> requirements should be available for all protocols in Swift. While
>>>>> this direction is implementable, each time this is discussed there
>>>>> is significant feedback that optional requirements are not a feature
>>>>> we want in Swift. They overlap almost completely with default
>>>>> implementations of protocol requirements, which is a more general
>>>>> feature, and people seem to feel that designs based around default
>>>>> implementations and refactoring of protocol hierarchies are overall
>>>>> better.
>>>>> The main concern with removing optional requirements from Swift is their impact on Cocoa: Objective-C protocols, especially for delegates and data sources, make heavy use of optional requirements. Moreover, there are no default implementations for any of these optional requirements: each caller effectively checks for the presence of the method explicitly, and implements its own logic if the method isn’t there.
>>>>> 
>>>>> A Non-Workable Solution: Import as optional property requirements One suggestion that’s come up to map an optional requirement to a property with optional type, were “nil” indicates that the requirement was not satisfied. For example,
>>>>> 
>>>>> @protocol NSTableViewDelegate @optional - (nullable NSView *)tableView:(NSTableView *)tableView viewForTableColumn:(NSTableColumn *)tableColumn row:(NSInteger)row; - (CGFloat)tableView:(NSTableView *)tableView heightOfRow:(NSInteger)row; @end
>>>>> 
>>>>> currently comes in as
>>>>> 
>>>>> @objc protocol NSTableViewDelegate { optional func tableView(_: NSTableView, viewFor: NSTableColumn, row: Int) -> NSView?  optional func tableView(_: NSTableView, heightOfRow: Int) -> CGFloat }
>>>>> 
>>>>> would come in as:
>>>>> 
>>>>> @objc protocol NSTableViewDelegate { var tableView: ((NSTableView, viewFor: NSTableColumn, row: Int) -> NSView?)? { get } var tableView: ((NSTableView, heightOfRow: Int) -> CGFloat)? { get } }
>>>>> 
>>>>> with a default implementation of “nil” for each. However, this isn’t practical for a number of reasons:
>>>>> 
>>>>> a) We would end up overloading the property name “tableView” a couple dozen times, which doesn’t actually work.
>>>>> 
>>>>> b) You can no longer refer to the member with a compound name, e.g., “delegate.tableView(_:viewFor:row:)” no longer works, because the name of the property is “tableView”.
>>>>> 
>>>>> c) Implementers of the protocol now need to provide a read-only property that returns a closure. So instead of
>>>>> 
>>>>> class MyDelegate : NSTableViewDelegate { func tableView(_: NSTableView, viewFor: NSTableColumn, row: Int) -> NSView? { … } }
>>>>> 
>>>>> one would have to write something like
>>>>> 
>>>>> class MyDelegate : NSTableViewDelegate { var tableView: ((NSTableView, viewFor: NSTableColumn, row: Int) -> NSView?)? = { … except you can’t refer to self in here unless you make it lazy ...  } }
>>>>> 
>>>>> d) We’ve seriously considered eliminating argument labels on function types, because they’re a complexity in the type system that doesn’t serve much of a purpose.
>>>>> 
>>>>> One could perhaps work around (a), (b), and (d) by allowing compound (function-like) names like tableView(_:viewFor:row:) for properties, and work around (c) by allowing a method to satisfy the requirement for a read-only property, but at this point you’ve invented more language hacks than the existing @objc-only optional requirements. So, I don’t think there is a solution here.
>>>>> 
>>>>> Proposed Solution: Caller-side default implementations
>>>>> 
>>>>> Default implementations and optional requirements differ most on the caller side. For example, let’s use NSTableView delegate as it’s imported today:
>>>>> 
>>>>> func useDelegate(delegate: NSTableViewDelegate) { if let getView = delegate.tableView(_:viewFor:row:) { // since the requirement is optional, a reference to the method produces a value of optional function type // I can call getView here }
>>>>> 
>>>>> if let getHeight = delegate.tableView(_:heightOfRow:) { // I can call getHeight here } }
>>>>> 
>>>>> With my proposal, we’d have some compiler-synthesized attribute (let’s call it @__caller_default_implementation) that gets places on Objective-C optional requirements when they get imported, e.g.,
>>>>> 
>>>>> @objc protocol NSTableViewDelegate { @__caller_default_implementation func tableView(_: NSTableView, viewFor: NSTableColumn, row: Int) -> NSView?  @__caller_default_implementation func tableView(_: NSTableView, heightOfRow: Int) -> CGFloat }
>>>>> 
>>>>> And “optional” disappears from the language. Now, there’s no optionality left, so our useDelegate example tries to just do correct calls:
>>>>> 
>>>>> func useDelegate(delegate: NSTableViewDelegate) -> NSView? { let view = delegate.tableView(tableView, viewFor: column, row: row) let height = delegate.tableView(tableView, heightOfRow: row) }
>>>>> 
>>>>> Of course, the code above will fail if the actual delegate doesn’t implement both methods. We need some kind of default implementation to fall back on in that case. I propose that the code above produce a compiler error on both lines *unless* there is a “default implementation” visible. So, to make the code above compile without error, one would have to add:
>>>>> 
>>>>> extension NSTableViewDelegate { @nonobjc func tableView(_: NSTableView, viewFor: NSTableColumn, row: Int) -> NSView? { return nil }
>>>>> 
>>>>> @nonobjc func tableView(_: NSTableView, heightOfRow: Int) -> CGFloat { return 17 } }
>>>>> 
>>>>> Now, the useDelegate example compiles. If the actual delegate implements the optional requirement, we’ll use that implementation. Otherwise, the caller will use the default (Swift-only) implementation it sees. From an implementation standpoint, the compiler would effectively produce the following for the first of these calls:
>>>>> 
>>>>> if delegate.responds(to: #selector(NSTableViewDelegate.tableView(_:viewFor:row:))) { // call the @objc instance method with the selector tableView:viewForTableColumn:row: } else { // call the Swift-only implementation of tableView(_:viewFor:row:) in the protocol extension above }
>>>>> 
>>>>> There are a number of reasons why I like this approach:
>>>>> 
>>>>> 1) It eliminates the notion of ‘optional’ requirements from the language. For classes that are adopting the NSTableViewDelegate protocol, it is as if these requirements had default implementations.
>>>>> 
>>>>> 2) Only the callers to these requirements have to deal with the lack
>>>>> of default implementations. This was already the case for optional
>>>>> requirements, so it’s not an extra burden in principle, and it’s
>>>>> generally going to be easier to write one defaulted implementation
>>>>> than deal with it in several different places. Additionally, most of
>>>>> these callers are probably in the Cocoa frameworks, not application
>>>>> code, so the overall impact should be small.
>>>>> 
>>>>> Thoughts?
>>>>> 
>>>>> 	- Doug
>>>>> 
>>>>> _______________________________________________ swift-evolution
>>>>> mailing list swift-evolution at swift.org
>>>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>>> 
>>>> _______________________________________________
>>>> swift-evolution mailing list
>>>> swift-evolution at swift.org
>>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>> 
>>> -- 
>>> Dave
>>> 
>>> _______________________________________________
>>> swift-evolution mailing list
>>> swift-evolution at swift.org
>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>> 
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution
> 
> -- 
> Dave
> 
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution