[swift-evolution] [Idea] How to eliminate 'optional' protocol requirements

[Dave Abrahams]

on Sun Apr 10 2016, Dietmar Planitzer <dplanitzer-AT-q.com> wrote:

>> On Apr 10, 2016, at 11:46, Dave Abrahams via swift-evolution
> <swift-evolution at swift.org> wrote:
>> 
>> 
>> on Sun Apr 10 2016, Dietmar Planitzer <swift-evolution at swift.org>
> wrote:
>> 
>>> I’m not sure whether you’ve read the conclusion of my mail since
>>> you’ve only commented on the introductory part. In the conclusion I
>>> wrote that a possible approach for the replacement of ObjC-style
>>> optional protocol methods would be:
>>> 
>>> 1) the default implementation of a protocol method must be defined
> in
>>> the protocol (so just like in native Swift protocols today).
>> 
>> ? They can and must be defined in protocol extensions today.
>
> I know.
>
>>> 
>>> 2) we add a way for a protocol provider to check whether the
> protocol
>>> adopter has provided an “override” of the default method.
>> 
>> I object to this part.
>
> You object why? I do understand why you object to the ObjC model since
> there is not necessarily an implementation of the protocol method and
> thus the protocol provider has to guard every call with an existence
> check. But in this model here we would be guaranteed that there would
> be an implementation of the protocol method and thus guarding the call
> wouldn’t be necessary.

Because it's a needless complication that will encourage protocol and
algorithm designers to create inefficient programs because they know the
user can fall back on this hack.  Nobody thinks that classes need the
ability to check whether a given method is overridden.  Why should this
be needed for protocols?

>>> 3) we improve the Xcode interface generator so that it clearly shows
>>> whether a protocol method comes with a default or whether it
>>> doesn’t.
>> 
>> Obvious goodness, long overdue.
>> 
>>> (1) should address your main concern since it would guarantee that
>>> the protocol provider is always able to call the protocol method
>>> without having to do any checks. (2) would address the main concern
>>> of protocol providers who need to guarantee that the protocol using
>>> type achieves a certain minimum speed and does not use more than a
>>> certain amount of memory for its internal book-keeping.
>> 
>> I don't how (2) can possibly help with that.
>
> It helps because it allows the protocol provider to *understand*
> whether the protocol adopter is actually using a certain feature or
> isn’t. 

If they need to understand that, they can make the indicator of that
fact a separate protocol requirement.

> Here is the table view example again:
>
> func useDelegate(delegate: NSTableViewDelegate) {
>
>   if has_override(delegate, tableView(_:, heightForRow:)) {
>      // call tableViewNumberOfRows() on the delegate
>      // allocate the geometry cache (1 entry per row)
>      // call tableView(_:, heightForRow:) for each row
>   } else {
>     // nothing to do here since here all rows have the same height
>    }
> }
>
> Note that has_override() is just a placeholder syntax because I’ve not
> had a good idea yet of how to express this in a Swiftier way.

   if delegate.hasVariableSizedRows { ... }

   if !(delegate is NSUniformTableViewDelegate) { ... }

etc.
>
> In this example the table view is able to check whether the protocol
> adopter has actually “overriden” the default implementation of
> tableView(_:, heightForRow:). 

Which, IMO, is a terrible way to indicate that a view has variable row
heights.  It's indirect and maybe even inaccurate (I can imagine a table
view that is uniform and has its height set up once at construction
time, therefore it needs to override heightForRow).

> If the adopter did, then the table view knows that the adopter wants
> variable row heights and thus the table view can now create a cache of
> row heights and it can enable the layouting code that knows how to lay
> out rows with different heights. If however the adopter did not
> provide its own implementation of this method then the table view does
> not need to create a geometry cache and it can switch over to the
> simpler fixed-row-height layout code. The reason why we want to cache
> the row heights in the table view is because computing those heights
> can be nontrivial and the layout code needs to access those height
> values in every layoutSubviews() call. And layoutSubviews() is invoked
> 60 times per second while the user is scrolling. Also keep in mind
> that, if we would not cache the row heights, then the row height
> computation would end up competing for CPU cycles with the code that
> properly configures the views for each row.
>
> Without the ability to do this check on the protocol provider side, we
> are forced to increase the API surface so that the protocol adopter
> can explicitly tell us which layouting model he wants. 

That's exactly what one should do.  If layout model is an important
feature, the adopter should be explicit abou tit.

> But this also means that the protocol adopter now has to remember that
> he needs to configure the layouting option correctly in order to get a
> working and efficiently working table view. So the end result would be
> a table view that’s hard to use correctly.
>
> Regards,
>
> Dietmar Planitzer
>
>>> 
>>> (3) is important because it would fix one of the many aspects that
>>> make Swift protocols confusing for people who are new to the
> language.
>>> 
>>> Finally, let me restate that the goal should really be that we
>>> completely remove the syntactical differences between @objc and
> native
>>> Swift protocols. There should be one concept of protocol in Swift
> and
>>> we should be able to cover the use cases of formal and informal
> ObjC
>>> Protocols with them. The use case of formal protocols is already
>>> covered today. The use case of informal protocols could be covered
>>> with the approach above.
>>> 
>>> So is this an approach that would be acceptable to you?
>>> 
>>> Regards,
>>> 
>>> Dietmar Planitzer
>>> 
>>>> 
>>>> On Apr 10, 2016, at 10:29, Dave Abrahams via swift-evolution
> <swift-evolution at swift.org> wrote:
>>>> 
>>>> 
>>>> on Fri Apr 08 2016, Dietmar Planitzer <swift-evolution at swift.org>
> wrote:
>>>> 
>>>>> The biggest missing part with this model is that we are still not
> able
>>>>> to enable macro-level optimizations in the delegating type by
> checking
>>>>> whether the delegate does provide his own implementation of an
>>>>> optional method or doesn’t. However, this is an important
> advantage of
>>>>> the ObjC model that we should not lose.
>>>>> 
>>>>> Maybe it’s time to take a big step back and ignore the question
> of how
>>>>> to implement things for a moment and to instead focus on the
> question
>>>>> of what the conceptual differences are between ObjC protocols
> with
>>>>> optional methods and Swift protocols with default
>>>>> implementations. There are two relevant viewpoints here:
>>>>> 
>>>>> 1) From the viewpoint of a protocol adaptor:
>>>>> 
>>>>> ObjC:
>>>>> 
>>>>> 1a) adopter may provide his own implementation of the protocol
> method,
>>>>> but he is no required to.
>>>>> 
>>>>> 1b) adopter can see in the protocol declaration for which methods
> he
>>>>> must provide an implementation. Those methods do not have the
>>>>> “optional” keyword in front of them while optional methods do.
>>>>> 
>>>>> Swift:
>>>>> 
>>>>> 1c) same as (1a).
>>>>> 
>>>>> 1d) opening a binary-only Swift file in Xcode with a protocol
>>>>> definition in it which contains methods with default
> implementations
>>>>> will not give any indication of which method has a default
>>>>> implementation and which doesn’t. It’s only possible to see a
>>>>> difference on the syntax level if you have access to the sources.
>>>> 
>>>> This visibility problem is something we aim to correct in Swift,
> but
>>>> that is a question of syntax, documentation, and “header”
> generation,
>>>> and really orthogonal to what's fundamental about “optional
>>>> requirements:” 
>>>> 
>>>> 1. The ability to “conform” to the protocol without a
>>>>  default implementation of the requirement have been provided
>>>>  anywhere.
>>>> 
>>>> 2. The ability to dynamically query whether a type actually
> provides the
>>>>  requirement.
>>>> 
>>>> Both of these “features,” IMO, are actually bugs.
>>>> 
>>>>> So from the viewpoint of the protocol adopter, there isn’t much
> of a
>>>>> difference. The only relevant difference is that its always
> possible
>>>>> in ObjC to tell whether a protocol method must be implemented by
> the
>>>>> adopter or whether a method already has a default behavior. We
>>>>> shouldn’t actually have to change anything on the syntax-level in
>>>>> Swift to fix this problem. It should be sufficient to improve the
>>>>> Swift interface generator in Xcode so that it gives an indication
>>>>> whether a protocol method has a default implementation or
> doesn’t. Eg
>>>>> if we want to ensure that the generated interface is valid syntax
> then
>>>>> we could do this:
>>>>> 
>>>>> protocol Foo {
>>>>> 
>>>>>  func void bar() -> Int /* has default */
>>>>> 
>>>>> }
>>>>> 
>>>>> or if we say that it is fine that the generated interface is not
> valid
>>>>> syntax (I think it already shows "= default” for function
> arguments
>>>>> with a default value which I don’t think is valid syntax), then
> we
>>>>> could do this:
>>>>> 
>>>>> protocol Foo {
>>>>> 
>>>>>  func void bar() -> Int {…}
>>>>> 
>>>>> }
>>>>> 
>>>>> Now on to the other side of the equation.
>>>>> 
>>>>> 2) From the viewpoint of the protocol provider (the person who
> defines
>>>>> the protocol and the type that will invoke the protocol methods):
>>>>> 
>>>>> ObjC:
>>>>> 
>>>>> 2a) provider has freedom in deciding where to put the default
>>>>> implementation and he can put the default implementation in a
> single
>>>>> place or spread it out if necessary over multiple places. So has
> the
>>>>> freedom to choose whatever makes the most sense for the problem
> at
>>>>> hand.
>>>> 
>>>> But freedom for protocol implementors reduces predictability for
> protocol
>>>> clients and adopters.
>>>> 
>>>>> 2b) provider can detect whether the adopter provides his own
> protocol
>>>>> method implementation without compromising the definition of the
>>>>> protocol (compromising here means making return values optional
> when
>>>>> they should not be optional based on the natural definition of
> the
>>>>> API). This enables the provider to implement macro-level
> optimizations
>>>>> (eg table view can understand whether fixed or variable row
> heights
>>>>> are desired).
>>>>> 
>>>>> Swift:
>>>>> 
>>>>> 2c) provider is forced to put the default implementation in a
> specific
>>>>> place.
>>>>> 
>>>>> 2d) provider has no way to detect whether the adopter has
> provided his
>>>>> own implementation of the protocol method.
>>>>> 
>>>>> I do think that (2a) would be nice to have but we can probably do
>>>>> without it if it helps us to make progress with this
> topic. However,
>>>>> the ability to detect whether a protocol adopter provides his own
>>>>> implementation of a protocol method which comes with a default is
> a
>>>>> useful and important feature which helps us in optimizing the
>>>>> implementation of types and which allows us to keep the API
> surface
>>>>> smaller than it would be without this ability. Just go and
> compare eg
>>>>> UITableView to the Android ListView / RecyclerView to see the
>>>>> consequences of not having that ability and how it inflates the
> API
>>>>> surface (and keep in mind that the Android equivalents provide a
>>>>> fraction of the UITableView functionality).
>>>>> 
>>>>> The important point about (2b) is actually that we are able to
> detect
>>>>> whether an “override” (I’ll just call this overriding for now) of
> the
>>>>> default implementation exists or does not exist. 
>>>> 
>>>> IMO the important point about (2b) is that it leads to protocol
> designs
>>>> that create work and complexity for clients of the protocol, and
> being
>>>> constrained to make your protocol work so that clients don't have
> to do
>>>> these kinds of checks is a Very Good Thing™.
>>>> 
>>>>> In ObjC we make this distinction by checking whether an
> implementation
>>>>> of the method exists at all. But we don’t have to do it that
> way. An
>>>>> alternative approach could be based on a check that sees whether
> the
>>>>> dispatch table of the delegate contains a pointer to the default
>>>>> implementation of the protocol method or to some other method. So
>>>>> conceptually what we want is an operation like this:
>>>>> 
>>>>> func void useDelegate(delegate: NSTableViewDelegate) {
>>>>> 
>>>>> if has_override(delegate, tableView(_:, heightOfRow:)) { // ask
> the
>>>>> delegate how many rows it has // allocate the geometry cache //
> fill
>>>>> in the geometry cache by calling tableView(_:, heightForRow:) for
> each
>>>>> row } else { // nothing to do here } }
>>>>> 
>>>>> Which would get the job done but doesn’t look good. Maybe someone
> has
>>>>> a better idea of how the syntax such an operator could look.
>>>>> 
>>>>> So my point here is that what we care about is the ability to
> detect
>>>>> whether the adopter provides an implementation of a protocol
> method
>>>>> which comes with a default implementation. The point is not that
> Swift
>>>>> protocols should work the exact same way that ObjC protocols have
> been
>>>>> working under the hood. But I do think that we want to eventually
> get
>>>>> to a point where the @objc attribute disappears and that we get a
>>>>> truly unified language on the syntactical level. An approach
> where:
>>>>> 
>>>>> I) we accept that the default behavior of a protocol method has
> to be
>>>>> provided by the protocol itself
>>>>> 
>>>>> II) the language is extended with a mechanism that makes it
> possible
>>>>> for a protocol provider to detect whether the adopter has
> “overridden”
>>>>> the default implementation
>>>>> 
>>>>> III) we improve the Xcode Swift interface generator so that it
> gives a
>>>>> clear indication whether a protocol method does come with a
> default
>>>>> implementation
>>>>> 
>>>>> would give us all the relevant advantages of ObjC-style optional
>>>>> protocol methods and it should allow us to create a unified
> syntax
>>>>> where there is no longer a visible difference between an optional
>>>>> protocol method that was imported from ObjC and a native Swift
>>>>> protocol with default implementations.
>>>>> 
>>>>> Regards,
>>>>> 
>>>>> Dietmar Planitzer
>>>>> 
>>>>>> On Apr 7, 2016, at 17:12, Douglas Gregor via swift-evolution
>>>>>> <swift-evolution at swift.org> wrote:
>>>>>> 
>>>>>> Hi all,
>>>>>> 
>>>>>> Optional protocol requirements in Swift have the restriction
> that
>>>>>> they only work in @objc protocols, a topic that’s come up a
> number
>>>>>> of times. The start of these threads imply that optional
>>>>>> requirements should be available for all protocols in
> Swift. While
>>>>>> this direction is implementable, each time this is discussed
> there
>>>>>> is significant feedback that optional requirements are not a
> feature
>>>>>> we want in Swift. They overlap almost completely with default
>>>>>> implementations of protocol requirements, which is a more
> general
>>>>>> feature, and people seem to feel that designs based around
> default
>>>>>> implementations and refactoring of protocol hierarchies are
> overall
>>>>>> better.
>>>>>> The main concern with removing optional requirements from Swift
> is their impact on Cocoa: Objective-C protocols, especially for
> delegates and data sources, make heavy use of optional
> requirements. Moreover, there are no default implementations for any
> of these optional requirements: each caller effectively checks for the
> presence of the method explicitly, and implements its own logic if the
> method isn’t there.
>>>>>> 
>>>>>> A Non-Workable Solution: Import as optional property
> requirements One suggestion that’s come up to map an optional
> requirement to a property with optional type, were “nil” indicates
> that the requirement was not satisfied. For example,
>>>>>> 
>>>>>> @protocol NSTableViewDelegate @optional - (nullable NSView
> *)tableView:(NSTableView *)tableView viewForTableColumn:(NSTableColumn
> *)tableColumn row:(NSInteger)row; - (CGFloat)tableView:(NSTableView
> *)tableView heightOfRow:(NSInteger)row; @end
>>>>>> 
>>>>>> currently comes in as
>>>>>> 
>>>>>> @objc protocol NSTableViewDelegate { optional func tableView(_:
> NSTableView, viewFor: NSTableColumn, row: Int) -> NSView?  optional
> func tableView(_: NSTableView, heightOfRow: Int) -> CGFloat }
>>>>>> 
>>>>>> would come in as:
>>>>>> 
>>>>>> @objc protocol NSTableViewDelegate { var tableView:
> ((NSTableView, viewFor: NSTableColumn, row: Int) -> NSView?)? { get }
> var tableView: ((NSTableView, heightOfRow: Int) -> CGFloat)? { get } }
>>>>>> 
>>>>>> with a default implementation of “nil” for each. However, this
> isn’t practical for a number of reasons:
>>>>>> 
>>>>>> a) We would end up overloading the property name “tableView” a
> couple dozen times, which doesn’t actually work.
>>>>>> 
>>>>>> b) You can no longer refer to the member with a compound name,
> e.g., “delegate.tableView(_:viewFor:row:)” no longer works, because
> the name of the property is “tableView”.
>>>>>> 
>>>>>> c) Implementers of the protocol now need to provide a read-only
> property that returns a closure. So instead of
>>>>>> 
>>>>>> class MyDelegate : NSTableViewDelegate { func tableView(_:
> NSTableView, viewFor: NSTableColumn, row: Int) -> NSView? { … } }
>>>>>> 
>>>>>> one would have to write something like
>>>>>> 
>>>>>> class MyDelegate : NSTableViewDelegate { var tableView:
> ((NSTableView, viewFor: NSTableColumn, row: Int) -> NSView?)? = { …
> except you can’t refer to self in here unless you make it lazy ...  }
> }
>>>>>> 
>>>>>> d) We’ve seriously considered eliminating argument labels on
> function types, because they’re a complexity in the type system that
> doesn’t serve much of a purpose.
>>>>>> 
>>>>>> One could perhaps work around (a), (b), and (d) by allowing
> compound (function-like) names like tableView(_:viewFor:row:) for
> properties, and work around (c) by allowing a method to satisfy the
> requirement for a read-only property, but at this point you’ve
> invented more language hacks than the existing @objc-only optional
> requirements. So, I don’t think there is a solution here.
>>>>>> 
>>>>>> Proposed Solution: Caller-side default implementations
>>>>>> 
>>>>>> Default implementations and optional requirements differ most on
> the caller side. For example, let’s use NSTableView delegate as it’s
> imported today:
>>>>>> 
>>>>>> func useDelegate(delegate: NSTableViewDelegate) { if let getView
> = delegate.tableView(_:viewFor:row:) { // since the requirement is
> optional, a reference to the method produces a value of optional
> function type // I can call getView here }
>>>>>> 
>>>>>> if let getHeight = delegate.tableView(_:heightOfRow:) { // I can
> call getHeight here } }
>>>>>> 
>>>>>> With my proposal, we’d have some compiler-synthesized attribute
> (let’s call it @__caller_default_implementation) that gets places on
> Objective-C optional requirements when they get imported, e.g.,
>>>>>> 
>>>>>> @objc protocol NSTableViewDelegate {
> @__caller_default_implementation func tableView(_: NSTableView,
> viewFor: NSTableColumn, row: Int) -> NSView?
> @__caller_default_implementation func tableView(_: NSTableView,
> heightOfRow: Int) -> CGFloat }
>>>>>> 
>>>>>> And “optional” disappears from the language. Now, there’s no
> optionality left, so our useDelegate example tries to just do correct
> calls:
>>>>>> 
>>>>>> func useDelegate(delegate: NSTableViewDelegate) -> NSView? { let
> view = delegate.tableView(tableView, viewFor: column, row: row) let
> height = delegate.tableView(tableView, heightOfRow: row) }
>>>>>> 
>>>>>> Of course, the code above will fail if the actual delegate
> doesn’t implement both methods. We need some kind of default
> implementation to fall back on in that case. I propose that the code
> above produce a compiler error on both lines *unless* there is a
> “default implementation” visible. So, to make the code above compile
> without error, one would have to add:
>>>>>> 
>>>>>> extension NSTableViewDelegate { @nonobjc func tableView(_:
> NSTableView, viewFor: NSTableColumn, row: Int) -> NSView? { return nil
> }
>>>>>> 
>>>>>> @nonobjc func tableView(_: NSTableView, heightOfRow: Int) ->
> CGFloat { return 17 } }
>>>>>> 
>>>>>> Now, the useDelegate example compiles. If the actual delegate
> implements the optional requirement, we’ll use that
> implementation. Otherwise, the caller will use the default
> (Swift-only) implementation it sees. From an implementation
> standpoint, the compiler would effectively produce the following for
> the first of these calls:
>>>>>> 
>>>>>> if delegate.responds(to:
> #selector(NSTableViewDelegate.tableView(_:viewFor:row:))) { // call
> the @objc instance method with the selector
> tableView:viewForTableColumn:row: } else { // call the Swift-only
> implementation of tableView(_:viewFor:row:) in the protocol extension
> above }
>>>>>> 
>>>>>> There are a number of reasons why I like this approach:
>>>>>> 
>>>>>> 1) It eliminates the notion of ‘optional’ requirements from the
> language. For classes that are adopting the NSTableViewDelegate
> protocol, it is as if these requirements had default implementations.
>>>>>> 
>>>>>> 2) Only the callers to these requirements have to deal with the
> lack
>>>>>> of default implementations. This was already the case for
> optional
>>>>>> requirements, so it’s not an extra burden in principle, and it’s
>>>>>> generally going to be easier to write one defaulted
> implementation
>>>>>> than deal with it in several different places. Additionally,
> most of
>>>>>> these callers are probably in the Cocoa frameworks, not
> application
>>>>>> code, so the overall impact should be small.
>>>>>> 
>>>>>> Thoughts?
>>>>>> 
>>>>>> 	- Doug
>>>>>> 
>>>>>> _______________________________________________ swift-evolution
>>>>>> mailing list swift-evolution at swift.org
>>>>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>>>> 
>>>>> _______________________________________________
>>>>> swift-evolution mailing list
>>>>> swift-evolution at swift.org
>>>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>>> 
>>>> -- 
>>>> Dave
>>>> 
>>>> _______________________________________________
>>>> swift-evolution mailing list
>>>> swift-evolution at swift.org
>>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>> 
>>> _______________________________________________
>>> swift-evolution mailing list
>>> swift-evolution at swift.org
>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>> 
>> -- 
>> Dave
>> 
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution

-- 
Dave