[swift-evolution] static vs. dynamic dispatch of protocol methods

[Michael Peternell]

(renamed subject line from "Re: [swift-evolution] Proposal SE-0009 Reconsideration" to "static vs. dynamic dispatch of protocol methods")

> Am 19.05.2016 um 19:59 schrieb Vladimir.S via swift-evolution <swift-evolution at swift.org>:
> 
> And... is this all OK? No one think this is a broken design?

I do. I think this has not been done on purpose. But a solution to this problem is far from obvious, I fear.

***

May I explain something, that may bring us closer to understanding the problem?

I think the difficulty of dispatching everything dynamically is that it would require to "duck type" everything. (Google: "duck typing")

E.g.:

// Module: ObjSupportLibrary
// ObjectPlayground.swift ...

class A {
   func foo() { ... }
}

class B {
   func foo() { ... }
}

class C {
   func bar() { ... }
}

If I just want to call foo(), but I don't want the type to be known at compile time, I can just do

// Module: Main
// CanFooSupport.swift ...

protocol CanFoo {
    func foo()
}

extension AnyObject: CanFoo [named Extension12] {
    func foo() { ... }
}

So... it is clear that the name `foo` has to be available at runtime, in class `A` and class `B`, because the compiler cannot know if some other module will make a `CanFoo`-protocol when it compiles module ObjSupportLibrary. And if someone implements a `CanFoo`-protocol, the information that `A.foo` and `B.foo` have the same signature has to be available at runtime. At least there has to be some function `S` with the property that `S(CanFoo.foo) == S(A.foo) == S(B.foo)` and a dispatch-table for `A` with `a.dispatchTable(S(CanFoo.foo)) == A.foo` and a dispatch table for `C` with `c.dispatchTable(S(CanFoo.foo)) == nil`, (a: A, c: C), I suspect that this is not the case yet, but maybe someone who knows better about the current ABI can provide more insight here? At the call site, for a variable `b` of runtime type `B` and inferred type `CanFoo`, a call to `b.foo()` has to do something like

    if let f = b.dispatchTable(S(CanFoo.foo)) {
        f()
    } else {
        Extension12.foo(self: b)
    }

The above is unlike any `objc_messageSend()` call! It's more like `objc_messageSendWithFallbackIsaPointers([Extension12.class], b)`.
You cannot ask the class-object of `b` (which is `B`) to "just invoke foo", because from the declaration `b: CanFoo`, you cannot tell if `b.dynamicType` has overridden `foo()`. You have to check if `b` implements `foo()`, and if it does, just call it, and if not, call the default implementation instead. This is not an algorithm that the class object can do. After all, there could be a different protocol `CanFoo2` that works the same but has a different default implementation. How can the class-object of `b` know if the inferred type of the variable holding it is `CanFoo` or `CanFoo2`? It cannot. The default implementation from the extension has to be statically dispatched.

***

What can be done about this?

Maybe we should define:

If you have a variable `x` with inferred type `P` (where `P` is a protocol) and you want to call a function `f` on it:
1) try to look up the function `f()` in the dispatch table of `x`'s class object. If `f()` is defined there with an `override(P)` annotation, just call it.
2) if 1) didn't work, call the protocol extensions default method instead. This means that a method defined in a protocol is only dynamically dispatched if it is annotated with `override(P)`.

This also means that all protocol-implementations would have to be annotated with `override(Protocol_name)` (or some other, equivalent syntax) (e.g. `extension X: P { func bar() { ... } }` would need to change to `extension X: P { override(P) func bar() { ... } }` too. (there are funny edge-cases if you don't require that.))

Dynamically overriding something from a protocol extension will just not be possible, because you cannot name a protocol extension. (I named the extension above `Extension12` just because I had to refer to it, but it's not valid Swift syntax.) Unless of course, that method is also defined in the protocol itself! If protocol extensions can be given names (there is already another proposal/pitch for that), this may be possible though.

I don't know if this would be feasible, or how much effort it would be to implement this. But maybe it's a starting point? I know there is still more design work to be done here, for example I have swept generics under the mat. That said, I think I understand now why no one has really solved this problem yet. It's just really hard.

***

Why wouldn't we want to just dispatch everything dynamically?

The problem with dispatching everything dynamically is that it would mean that we could accidentally override methods from protocols. The specialized method may have a totally different contract, and the developer of method `fooo()` cannot know if some protocol will eventually declare and implement a `fooo()`-method too.

For a more realistic example:

class Diagram {
    /// should draw to the screen (default UI-graphics-context)
    func draw() {
        // default implementation does nothing
    }
    func toXML() -> String {
        return "<diagram></diagram>"
    }
}

class BluePolygonDiagram: Diagram {
    override func draw() { ... }
    override func toXML() { ... }
    /// changes the blue-ness of the diagram
    /// - parameter v: should be in the open range (-100.0 .. +100.0).
    func makeBlue(v: Float) { ... }
}

// imagine 32 other Diagram-subclasses here...

// and in another place:

extension Diagram {
    /// will draw a blue-and-white version of the diagram to the screen
    /// - parameter v: how much blue you want it.. 0="black and white", 1="blue and white", and everything in-between is possible too
    func makeBlue(v: Float) {
        ...
    }
}

// then at some other place

func drawDiagramsNextToEachOther(diagrams: Array<Diagram>, withBlueness: Float? = nil) {
    for d in diagrams {
        if let blueness = withBlueness {
            d.makeBlue(blueness)
        } else {
            d.draw()
        }
    }
}

Therefore:
- can the developer of BluePolygonDiagram be blamed for not knowing that some other developer will be making an extension that defines makeBlue(_:) in a different way?
- can the developer of the makeBlue-Diagram-extension blamed for not reading through every class in the Diagram-standard-library, in order to know that there is already a makeBlue-method implemented somewhere?
=> I think the answers to both questions have to be "No". It follows that you can also have *too much* dynamic dispatch, not only *too little*.

***

Maybe someone wants to write up a proposal draft?

Regards,
Michael