[swift-evolution] Drafting a proposal: Make conflicting with protocol extension methods an error

[Brent Royal-Gordon]

This proposal comes out of the threads "Proposal: Universal dynamic dispatch for method calls” and “Proposal: Require explicit modifier for statically dispatched extension methods”. I’m posting it now as a new thread because it needs some attention from people who haven’t been following these long and detailed threads.

This proposal is intended to address a Swift behavior which surprises some people. Suppose you write a protocol and extension like this:

	protocol Turnable {
	    func turning() -> Self
	    mutating func turn()
	}
	extension Turnable {
	    mutating func turn() {
	        self = turning()
	    }
	
	    func turningRepeatedly(additionalTurns: Int) -> Self {
	        var turnedSelf = self
	        for _ in 1...additionalTurns {
	            turnedSelf.turn()
	        }
	        return turnedSelf
	    }
	}

Now you want to write a conforming type, `SpimsterWicket`. There are three different rules about whether your type has to, or can, implement its own versions of these methods.

1. `turning()` is a “protocol method”: it is listed in the protocol but is not included in the extension. You *must* implement `turning()` to conform to `Turnable`.
2. `turn()` is a “defaulted protocol method”: it is listed in the protocol but there is also an implementation of it in the extension. You *may* implement `turn()`; if you don’t, the protocol extension’s implementation will be used.
3. `turningRepeatedly(_: Int)` is a “protocol extension method”: it is *not* listed in the protocol; it is only in the protocol extension. This is the case we are trying to address.

Currently, in case 3, Swift permits you to implement your own `turningRepeatedly(_: Int)`. However, your implementation may not be called in every circumstance that you expect. If you call `turningRepeatedly` on a variable of type `SpimsterWicket`, you’ll get `SpimsterWicket`’s implementation of the method; however, if you call `turningRepeatedly` on a variable of type `Turnable`, you’ll get `Turnable`’s implementation of the method.

	var wicket: SpimsterWicket = SpimsterWicket()
	var turnable: Turnable = wicket
	
	wicket.turn()					// Calls SpimsterWicket.turn()
	turnable.turn()					// Also calls SpimsterWicket.turn()
	
	wicket.turningRepeatedly(5)		// Calls SpimsterWicket.turningRepeatedly(_:)
	turnable.turningRepeatedly(5)	// Calls Turnable.turningRepeatedly(_:)

In most parts of Swift, casting an instance or assigning it to a variable of a different type doesn’t change which implementation will be called when you put it on the left-hand side of a dot. (I’m leaving aside Objective-C bridging, like `Int` to `NSNumber`, which is really a different operation being performed with the same syntax.) If you put a `UIControl` into a variable of type `UIView`, and then call `touchesBegan()` on that variable, Swift will still call `UIControl.touchesBegan()`. The same is true of defaulted protocol methods—if you call `turn()` on `turnable`, you’ll get `SpimsterWicket.turn()`.

But this is not true of protocol extension methods. There, the static type of the variable—the type known at compile time, the type that the variable is labeled with—is used. Thus, calling `turningRepeatedly(_:)` on `wicket` gets you `SpimsterWicket`’s implementation, but calling it on `turnable`—even though it's merely the same instance casted to a different type—gets you `Turnable`’s implementation.

This creates what I call an “incoherent” dispatch, and it occurs nowhere else in Swift. In most places in Swift, dispatch is either based on the runtime type (reference types, normal protocol members), or the design of the language ensures there’s no difference between dispatching on the compile-time type and the runtime type (value types, `final` members). But in protocol extension members, dispatch is based on the compile-time type even though the runtime type might produce different behavior.

The lack of a warning on this is currently considered a bug <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20151207/001861.html>, but I think we should go further. In brief, here’s what I propose.


MARK PROTOCOL EXTENSION MEMBERS AS FINAL AND PREVENT CONFLICTS
------------------------------------------------------------------------------------------------------------------

I think conflicting with a protocol extension member should be an error. But I also think the source code should tell you which protocol extension members this rule applies to.

What I think we should do is require protocol extension methods (that is, non-default ones) to be marked `final`. The current meaning of `final` is specific to classes: “Subclasses must use this as-is”. I think we can extend that to protocols by making it mean, more generally, “Subtypes must use this as-is”.

This also has the handy effect of making it obvious, without reading the original protocol definition, which members in a protocol extension are default implementations and which are not:

	extension Turnable {
	    mutating func turn() {
	        self = turning()
	    }
	
	    final func turningRepeatedly(additionalTurns: Int) -> Self {
	        var turnedSelf = self
	        for _ in 1...additionalTurns {
	            turnedSelf.turn()
	        }
	        return turnedSelf
	    }
	}

With this in place, I would then make shadowing a visible `final` protocol extension member illegal. That means:

1. It is an error to conform a type* to a protocol which includes a visible `final` member matching one of its own members. (You can’t conform `SpimsterWicket` to `Turnable` if it has a `turningRepeatedly(_:)` method.)

2.  It is an error to extend a type to add a member matching a visible `final` member in any protocol it conforms to. (You can’t extend `SpimsterWicket` to add a `turningRepeatedly(_:)` method if it conforms to `Turnable`.)

3. It is an error to extend a protocol with a member that conflicts with any visible member of a conforming type. (You can’t extend `Turnable` to add a final `turningRepeatedly(_:)` method if `SpimsterWicket` has one.)

4. It is an error to conform a type to two protocols which both have `final` protocol extension members with the same name and signature. (You can’t extend `SpimsterWicket` with both `Turnable` and `VampireVictimizable` if they both have extensions with `final func turningRepeatedly(_: Int) -> Self`.)

5. It is an error to import a combination of modules which causes any of the above conflicts. (If the `Turnable` protocol is in `TurnableKit`, the `SpimsterWicket` conformance is in `LibWickets`, and the `Turnable` protocol extension is in `TurnableExtensions`, you cannot import both `SpimsterWicket` and `TurnableExtensions` into the same file.)

* “Type” in this list means a class, struct, enum, or another protocol, whether existential or generic.

Though there are many permutations (and I may have missed some), these all express one simple rule: it should not be legal to have both a `final` protocol extension member and a member which conflicts with it visible in the same place.


SELECTIVELY PERMIT CONFLICTS WITH AN ATTRIBUTE
---------------------------------------------------------------------------------

This part I’m a little less sure about; we may want to drop it.

The enforcement of the above rule prevents bugs, but in some cases it may also prevent desirable conformances. I propose that we provide an escape hatch, which I’m calling the `@incoherent` attribute.

`@incoherent` basically acknowledges the conflict and explicitly declares that you are expecting the current behavior, where different variable types may call different implementations on the same instance. It may be used in any of the contexts listed above.

An `@incoherent` attribute should always make clear both the conforming type and the protocol it conforms to. However, in some contexts, one or both of these types are obvious, and can be left out. So it has several different syntactic forms.

On a protocol conformance declaration:

	extension SpimsterWicket: @incoherent Turnable { … }

On a protocol extension adding a member which conflicts with an existing conforming type:

	@incoherent(SpimsterWicket) extension Turnable {…}

On an extension to a conforming type which conflicts with an existing final protocol extension method:

	@incoherent(Turnable) extension SpimsterWicket {…}

On an import statement which imports extensions that conflict with a conformance:

	@incoherent(SpimsterWicket: Turnable) import TurnableExtensions

(Incidentally, we could make `@incoherent` also work when inheriting from a class with `final` members, with the same caveat: casting to the superclass may change the instance’s behavior. The result would be something like overriding a non-virtual member in C++.)

Keep in mind that, like `try!` or the various `Unsafe` APIs, `@incoherent` is not really a desirable feature—it is an escape hatch for difficult situations. The preferred solution is to rename conflicting members or add them to the protocol definition so that they can be non-final. I would suggest that Swift *not* offer a fix-it inserting an `@incoherent` annotation; people would use it without realizing what it meant.


A NOTE ON FUTURE EXPANSION
-----------------------------------------------

Some people have suggested that protocol extension methods should get the same dynamic dispatch as methods listed in the protocol itself, effectively erasing the distinction between defaulted members and other protocol extension members. I’m currently agnostic on this proposal, other than noting that implementation might be complicated. However, even if we do that, we may want `final` protocol extension members anyway, to give us better dispatch performance on certain calls.


So—any thoughts? Particularly, is `@incoherent` needed or wanted, or should we just error out when we see a conflict and demand that users disambiguate their code?

-- 
Brent Royal-Gordon
Architechies