[swift-evolution] [Proposal Draft] automatic protocol forwarding

Wed Dec 30 21:12:13 CST 2015

> On Dec 29, 2015, at 5:44 PM, Dave Abrahams via swift-evolution <swift-evolution at swift.org> wrote:
> 
> 
>> On Dec 29, 2015, at 12:06 PM, Kevin Ballard via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>> 
>> I briefly skimmed your proposal, so I apologize if you already addressed this, but it occurs to me that we could support automatic protocol forwarding today on a per-protocol basis simply by declaring a separate protocol that provides default implementations doing the forwarding. Handling of Self return types can then be done by adding a required initializer (or just not implementing that method, so the concrete type is forced to deal with it even though everything else is forwarded).
>>  
>> For example, if I want to automatically forward SequenceType to a member, I can do something like
>>  
>> protocol SequenceTypeForwarder : SequenceType {
>>     typealias ForwardedSequenceType : SequenceType
>> 
>>     var forwardedSequence : ForwardedSequenceType { get }
>> }
>> 
>> extension SequenceTypeForwarder {
>>     func generate() -> ForwardedSequenceType.Generator {
>>         return forwardedSequence.generate()
>>     }
>> 
>>     func underestimateCount() -> Int {
>>         return forwardedSequence.underestimateCount()
>>     }
>> 
>>     func map<T>(@noescape transform: (ForwardedSequenceType.Generator.Element) throws -> T) rethrows -> [T] {
>>         return try forwardedSequence.map(transform)
>>     }
>> 
>>     func filter(@noescape includeElement: (ForwardedSequenceType.Generator.Element) throws -> Bool) rethrows -> [ForwardedSequenceType.Generator.Element] {
>>         return try forwardedSequence.filter(includeElement)
>>     }
>> 
>>     func forEach(@noescape body: (ForwardedSequenceType.Generator.Element) throws -> Void) rethrows {
>>         return try forwardedSequence.forEach(body)
>>     }
>> 
>>     func dropFirst(n: Int) -> ForwardedSequenceType.SubSequence {
>>         return forwardedSequence.dropFirst(n)
>>     }
>> 
>>     func dropLast(n: Int) -> ForwardedSequenceType.SubSequence {
>>         return forwardedSequence.dropLast(n)
>>     }
>> 
>>     func prefix(maxLength: Int) -> ForwardedSequenceType.SubSequence {
>>         return forwardedSequence.prefix(maxLength)
>>     }
>> 
>>     func suffix(maxLength: Int) -> ForwardedSequenceType.SubSequence {
>>         return forwardedSequence.suffix(maxLength)
>>     }
>> 
>>     func split(maxSplit: Int, allowEmptySlices: Bool, @noescape isSeparator: (ForwardedSequenceType.Generator.Element) throws -> Bool) rethrows -> [ForwardedSequenceType.SubSequence] {
>>         return try forwardedSequence.split(maxSplit, allowEmptySlices: allowEmptySlices, isSeparator: isSeparator)
>>     }
>> }
> 
> FWIW,
> 
> https://github.com/apple/swift/blob/master/stdlib/public/core/SequenceWrapper.swift <https://github.com/apple/swift/blob/master/stdlib/public/core/SequenceWrapper.swift>
> 
> Though I don’t know why we still have this; it’s not used anywhere and should probably be removed.  I think it was supposed to be part of the new lazy sequence/collection subsystem but it was never incorporated.

Dave, thanks for pointing me to the Lazy Collections subsystem.  It made for a really great case study!  

A lot of what is happening in there is not directly forwarding related.  But I do think the implementation of the parts that involve forwarding is improved by using the forwarding mechanism in this proposal.  It is more clear and more robust than the current implementation. 

As it turns out, _SequenceWrapperType and the extension to SequenceType in SequenceWrapper.swift actually are still in use.  They contain an implementation of the forwarding mechanism Kevin Ballard suggested in this thread.  _CollectionWrapperType and the extension to CollectionType are not in use.  LazyCollection uses manual forwarding in the type itself which avoids some of the drawbacks of the protocol extension approach.  Of course this begs the question of why two different mechanisms are in use and which is actually preferred.

I am working on a new draft of the proposal with a greatly expanded motivation section.  I don’t have that completed yet, but I have completed a first pass of the section on the lazy collection subsystem.  I am including the current draft here.  I hope you find it interesting.  I am interested in your thoughts on it.

Matthew
Motivation

Delegation is a robust, composition oriented design technique that keeps interface and implementation inheritance separate. The primary drawback to this technique is that it requires a lot of manual boilerplate to forward implemenation to the implementing member. This proposal eliminates the need to write such boilerplate manually, thus making delegation-based designs much more convenient and attractive.

This proposal may also serve as the foundation for a future enhancement allowing a very concise “newtype” declaration. In the meantime, it facilitates similar functionality, although in a slightly more verbose manner.

Examples

Several examples follow. 

The first two show how this proposal could improve how forwarding is implemented by the lazy collection subsystem of the standard library. This makes an interesting case study as each example employs a different forwarding mechanism.

LazySequence

The relevant portion of the current implementation of LazySequence looks like this (with comments removed and formatting tweaks):

// in SequenceWrapper.swift:

public protocol _SequenceWrapperType {
  typealias Base : SequenceType
  typealias Generator : GeneratorType = Base.Generator

  var _base: Base {get}
}

extension SequenceType
  where Self : _SequenceWrapperType, Self.Generator == Self.Base.Generator {

  public func generate() -> Base.Generator {
    return self._base.generate()
  }

  public func underestimateCount() -> Int {
    return _base.underestimateCount()
  }

  @warn_unused_result
  public func map<T>(
    @noescape transform: (Base.Generator.Element) throws -> T
  ) rethrows -> [T] {
    return try _base.map(transform)
  }

  @warn_unused_result
  public func filter(
    @noescape includeElement: (Base.Generator.Element) throws -> Bool
  ) rethrows -> [Base.Generator.Element] {
    return try _base.filter(includeElement)
  }

  public func _customContainsEquatableElement(
    element: Base.Generator.Element
  ) -> Bool? { 
    return _base._customContainsEquatableElement(element)
  }

  public func _preprocessingPass<R>(@noescape preprocess: (Self) -> R) -> R? {
    return _base._preprocessingPass { _ in preprocess(self) }
  }

  public func _copyToNativeArrayBuffer()
    -> _ContiguousArrayBuffer<Base.Generator.Element> {
    return _base._copyToNativeArrayBuffer()
  }

  public func _initializeTo(ptr: UnsafeMutablePointer<Base.Generator.Element>)
    -> UnsafeMutablePointer<Base.Generator.Element> {
    return _base._initializeTo(ptr)
  }
}

// in LazySequence.swift:

public struct LazySequence<Base : SequenceType>
  : LazySequenceType, _SequenceWrapperType {

  public init(_ base: Base) {
    self._base = base
  }

  public var _base: Base
  public var elements: Base { return _base }
}
LazySequence is using the approach to forwarding mentioned by Kevin Ballard on the mailing list <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20151228/004755.html>in response to this proposal. This approach has several deficiencies that directly impact LazySequence:

LazySequence must publicly expose implementation details. Both its _base property as well as its conformance to _SequenceWrapperType.
The forwarding members must be manually implemented. They are trivial, but mistakes are still possible. In this case, @warn_unused_result is missing in some places where it should probably be specified (and would be synthesized using the approach in this proposal due to its presence in the protocol member declarations).
It is not immediately apparent that _SequenceWrapperType and the corresponding extension only provide forwarding members. Even if the name clearly indicates that it is possible that the code does something different. It is possible for somebody to come along after the initial implementation and add a new method that does something other than simple forwarding.
Because the forwarding is implemented via a protocol extension as default methods it can be overriden by an extension on LazySequence.
Here is an alternative implemented using the current proposal:

// _LazySequenceForwarding redeclares the subset of the members of SequenceType we wish to forward.
// The protocol is an implementation detail and is marked private.
private protocol _LazySequenceForwarding {

  typealias Generator : GeneratorType

  @warn_unused_result
  func generate() -> Generator

  @warn_unused_result
  func underestimateCount() -> Int

  @warn_unused_result
  func map<T>(
    @noescape transform: (Generator.Element) throws -> T
  ) rethrows -> [T]

  @warn_unused_result
  func filter(
    @noescape includeElement: (Generator.Element) throws -> Bool
  ) rethrows -> [Generator.Element]

  @warn_unused_result
  func _customContainsEquatableElement(
    element: Generator.Element
  ) -> Bool?

  func _copyToNativeArrayBuffer() -> _ContiguousArrayBuffer<Generator.Element>

  func _initializeTo(ptr: UnsafeMutablePointer<Generator.Element>)
    -> UnsafeMutablePointer<Generator.Element>
}

public struct LazySequence<Base : SequenceType> : LazySequenceType {

  public init(_ base: Base) {
    self._base = base
  }

  // NOTE: _base is now internal
  internal var _base: Base
  public var elements: Base { return _base }

  public forward _LazySequenceForwarding to _base

  // The current proposal does not currently support forwarding 
  // of members with nontrivial Self requirements.
  // Because of this _preprocessingPass is forwarded manually.
  // A future enhancement may be able to support automatic
  // forwarding of protocols with some or all kinds of 
  // nontrivial Self requirements.
  public func _preprocessingPass<R>(@noescape preprocess: (Self) -> R) -> R? {
    return _base._preprocessingPass { _ in preprocess(self) }
  }
}
This example takes advantage of a very important aspect of the design of this proposal. Neither Base nor LazySequence are required to conform to _LazySequenceForwarding. The only requirement is that Base contains the members specified in _LazySequenceForwarding as they will be used in the synthesized forwarding implementations. 

The relaxed requirement is crucial to the application of the protocol forwarding feature in this implementation. We cannot conform Base to _LazySequenceForwarding. If it were possible to conform one protocol to another we could conform SequenceType to _LazySequenceForwarding, however it is doubtful that we would want that conformance. Despite this, it is clear to the compiler that Base does contain the necessary members for forwarding as it conforms to LazySequence which also declares all of the necessary members. 

This implementation is more robust and more clear:

We no longer leak any implementation details.
There is no chance of making a mistake in the implementation of the forwarded members. It is possible that a mistake could be made in the member declarations in _LazySequenceForwarding. However, if a mistake is made there a compiler error will result.
The set of forwarded methods is immediately clear, with the exception of _preprocessingPass because of its nontrivial Self requirement. Removing the limitation on nontrivial Self requirements is a highly desired improvement to this proposal or future enhancement to this feature.
The forwarded members cannot be overriden in an extension on LazySequence. If somebody attempts to do so it will result in an ambiguous use error at call sites.
LazyCollection

The relevant portion of the current implementation of LazyCollection looks like this (with comments removed and formatting tweaks):

// in LazyCollection.swift:

public struct LazyCollection<Base : CollectionType>
  : LazyCollectionType {

  public typealias Elements = Base
  public var elements: Elements { return _base }

  public typealias Index = Base.Index

  public init(_ base: Base) {
    self._base = base
  }

  internal var _base: Base
}

extension LazyCollection : SequenceType {

  public func generate() -> Base.Generator { return _base.generate() }

  public func underestimateCount() -> Int { return _base.underestimateCount() }

  public func _copyToNativeArrayBuffer() 
     -> _ContiguousArrayBuffer<Base.Generator.Element> {
    return _base._copyToNativeArrayBuffer()
  }

  public func _initializeTo(
    ptr: UnsafeMutablePointer<Base.Generator.Element>
  ) -> UnsafeMutablePointer<Base.Generator.Element> {
    return _base._initializeTo(ptr)
  }

  public func _customContainsEquatableElement(
    element: Base.Generator.Element
  ) -> Bool? { 
    return _base._customContainsEquatableElement(element)
  }
}

extension LazyCollection : CollectionType {

  public var startIndex: Base.Index {
    return _base.startIndex
  }

  public var endIndex: Base.Index {
    return _base.endIndex
  }

  public subscript(position: Base.Index) -> Base.Generator.Element {
    return _base[position]
  }

  public subscript(bounds: Range<Index>) -> LazyCollection<Slice<Base>> {
    return Slice(base: _base, bounds: bounds).lazy
  }

  public var isEmpty: Bool {
    return _base.isEmpty
  }

  public var count: Index.Distance {
    return _base.count
  }

  public func _customIndexOfEquatableElement(
    element: Base.Generator.Element
  ) -> Index?? {
    return _base._customIndexOfEquatableElement(element)
  }

  public var first: Base.Generator.Element? {
    return _base.first
  }
}
LazyCollection is using direct manual implementations of forwarding methods. It corresponds exactly to implementations that would be synthesized by the compiler under this proposal. This approach avoids some of the problems with the first approach:

It does not leak implementation details. This is good!
The forwarded members cannot be overriden.
Unfortunately it still has some drawbacks:

It is still possible to make mistakes in the manual forwarding implementations.
The set of forwarded methods is even less clear than under the first approach as they are now potentially interspersed with custom, nontrivial member implementations, such as subscript(bounds: Range<Index>) -> LazyCollection<Slice<Base>> in this example.
This approach requires reimplementing the forwarded members in every type which forwards them and is therefore less scalable than the first approach and this proposal. This may not matter for LazyCollection but it may well matter in other cases.
One intersting difference to note between LazySequence and LazyCollection is that LazySequence forwards three members which LazyCollection does not: map, filter, and _preprocessingPass. It is unclear whether this difference is intentional or not. 

This difference is particularly interesting in the case of _preprocessingPass. LazyCollectionappears to be using the default implementation for CollectionType in Collection.swift, which results in _base._preprocessingPass not getting called. It is not apparent why this behavior would be correct for LazyCollection and not for LazySequence.

I wonder if the difference in forwarded members is partly due to the fact that the set of forwarded members is not as clear as it could be. 

Here is an alternate approach implemented using the current proposal. It assumes that the same SequenceType members that are forwarded by LazySequence should also be forwarded by LazyCollection, allowing us to reuse the _LazySequenceForwarding protocol declared in the first example.

// _LazyCollectionForwarding redeclares the subset of the members of Indexable and CollectionType we wish to forward.
// The protocol is an implementation detail and is marked private.
private protocol _LazyCollectionForwarding: _LazySequenceForwarding {
  typealias Index : ForwardIndexType
  var startIndex: Index {get}
  var endIndex: Index {get}

  typealias _Element
  subscript(position: Index) -> _Element {get}

  var isEmpty: Bool { get }
  var count: Index.Distance { get }
  var first: Generator.Element? { get }

  @warn_unused_result
  func _customIndexOfEquatableElement(element: Generator.Element) -> Index??
}

public struct LazyCollection<Base : CollectionType>
  : LazyCollectionType {

  public typealias Elements = Base
  public var elements: Elements { return _base }

  public init(_ base: Base) {
    self._base = base
  }

  internal var _base: Base

  public forward _LazyCollectionForwarding to _base

  // It may be the case that LazyCollection should forward _preprocessingPass 
  // in the same fashion that LazySequence uses, which cannot yet be automated
  // under the current proposal.
}

extension LazyCollection : CollectionType {
  // This implementation is nontrivial and thus not forwarded
  public subscript(bounds: Range<Index>) -> LazyCollection<Slice<Base>> {
    return Slice(base: _base, bounds: bounds).lazy
  }
}
This approach to forwarding does not exhibit any of the issues with the manual approach and only takes about half as much code now that we are able to reuse the previous declaration of _LazySequenceForwarding.

NOTE: LazyMapCollection in Map.swift uses the same manual forwarding approach as LazyCollection to forward a handful of members and would therefore also be a candidate for adopting the new forwarding mechanism as well.

> 
>> With this protocol declared, I can then say something like
>>  
>> struct Foo {
>>     var ary: [Int]
>> }
>> 
>> extension Foo : SequenceTypeForwarder {
>>     var forwardedSequence: [Int] { return ary }
>> }
>>  
>> and my struct Foo now automatically implements SequenceType by forwarding to its variable `ary`.
>>  
>> The downside to this is it needs to be manually declared for each protocol. But I wager that most protocols actually aren't really amenable to forwarding anyway.
>>  
>> -Kevin Ballard
>> 
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-evolution
> 
> 
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20151230/a1d01c17/attachment-0001.html>