[swift-evolution] [Pitch] Introduce User-defined "Dynamic Member Lookup" Types

[Brent Royal-Gordon]

> On Nov 16, 2017, at 1:44 PM, Paul Cantrell <paul at bustoutsolutions.com> wrote:
> 
>> On Nov 16, 2017, at 12:00 AM, Brent Royal-Gordon via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>> 
>> * Ruby and Perl don't have the "call a method by fetching a closure property and invoking it" behavior you're relying on here. Instead, Ruby has a syntax for settable "overloads" of methods (i.e. you can write `def someMember` and `def someMember= (newValue)`), while Perl supports lvalue methods (but sometimes uses getter and setter method pairs instead). How do you envision these behaviors being bridged to Swift? I worry that this protocol may not be sufficient, and that we may need a design which can distinguish between looking up methods and looking up properties.
> 
> I’ve never pried the lid of Ruby’s implementation of method dispatch, but I’m pretty sure that if foo defines a bar method, then
> 
>     foo.bar(…args…)
> 
> is fully equivalent to:
> 
>     foo.method(:bar).call(…args…)
> 
> IOW, there is an intermediate Method object that would fit the shape of the proposed callable protocol.
> 
> If foo instead doesn’t actually declare the bar method, but instead handles it via method_missing or __send__, then foo.method(:bar) raises an exception. However, it would be trivial to write a deferred invocation wrapper that also fits the shape of the proposed protocols and calls foo.send(“bar”, …args…) at the appropriate time.
> 
> In short, I don’t think there’s a problem here.

True. Ruby doesn't dispatch everything through a Method object…but I guess Swift sort of does, and we're bridging semantics into Swift here.

> In the example you bring up:
> 
>> you can write `def someMember` and `def someMember= (newValue)`)
> 
> …there is no overloading. The = is _part of the method name_, i.e. there is a `someMember` method and a `someMember=` method.

You're right—I was speaking imprecisely when I used the word "overloading". Nevertheless, Ruby doesn't quite directly interpret `x.someMember = y` as `x.someMember= (y)`—it supports operators like `+=`, which do a getter-operation-setter dance.

> The following are equivalent:
> 
>     foo.bar = 3  # just sugar
>     foo.bar=(3)
>     foo.send("bar=", 3)
> 
> Ruby allows ?, !, and = as the last char of method names, and AFAIK other than the special sugar around setters, they are just parts of the method name with no further semantic significance.

You're correct that, with this design, you could access Ruby accessors from Swift with syntax like:

	myObj.name()
	myObj.`name=`("Chris")		// If we loosened the characters allowed in backticks

My point is simply that this is a poor mapping, for much the same reason `dog["add_trick"].call(…)` is a poor mapping. It's technically correct and exposes the functionality, but it's awkward and doesn't match the user's mental model.

If we had separate subscripts for methods and properties, then the property subscript could immediately call the appropriate getters and setters, while the method subscript could return a ready-to-call `Method` object. This would prevent you from fetching uncalled methods using the `x.method` syntax (as opposed to `x.method(_:)`, which could be made to work), but that seems a lot better than a mapping that's technically correct but breaks the mental model.

>> * Let's step away from bridging entirely and just think about Swift for a moment. There are cases where we'd like to make *semi*-dynamic proxies which wrap another type and allow operations based on what's statically known about that type. Think, for example, of the appearance proxy in UIKit: This is an object attached to UIView subclasses which lets you (in essence) set default values for all instances. We currently just pretend it's an instance of `Self`, which mostly works because of Objective-C, but a Swift-native version would probably prefer to return a `UIAppearance<Self>` object which used its knowledge of `Self` to expose `Self`'s properties on itself. Is there a way we could design this feature, or a related feature, to cover that kind of use case? That is, to allow a limited set of keys—perhaps even key-path-based when you want static control—with a different type for each key, *or* to allow any key with some common type, depending on your type's needs?
> 
> Per my question about whether native methods shadow dynamic ones, one might be able to achieve some of this using a mix of statically typed, statically declared methods + dynamic members.

So, let me sketch a vague idea of how this might work. This is definitely not fully baked, but it might give you an idea.

Imagine you want to write an ORM. Its root class, `Record`, should expose a property for each field in a given record. You could make the ORM generate an enum like this:

	enum PersonProperty: String {
		case id, name, birthDate, address
	}

And then make `Record` dynamically gain a property for each enum case by defining the `subscript(additionalProperty:)` subscript:

	class Record<Property> where Property: RawRepresentable, Property.RawValue == String {
		…
		subscript(additionalProperty property: Property) -> Any {
			get { … }
			set { … }
		}
	}

Swift will notice this overload and essentially augment `Record<T>` with a property for each case of `T`, giving it type `Any`. Attempting to use any of these properties will pass through this subscript. (Presumably, if you generated the key path `\Record<PersonProperty>.name`, you'd actually end up with the key path for `\Record<PersonProperty>.[additionalProperty: PersonProperty.name]`.)

With a more sophisticated (and convoluted) design, our ORM could give the fields more specific types than `Any`:

	struct Property<RecordType: Record, Value> {
		let name: String
	}
	extension Property where RecordType == Person, Value == Int {
		static let id = Property(name: "id")
	}
	extension Property where RecordType == Person, Value == String {
		static let name = Property(name: "name")
		static let address = Property(name: "address")
	}
	extension Property where RecordType == Person, Value == Date {
		static let birthDate = Property(name: "birthDate")
	}

And we could then expose typed, automatically created properties:

	protocol Record {
		…
	}
	extension Record {
		subscript<T>(additionalProperty property: Property<Self, T>) -> T {
			get { … }
			set { … }
		}
	}
	
	struct Person: Record {}

That would work for arbitrary fixed sets of properties, but we can extend this to wrapper types. Imagine you want to write the `UIAppearance` class I mentioned previously. (Actually, we'll call it `UIAppearanceProxy` to avoid names already used in UIKit.) Your basic structure looks like this:

	class UIAppearanceProxy<View: UIAppearance> {
		let containers: [UIAppearanceContainer.Type]
		let traits: UITraitCollection
		
		var properties: [PartialKeyPath<View>: Any] = [:]
	}

Now, to make all properties of the `View` class settable on this class, you can overload the `additionalProperty` subscript to accept `View` keypaths:

	extension UIAppearanceProxy {
		subscript<T>(additionalProperty viewKeyPath: KeyPath<View, T>) -> T? {
			get {
				return properties[viewKeyPath] as! T?
			}
			set {
				properties[viewKeyPath] = newValue
			}
		}
	}

Swift would notice this overload and allow any `View` property to be set on `UIAppearanceProxy`. The subscript gives its return type as `T?`, so when it does so, it will add an extra level of optionality—since `UITextField.font` is of type `UIFont?`, `UIAppearanceProxy<UITextField>.font` will be of type `UIFont??`. You can modify types like that in any way the type system permits.

For the totally dynamic use case, like Python, you could overload `subscript(additionalProperty:)` to take a `String` (or any other `ExpressibleByStringLiteral` type, like a `RubySymbol`):

	extension PyVal {
		subscript(additionalProperty member: String) -> PyVal {
			get {
				let result = PyObject_GetAttrString(borrowedPyObject, member)!
				return PyRef(owned: result)  // PyObject_GetAttrString returns +1 result.
			}
			set {
				PyObject_SetAttrString(borrowedPyObject, member, 
					newValue.toPython().borrowedPyObject)
			}
		}
	}

Swift would map any completely unknown property to this key path, if present. 

* * *

Methods, I think, could be handled analogously. If you wanted a fixed but arbitrary set of automatically-"generated" methods, you might say something like:

	enum PersonMethod: Method {
		case insertJob(Job.Type, at: Int.Type)
		case removeJob(at: Int.Type)

		func implementation<P>(for record: Record<P, Self>) -> (Any...) -> Any { … }
	}

	class Record<PropertyType: Property, MethodType: Method> … {
		…
		subscript (additionalMethod method: MethodType) -> (Any...) -> Any {
			get { return method.implementation(for: self) }
		}
	}

Swift will notice this overload and essentially augment `Record`'s methods with ones corresponding to the static methods of `MemberType`, so long as all of their parameters are metatypes. Attempting to use any of these methods will pass through this subscript. So `myPerson.insertJob(myJob, at: 0)` gets compiled into `myPerson[additionalMethods: PersonMethod.insertJob(Job.self, at: Int.self)](myJob, 0)`.

If you wanted stronger typing, you could do something like this:

	struct Method<RecordType: Record, ParameterTypes, ReturnType>: Method {
		enum Behavior {
			case insert
			case remove
		}
		
		let field: String
		let behavior: Behavior
		
		func implementation(for record: RecordType) -> (ParameterTypes) -> ReturnType { … }
	}
	extension PersonMethod where RecordType == Person, ParameterTypes == (Job.Type, Int.Type), ReturnType == Job {
		static func insertJob(Job.Type, at _: Int.Type) -> PersonMethod {
			return PersonMethod(field: "job", behavior: .insert)
		}
	}
	extension PersonMethod where RecordType == Person, ParameterTypes == (Int.Type), ReturnType == Void {
		static func removeJob(at _: Int.Type) -> PersonMethod {
			return PersonMethod(field: "job", behavior: .remove)
		}
	}

	protocol Record {
		…
	}
	extension Record {
		subscript <ParameterTypes, ReturnType>(additionalMethod method: Method<Self, ParameterTypes, ReturnType>) -> (ParameterTypes) -> ReturnType {
			get { return method.implementation(for: self) }
		}
	}

(This would require that `subscript(additionalMethod:)` be allowed to pack parameter types into a single tuple, unless we were willing to wait for variadic generics.)

We could perhaps add a special case for wrapping methods which would leverage the unbound methods on a type (or whatever future replacement for that feature we devise):

	class UIAppearanceProxy<View: UIAppearance> {
		let containers: [UIAppearanceContainer.Type]
		let traits: UITraitCollection
		
		var properties: [PartialKeyPath<View>: Any] = [:]
		var calls: [(View) -> Void]
	}

	extension UIAppearanceProxy {
		subscript <ParameterTypes, ReturnType>(additionalMethod method: (View) -> (ParameterTypes) -> ReturnType) -> (ParameterTypes) -> Void {
			return { params in
				self.calls.append({ view in _ = method(view)(params) })
			}
		}
	}

Note that here, we're completely changing the type of the method we return! Since we're deferring the call until later, we throw away the original return type and substitute `Void` instead. When you try to pull similar tricks in Objective-C (for instance, with `-[NSUndoManager prepareWithInvocationTarget:]`), you end up with invalid return values.

And of course, for Python and the like, you can use `ExpressibleByStringLiteral` types:

	extension PyVal {
		subscript (additionalMethod member: String) -> (PythonConvertible...) -> PyVal {
			let (baseName, labels) = splitMethodName(member)
			
			// Python has a unified namespace for methods and properties, so we'll just leverage the property lookup.
			let method = self[additionalProperty: baseName]
			return { arguments in method.dynamicCall(arguments: zip(labels, arguments)) }
		}
		
		private func splitMethodName(_ name: String) -> (baseName: Substring, labels: [Substring]) {
			guard let argListIndex = name.index(of: "(") else {
				return (name, [])
			}
			
			let argListSansParens = name[argListIndex...].dropFirst().dropLast()
			return (name[...argListIndex], argListSansParens.split(separator: ":")
		}
	}

For types using both mechanisms, `subscript(additionalMethod:)` would be called whenever there was some sign—like a parameter list (with or without arguments) or an assignment to a function type—that we were looking up a method; `subscript(additionalProperty:)` would be called when there was no such sign. Python doesn't really need the two cases to be split up, but this would help a Ruby bridge differentiate between property and method accesses, and I think it would help with the more static cases as well.

* * *

This is a complex design, but I really like how much ground it covers. Python bridging is just the beginning—it can cover a *ton* of use cases with only a couple of variations. By not using protocols, it permits overloading and the use of generics tricks that might not be accessible to a protocol. 

To begin with, we could support only `subscript(additionalProperty: String)` and `subscript(additionalMethod: String)`, and then expand over time to cover the more static use cases. That would get 

-- 
Brent Royal-Gordon
Architechies

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20171116/c88dca31/attachment.html>