[swift-evolution] Proposal: Introduce User-defined "Dynamic Member Lookup" Types
Joe DeCapo
snoogansbc at gmail.com
Sun Dec 3 11:25:01 CST 2017
I like Chris's addition to the proposal about only allowing conformance on the original type definition, rather than requiring a class type.
https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438 <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438>
> In the discussion cycle, there was significant concern about abuse of this feature, particularly if someone retroactively conforms a type to DynamicMemberLookupProtocol. For this reason, the compiler only permits conformance of this protocol on the original type definition, not extensions. While the potential for abuse has never been a strong guiding principle for Swift features (and many features can be abused, e.g. operator overloading, emoji identifiers, AnyObject lookup, ImplicitlyUnwrappedOptional, and many more) there is a strong argument that dynamic behavior is a core property of a type that should be part of its declaration. If for some reason there becomes a reason to relax this requirement, we can evaluate that as a separate proposal based on its own merits. See the "Alternatives Considered" section below for further ways to reduce potential for abuse.
It would be a shame if structs were excluded from these capabilities.
> On Dec 3, 2017, at 11:18 AM, Jose Cheyo Jimenez via swift-evolution <swift-evolution at swift.org> wrote:
>
>
>
> On Dec 3, 2017, at 8:41 AM, David Hart via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>
>>
>>
>>> On 3 Dec 2017, at 04:11, Matthew Johnson via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>>
>>>
>>>
>>> Sent from my iPad
>>>
>>> On Dec 2, 2017, at 7:40 PM, Chris Lattner <clattner at nondot.org <mailto:clattner at nondot.org>> wrote:
>>>
>>>> On Dec 2, 2017, at 2:13 PM, Matthew Johnson <matthew at anandabits.com <mailto:matthew at anandabits.com>> wrote:
>>>>>> For all those reasons, we really do need something like AnyObject dispatch if we care about working with dynamically typed languages. The design I’m suggesting carefully cordons this off into its own struct type, so it doesn’t infect the rest of the type system, and is non-invasive in the compiler.
>>>>>
>>>>> I am quite familiar with dynamic languages and agree that this is necessary if we are going to fully open up access to these languages from Swift.
>>>>
>>>> Ok, then it appears you agree that something like anyobject dispatch is necessary for effective dynamic language interop.
>>>>
>>>>>>> I strongly urge you to reconsider the decision of that dynamic members must be made available with no indication at usage sites. An indication of dynamic lookup at usage sites aligns very well (IMO) with the rest of Swift (AnyObject lookup aside) by calling attention to code that requires extra care to get right.
>>>>>>
>>>>>> I don’t understand this. The proposal is fully type safe, and this approach is completely precedented by AnyObject. Swift’s type system supports many ways to express fallibility, and keeping those decisions orthogonal to this proposal is the right thing to do, because it allows the author of the type to decide what model makes sense for them.
>>>>>
>>>>> Allowing the author of the type to choose whether the mechanism is hidden or visible is exactly what I don’t want to allow. I think you have the right design regarding types and semantics - the author chooses. But I don’t want these calls to look like ordinary member lookup when I’m reading code.
>>>>>
>>>>> They inherently have a much greater chance of failure than ordinary member lookup. Further, authors are likely to choose immediate traps or nil IUO as failure modes as forcing users to deal with Optional on every call is likely to be untenable. I believe this behavior should be represented by some kind of syntax at the usage site. I don’t believe it is an undue burden. It would make the dynamic lookup semantic clear to all readers and would help to discourage abuse.
>>>>
>>>> I believe that adding explicit syntax would be counterproductive to your goals, and would not make dynamic lookup syntax more clear. I assume that you would also want the same thing for DynamicCallable too, and operator overloads, subscripts, and every other operation you perform on these values, since they all have the exact same behavior.
>>>>
>>>> If we required some syntax even as minimal as “foo.^bar” and "baz^(42)”, that change would turn this (which uses runtime failing or IUO return values like AnyObject):
>>>>
>>>> let np = Python.import("numpy")
>>>> let x = np.array([6, 7, 8])
>>>> let y = np.arange(24).reshape(2, 3, 4)
>>>>
>>>> let a = np.ones(3, dtype: np.int32)
>>>> let b = np.linspace(0, pi, 3)
>>>> let c = a+b
>>>> let d = np.exp(c)
>>>> print(d)
>>>>
>>>> into:
>>>>
>>>> let np = Python.import("numpy")
>>>> let b = np^.array^([6, 7, 8])
>>>> let y = np^.arange^(24)^.reshape^(2, 3, 4)
>>>>
>>>> let a = np^.ones^(3, dtype: np^.int32)
>>>> let b = np^.linspace^(0, pi, 3)
>>>> let c = a+^b
>>>> let d = np^.exp^(c)
>>>>
>>>> This does not improve clarity of code, it merely serves to obfuscate logic. It is immediately apparent from the APIs being used, the API style, and the static types (in Xcode or through static declarations) that this is all Python stuff.
>>>
>>> It may be immediately apparent when the types involved are obviously dynamic, such as in this example where Python.import is explicitly used. However, my concern is less about the intended use case of dynamic language interop than I am that this feature will be generally available to all types in Swift.
>>>
>>> This is big change from AnyObject dispatch. It opens up the dynamism to types and contexts that are not necessarily obviously using dynamic lookup, callable, etc. Maybe this won’t turn out to be a problem in practice but I still think it’s a legitimate concern.
>>
>> If dynamism if restricted to subclasses of a DynamicObject type, like Xiaodi suggested earlier, then we can protect ourselves from this dynamic dispatch being generally available to all types in Swift.
>
> I like the idea of a class type as the marker.
>
> If we did expose the protocols that DynamicObject conform to, then I would like to see an attribute that would be required at the declaration site.
>
> @dynamic
> class MyTypo: SomeDynamicProtocol {...}
>
> I also like the idea of having
> @dynamic subscript(...)
>
> I’d like the declaration site to be explicit.
>
> The calling site should just be dot syntax like AnyObject.
>
>>
>>>> When you start mixing in use of native Swift types like dictionaries (something we want to encourage because they are typed!) you end up with an inconsistent mismash where people would just try adding syntax or applying fixits continuously until the code builds.
>>>>
>>>> Beyond that, it is counterproductive to your goals, because it means that people are far less likely to use to use optional returns. Doing so (which produces a safer result) would cause a double tax in syntax, and would be a confusing jumble. I can’t bring myself to do the whole example above, one line - just converting member lookup syntax but not callable syntax - would end up:
>>>>
>>>> let y = np^.arange?^(24)^.reshape^?(2, 3, 4)
>>>>
>>>> If you made DynamicCallable also return optional it would be:
>>>>
>>>> let y = np^.arange?^(24)?^.reshape^?(2, 3, 4)!
>>>>
>>>> or something. This is such madness that no one would do that.
>>>
>>> Yes, I agree. The interaction with optional chaining makes it unworkable. I hadn’t thought that all the way through. Thank you for indulging in the discussion about this idea.
>>>
>>> I’m uncertain what the right answer is. I’m still not really comfortable with opening up dynamic lookup to any user-defined type without some way to indicate to readers that dynamic lookup is happening in a piece of code. Maybe there is a less localized annotation that would indicate dynamic lookup is in effect for a larger chunk of code.
>>
>> I think that making dynamic calls syntactically different to readers is going too far in the direction of safety. Plus, it makes the language inconsistent as we already have AnyObject dispatch with exactly the same syntax. But I understand why you would want it if *any* type could end up being conformed to a dynamic lookupable/callable protocol. Like said above, I think that a DynamicObject type is enough protection to not bother making the syntax heavier at the point of use.
>>
>>> One idea that hasn’t been explored yet is introducing a dynamic lookup effect. That would provide clean syntax at the expression level while still making it clear that dynamic lookup is happening. This probably isn’t the right approach as it would be viral and wouldn’t even indicate that the code in a specific function body even contains a dynamic lookup. I mention it mostly to broaden the conversation and thought space about what kind of approach might address my concerns without cluttering up expressions.
>>>
>>> Another, potentially more viable approach would be to use an approach similar to try and the proposed async of a statement modifier (possibly also allowed at the expression level as with try and async). This could be used in conjunction with an effect as mentioned above but could also be used independently even though there isn’t a current precedent for that. Your example written with this approach might be:
>>>
>>> let np = Python.import("numpy")
>>> let b = dynamic np.array([6, 7, 8])
>>> let y = dynamic np.arange(24).reshape(2, 3, 4)
>>>
>>> let a = dynamic np.ones(3, dtype: np.int32)
>>> let b = dynamic np.linspace(0, pi, 3)
>>> let c = dynamic a + b
>>> let d = dynamic np.exp(c)
>>>
>>> Note: `dynamic` is just a straw man here. This is obviously a lot of boilerplate repetition of the same modifier. I would also be perfectly happy allowing this annotation to apply to a block of code or even a whole function body. Then it might be:
>>>
>>> dynamic {
>>> let np = Python.import("numpy")
>>> let b = np.array([6, 7, 8])
>>> let y = np.arange(24).reshape(2, 3, 4)
>>>
>>> let a = np.ones(3, dtype: np.int32)
>>> let b = np.linspace(0, pi, 3)
>>> let c = a + b
>>> let d = np.exp(c)
>>> }
>>>
>>> The most obvious objection to this is that it introduces nesting and does so in a way that only very indirectly influences control flow (through the potential for dynamic failure).
>>>
>>> My objective certainly isn’t to make code ugly and obfuscate logic. It is simply to make the usage of dynamic features that are prone to failure at runtime immediately clear to a reader. This should be as lightweight as possible while still providing valuable information to the reader.
>>>
>>>>
>>>>
>>>>>> Swift already has a dynamic member lookup feature, "AnyObject dispatch" which does not use additional punctuation, so this would break precedent.
>>>>> I would prefer if dynamic lookup were visible with AnyObject as well. For that reason I don’t believe it makes a good precedent to follow. In fact, I would prefer to see us go the other direction and perhaps even consider revising dynamic lookup syntax for AnyObject in the future.
>>>>
>>>> This is definitely not going to happen. The change Doug mentioned is to have AnyObject lookup return optional instead of IUO, which forces ? on the clients. Adding other syntax (like you’re suggesting) is certainly not going to happen.
>>>>
>>>> The entire point of AnyObject dispatch is to improve syntactic elegance and clarity of code using it. There is no other reason to exist. Making code that uses it syntactically onerous completely defeats the point of having it in the first place, as I’ve mentioned before.
>>>
>>> I agree. The interaction with optional chaining is significantly more onerous than I had considered. I should have worked through an example on my own. I do hope you will consider a less localized approach to usage-site annotation though.
>>>
>>>>
>>>>
>>>> Furthermore, your premise that Swift does not have invisibly failable operations is plainly wrong. Array subscript and even integer addition can fail.
>>>
>>> I am well aware of these behaviors. The difference IMO is that programmers tend to be well aware of these preconditions even if they also often choose to ignore them. Dynamic lookup will not be so clear. This is especially true if people use it with types that also have an API available through static lookup.
>>>
>>>> Even the behavior of AnyObject was carefully designed and considered, and were really really good reasons for it returning IUO.
>>>
>>> I am not trying to call into question the choices made in the past. Swift wouldn’t be the great language with a bright future that it is today without an incredibly successful migration of a large user base from Objective-C to Swift. This is a huge accomplishment and couldn’t have happened without making really good decisions about some really hard tradeoffs.
>>>
>>>>
>>>> -Chris
>>>>
>>> _______________________________________________
>>> swift-evolution mailing list
>>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20171203/493622f7/attachment.html>
More information about the swift-evolution
mailing list