[swift-evolution] [Pitch] Introduce User-defined "Dynamic Member Lookup" Types
Nathan Gray
n8gray at n8gray.org
Wed Nov 15 15:12:48 CST 2017
How would this work for a property access that caused a python exception to
be thrown? Are we assuming that only "good" non-throwing properties will
be bridged in this way?
```
>>> x = "hello"
>>> x.foobar
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'str' object has no attribute 'foobar'
```
On Tue, Nov 14, 2017 at 11:29 PM, Chris Lattner via swift-evolution <
swift-evolution at swift.org> wrote:
> Hi All,
>
> As a peer to the DynamicCallable proposal (https://gist.github.com/
> lattner/a6257f425f55fe39fd6ac7a2354d693d), I’d like to get your feedback
> on making member lookup dynamically extensible. My primary motivation is
> to improve interoperability with dynamic languages like Python, Perl, Ruby,
> Javascript, etc, but there are other use cases (e.g. when working with
> untyped JSON).
>
> In addition to being a high impact on expressivity of Swift, I believe an
> implementation can be done in a way with changes that are localized, and
> thus not have significant impact on the maintainability of the compiler as
> a whole. Once the pitch phase of this proposal helps refine the details,
> I’ll be happy to prepare an implementation for consideration.
>
> In case it is useful, I’m working on cleaning up my current prototype
> Python bindings. I’ll share them in the next day or two in case they are
> useful to provide context. It is amazingly simple: less than 500 lines of
> Swift code (plus some small additional C header glue to work around clang
> importer limitations) enables impressive interoperability. The only
> problems are the verbosity addressed by this proposal and the peer
> DynamicCallable proposal.
>
>
> Here is the canonical proposal URL:
> https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438
>
> A snapshot of the proposal is included below in case it is useful. Thanks
> in advance for help improving the proposal!
>
> -Chris
>
>
> Introduce User-defined "Dynamic Member Lookup" Types
>
> - Proposal: SE-NNNN
> <https://gist.github.com/lattner/NNNN-DynamicMemberLookup.md>
> - Author: Chris Lattner <https://github.com/lattner>
> - Review Manager: TBD
> - Status: Awaiting implementation
>
>
> <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#introduction>
> Introduction
>
> This proposal introduces a new DynamicMemberLookupProtocol type to the
> standard library. Types that conform to it provide "dot" syntax for
> arbitrary names which are resolved at runtime. It is simple syntactic sugar
> which allows the user to write:
>
> a = someValue.someMember
> someValue.someMember = a
>
> and have it be interpreted by the compiler as:
>
> a = someValue[dynamicMember: "someMember"]
> someValue[dynamicMember: "someMember"] = a
>
> Many other languages have analogous features (e.g. the composition of
> Objective-C's explicit properties
> <https://developer.apple.com/library/content/documentation/General/Conceptual/DevPedia-CocoaCore/DeclaredProperty.html> and
> underlying messaging infrastructure
> <https://developer.apple.com/library/content/documentation/Cocoa/Conceptual/ObjCRuntimeGuide/Articles/ocrtHowMessagingWorks.html>).
> This sort of functionality is great for implementing dynamic language
> interoperability, dynamic proxy APIs
> <https://developer.apple.com/library/content/documentation/Cocoa/Conceptual/ObjCRuntimeGuide/Articles/ocrtForwarding.html>,
> and other APIs (e.g. for JSON processing).
>
> Swift-evolution thread: Discussion thread topic for that proposal
> <https://lists.swift.org/pipermail/swift-evolution/>
>
> <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#motivation-and-context>Motivation
> and Context
>
> Swift is well known for being exceptional at interworking with existing C
> and Objective-C APIs, but its support for calling APIs written in scripting
> languages like Python, Perl, and Ruby is quite lacking.
>
> C and Objective-C are integrated into Swift by expending a heroic amount
> of effort into integrating Clang ASTs, remapping existing APIs in an
> attempt to feel "Swifty", and by providing a large number of attributes and
> customization points for changing the behavior of this integration when
> writing an Objective-C header. The end result of this massive investment of
> effort is that Swift provides a *better* experience when programming
> against these legacy APIs than Objective-C itself did.
>
> When considering the space of dynamic languages, three things are clear:
> 1) there are several different languages of interest, and they each have
> significant interest in different quarters: for example, Python is big in
> data science and machine learning, Ruby is popular for building server side
> apps, and even Perl is in still widely used. 2) These languages have
> decades of library building behind them, sometimes with significant
> communities <https://pandas.pydata.org/> and 3) there are one or two
> orders of magnitude more users of these libraries than there are people
> currently using Swift.
>
> While it is theoretically possible to expend the same level of effort on
> each of these languages and communities as has been spent on Objective-C,
> it is quite clear that this would both ineffective as well as bad for
> Swift: It would be ineffective, because the Swift community has not
> leverage over these communities to force auditing and annotation of their
> APIs. It would be bad for Swift because it would require a ton of
> language-specific support (and a number of third-party dependencies) onto
> the compiler and runtime, each of which makes the implementation
> significantly more complex, difficult to reason about, difficult to
> maintain, and difficult to test the supported permutations. In short, we'd
> end up with a mess.
>
> Fortunately for us, these scripting languages provide an extremely dynamic
> programming model where almost everything is discovered at runtime, and
> many of them are explicitly designed to be embedded into other languages
> and applications. This aspect allows us to embed APIs from these languages
> directly into Swift with no language support at all - without not the level
> of effort, integration, and invasiveness that Objective-C has benefited
> from. Instead of invasive importer work, we can write some
> language-specific Swift APIs, and leave the interop details to that library.
>
> This offers a significant opportunity for us - the Swift community can
> "embrace" these dynamic language APIs (making them directly available in
> Swift) which reduces the pain of someone moving from one of those languages
> into Swift. It is true that the APIs thus provided will not feel "Swifty",
> but if that becomes a significant problem for any one API, then the
> community behind it can evaluate the problem and come up with a solution
> (either a Swift wrapper for the dynamic language, or a from-scratch Swift
> reimplementation of the desired API). In any case, if/when we face this
> challenge, it will be a good thing: we'll know that we've won a significant
> new community of Swift developers.
>
> While it is possible today to import (nearly) arbitrary dynamic language
> APIs into Swift today, the resultant API is unusable for two major reasons:
> member lookup is too verbose to be acceptable, and calling behavior is
> similarly too verbose to be acceptable. As such, we seek to provide two
> "syntactic sugar" features that solve this problem. These sugars are
> specifically designed to be dynamic language independent and, indeed,
> independent of dynamic languages at all: we can imagine other usage for the
> same primitive capabilities.
>
> The two proposals in question are the introduction of the DynamicCallable
> <https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d> protocol
> and a related DynamicMemberLookupProtocol proposal (this proposal). With
> these two extensions, we think we can eliminate the need for invasive
> importer magic by making interoperability with dynamic languages ergonomic
> enough to be acceptable.
>
> For example, consider this Python code:
>
> class Dog:
> def __init__(self, name):
> self.name = name
> self.tricks = [] # creates a new empty list for each dog
>
> def add_trick(self, trick):
> self.tricks.append(trick)
>
> we would like to be able to use this from Swift like this (the comments
> show the corresponding syntax you would use in Python):
>
> // import DogModule // import DogModule.Dog as Dog // an alternate let Dog = Python.import(“DogModule.Dog") // dog = Dog("Brianna") let dog = Dog("Brianna") // dog.add_trick("Roll over") dog.add_trick("Roll over") // dog2 = Dog("Kaylee").add_trick("snore") let dog2 = Dog("Kaylee").add_trick("snore")
>
> Of course, this would also apply to standard Python APIs as well. Here is
> an example working with the Python pickleAPI and the builtin Python
> function open:
>
> // import pickle let pickle = Python.import("pickle")
>
> // file = open(filename) let file = Python.open(filename)
>
> // blob = file.read() let blob = file.read()
>
> // result = pickle.loads(blob) let result = pickle.loads(blob)
>
> This can all be expressed today as library functionality written in Swift,
> but without this proposal, the code required is unnecessarily verbose and
> gross. Without it (but *with* the related DynamicCallable proposal
> <https://gist.github.com/lattner/a6257f425f55fe39fd6ac7a2354d693d> the
> code would have explicit member lookups all over the place:
>
> // import pickle let pickle = Python.get(member: "import")("pickle")
>
> // file = open(filename) let file = Python.get(member: "open")(filename)
>
> // blob = file.read() let blob = file.get(member: "read")()
>
> // result = pickle.loads(blob) let result = pickle.get(member: "loads")(blob)
>
> // dog2 = Dog("Kaylee").add_trick("snore") let dog2 = Dog("Kaylee").get(member: "add_trick")("snore")
>
> While this is a syntactic sugar proposal, we believe that this expands
> Swift to be usable in important new domains. In addition to dynamic
> language interoperability, this sort of functionality is useful for other
> APIs, e.g. when working with dynamically typed unstructured data like JSON,
> which could provide an API like jsonValue?.jsonField1?.jsonField2where
> each field is dynamically looked up.
>
> <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#proposed-solution>Proposed
> solution
>
> We propose introducing this protocol to the standard library:
>
> protocol DynamicMemberLookupProtocol {
> associatedtype DynamicMemberLookupValue
>
> subscript(dynamicMember name: String) -> DynamicMemberLookupValue { get set }
> }
>
> It also extends the language such that member lookup syntax (x.y) - when
> it otherwise fails (because there is no member y defined on the type of x)
> and when applied to a value which conforms to DynamicMemberLookupProtocol-
> is accepted and transformed into a call to the subscript in the protocol.
> This ensures that no member lookup on such a type ever fails.
>
> <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#example-usage>Example
> Usage
>
> While there are many potential uses of this sort of API (e.g. resolving
> JSON members to named results, producing optional bindings) a motivating
> example comes from a prototype Python interoperability layer. There are
> many ways to implement this, and the details are not particularly
> important, but it is perhaps useful to know that this is directly useful to
> address the motivation section described above. Given a currency type of
> PyVal (and a conforming implementation named PyRef), an implementation
> may look like:
>
> extension PyVal {
> subscript(dynamicMember member: String) -> PyVal {
> get {
> let result = PyObject_GetAttrString(borrowedPyObject, member)!
> return PyRef(owned: result) // PyObject_GetAttrString returns +1 result. }
> set {
> PyObject_SetAttrString(borrowedPyObject, member,
> newValue.toPython().borrowedPyObject)
> }
> }
> }
>
>
> <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#source-compatibility>Source
> compatibility
>
> This is a strictly additive proposal with no source breaking changes.
>
> <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#effect-on-abi-stability>Effect
> on ABI stability
>
> This is a strictly additive proposal with no ABI breaking changes.
>
> <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#effect-on-api-resilience>Effect
> on API resilience
>
> This has no impact on API resilience which is not already captured by
> other language features.
>
> <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#alternatives-considered>Alternatives
> considered
>
> A few alternatives were considered:
>
> <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#add-ability-to-provide-read-only-members>Add
> ability to provide read-only members
>
> The implementation above does not allow an implementation to statically
> reject members that are read-only. If this was important to add, we could
> add another protocol to model this, along the lines of:
>
> protocol DynamicMemberLookupGettableProtocol {
> associatedtype DynamicMemberLookupValue
>
> // gettable only subscript(dynamicMember name: String) -> DynamicMemberLookupValue { get }
> }
> protocol DynamicMemberLookupProtocol : DynamicMemberLookupGettableProtocol {
> // gettable and settable. subscript(dynamicMember name: String) -> DynamicMemberLookupValue { get set }
> }
>
> This would allow a type to implement one or the other based on their
> capabilities. This proposal starts with a very simple design based on the
> requirements of dynamic languages (which have no apparent immutability
> model), but if there is demand for this (e.g. because we want input JSON
> values to be gettable but not settalbe), the author is happy to switch to
> this more general model.
> <https://gist.github.com/lattner/b016e1cf86c43732c8d82f90e5ae5438#naming>
> Naming
> There is a lot of grounds to debate naming of the protocol and methods in
> this type. Suggestions (along with rationale to support them) are more than
> welcome.
>
>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
>
--
Functional Programmer, iOS Developer, Surfs Poorly
http://twitter.com/n8gray
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20171115/dc40742c/attachment.html>
More information about the swift-evolution
mailing list