<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">I’ve been going back and forth on whether I’m for or against the current revision of the proposal and I’d have to say that I am in favor of it as is for the following reasons (many of which have already been stated):<div class=""><br class=""></div><div class="">It was stated early on in this proposal that Python is not going to be given the same first class support that C/Objective-C have in the compiler. It would be unreasonable and would set a precedent that all languages would get the same treatment when the community wants to add interoperability. Many arguments have been made that we NEED to have static typing for dynamic languages. I don’t see static typing coming without first class compiler support. </div><div class=""><br class=""></div><div class="">Also, the very nature of these dynamic languages means static typing would violate basic principles of the dynamic language. I think it would be much worse and scare away python developers who are suddenly forced to statically type things that they may know to be dynamic types in their python APIs. Of course, the other side of the coin is that swift developers would be scared away by the lack of static typing when using Python interop. I see the latter as a non-issue though because anyone using python knows to expect things without static types. </div><div class=""><br class=""></div><div class="">Forcing static types where there are no static types sounds like a really bad idea to me. So I’m ok with the proposed PyVal type for all python types. Extensions can be written to make casting a PyVal object to the various swift types easy (if/when desired).</div><div class=""><br class=""></div><div class="">There has also been much discussion on the potential for abuse. There was an example given of adding conformance to NSObject I believe. Chris and Xiaodi both proposed solutions to this that would minimize (and possibly eliminate) the potential for abuse. </div><div class=""><br class=""></div><div class="">However, there will always be potential for a feature to be misused. The potential does not mean it is an absolute given that it will be misused (I’m an optimist if you can’t tell ;).</div><div class=""><br class=""></div><div class="">I don’t view this protocol as a beginner user feature. This is not a protocol that will be commonly used or even commonly desired. It has a very specific set of uses and I strongly believe that the community will continue to do a good job at suggesting the right tool for the right job. The advanced users that are aware of this protocol will most likely properly recommend this protocol when applicable. Your average Joe Shmoe isn’t going to even be aware of this. Those who are curious, will probably ask about it. There will be StackOverflow responses describing it and what it does and when/why to use it. (This is just my two cents based on what I’ve witnessed in the swift community and on StackOverflow in the past). I’m sure that if the documentation on this protocol says “Used to provide interoperability with dynamic languages” then most users are going to steer clear of it unless they are writing an interop layer with a dynamic language.</div><div class=""><br class=""></div><div class="">I don’t think the goal here is to write Python (or ruby or javascript or [dynamic language]) in swift. The aim of this proposal it to make it easy and possible to use python from swift. I don’t expect to write a full blown python project in swift. I expect to call the few python APIs from module xyz and be done with python. I’m trying to write a swift project not a python one. If I wanted to use python extensively then I would just stick with python (I work as a python developer and Python is definitely not my language of choice, but it does have its upsides). I don’t want to write my own swift implementation for something where a python module already exists so I import the python module and do the few things with it that I actually need. I don’t need or expect IDE integration with the language I’m using at that moment. I can read docs for my 2-3 python API calls while I’m developing it.</div><div class=""><br class=""></div><div class="">Even with this proposal, people could write their own swift libraries that are just statically typed wrappers around a python library. This proposal does not limit people in any way. It does however, open the door for interoperability with a large set of dynamic languages.<br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Dec 1, 2017, at 10:37 AM, Chris Lattner via swift-evolution <<a href="mailto:swift-evolution@swift.org" class="">swift-evolution@swift.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">On Dec 1, 2017, at 12:26 AM, Douglas Gregor <<a href="mailto:dgregor@apple.com" class="">dgregor@apple.com</a>> wrote:<div class=""><blockquote type="cite" class=""><div class=""><meta http-equiv="Content-Type" content="text/html; charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><blockquote type="cite" class=""><div class="">On Nov 30, 2017, at 10:05 PM, Chris Lattner <<a href="mailto:clattner@nondot.org" class="">clattner@nondot.org</a>> wrote:</div><div class=""><meta http-equiv="Content-Type" content="text/html charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">Hi Doug,</div><div class=""><br class=""></div><div class="">Thank you for the detailed email. I have been traveling today, so I haven’t had a chance to respond until now. I haven’t read the down-thread emails, so I apologize if any of this was already discussed:</div><div class=""><br class=""></div><div class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class="">I think better interoperability with Python (and other OO languages in widespread use) is a good goal, and I agree that the implementation of the feature described is straight-forward and not terribly invasive in the compiler.</div></div></div></blockquote><div class=""><br class=""></div><div class="">Fantastic, I’m really pleased to hear that! I only care about solving the problem, so if we can find a good technical solution to the problems than I’ll be happy.</div><div class=""></div></div></div></div></blockquote></div></div></div></blockquote><div class=""><br class=""></div><div class="">After thinking about your email a bit more, I think I might understand the disconnect we’re having. I think we have different versions in mind of what “success” looks like:</div><div class=""><br class=""></div><div class="">I believe your view is that people start using Python APIs from their Swift code, but gradually add type annotations (either inline or in a sidecar database) to make those APIs progressively more Swifty. New features will be added to Python (its type system, compilers, databases for common APIs, etc) to improve this interoperability, along the lines of what we’ve done for Objective-C over the years. Fast forward several years, and large Python libraries would be nice to use from Swift - perhaps nicer than they are to use from Python itself. This view aligns with what happened with Objective-C <-> Swift interoperability.</div><div class=""><br class=""></div><div class="">In contrast, I’m specifically interested in developers in certain large domains (e.g. data science and ML) which are “forced” to use Python because that is where all the libraries are. I have spoken to many of these sorts of people, and a large number of them really *dislike* using Python for all the obvious reasons (including the tooling issues you point out). My view of success is that we allow them to write all of *their code* in Swift, which will lead to a massive quality of life benefit for these frustrated people. You’re right that they will still chaff when using imported Python APIs (which don’t feel “swifty” for LOTS of reasons - e.g. method naming and design patterns), but my view is that this will provide incentive for these people to provide a proper Swift implementation for these libraries over time.</div><div class=""><br class=""></div><div class="">In short, your end game is a pervasively intertwined Swift/Python world (like ObjC and Swift are). My view is that Swift holds Python at arm's length, and wins over the hearts and minds of developers, leading to new Swift APIs designed for Swift.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">I’m not sure if you agree with that portrayal of your position, and if not, I’m sorry and will try to understand another way. However, if I’m close, then I have several concerns about that vision and don’t believe the end-game is achievable or better than what I’m proposing. Consider:</div><div class=""><br class=""></div><div class="">- I don’t think there will be a lot of success getting people who *actually love* Python to use Swift, unless there is already an extrinsic reason for them to use it.</div><div class="">- Type hints are not widely used in Python, and there are believable reasons that they won’t ever be. Because they are specifically poorly suited for libraries, their use seems to be in “user’s own code” - but my proposal solves this already! See below for examples.</div><div class="">- Even if type hints were widely adopted, it would require massive extensions to them and to Python to make them be "good enough” to provide value for the things you’re envisioning. Analogs to instancetype, objc generics, lots of the C macros, and many of the other things we’ve added to ObjC would have to be added.</div><div class="">- The Python community has no reason to do this work for the Swift community, or accept changes to Python that are required to make this actually great.</div><div class="">- Beyond the core type system, the Python and Objective-C languages work extremely differently in other ways (e.g. lack of umbrella headers making AnyObject-style dispatch questionable).</div><div class=""><div class="">- Even with those annotations and all the work, the APIs we’d end up with are not going to be good Swift APIs. I don’t think that “renamification” and "IUO audits" would ever actually happen in practice, for example.</div><div class="">- I believe the engineering effort required to implement this vision is so massive (including the changes to Swift, Python, and Python libraries) that it simply will never actually happen.</div><div class=""><br class=""></div><div class=""><br class=""></div></div><div class="">More concerning to me is that your design of using the existing AnyObject type presents a really concerning technical problem, scalability: It is not simply a Swift/ObjC/Python world, Javascript is also super important. There are also a number of other less-widely used dynamic languages that are interesting. Your design leads to them all being mashed together into a common runtime system. I cannot imagine how this would end up being a good thing for system complexity, and would lead to each of them being jeopardized in different ways. As I have mentioned before numerous times, it also opens the door for enormous Swift compiler complexity, as each of the object models need to be supported in the compiler (so you can subclass each languages’ types), and many other complexities.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">If you are serious about improving the tooling situation for people using Python APIs in Swift, the most natural way to do so is to follow the approach that the MyPy community (they are the ones who have thought about this the most) is using: provide progressive typing extensions, use data flow analysis to propagate around types, and enhance the tooling to have support this. I’m skeptical that this will ever be worthwhile, but a sketch could look like this:</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Consider the first example from: <a href="http://mypy-lang.org/examples.html" class="">http://mypy-lang.org/examples.html</a></div><div class=""><br class=""></div><div class="">With my proposals you could write the dictionary part very similar to the Python part (other pieces of the example, e.g. list comprehensions are uninteresting to me):</div><div class=""><br class=""></div><div class=""></div></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class=""><div class="">let d: PyVal = {:}</div></div><div class=""><div class="">…</div></div><div class=""><div class=""><div class="">d[word] = d.get(word, 0) + 1</div></div></div></blockquote><div class=""><div class=""><br class="Apple-interchange-newline"></div><div class="">We can teach the compiler simple data flow analysis, to know that ‘d’ is a Python dictionary. We could then allow the user to go further, with some new syntax along the lines of:</div><div class=""><br class=""></div><div class=""><blockquote style="margin: 0px 0px 0px 40px; border: none; padding: 0px;" class=""><div class=""><div class="">let d: @type(Dict<String, Int>) PyVal = {:} // Lots of other syntactic choices possible.</div></div><div class=""><div class="">…</div></div><div class=""><div class="">d[word] = d.get(word, 0) + 1</div></div></blockquote><div class=""></div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">However, the really important thing to recognize about these examples it that the type annotations are really poor at representing “generic” code like common Python libraries. They are designed for “user code” - like that on the mypy web page - which use concrete types.</div><div class=""><br class=""></div><div class="">As it turns out, my proposal provides a solution for this part of the problem, because we’re allowing user code to be written in Swift! A Swift programmer working with Python APIs would actually write this as:</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><blockquote style="margin: 0px 0px 0px 40px; border: none; padding: 0px;" class=""><div class=""><div class="">let d: Dictionary<String,Int> = {:}</div></div><div class=""><div class="">…</div></div><div class=""><div class="">d[word, default: 0] += 1</div></div></blockquote><div class=""></div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Similarly, there is no reason to do anything to make the second and third examples work nicely in Swift: you’d just define a Swift class, and a Swift function.</div><div class=""><br class=""></div><div class="">MyPy:</div><div class=""><br class=""></div><div class=""><pre class="ex" style="margin-top: 1.3em; margin-bottom: 1.3em; color: rgb(60, 60, 60); padding: 0.9em; background-color: rgb(238, 238, 238); border: 1px solid rgb(221, 221, 221); font-size: 13.0321px; font-variant-ligatures: normal; orphans: 2; widows: 2; font-family: 'Bitstream Vera Sans Mono', 'DejaVu Sans Mono', Monaco, Courier, monospace !important;"><span class="kw" style="color: rgb(92, 92, 160);">class</span> BankAccount:
<span class="kw" style="color: rgb(92, 92, 160);">def</span> __init__<span class="hili" style="background-color: rgb(223, 223, 223);">(self, initial_balance: <span class="pr" style="color: rgb(92, 92, 160);">int</span> = 0) -> None</span>:
self.balance = initial_balance
<span class="kw" style="color: rgb(92, 92, 160);">def</span> deposit<span class="hili" style="background-color: rgb(223, 223, 223);">(self, amount: <span class="pr" style="color: rgb(92, 92, 160);">int</span>) -> None</span>:
self.balance += amount
<span class="kw" style="color: rgb(92, 92, 160);">def</span> withdraw<span class="hili" style="background-color: rgb(223, 223, 223);">(self, amount: <span class="pr" style="color: rgb(92, 92, 160);">int</span>) -> None</span>:
self.balance -= amount
<span class="kw" style="color: rgb(92, 92, 160);">def</span> overdrawn<span class="hili" style="background-color: rgb(223, 223, 223);">(self) -> <span class="pr" style="color: rgb(92, 92, 160);">bool</span></span>:
<span class="kw" style="color: rgb(92, 92, 160);">return</span> self.balance < 0
my_account = BankAccount(15)
my_account.withdraw(5)
<span class="pr" style="color: rgb(92, 92, 160);">print</span>(my_account.balance)</pre><div class="">Swift:</div><div class=""><br class=""></div><div class="">class BankAccount { </div><div class=""> var balance : Int</div><div class=""> init(initialBalance: Int) { … }</div></div><div class=""> func deposit(….</div><div class="">}</div><div class=""><br class=""></div><div class="">Even the proposals that I’m making - simple and isolated though they are - are enough to provide a major quality of life improvement for people writing large amounts of code against Python APIs. Beyond that, they are small extensions with low complexity, scale to supporting many different dynamic languages over time, require a level of engineering effort that is plausible to be built, and do not require some sort of "executive buy in” from the Python community.</div><div class=""><br class=""></div><div class="">-Chris</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div></div></div>_______________________________________________<br class="">swift-evolution mailing list<br class=""><a href="mailto:swift-evolution@swift.org" class="">swift-evolution@swift.org</a><br class="">https://lists.swift.org/mailman/listinfo/swift-evolution<br class=""></div></blockquote></div><br class=""></div></body></html>