[swift-evolution] [Pitch] Introduce user-defined dynamically "callable" types

[Chris Lattner]

On Nov 11, 2017, at 6:24 PM, Joe Groff <jgroff at apple.com> wrote:
>> In contrast, it does *not* seem like it would help with the problem of resolving members of “id” typed values in Objective-C, which is the crux of the dynamic language problem.  If my understanding is correct, type providers seem completely unhelpful for python interoperability, because you don’t know the types of python values: anything loaded from a Python property or returned by a Python function is the equivalent of “id”.
>> 
>> It’s entirely possible I’m missing something big.  If so, I’d appreciate a pointer to something to read that explains how this works.  Thanks!
> 
> A type provider should let you provide custom logic for something the compiler sees as a module, which can answer name lookup queries by producing decls based on externally-available information.

Right.

> While Python is dynamic, Python culture doesn't go as wild with dynamic shenanigans as other languages do;

I’m not sure what you mean by this, but it is not my experience that Python programs are widely monomorphic.  In fact, several Python API design experts I respect have mentioned that a natural effect of dynamic languages is “interface creep” where APIs end up accepting broader and broader sets of stuff to make them “just work” in more cases.

Static languages solve this heads on by allowing API authors to express their requirements.  I prefer this approach of course, but we’re talking about a real existing world here, not my preference :-)

> there are established idioms for communicating some level of type information in doc comments (or more recently with first class syntax in Python 3),

Yes, Python 3 does have optional type annotations.  I’m not sure how widely used they are though, and given that the Python 2 community is (far?) larger than Python 3, this isn’t something we can depend on IMO.

> and people like code completion to work with their Python text editors, so there's incentive to use them. A type provider in Swift could address not only the syntax gap but the tooling gap from directly writing Python, since it could answer name lookup requests into an imported Python type by static analysis, and it could bring that declaration into Swift with doc comments and other metadata intact. This would let Python decls get wired up to Swift's code completion, quick help, and other IDE tooling.

Absolutely, I can see how type providers would provide nice refinement on top of basic “anyobject” style dispatch.  We still need to solve the basic problem though before providing refinements.

I’m also extremely concerned about how type providers work in practice.  With the disclaimer that I barely understand what is going on, it seems like they cause an extreme amount of compile-time work.  Implementing them seems to require implementing an interpreter into the compiler.  This is extremely concerning to me: for the same amount of complexity, we could burn many languages worth of special cases into Swift.

Since you’re apparently familiar with F#, are you aware of any efforts to use F# type providers to interoperate with dynamic languages?  How well does that actually work in practice?

> Since statically-available type info still isn't going to be perfect in Python, and we don't want to be boxed in to type-annotated APIs, a type provider could be lax with its name lookup behavior and allow any known Python method to be found on any Python type, much like AnyObject's behavior for ObjC methods (and, like AnyObject, maybe tag them with IUO types so that type safety aficionados can test for their presence, and prioritizing the "good" type-matching methods in code completion results). 

Right, but there is a very simple and closed problem to solve here.  Type providers don’t solve it, and so now you’re basically saying that type providers could be extended to provide exactly the same functionality I’m describing, but in a far more opaque, difficult to use, and significantly more compile time intensive approach.

Type providers may be a cool thing in principle, but if you’re going to advocate for them, I think we should start with a discussion about whether they would ever be realistically implemented in Swift (ignoring schedule).  Further, whatever extension you’re imagining to solve the “anyobject dispatch” problem will require making Swift types “Callable" anyways (so the type provider can provide that type).  

At that point, I would ask why type providers are in the critical path.  It seems obvious to implement the underlying type that the providers would provide first.  It’s not like we said we should wait to introduce structs until type providers exist.

-Chris



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20171111/cc7f7e38/attachment.html>