[swift-evolution] Proposal: Introduce User-defined "Dynamic Member Lookup" Types

[Vincent Esche]

Paul Cantrell wrote:

It seems like this fundamentally alters Swift’s aesthetic of “either an
> operation is type-safe, or it’s clear at the point of use that it’s not.”
> Should this use an operator other than a period, e.g. `pickle->loads(blob)`?
>

This is a major concern of mine as well. What you end up by camouflaging
dynamic, unchecked APIs in what looks like normal (i.e. static, checked)
Swift is a wolf in sheeps’ clothing. One would not be able to tell one from
the other, yet would have to juggle two completely orthogonal language
models in one’s mind when reading/writing Swift code.

For me for such a proposal to even be remotely acceptable such dynamic
calls would have to have a clearly distinguishable syntax (`foo.bar?(…)`
would have been a great fit for this, but that ship has sailed).

In order to prevent such dynamic code from getting sprinkled all over the
place (dirt gets everywhere if given enough time) I’d further more expect
entire code blocks to be marked with a label that leans more towards
discouragement, than encouragement, such as:

unchecked {
    foo.bar?!(…)
}


Brent Royal-Gordon wrote:

* Let's step away from bridging entirely and just think about Swift for a
> moment. There are cases where we'd like to make *semi*-dynamic proxies
> which wrap another type and allow operations based on what's statically
> known about that type. Think, for example, of the appearance proxy in
> UIKit: This is an object attached to UIView subclasses which lets you (in
> essence) set default values for all instances. We currently just pretend
> it's an instance of `Self`, which mostly works because of Objective-C, but
> a Swift-native version would probably prefer to return a
> `UIAppearance<Self>` object which used its knowledge of `Self` to expose
> `Self`'s properties on itself. Is there a way we could design this feature,
> or a related feature, to cover that kind of use case? That is, to allow a
> limited set of keys—perhaps even key-path-based when you want static
> control—with a different type for each key, *or* to allow any key with some
> common type, depending on your type's needs?


This particular use-case (along with NSProxy) looks to me like a perfect us
of a new-type/derive as known from Haskell or Rust, rather than dynamic.
(Albeit there are problems with new-type and types making use of `Self` as
associated types.)

Doug Gregor wrote:

Swift is, unabashedly, a strong statically-typed language.


And thankfully so, if I may say so. There a few places in the language
(other than objc-bridging) where one needs to make us of dynamic features.
But most of these are caused by the fact that Swift’s type system is still
rather limited and certain type relations simply cannot be expressed yet,
forcing one to make use of the type-safety escape hook that is
Any/AnyObject.

IMO, this proposal is a significant departure from the fundamental
> character of Swift, because it allows access to possibly-nonexistent
> members […] without any indication that the operation might fail.


Unless there would be a corresponding compiler flag for marking use of such
dynamic calls as warnings if not errors, the addition of this proposal
would eliminate many of the safety guarantees that Swift gives me for code
written in it for safety critical programs. If one cannot judge about the
safety of a piece of code in Swift by merely looking at it, then what is
the use of a remaining half-baked safety promise to begin with?

Matt Diephouse wrote:

I personally don’t want to work with APIs like this. And while I can
> understand the appeal of easier interop—especially if you’ve come into a
> large library of Python code—IMO this would inevitably lead to other
> misuses of dynamic lookups in 1st- or popular 3rd-party libraries.


Neither do I. The still unfinished nature of Swift’s type checking/system
and lack of meta-programming capabilities will drive users of the language,
who don’t know any better (especially when coming from dynamic languages),
to a wide-spread misuse of such features.

There also is another aspect to it:

The main driving force behind this proposal according to this discussion is
“interop with dynamic languages”.
The majority of language users however work in an environment where such
inter is prohibited: iOS.
Releasing a feature that’s basically shouting “MISUSE ME, MISUSE ME!” at
hundreds of thousands of users who’s pretty much only possible use _is_ a
misuse, and of which many are coming from ObjC or Java and still at odds
with, if not outright denial of Swift’s more strict type rules, is a
terrible foundation, I think.

I also think that we should rather have a syntax sugar freeze until Swift 6
(more on that: http://bit.ly/2BDOFxu
<https://blog.definiteloops.com/syntactic-diabetes-bbf54b3b4487>), rather
than adding yet another to the mix without really thinking about the
consequences (lack of coherence, e.g.).

Benjamin G wrote:

A few examples of the kind of things that i would think about :
> - use different file names for code using dynamic features.
> - have every dynamic code included in a dynamic{} block
> - have a different syntax for dynamic function calls, using ! or !!!
> - Make the dynamic calls work only of subclasses of a base "PythonObject".
> That would make the goal clear.
> etc.
> Now, i think those are all bad ideas (and that's why i don't like the
> proposal in the first place). But i really think the concern should be
> addressed.


Two of these (!, !!! and dynamic {}) are similar in spirit to what I
proposed above (?, ?! and unchecked {}).

And I too think that those are all bad ideas (and that's why i don't like
the proposal in the first place).

Karl Wagner wrote:

My worry is that in eagerness to try and scoop up developers of other
> languages, that we diminish the Swift language to the point where it
> doesn’t really provide a benefit for them. Python developers will be
> writing the same unsafe, non-checked code using Swift that they do in
> Python; where’s the win?


I agree with this 100%.

A language should never have growth (beyond what is necessary for
sustainable as a language!) be its driving force (unlike the often quoted
“world dominance” of Swift). Instead a language should strive to provide
the best tools for its use-case. Quality over quantity. Otherwise you’re
gaining the world and losing your soul.

Especially when it comes to Python and JS I’m not sure I’d want their
ecosystems' practices to get carried over to Swift.

Python is infamous for its liberal use of “putting everything and the
kitchen sink into stringly typed dicts”.
Please, please let’s not have this in Swift (which DynamicMemberLookup
would enable all too conveniently).

I’m also rather skeptical about the influences of the Javascript ecosystem
on Swift. The JS ecosystem is "the best we could have from nothing”, but
still utterly broken by design, with abominations such as leftpad being
more the norm, than the exception.

Before opening the floodgates to external influences (from communities that
dwarf the Swift community in sheer size) shall we please first wait long
enough until we as a community have come to a consensus about what actually
is idiomatic Swift? (Case in point: Apple claimed to have invented [sic!]
Protocol-oriented Programming with Swift and touted it as the driving force
behind its design. Yet a brief look at the stdlib reveals little to no use
of it. It’s concrete types all over the place. Arrays all over the place.)

When we consider interoperability with other languages, it should be from
> the perspective of mapping their features to Swift’s philosophies.
> This proposal takes the reverse approach, and makes Swift like Python, so
> I’m against it.


This. I am further more of the opinion that in a pair of interoperable
languages it should always be the dynamic one that’s abstracting over the
static one, not the other way round. Statically typed languages are—by
nature—more concrete than dynamic ones. They have to be.

Jon Gilbert wrote:

However, I believe that you bring up an extremely valid concern. To
> extrapolate from your points, this proposal seems to have a high potential
> to erode what makes Swift special, and it provides a way to get around
> Swift’s safe-guards, avoid writing Swift versions of libraries, and
> ultimately, avoid hiring real Swift developers.


This is another valid and often overlooked point. Management will find the
path of least (initial) financial resistance. Every single time.
It's the very same reasons that we have more and more apps build on the
technically inferior (as in much more memory & power-hungry, non-native
feel, etc.) Electron apps these days. And while at the current time they
might look like the right solution they will be looked upon in similar ways
as one looks at Flash these days. How did we manage to obsolete Flash? By
extending the capabilities and expressiveness of HTML and our browsers.
Let’s extend the capabilities and expressiveness of Swift, instead of
dividing the ecosystem as Flash did with the web.

Why compromise Swift’s static-ness and roll out the red carpet for the
> dynamic, unwashed masses?


Swift’s static-ness is the unique selling point of the language. As is for
Kotlin, Rust, Haskell, and the like (all with varying degrees of
static-ness). Overall we’re seeing type-systems spread through languages
(in many instances in form of gradual typing). There hardly is a language
out there that did not gain some kind of typing recently. Python, Ruby,
Javascript, Racket, … you name it.

Everything in life and engineering is a trade-off. Swift’s trademark is
> that it consistently sacrifices the convenience of dynamic-ness for the
> safety and performance of static-ness. This proposal does seem to do the
> opposite, and we’d be naive not to fear the effect this will have.


ThisI can’t help but read this proposal as a bit of a “Wait, I actually
_did_ mean Swift to be Objective-C without the C, please fix!” So does this
mean Craig was right after all? ;)

Or why it would be bad for a Swift project to depend upon a stack of
> third-language dependencies that CocoaPods/Carthage can’t manage, XCode
> can’t analyze, stack traces can’t delve into (or can it?), lldb can’t step
> through, etc.


This is another big one. Which is especially worrying given the state of
Swift’s tooling today: there literally is none.
We have a compiler that still provides utterly unhelpful diagnostics every
other day with no proper official swift-env handling, no proper
cross-compilation tooling, no proper unit testing, no proper formatting, no
proper lints, no proper plugins, … nothing.

All we have is a compiler that has much potential but is still very much
unfinished. Embedded in an IDE that completely breaks down when asked to
provide auto-completion and debugging support leaving a lot to be desired,
too. Even in Xcode 9. Regularly. With a package manager that up until
recently featured a fatal remote code execution exploit (which afaik is
still there on platforms other than macOS).

I think we have more urgent things to work on at this point, than interop
with other languages—which, as Chris pointed out don’t give a damn about
Swift to begin with.

Chris Lattner wrote:


3) Many Python APIs are wrappers for C APIs.  “Swiftizing” a Python API in
> this case means writing a new Swift wrapper for the API, not adding type
> annotations.


Especially in the context of interoperation with machine learning /
scientific computing libraries from Python I don’t get why one would want
to bridge to Python.

Tensorflow, Scipy, Numpy, … they all boil down to being shallow wrappers
around native language extensions written in C or C++, just as you wrote
yourself.

What I would want is to have these be ported to/wrapped with Swift, instead
of Python.

In general cannot see a single scenario where I would want to directly
interact (as not just “please evaluate the entire file Foo.py”) with a
scripting language from Swift, where it wouldn’t have been a better idea to
implement either side in the given other language to begin with.

You want to make use of Tensorflow from Swift? Simple, use the C-API. It
exists for _exactly_ this reason.

If I am coding in a low-level language to begin with, then why would I want
to interface with 3rd-party languages/library on a layer that’s any higher
than necessary?

I’m not opposed to going further over time, but I’d like to get started at
> some point :-).  I’m not in a super urgent hurry to get this in in the next
> week or month or anything like that, but I also don’t want to wait until
> Swift 10.


>From my point of view Swift 10 would be a perfect time to revisit the
prospect of extending Swift to such dynamic use-cases. At a point where its
strengths, weaknesses and idiomatic patterns have had plenty of time to be
investigated.

- Vincent
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20171202/f82ecfdb/attachment.html>