[swift-evolution] Notes from Swift core team 2016-04-05 design discussion

[Alex Martini]

To help keep proposals moving forward, the Swift core team has set aside some time specifically for design discussions of upcoming proposals.  Below are some rough notes from the yesterday's discussion.

These are informal comments, intended to guide the proposals in directions that draw constructive feedback. You are welcome to ignore the feedback, agree with it, or disagree with it.  As always, the formal decision doesn't happen until after the review period ends.




SE-0058: Allow Swift types to provide custom Objective-C representations <file:///Users/alexmartini/DevPubs%20Git%20Repositories/Swift%20Language%20Review/_build/html/LR_MeetingNotes/2016-04-05.html#se-0058-allow-swift-types-to-provide-custom-objective-c-representations>
https://github.com/apple/swift-evolution/0058-objectivecbridgeable.md <https://github.com/apple/swift-evolution/0058-objectivecbridgeable.md>
This is essentially cleaning up the existing underscored protocol and blessing it as the way to bridge Obj-C types over to Swift value types. Mechanically, all the work is essentially done in the underscored version.

There are some C++ interop hacks to make things handle in-out. Calling the initializers from the C++ part of the runtime is really hairy; we use the extension’s function versions that use in-out instead.

There is a “Swift indirect result” attribute that we can use for pointers like this. It’s in Clang — this is what you do if you want to directly call Swift code. We should be able to do this with John’s changes.

There’s a level of hackiness here, but it’s part of the standard library, so we expect we can keep it working. If we had more generalized existentials, that C++ could move into Swift code. Or we could write it in SIL.

Question on the list about defining both the bridged type and the value type in Swift. The bridging gets interesting when you export back to Objective-C. We could make it work both ways — we’re probably close to getting it to work — it’s a matter of making sure we find the value type and re-import the generated header. We have to make sure that when you see the Obj-C version of the Swift type come in again via Clang, that you know it’s already wired up.

We have some C bridging around Darwin, but it’s very ad-hoc right now. Right now, the dynamic cast machinery in all three places that implement it only have to care about class and struct types, but it’s unfortunate to have this. There are class-to-class bridging because of error types: you can throw an instance of a class. It’s a little different because you’re not turning the Swift error type as NSError to Obj-C, although it has been proposed that we should bridge between ErrorProtocol and NSError.

To prove that something fails, you’re always doing this work. On the other hand, it’s a dynamic cast, so we don’t expect it to be fast. Making value-to-value casting require protocol lookup would be unfortunate.

There are a lot of people who want to do things like hiding members. Doing that via API notes seems much preferred over bridging it to another C type.

You might want to do something like this around sockaddr types that use pointer punning, mapping them to the Swift idiom of protocols. But the most common thing people want to do is take advantage of Swift-only features but still projects out to Objective-C in a way that makes sense.

There are some objective-C specific hacks such as the optional parameter in init(unconditionallyBridgedFromObjectiveC:) which handles the case of returning nilfrom a method that said it wouldn’t return nil. The contract of treating nil and the empty collection as the same makes sense. This hack supports it. A general C bridging protocol might be a little different.

We have C bridged types, such as bool, where C has a type with the same meaning but a memory layout that’s not compatible.

There are some APIs in there that could have better names. In the argument labels, the ‘-ed’ should probably be ‘-ing’.

The main difference from the underscored version is the removal of underscores and using an initializer instead of a static method.

Do we want a standard shorthand for these long names in the mangler?







Side-car annotations of C libraries (i.e., API notes as a first-class feature) <file:///Users/alexmartini/DevPubs%20Git%20Repositories/Swift%20Language%20Review/_build/html/LR_MeetingNotes/2016-04-05.html#side-car-annotations-of-c-libraries-i-e-api-notes-as-a-first-class-feature>
API notes is some YAML input that lets you add certain kinds of attributes to Clang declarations. We created it for specific problems, but it is more generally useful. It seems totally reasonable that some package could wrap an existing C library via sidecar data and provide a more Swift idiomatic interface to it.

For example, dyspatch_async() turns into a member named async(_:) on a class named OS_dispatch_queue. You can form properties like how dispatch_get_main_queue() maps over. This lets the overlay for that API stay small.

The major downside of API notes is that it’s a limited format with a baroque implementation; every time we want to add an attribute, we have to update the YAML parser and then Clang and then the serialized form of the API notes. For this to be a “real thing” we should intentionally design the format. A simple text-based format is great, but this wouldn’t be able to do things like add C++ attributes as it currently is build.

An issue with making this just a way to add Clang attributes is that it wouldn’t let you add attributes without updating Clang.

We have created a lot of attributes on the Clang side as part of the mapping into Swift. We always thought that API notes were a transitional technology and we wanted to be able to push the changes back into the headers, so that the headers contain the truth. However, that doesn’t work when you don’t control the underlying C libraries.

As the things we need to do become more complex, it can become difficult to determine the Swift interface by reading the annotations on a header anyhow.

It’s important to be able to control the way that a C API comes into Swift, and it’s important for people to be able to author these as part of the package manager. Do we think that API notes is a better solution than the overlay? People will start by importing a C library, and then writing more Swift code to make that interface better. If the tools were exposed at the Swift level to re-export a library, maybe that would be better. If we can express it in Swift, inventing a new language to express it seems like a bad idea.

The API notes approach is a lot less work today than writing an overlay.

Is this backwards? Should you say “here’s the Swift interface I want” and then have some attributes that wire it up to the existing C API?

The two things you get today from API notes are a faster calling convention and hiding the C version. If it was marked as inline-always, you should get the faster calling convention.

Implementing something like the sockaddr example above in API notes seems like it would be a ton of work.

We could generate a Swift file that shows the interface — handle the retyping of all the call forwarding. That would give you a path to transition away from API notes.

One difference between this and an overlay is that doing unsafe things is much more straightforward. For example, marking a function as @noescape is trivial to do in this and frustrating to do in an overlay, because it’s as unsafe operation.

You could create a library of declarative operations for doing this, similar to Boost.Python. It’s basically an FFI.

We’re interoperating with an unsafe language, so we have to trust the declarations. This is different from adding a Swift API for operations like “ignore @noescape” which opens a type safety hole. However, APIs like that already do exist.

Most of our mappings are not very direct. They go through thunks, they add extra retains and releases, and so on. So although there is value in being able to see mappings as direct and understand them, that case is uncommon.

In the import process, you want to import something without exposing all of it to your clients. There are two approaches: list things that should be exposed, or list things that shouldn’t be exposed.

Boost.Python works because Python is runtime reflective. You can’t do that in Swift.

People want to start with the C interface and then edit it. So you could start by having a tool that generated the overlay with minimal mapping, and then edit the overlay.

You could add a @shadowing_c attribute. If you omit the body of the declaration, listing the parameters in the attribute would let you do the following:

// C declaration
void dispatch_sync(dispatch_queue_t queue, void (^block)(void));

// API notes
- Name: dispatch_apply
  SwiftName: 'OS_dispatch_queue.apply(_:self:_:)'

// Default
@shadowing_c(dispatch_sync(self, block))
func dispatch_sync(block: (@convention(block) () -> Void)!)

// Desired
extension OS_dispatch_queue {
    @shadowing_c(dispatch_sync(self, block))
    func sync(@noescape block: @convention(block) () -> Void)
}
If the overlay hides everything automatically is that later-added new APIs aren’t exposed unless the overlay is changed. An additive approach, where you can call the newly added APIs using the default (ugly) names, would be better.

A wrapper is not an overlay. If the Swift types wrap a C type rather than just renaming them and adjusting them, that would prevent the “calling with ugly name” workaround as above, because the types are different instead of just having mapped names.

One problem: only one person can write the overlay for a given C API.

If you’re going to write a wrapper, that is something else. That is another package on top. The overlay should be its own thing, and there should be just one. And it should be pretty clear how to write the overlay package. If you’re doing the mapping in code, there’s a much greater temptation to blur the line and start doing some wrapping in addition to the overlay.

Upstream projects are more likely to take the minimal overlay package as opposed to a wrapper package. The project is much more likely to take those if the files don’t change often.

What’s wrong with the way things ingest today? The package manager doesn’t have the information to ingest them today; for example, it doesn’t read the autoconf file.

A three-part approach: the basic thing needed to get it to build, which you try to upstream, the overlay, and the wrapper.

Stepping back, it seems like there’s a lot of background and goals in many different areas that need to be better established.

Making this usable on Linux is important. On Apple platforms, most of the Objective-C APIs come through reasonably well without a lot of work. Linux APIs have more C pointers that need more work to make them usable in Swift.

It seems like pushing API note out into the world is not helpful, especially in their current form. We should avoid introducing a whole new technology. The least damaging way for people to do those things seems to be using regular Swift modules.

No, it’s not that simple. If we don’t push out API notes, people invent their own minimal simple wrapper in every case. There are negative impacts from both approaches.

It would be bad for the ecosystem to have the overlay for things like sockaddr being buried in an opinionated wrapper. However, sockaddr is never going to come through well in Swift with annotations or automation; it would need a full wrapper to come through well.

What if we define “overlay” to mean “the minimal amount of work needed to make an interface work with Swift, without unsafe behavior, and shadows the C module”? Different opinions about whether it’s a goal for the overlay to remove unsafe behavior. It would be good to have the overlay not require casting away @noescape. There is already a technical restriction from the compiler that you get one and only one of these, and that’s the overlay. Establishing the boundary of what goes in the overlay is important.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20160405/4c99d67b/attachment.html>