[swift-evolution] SR-104: Improve Crash-Safety when Importing Objective-C Code Without Nullability Attributes

Fri Dec 11 02:08:23 CST 2015

On 10.12.2015, at 23:28, Douglas Gregor <dgregor at apple.com<mailto:dgregor at apple.com>> wrote:

On Dec 10, 2015, at 1:22 PM, Fabian Ehrentraud <Fabian.Ehrentraud at willhaben.at<mailto:Fabian.Ehrentraud at willhaben.at>> wrote:

Too bad to hear you are opposed to the change.
Thank you for the hint at the API dumps - this is really something that should be referenced in the proposal.

First of all, this affects a lot of APIs. I went ahead and grepped through the API dumps [...] to see how often implicitly-unwrapped optionals came through in the Cocoa APIs, and there are lots of them. For example, on OS X, grepping for !’s finds:

We should come to a common way to count the occurrences of affected APIs (despite that users would also need to adapt for un-annotated 3rd-party frameworks, and the own mixed&matched application code).
When I counted, it resulted in less than half the numbers for initializers and functions/methods.

There are many comment lines in the API dumps that contain /*! - we should not count those.

I didn’t, FWIW. Just “func”, “init”, or “var” lines that include !.

Some comment lines are included, yes, like this one for example:
./OSX/WebKit.swift:  /*! @abstract The URL of the initial request that created this item.

Also for functions/methods, input-parameters do not matter, due to automatic Optional wrapping on the Swift side. Maybe closure parameters that have implicitly unwrapped optional arguments themselves should be counted too. (anything else?)

Input parameters do matter, because their type communicates something: an optional input parameter says that “nil” has a meaning. An implicitly unwrapped optional input parameter says “check the docs!”. If we apply your proposal, you’re communicating meaning that “nil” is accepted in these input parameters when in fact the original (Objective-)C API never said such a thing, and very likely doesn’t permit “nil”. That actually makes it harder to use these APIs safely; at least when there are IUOs, we’re communicating “don’t know” to let the programmer decide what to do.

Ok this is a valid point. As keeping input parameters as implicitly unwrapped would not reduce the desired improvement of crash-resilience of this proposal, this could be explicitly exempted in the proposal.
Could you help me out? Which other kinds of parameters would benefit from defaulting to Optional? For one I guess closures which have non-annotated parameters themselves. In-Out Parameters? Is there a concrete method that springs to your mind as an example?

Initializers:

Should we only count "init!" instances?
$ grep init! . | wc -l
=> OSX 152, iOS 42

Other initializers containing ! in another place (I guess not relevant for the proposal)
$ grep -r ! . | grep init | grep -v '\/\*\!' | grep -v init!

Functions/Methods:

Only counting those that have a return type with an implicitly unwrapped optional. This does not take into account closure parameters that have implicitly unwrapped optional arguments themselves though.
$ grep -r ! . | grep func | grep '\->' | grep -v '\/\*\!' | grep -e '\!$' | wc -l
=> OSX 1577, iOS 452

Properties:

$ grep -r ! . | grep -e "let " | grep -v '\/\*\!' | wc -l
=> OSX 1120, iOS 336

This is still a large number of APIs that will require additional effort, where the common case is that you don’t care (because most things should be non-optional).

Yes, but "don't care" should be a decision, not an oversight. Writing that ! yourself reminds you of that.

Please also consider non-API use cases of this proposal. If you have a big Objective-C app, and try to add some Swift code, e.g. by adding an Extension in Swift, your new Swift code is bound to call a lot of you own old Objective-C code. You do not get to see the methods the way Swift sees it.

E.g. you have a code snippet:

func someSwiftFuncInAnExtension() {

myDataModel.someProperty.someMethod()
}

You do not immediately see if myDataModel, someProperty or someMethod is written in Objective-C and missing nullability annotations. Swift should help you writing safe code.

Constants:

I noticed there are many constants that get translated to implicitly unwrapped optionals (e.g. in iOS CoreFoundation.swift). Would they be affected by the change in this proposal? Or are they "explicitly set to implicitly unwrapped" by the importer?

You can grep for “ull_unspecified” in the SDK to find the cases where things are explicitly set to implicitly unwrapped optionals; I doubt there are many.

Ok, thank you, so we do not need to consider those. I found 12 in iOS, in the 3 headers "NSExtensionContext.h", "NSExtensionItem.h" and "NSItemProvider.h".

Most of the implicitly unwrapped optionals we currently get from (Objective-)C APIs can, in fact, never be nil. Forcing programmers to deal with all of these values as full optionals leads to a ton of optional-related boilerplate that doesn’t actual help users write better code. Rather, it dilutes the effectiveness of optionals as a language feature: optionals should mean “nil matters here, think about it carefully!”. But if some large number of optionals people deal with on a day-to-day basis are instead “the compiler didn’t know, so you have to write !”, the useful meaning of the “true” optionals gets lost in the noise.

I think exactly the opposite. If an API has not yet been audited, the client needs to consult the documentation if a return value can ever be nil. If not, she might explicitly and knowingly unwrap the value safely or non-safely. I deem this a good thing, as it prevents unknowingly unsafe use. I do not think that it would get in the way, as the syntactic sugar for Optionals is quite minimal. Also when the API gets audited for nullability later on, the compiler will hint which optional unwrappings have been rendered unnecessary.

You’re discounting my main argument, which is that the safety provided by optionals is contingent on the nil case being interesting most of the time. That’s generally true in “pure” Swift code—Swift programmers use optionals when nil matters—and it’s generally true in imported (Objective-)C code that has been audited, but your change would create a lot of optionals where “nil” isn’t actually interesting in practice.

Have I understood it right that your argument is that defaulting to Optionals would would make them loose their meaning and result in programmers just use ! too often, also in cases where the nil case should properly be handled? I see the point. And I'm thankful for this constructive discussion.

Nonetheless I think that Swift preventing unintentional mistakes would be something to strive for. Have you got alternative ideas on how to deal with the issue this proposal tries to address?

- Doug

On 10.12.2015, at 19:27, Douglas Gregor <dgregor at apple.com<mailto:dgregor at apple.com>> wrote:

On Dec 10, 2015, at 3:42 AM, Fabian Ehrentraud via swift-evolution <swift-evolution at swift.org<mailto:swift-evolution at swift.org>> wrote:

Are there any objections to variant 2 "Import un-annotated code as Optional"? If not, I would create the proposal.

I’m opposed to this.

First of all, this affects a lot of APIs. I went ahead and grepped through the API dumps over at

https://github.com/apple/swift-3-api-guidelines-review

to see how often implicitly-unwrapped optionals came through in the Cocoa APIs, and there are lots of them. For example, on OS X, grepping for !’s finds:

4132 functions/methods
2281 properties
409 initializers

On iOS, which has a smaller number of overall APIs, the numbers are still pretty big:
1071 functions/methods
320 properties
134 initializers

This may be a slight oversampling of the data—some of those might be explicitly annotated as _Null_unspecified in the (Objective-)C headers, some might be unavailable—but the scale of the impact here is very, very large.

Most of the implicitly unwrapped optionals we currently get from (Objective-)C APIs can, in fact, never be nil. Forcing programmers to deal with all of these values as full optionals leads to a ton of optional-related boilerplate that doesn’t actual help users write better code. Rather, it dilutes the effectiveness of optionals as a language feature: optionals should mean “nil matters here, think about it carefully!”. But if some large number of optionals people deal with on a day-to-day basis are instead “the compiler didn’t know, so you have to write !”, the useful meaning of the “true” optionals gets lost in the noise.

That said, I’d love for implicitly-unwrapped optionals to be used less often, and getting them further out of the type system would be beneficial for the Swift experience.

- Doug

Am 09.12.2015 um 07:11 schrieb Chris Lattner <clattner at apple.com>:

On Dec 8, 2015, at 4:09 AM, Fabian Ehrentraud via swift-evolution <swift-evolution at swift.org> wrote:

Swift code accessing Objective-C methods can easily crash - if the Objective-C code does not include nullability attributes, the code is brought as implicitly unwrapped into Swift, where unsafe accesses do not produce compiler warnings.

I created issue SR-104 for this, but as suggested by Jordan Rose it should be discussed on this mailing list first.

Short example:

// Objective-C
- (NSString *)giveMeAString { return nil; }

// Swift
func thisWillCrash() {
let string = someObjectiveCObject.giveMeAString()
let length = string.length // crash
}

The ClangImporter could handle this issue in several safer ways:

1. Only import declarations that are nullability annotated (as proposed by Greg Parker)
Positive side effect would be that it would motivate to write nullability annotations.

This seems unfortunate to me, because it would severely hamper interoperating with Objective-C and *C* APIs that have not been audited.  While Apple is keen to audit their APIs, I doubt that all the linux system headers will get updated in a timely manner and users don’t generally have a way to do anything about that.

2. Import un-annotated code as Optional
Values would need to be unwrapped every time, which does not hurt too much due to the easy use of the `?` syntactic sugar.

This is a very interesting idea, one I haven’t considered recently.  I agree with you that this is worth considering, and I would love to see IUO just get summarily deleted :-)

-Chris
_______________________________________________
swift-evolution mailing list
swift-evolution at swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20151211/f4a1219b/attachment.html>