[swift-evolution] Making pointer nullability explicit (using Optional)
Jordan Rose
jordan_rose at apple.com
Mon Mar 21 15:52:06 CDT 2016
I'm not sure what you mean by this. (I think we agree, but I'm not sure.) A null pointer (in C) is not a pointer with a zero representation; it's a pointer that is guaranteed not to point to any valid object. By using Optional, Swift gets that value out of the type called "pointer", but can still represent it efficiently by claiming a platform-dependent bit pattern that won't be mistaken for a valid pointer. (That is, we don't need a separate flag to represent the type 'UnsafePointer<Int>?', which is hardly an uncommon type.)
In practice, people often pass all sorts of non-pointer values through parameters of type 'void *', so Swift can't be too aggressive about assuming particular bit patterns are or are not valid. Swift does have to understand C's representation of an invalid pointer because you want to be able to assign to imported struct fields with pointer type.
And correct, using a very large subscript on a pointer to navigate to the zero value will not result in a precondition failure.
Jordan
> On Mar 21, 2016, at 13:40 , David Waite <david at alkaline-solutions.com> wrote:
>
> I’m +1 on this idea, but I think we should be more careful in the introduction. I don’t think at the swift level the concept of a null pointer should exist at all.
>
> It is the platform and/or languages Swift integrates with which define a zero pointer as indicating ‘not pointing to anything’. It is the bridging between the platform or languages which should understand that a parameter or resulting value may optionally have on pointer, and how that pointer is represented by the underlying swift runtime.
>
> I imagine that using RandomAccessIndexType methods to navigate to the zero value will not result in a precondition failure?
>
> -DW
>
>> On Mar 17, 2016, at 7:59 PM, Jordan Rose via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>
>> Hey, everyone. If you're like me, you're sick of the fact that 'UnsafePointer<Int>' doesn't tell you whether or not the pointer can be nil. Why do we need to suffer this indignity when reference types—including function pointers!—can distinguish "present" from "absent" with the standard type 'Optional'? Well, good news: here's a proposal to make pointer nullability explicit. 'UnsafePointer<Int>?' can be null (nil), while 'UnsafePointer<Int>' cannot. Read on for details!
>>
>> https://github.com/jrose-apple/swift-evolution/blob/optional-pointers/proposals/nnnn-optional-pointers.md <https://github.com/jrose-apple/swift-evolution/blob/optional-pointers/proposals/nnnn-optional-pointers.md>
>>
>> Bonus good news: I've implemented this locally and updated nearly all the tests already. Assuming this is accepting, the actual changes will go through review as a PR on GitHub, although it's mostly going to be one big mega-patch because the core change has a huge ripple effect.
>>
>> Jordan
>>
>> ---
>>
>> Make pointer nullability explicit using Optional
>>
>> Proposal: SE-NNNN <https://github.com/apple/swift-evolution/blob/master/proposals/NNNN-name.md>
>> Author(s): Jordan Rose <https://github.com/jrose-apple>
>> Status: Awaiting review
>> Review manager: TBD
>> <https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#introduction>Introduction
>>
>> In Objective-C, pointers (whether to objects or to a non-object type) can be marked as nullable or nonnull, depending on whether the pointer value can ever be null. In Swift, however, there is no such way to make this distinction for pointers to non-object types: an UnsafePointer<Int> might be null, or it might never be.
>>
>> We already have a way to describe this: Optionals. This proposal makes UnsafePointer<Int> represent a non-nullable pointer, and UnsafePointer<Int>? a nullable pointer. This also allows us to preserve information about pointer nullability available in header files for imported C and Objective-C APIs.
>>
>> <https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#motivation>Motivation
>>
>> Today, UnsafePointer and friends suffer from a problem inherited from C: every pointer value could potentially be null, and code that works with pointers may or may not expect this. Failing to take the null pointer case into account can lead to assertion failures or crashes. For example, pretty much every operation on UnsafePointer itself requires a valid pointer (reading, writing, and initializing the pointee or performing arithmetic operations).
>>
>> Fortunately, when a type has a single invalid value for which no operations are valid, Swift already has a solution: Optionals. Applying this to pointer types makes things very clear: if the type is non-optional, the pointer will never be null, and if it isoptional, the developer must take the "null pointer" case into account. This clarity has already been appreciated in Apple's Objective-C headers, which include nullability annotations for all pointer types (not just object pointers).
>>
>> This change also allows developers working with pointers to take advantage of the many syntactic conveniences already built around optionals. For example, the standard library currently has a helper method on UnsafeMutablePointer called _setIfNonNil; with "optional pointers" this can be written simply and clearly:
>>
>> ptr?.pointee = newValue
>> Finally, this change also reduces the number of types that conform to NilLiteralConvertible, a source of confusion for newcomers who (reasonably) associate nil directly with optionals. Currently the standard library includes the following NilLiteralConvertible types:
>>
>> Optional
>> ImplicitlyUnwrappedOptional (subject of a separate proposal by Chris Willmore)
>> _OptionalNilComparisonType (used for optionalValue == nil)
>> UnsafePointer
>> UnsafeMutablePointer
>> AutoreleasingUnsafeMutablePointer
>> OpaquePointer
>> plus these Objective-C-specific types:
>>
>> Selector
>> NSZone (only used to pass nil in Swift)
>> All of the italicized types would drop their conformance to NilLiteralConvertible; the "null pointer" would be represented by a nil optional of a particular type.
>>
>> <https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#proposed-solution>Proposed solution
>>
>> Have the compiler assume that all values with pointer type (the italicized types listed above) are non-null. This allows the representation of Optional.none for a pointer type to be a null pointer value.
>>
>> Drop NilLiteralConvertible conformance for all pointer types.
>>
>> Teach the Clang importer to treat _Nullable pointers as Optional (and _Null_unspecified pointers as ImplicitlyUnwrappedOptional).
>>
>> Deal with the fallout, i.e. adjust the compiler and the standard library to handle this new behavior.
>>
>> Test migration and improve the migrator as necessary.
>>
>> This proposal does not include the removal of the NilLiteralConvertible protocol altogether; besides still having two distinct optional types, we've seen people wanting to use nil for their own types (e.g. JSON values). (Changing this in the future is not out of the question; it's just out of scope for this proposal.)
>>
>> <https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#detailed-design>Detailed design
>>
>> <https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#api-changes>API Changes
>>
>> Conformance to NilLiteralConvertible is removed from all types except Optional, ImplicitlyUnwrappedOptional, and _OptionalNilComparisonType, along with the implementation of init(nilLiteral:).
>>
>> init(bitPattern: Int) and init(bitPattern: UInt) on all pointer types become failable; if the bit pattern represents a null pointer, nil is returned.
>>
>> Process.unsafeArgv is a pointer to a null-terminated C array of C strings, so its type changes from UnsafeMutablePointer<UnsafeMutablePointer<Int8>> to UnsafeMutablePointer<UnsafeMutablePointer<Int8>?>, i.e. the inner pointer type becomes optional. It is then an error to access Process.unsafeArgv before entering main. (Previously you would get a null pointer value.)
>>
>> NSErrorPointer becomes optional:
>>
>> -public typealias NSErrorPointer = AutoreleasingUnsafeMutablePointer<NSError?>
>> +public typealias NSErrorPointer = AutoreleasingUnsafeMutablePointer<NSError?>?
>> A number of methods on String that came from NSString now have optional parameters:
>> public func completePathIntoString(
>> - outputName: UnsafeMutablePointer<String> = nil,
>> + outputName: UnsafeMutablePointer<String>? = nil,
>> caseSensitive: Bool,
>> - matchesIntoArray: UnsafeMutablePointer<[String]> = nil,
>> + matchesIntoArray: UnsafeMutablePointer<[String]>? = nil,
>> filterTypes: [String]? = nil
>> ) -> Int {
>> public init(
>> contentsOfFile path: String,
>> - usedEncoding: UnsafeMutablePointer<NSStringEncoding> = nil
>> + usedEncoding: UnsafeMutablePointer<NSStringEncoding>? = nil
>> ) throws {
>>
>> public init(
>> contentsOfURL url: NSURL,
>> - usedEncoding enc: UnsafeMutablePointer<NSStringEncoding> = nil
>> + usedEncoding enc: UnsafeMutablePointer<NSStringEncoding>? = nil
>> ) throws {
>> public func linguisticTags(
>> in range: Range<Index>,
>> scheme tagScheme: String,
>> options opts: NSLinguisticTaggerOptions = [],
>> orthography: NSOrthography? = nil,
>> - tokenRanges: UnsafeMutablePointer<[Range<Index>]> = nil
>> + tokenRanges: UnsafeMutablePointer<[Range<Index>]>? = nil
>> ) -> [String] {
>> NSZone's no-argument initializer is gone. (It probably should have been removed already as part of the Swift 3 naming cleanup.)
>>
>> A small regression: optional pointers can no longer be passed using withVaList because it would require a conditional conformance to the CVarArg protocol. For now, using unsafeBitCast to reinterpret the optional pointer as an Int is the best alternative; Int has the same C variadic calling conventions as a pointer on all supported platforms.
>>
>> <https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#conversion-between-pointers>Conversion between pointers
>>
>> Currently each pointer type has initializers of this form:
>>
>> init<OtherPointee>(_ otherPointer: UnsafePointer<OtherPointee>)
>> This simply makes a pointer with a different type but the same address as otherPointer. However, in making pointer nullability explicit, this now only converts non-nil pointers to non-nil pointers. In my experiments, this has led to this idiom becoming very common:
>>
>> // Before:
>> let untypedPointer = UnsafePointer<Void>(ptr)
>>
>> // After:
>> let untypedPointer = ptr.map(UnsafePointer<Void>.init)
>>
>> // Usually the pointee type is actually inferred:
>> foo(ptr.map(UnsafePointer.init))
>> I consider this a bit more difficult to understand than the original code, at least at a glance. We should therefore add new initializers of the following form:
>>
>> init?<OtherPointee>(_ otherPointer: UnsafePointer<OtherPointee>?) {
>> guard let nonnullPointer = otherPointer else {
>> return nil
>> }
>> self.init(nonnullPointer)
>> }
>> The body is for explanation purposes only; we'll make sure the actual implementation does not require an extra comparison.
>>
>> (This would need to be an overload rather than replacing the previous initializer because the "non-null-ness" should be preserved through the type conversion.)
>>
>> The alternative is to leave this initializer out, and require the nil case to be explicitly handled or mapped away.
>>
>> <https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#open-issue-unsafebufferpointer>Open Issue: UnsafeBufferPointer
>>
>> The type UnsafeBufferPointer represents a bounded typed memory region with no ownership or lifetime semantics; it is logically a bare typed pointer (its baseAddress) and a length (count). For a buffer with 0 elements, however, there's no need to provide the address of allocated memory, since it can't be read from. Previously this case would be represented as a nil base address and a count of 0.
>>
>> With optional pointers, this now imposes a cost on clients that want to access the base address: they need to consider the nil case explicitly, where previously they wouldn't have had to. There are several possibilities here, each with their own possible implementations:
>>
>> Like UnsafePointer, UnsafeBufferPointer should always have a valid base address, even when the count is 0. An UnsafeBufferPointer with a potentially-nil base address should be optional.
>>
>> UnsafeBufferPointer's initializer accepts an optional pointer and becomes failable, returning nil if the input pointer is nil.
>>
>> UnsafeBufferPointer's initializer accepts an optional pointer and synthesizes a non-null aligned pointer value if given nil as a base address.
>>
>> UnsafeBufferPointer's initializer only accepts non-optional pointers. Clients such as withUnsafeBufferPointermust synthesize a non-null aligned pointer value if they do not have a valid pointer to provide.
>>
>> UnsafeBufferPointer's initializer only accepts non-optional pointers. Clients using withUnsafeBufferPointermust handle a nil buffer.
>>
>> UnsafeBufferPointer should allow nil base addresses, i.e. the baseAddress property will be optional. Clients will need to handle this case explicitly.
>>
>> UnsafeBufferPointer's initializer accepts an optional pointer, but no other changes are made.
>>
>> UnsafeBufferPointer's initializer accepts an optional pointer. Additionally, any buffers initialized with a count of 0 will be canonicalized to having a base address of nil.
>>
>> I'm currently leaning towards option (2i). Clients that expect a pointer and length probably shouldn't require the pointer to be non-null, but if they do then perhaps there's a reason for it. It's also the least work.
>>
>> Chris (Lattner) is leaning towards option (1ii), which treats UnsafeBufferPointer similar to UnsafePointer while not penalizing the common case of withUnsafeBufferPointer.
>> <https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#impact-on-existing-code>Impact on existing code
>>
>> Any code that uses a pointer type (including Selector or NSZone) may be affected by this change. For the most part our existing logic to handle last year's nullability audit should cover this, but the implementer should test migration of several projects to see what issues might arise.
>>
>> Anecdotally, in migrating the standard library to use this new logic I've been quite happy with nullability being made explicit. There are many places where a pointer really can't be nil.
>> <https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#alternatives-considered>Alternatives considered
>>
>> The primary alternative here would be to leave everything as it is today, with UnsafePointer and friends including the null pointer as one of their normal values. This has obviously worked just fine for nearly two years of Swift, but it is leaving information on the table that can help avoid bugs, and is strange in a language that makes fluent use of Optional. As a fairly major source-breaking change, it is also something that we probably should do sooner rather than later in the language's evolution.
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-evolution
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20160321/6c8ef848/attachment.html>
More information about the swift-evolution
mailing list