[swift-evolution] Making pointer nullability explicit (using Optional)

[Howard Lovatt]

+1

On Friday, 18 March 2016, Jordan Rose via swift-evolution <
swift-evolution at swift.org> wrote:

> Hey, everyone. If you're like me, you're sick of the fact that
> 'UnsafePointer<Int>' doesn't tell you whether or not the pointer can be
> nil. Why do we need to suffer this indignity when reference types—including
> function pointers!—can distinguish "present" from "absent" with the
> standard type 'Optional'? Well, good news: here's a proposal to make
> pointer nullability explicit. 'UnsafePointer<Int>?' can be null (nil),
> while 'UnsafePointer<Int>' cannot. Read on for details!
>
>
> https://github.com/jrose-apple/swift-evolution/blob/optional-pointers/proposals/nnnn-optional-pointers.md
>
> Bonus good news: I've implemented this locally and updated nearly all the
> tests already. Assuming this is accepting, the actual changes will go
> through review as a PR on GitHub, although it's mostly going to be one big
> mega-patch because the core change has a huge ripple effect.
>
> Jordan
>
> ---
>
> Make pointer nullability explicit using Optional
>
>    - Proposal: SE-NNNN
>    <https://github.com/apple/swift-evolution/blob/master/proposals/NNNN-name.md>
>    - Author(s): Jordan Rose <https://github.com/jrose-apple>
>    - Status: *Awaiting review*
>    - Review manager: TBD
>
>
> <https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#introduction>
> Introduction
>
> In Objective-C, pointers (whether to objects or to a non-object type) can
> be marked as nullable or nonnull, depending on whether the pointer value
> can ever be null. In Swift, however, there is no such way to make this
> distinction for pointers to non-object types: an UnsafePointer<Int> might
> be null, or it might never be.
>
> We already have a way to describe this: Optionals. This proposal makes
> UnsafePointer<Int> represent a non-nullable pointer, and
> UnsafePointer<Int>? a nullable pointer. This also allows us to preserve
> information about pointer nullability available in header files for
> imported C and Objective-C APIs.
>
> <https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#motivation>
> Motivation
>
> Today, UnsafePointer and friends suffer from a problem inherited from C:
> every pointer value could potentially be null, and code that works with
> pointers may or may not expect this. Failing to take the null pointer case
> into account can lead to assertion failures or crashes. For example, pretty
> much every operation on UnsafePointer itself requires a valid pointer
> (reading, writing, and initializing the pointee or performing arithmetic
> operations).
>
> Fortunately, when a type has a single invalid value for which no
> operations are valid, Swift already has a solution: Optionals. Applying
> this to pointer types makes things very clear: if the type is non-optional,
> the pointer will never be null, and if it *is*optional, the developer
> must take the "null pointer" case into account. This clarity has already
> been appreciated in Apple's Objective-C headers, which include nullability
> annotations for all pointer types (not just object pointers).
>
> This change also allows developers working with pointers to take advantage
> of the many syntactic conveniences already built around optionals. For
> example, the standard library currently has a helper method on
> UnsafeMutablePointer called _setIfNonNil; with "optional pointers" this
> can be written simply and clearly:
>
> ptr?.pointee = newValue
>
> Finally, this change also reduces the number of types that conform to
> NilLiteralConvertible, a source of confusion for newcomers who (reasonably)
> associate nil directly with optionals. Currently the standard library
> includes the following NilLiteralConvertible types:
>
>    - Optional
>    - ImplicitlyUnwrappedOptional (subject of a separate proposal by Chris
>    Willmore)
>    - _OptionalNilComparisonType (used for optionalValue == nil)
>    - *UnsafePointer*
>    - *UnsafeMutablePointer*
>    - *AutoreleasingUnsafeMutablePointer*
>    - *OpaquePointer*
>
> plus these Objective-C-specific types:
>
>    - *Selector*
>    - *NSZone* (only used to pass nil in Swift)
>
> All of the italicized types would drop their conformance to
> NilLiteralConvertible; the "null pointer" would be represented by a nil
> optional of a particular type.
>
> <https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#proposed-solution>Proposed
> solution
>
>    1.
>
>    Have the compiler assume that all values with pointer type (the
>    italicized types listed above) are non-null. This allows the representation
>    of Optional.none for a pointer type to be a null pointer value.
>    2.
>
>    Drop NilLiteralConvertible conformance for all pointer types.
>    3.
>
>    Teach the Clang importer to treat _Nullable pointers as Optional (and
>    _Null_unspecified pointers as ImplicitlyUnwrappedOptional).
>    4.
>
>    Deal with the fallout, i.e. adjust the compiler and the standard
>    library to handle this new behavior.
>    5.
>
>    Test migration and improve the migrator as necessary.
>
> This proposal does not include the removal of the NilLiteralConvertible
> protocol altogether; besides still having two distinct optional types,
> we've seen people wanting to use nil for their own types (e.g. JSON
> values). (Changing this in the future is not out of the question; it's just
> out of scope for this proposal.)
>
> <https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#detailed-design>Detailed
> design
> <https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#api-changes>API
> Changes
>
>    -
>
>    Conformance to NilLiteralConvertible is removed from all types except
>    Optional, ImplicitlyUnwrappedOptional, and _OptionalNilComparisonType,
>    along with the implementation of init(nilLiteral:).
>    -
>
>    init(bitPattern: Int) and init(bitPattern: UInt) on all pointer types
>    become failable; if the bit pattern represents a null pointer, nil is
>    returned.
>    -
>
>    Process.unsafeArgv is a pointer to a null-terminated C array of C
>    strings, so its type changes from
>    UnsafeMutablePointer<UnsafeMutablePointer<Int8>> to
>    UnsafeMutablePointer<UnsafeMutablePointer<Int8>?>, i.e. the inner
>    pointer type becomes optional. It is then an error to access
>    Process.unsafeArgv before entering main. (Previously you would get a
>    null pointer value.)
>    -
>
>    NSErrorPointer becomes optional:
>
> -public typealias NSErrorPointer = AutoreleasingUnsafeMutablePointer<NSError?>+public typealias NSErrorPointer = AutoreleasingUnsafeMutablePointer<NSError?>?
>
>
>    - A number of methods on String that came from NSString now have
>    optional parameters:
>
>    public func completePathIntoString(-    outputName: UnsafeMutablePointer<String> = nil,+    outputName: UnsafeMutablePointer<String>? = nil,
>      caseSensitive: Bool,-    matchesIntoArray: UnsafeMutablePointer<[String]> = nil,+    matchesIntoArray: UnsafeMutablePointer<[String]>? = nil,
>      filterTypes: [String]? = nil
>    ) -> Int {
>
>    public init(
>      contentsOfFile path: String,-    usedEncoding: UnsafeMutablePointer<NSStringEncoding> = nil+    usedEncoding: UnsafeMutablePointer<NSStringEncoding>? = nil
>    ) throws {
>
>    public init(
>      contentsOfURL url: NSURL,-    usedEncoding enc: UnsafeMutablePointer<NSStringEncoding> = nil+    usedEncoding enc: UnsafeMutablePointer<NSStringEncoding>? = nil
>    ) throws {
>
>    public func linguisticTags(
>      in range: Range<Index>,
>      scheme tagScheme: String,
>      options opts: NSLinguisticTaggerOptions = [],
>      orthography: NSOrthography? = nil,-    tokenRanges: UnsafeMutablePointer<[Range<Index>]> = nil+    tokenRanges: UnsafeMutablePointer<[Range<Index>]>? = nil
>    ) -> [String] {
>
>
>    -
>
>    NSZone's no-argument initializer is gone. (It probably should have
>    been removed already as part of the Swift 3 naming cleanup.)
>    -
>
>    A small regression: optional pointers can no longer be passed using
>    withVaList because it would require a conditional conformance to the
>    CVarArg protocol. For now, using unsafeBitCast to reinterpret the
>    optional pointer as an Int is the best alternative; Int has the same C
>    variadic calling conventions as a pointer on all supported platforms.
>
>
> <https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#conversion-between-pointers>Conversion
> between pointers
>
> Currently each pointer type has initializers of this form:
>
> init<OtherPointee>(_ otherPointer: UnsafePointer<OtherPointee>)
>
> This simply makes a pointer with a different type but the same address as
> otherPointer. However, in making pointer nullability explicit, this now
> only converts non-nil pointers to non-nil pointers. In my experiments, this
> has led to this idiom becoming very common:
>
> // Before:let untypedPointer = UnsafePointer<Void>(ptr)
> // After:let untypedPointer = ptr.map(UnsafePointer<Void>.init)
> // Usually the pointee type is actually inferred:
> foo(ptr.map(UnsafePointer.init))
>
> I consider this a bit more difficult to understand than the original code,
> at least at a glance. We should therefore add new initializers of the
> following form:
>
> init?<OtherPointee>(_ otherPointer: UnsafePointer<OtherPointee>?) {
>   guard let nonnullPointer = otherPointer else {
>     return nil
>   }
>   self.init(nonnullPointer)
> }
>
> The body is for explanation purposes only; we'll make sure the actual
> implementation does not require an extra comparison.
>
> (This would need to be an overload rather than replacing the previous
> initializer because the "non-null-ness" should be preserved through the
> type conversion.)
>
> The alternative is to leave this initializer out, and require the nil case
> to be explicitly handled or mapped away.
>
> <https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#open-issue-unsafebufferpointer>Open
> Issue: UnsafeBufferPointer
>
> The type UnsafeBufferPointer represents a bounded typed memory region
> with no ownership or lifetime semantics; it is logically a bare typed
> pointer (its baseAddress) and a length (count). For a buffer with 0
> elements, however, there's no need to provide the address of allocated
> memory, since it can't be read from. Previously this case would be
> represented as a nil base address and a count of 0.
>
> With optional pointers, this now imposes a cost on clients that want to
> access the base address: they need to consider the nil case explicitly,
> where previously they wouldn't have had to. There are several possibilities
> here, each with their own possible implementations:
>
>    1.
>
>    Like UnsafePointer, UnsafeBufferPointer should always have a valid
>    base address, even when the count is 0. An UnsafeBufferPointer with a
>    potentially-nil base address should be optional.
>    1.
>
>       UnsafeBufferPointer's initializer accepts an optional pointer and
>       becomes failable, returning nil if the input pointer is nil.
>       2.
>
>       UnsafeBufferPointer's initializer accepts an optional pointer and
>       synthesizes a non-null aligned pointer value if given nil as a base address.
>       3.
>
>       UnsafeBufferPointer's initializer only accepts non-optional
>       pointers. Clients such as withUnsafeBufferPointermust synthesize a
>       non-null aligned pointer value if they do not have a valid pointer to
>       provide.
>       4.
>
>       UnsafeBufferPointer's initializer only accepts non-optional
>       pointers. Clients *using* withUnsafeBufferPointermust handle a nil
>       buffer.
>       2.
>
>    UnsafeBufferPointer should allow nil base addresses, i.e. the
>    baseAddress property will be optional. Clients will need to handle
>    this case explicitly.
>    1.
>
>       UnsafeBufferPointer's initializer accepts an optional pointer, but
>       no other changes are made.
>       2.
>
>       UnsafeBufferPointer's initializer accepts an optional pointer.
>       Additionally, any buffers initialized with a count of 0 will be
>       canonicalized to having a base address of nil.
>
> I'm currently leaning towards option (2i). Clients that expect a pointer
> and length probably shouldn't require the pointer to be non-null, but if
> they do then perhaps there's a reason for it. It's also the least work.
> Chris (Lattner) is leaning towards option (1ii), which treats
> UnsafeBufferPointer similar to UnsafePointer while not penalizing the
> common case of withUnsafeBufferPointer.
>
>
> <https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#impact-on-existing-code>Impact
> on existing code
>
> Any code that uses a pointer type (including Selector or NSZone) may be
> affected by this change. For the most part our existing logic to handle
> last year's nullability audit should cover this, but the implementer should
> test migration of several projects to see what issues might arise.
>
> Anecdotally, in migrating the standard library to use this new logic I've
> been quite happy with nullability being made explicit. There are many
> places where a pointer really *can't* be nil.
>
>
> <https://github.com/jrose-apple/swift-evolution/tree/optional-pointers#alternatives-considered>Alternatives
> considered
> The primary alternative here would be to leave everything as it is today,
> with UnsafePointer and friends including the null pointer as one of their
> normal values. This has obviously worked just fine for nearly two years of
> Swift, but it is leaving information on the table that can help avoid bugs,
> and is strange in a language that makes fluent use of Optional. As a fairly
> major source-breaking change, it is also something that we probably should
> do sooner rather than later in the language's evolution.
>


-- 
-- Howard.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20160322/223c809c/attachment.html>