[swift-evolution] Proposed amendment to SE-0138: Normalize UnsafeRawBufferPointer Slices

Fri Mar 24 03:40:12 CDT 2017

> On 24 Mar 2017, at 07:49, Andrew Trick <atrick at apple.com> wrote:
> 
> 
>> On Mar 23, 2017, at 7:34 PM, Karl Wagner <razielim at gmail.com <mailto:razielim at gmail.com>> wrote:
>> 
>> Oh, one more thing (about this specific change): If we do this, we should add an “offset” parameter to UnsafeMutableBufferPointer.[move]initialize/assign (with a default of 0 for source compatibility). Otherwise, it becomes awkward to initialise a region of a buffer from another buffer.
>> 
>> What I want to write:
>> buffer.suffix(from: filled).initialize(from: newData)
>> 
>> If SubSequence is not another unsafe pointer, I’d have to do this:
>> buffer.baseAddress!.advanced(by: filled).initialize(from: newData)  // Warning: deprecated in Swift 4.0
> 
> Oh no. Don't do that. You're supposed to do:
> 
> UnsafeMutableRawBufferPointer(rebasing: buffer.suffix(from: filled)).initialize(from: newData)
> 
> I don't blame you for not wanting to write out the type name.
> 
>> 
>> So instead, we should have the ability to write this:
>> buffer.initialize(startingAt: filled, from: newData) // awkward labels, but src compat…
> 
> I agree that's worth doing as a result of changing the slice type. You
> shouldn't need to "rebase" the buffer each time you want to initialize
> a region. My concern is that there are a handful of APIs to fix
> (initialize, initializeMemory, copyBytes, load). We should handle them
> all in one proposal, and we may want time to bikeshed the API. I'm
> unconvinced this is worth pursuing for Swift 4. Can you file a bug?
> 

It might be related to the existing bug about unsafely initialising an Array’s storage (SR-3087). We could add these methods as a protocol extension or refined protocol on “UnsafelyInitializable”.

Then, in a similar fashion to SR-3631, which would introduce ContiguouslyStored and remove ArraySlice, Slice<T> could conditionally conform to that protocol when its base does. That would make the first example — buffer.suffix().initialize(from:), work.

https://bugs.swift.org/browse/SR-3087
https://bugs.swift.org/browse/SR-3631

Fleshing out those unsafe-initialisation protocols would be useful in general, not just for slices of unsafe buffers. For example, if you’re initialising such any such Collection from something which is ContiguouslyStored, we could optimise that operation quite a lot. Currently, the standard library has private, underscored methods on Sequence to do that: “_copyContentsToContiguousArray” and “_copyContents(initializing:)”.

I hope unsafe-initialisation is something which can make Swift 4. It would align well with other features in that release.

Where ownership is concerned, I feel we still need to express how code which wishes to avoid copies should get its data in to standard-library datatypes: the ownership manifesto only covers data at the Swift-type level, and provides no way to express ownership over an arbitrary buffer of allocated memory (that you might, for example, be able to safely wrap as an Array or String). Unsafe initialisation is a great answer to this - the standard library type owns its backing, which remains a private implementation detail, but you get the opportunity to have it allocate a buffer for you to fill up directly, using whichever unsafe operations you like and eliminating copies.

In particular, the new String model will expose its backing as a contiguous RandomAccessCollection of CodeUnits. It would be totally badass if we could unsafely initialise a standard Swift.String, using its backing storage directly as the buffer for file read/network recv operations.

- Karl

>> 
>> But yeah, this post just reminded me that there are a number of small consistency tweaks we could make to the unsafe-buffer API.
> 
> Yeah, but these are convenience issues, not correctness issues.
> 
> I think these are the ones that are important, obvious, and trivial
> enough that might make sense in Swift 4.
> 
> ** UnsafeBufferPointer should have init from mutable
> https://bugs.swift.org/browse/SR-3929 <https://bugs.swift.org/browse/SR-3929>
> 
> ** UnsafeMutableBufferPointer doesn't have an allocating init
> https://bugs.swift.org/browse/SR-3088 <https://bugs.swift.org/browse/SR-3088>
> 
> ** UnsafeBufferPointer needs a withMemoryRebound method
> https://bugs.swift.org/browse/SR-4340 <https://bugs.swift.org/browse/SR-4340>
> 
> ** Implicit Conversion: &Tuple to UnsafePointer<Tuple.Element>
> https://bugs.swift.org/browse/SR-3590 <https://bugs.swift.org/browse/SR-3590>
> 
> -Andy
> 
>> 
>> - Karl
>> 
>>> On 24 Mar 2017, at 03:22, Karl Wagner <karl.swift at springsup.com <mailto:karl.swift at springsup.com>> wrote:
>>> 
>>> The convenience initialiser should exist on all of the unsafe buffers, not just the raw (untyped) ones.
>>> 
>>> I’ve run in to this problem a few times, and I think it would get worse if we adopted a ContiguouslyStored protocol to formalise accessing the raw-pointers of generic collections. It would mean that you couldn’t write code that works with UnsafeRawBufferPointer/Data/DispatchData generically, or with UnsafeBufferPointer<T>/Array<T>.
>>> 
>>> Also, there seem to be some implicit conversions for the unsafe-pointer types, but UMBP -> UBP requires an awkward initialiser. We should introduce an implicit conversion for that case or add an “immutable” computed property to UMBP.
>>> 
>>> And while we’re on the subject, memory allocation/deallocation functions are weirdly dispersed. In order to allocate an UnsafeMutableBufferPointer<T>, for instance, you have to do:
>>> 
>>> var buffer: UnsafeMutableBufferPointer<T>
>>> init(length: Int) {
>>>   let b  = UnsafeMutablePointer<T>.allocate(capacity: length)
>>>   buffer = UnsafeMutableBufferPointer(start: b, count: length)
>>> }
>>> 
>>> Also, the deallocate API feels weird - since it deallocates n items from the head of the pointer, it is a consuming operation and I feel like it should return a new pointer (with @discardableResult). Once you’ve deallocated a memory address, you can never re-allocate that specific location so there is no reason to know about it any more.
>>> 
>>> - Karl
>>> 
>>>> On 21 Mar 2017, at 03:21, Andrew Trick via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>>> 
>>>> This proposal amends SE-0138: Normalize UnsafeRawBufferPointer Slices
>>>> to fix a design bug: https://github.com/apple/swift-evolution/pull/651 <https://github.com/apple/swift-evolution/pull/651>
>>>> 
>>>> The issue was discussed on swift-evolution in Nov/Dec:
>>>> See [swift-evolution] [Pitch] Normalize Slice Types for Unsafe Buffers
>>>> https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161128/029108.html <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20161128/029108.html>
>>>> 
>>>> The implementation of this fix is in PR #8222:
>>>> https://github.com/apple/swift/pull/8222 <https://github.com/apple/swift/pull/8222>
>>>> 
>>>> Fix: Change Unsafe[Mutable]RawBufferPointer's SubSequence type
>>>> 
>>>> Original: Unsafe[Mutable]RawBufferPointer.SubSequence = Unsafe[Mutable]RawBufferPointer
>>>> 
>>>> Fixed: Unsafe[Mutable]RawBufferPointer.SubSequence = [Mutable]RandomAccessSlice<Unsafe[Mutable]RawBufferPointer>
>>>> 
>>>> This is a source breaking bug fix that only applies to
>>>> post-3.0.1. It's extremely unlikely that any Swift 3 code would rely
>>>> on the SubSequence type beyond the simple use case of passing a
>>>> raw buffer subrange to an another raw buffer argument:
>>>> 
>>>> `takesRawBuffer(buffer[i..<j])`
>>>> 
>>>> A diagnostic message now instructs users to convert the slice to a
>>>> buffer using a `rebasing` initializer:
>>>> 
>>>> `takesRawBuffer(UnsafeRawBufferPointer(rebasing: buffer[i..<j]))`
>>>> 
>>>> To support this, the following `rebasing` initializers are added:
>>>> 
>>>> extension UnsafeRawBufferPointer {
>>>>  public init(rebasing slice: RandomAccessSlice<UnsafeRawBufferPointer>)
>>>>  public init(
>>>>    rebasing slice: MutableRandomAccessSlice<UnsafeMutableRawBufferPointer>
>>>>  )
>>>> }
>>>> 
>>>> extension UnsafeMutableRawBufferPointer {
>>>>  public init(
>>>>    rebasing slice: MutableRandomAccessSlice<UnsafeMutableRawBufferPointer>
>>>>  )
>>>> }
>>>> 
>>>> The source compatibility test builds are unnaffected by this change.
>>>> 
>>>> -Andy
>>>> _______________________________________________
>>>> swift-evolution mailing list
>>>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170324/619e13e2/attachment.html>