[swift-evolution] [swift-evolution-announce] SE-0138 UnsafeBytes

Tony Parker anthony.parker at apple.com
Fri Sep 2 15:34:47 CDT 2016 

Hi Andy,

> On Sep 2, 2016, at 1:08 PM, Andrew Trick <atrick at apple.com> wrote:
> 
>> 
>> On Sep 2, 2016, at 9:31 AM, Tony Parker via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>> 
>> Hi Andy, Dave,
>> 
>> I have two major objections to this proposal.
>> 
>> The first is timing. It is too late for us to evaluate this correctly for Swift 3.
>> 
>> The second is that this API overlaps too much with Foundation’s struct Data. We should be standardizing on a small number of common types, so that developers do not need to find ways to translate one API output into another API’s input.
>> 
>> I think we should instead focus on what needs to be added to struct Data (and other API, like Stream) to fill this role. The proposal touches only touches on this briefly, but in my opinion it is the direction we should pursue — and for Swift 4 or perhaps some Swift 3 update.
>> 
>> - Tony
> 
> Foundation Data needs an interface to UnsafePointers. UnsafeBytes
> perfectly fits that need. The same is true of any Stream or
> BinaryFormat abstraction that we design in the future. In fact,
> migrating Data itself to the raw pointer changes in Swift 3 is enough
> motivation to add UnsafeBytes.
> 
> It's really unfortunate that Data's interface can't take advantage of
> UnsafeBytes in Swift 3 because it would help with adoption of
> Data. There is currently a design flaw whenever a client of Data
> doesn't know the memory's original type. I'm not happy about that, but
> I think we can live with it for a while.
> 
> The issue at hand is the large amount of Swift code out there working
> with UnsafePointers. We urgently need to provide a migration strategy
> for that code. Obviously, we don't have an urgent need to migrate code
> already using Data, so improving it's interface can wait until Swift 4.
> 
> I've seen many attempts to migrate to Swift 3 (this has been my job
> for the past month). I can claim with certainty that if we don't
> give developers a natural way to replace their UnsafePointer<UInt8>,
> we are going to be left with a lot of incorrect Swift code.
> 
> Let me make it clear that there is no overlap between Data and UnsafeBytes.
> 
> For public APIs, UnsafeBytes is meant to replace those functions that
> currently take (UnsafePointer<UInt8>, Int) including Data's own
> interfaces. Otherwise, we're strongly encouraging users to write
> incorrect code on the client side. Won't NSStream, for example,
> continue to to support UnsafePointer for those developers who need it?
> If the developer does not need UnsafePointers, that's great, and
> that's what we should continue striving for. But when developers are
> using UnsafePointer, we need a natural way to use it correctly.
> 
> For general Swift code, Unsafe means something special and
> important. The ultimate goal of the standard library and frameworks is
> that application developers never need to do something Unsafe. If
> they do, it needs to be explicitly marked Unsafe. If we are promoting
> Foundation Data as the right way to solve problems for app developers,
> then it needs to *not* be Unsafe.
> 
> UnsafeBytes simply provides a missing bridge between Unsafe pointers
> and safe APIs like Data. By definition, Data and UnsafeBytes use cases
> don't overlap. You either need to use Unsafe pointers because you're
> programming at the systems level, or existing (safe) libraries do the
> job. In practice, there are points at which these worlds meet.
> 
> So...
> 
> - Today we need UnsafeBytes so that we can migrate existing Swift
>   code correctly to a well-defined memory model.
> 
> - In the future we need UnsafeBytes to safely implement the transitions
>   between "systems code" and "application code”.
> 
> -Andy

If the goal is to simplify the story for developers, so they can understand the complicated topic of the way that binding memory works in Swift, then introducing a new intermediate type feels to me like working in the opposite direction.

I would instead prefer to look at what we can do with adding API to the existing types to cover this use case. You point out in the proposal that it has become customary to use [UInt8] in API. I would prefer that we work towards a solution that makes it customary to use Data when you want to expose an API that uses Data. The example of migrated code in the proposal illustrates my concern. It suggests that the conversion for handleMessages should end here:

func handleMessages(_ bytes: UnsafeBytes) -> Int

I think instead handleMessages should take a Data argument. The input driver code should be able to use API on Data (or elsewhere, API that returns Data) to populate it with the contents of the file. For example, the existing:

public init(contentsOf url: URL, options: Data.ReadingOptions = []) throws 

Or by extending Stream, if necessary.

The JSON example advocates converting the struct Data to an NSData to access the bytes property. We specifically introduced the withUnsafeBytes argument to keep people from doing this. If there is an issue with it, we need to address it in struct Data, not by asking people to create another instance of a new type after bridging to the reference. Furthermore, with the lack of bridging on Linux plus the lack of the autoreleasepool that makes the bytes property safe in the first place, this example won’t even work there. 

Arguments of being out of time do not compel me, personally. Introducing a new type is effectively permanent. We just got started with introducing Swift API. I don’t want to accidentally saddle ourselves with additional complexity forever without giving ourselves an adequate opportunity to fully consider what the right long-term solution is. The fact that you’ve already described the mismatch between struct Data and this type as “unfortunate” sounds, to me, like a big problem.

- Tony

> 
>> 
>>> On Sep 1, 2016, at 3:18 PM, Andrew Trick <atrick at apple.com <mailto:atrick at apple.com>> wrote:
>>> 
>>> I’m resending this for Review Manager Dave A. because the announce list is dropping his messages...
>>> 
>>> Hello Swift community,
>>> 
>>> The review of "UnsafeBytes" begins now and runs through September
>>> 7th. This late addition to Swift 3 is a follow-up to SE-0107:
>>> UnsafeRawPointer. It addresses common use cases for UnsafeRawPointer,
>>> allowing developers to continue working with collections of UInt8 values,
>>> but now doing so via a type safe API. The UnsafeBytes API will not require 
>>> direct manipulation of raw pointers or reasoning about binding memory.
>>> 
>>> The proposal is available here:
>>> 
>>>  <https://github.com/apple/swift-evolution/blob/master/proposals/0138-unsafebytes.md <https://github.com/apple/swift-evolution/blob/master/proposals/0138-unsafebytes.md>>
>>> 
>>> Reviews are an important part of the Swift evolution process. All reviews
>>> should be sent to the swift-evolution mailing list at
>>> 
>>>  <https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>>
>>> 
>>> or, if you would like to keep your feedback private, directly to the
>>> review manager. When replying, please try to keep the proposal link at
>>> the top of the message:
>>> 
>>> Proposal link:
>>>  <https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>>
>>> 
>>> What goes into a review?
>>> 
>>> The goal of the review process is to improve the proposal under review
>>> through constructive criticism and, eventually, determine the direction of
>>> Swift. When writing your review, here are some questions you might want to
>>> answer in your review:
>>> 
>>>  * What is your evaluation of the proposal?
>>>  * Is the problem being addressed significant enough to warrant a
>>>    change to Swift?
>>>  * Does this proposal fit well with the feel and direction of Swift?
>>>  * If you have used other languages or libraries with a similar
>>>    feature, how do you feel that this proposal compares to those?
>>>  * How much effort did you put into your review? A glance, a quick
>>>    reading, or an in-depth study?
>>> 
>>> More information about the Swift evolution process is available at
>>> 
>>>  <https://github.com/apple/swift-evolution/blob/master/process.md <https://github.com/apple/swift-evolution/blob/master/process.md>>
>>> 
>>> Thank you,
>>> 
>>> -Dave Abrahams
>>> Review Manager
>>> _______________________________________________
>>> swift-evolution-announce mailing list
>>> swift-evolution-announce at swift.org <mailto:swift-evolution-announce at swift.org>
>>> https://lists.swift.org/mailman/listinfo/swift-evolution-announce <https://lists.swift.org/mailman/listinfo/swift-evolution-announce>
>> 
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20160902/ead2cdf7/attachment.html>

More information about the swift-evolution mailing list