[swift-evolution] [swift-evolution-announce] [Review] SE-0176: Enforce Exclusive Access to Memory

Fri May 5 00:11:35 CDT 2017

Thanks for these reassurances, John. This is the sort of feature that calls for an iterative design-implement-design sort of process, so it’s good to know you’ve already done the static analysis explorations you describe. Given what you wrote, and given the apparent importance of this proposal to the large memory vision, I’m now +1.

This feature calls to mind two things I remember Chris Lattner wrote in the early days of the list. The first is that he preferred that Swift stick with design ideas that have been proved out in other languages. The second is that he’s concerned with the “programmer model” that features create — the whole ecosystem of habits, best practices, tools, and mental models that form around a language’s approach.

This whole constellation of memory seems to me to be fairly novel. (Maybe a little Rust-like? But very much its own beast. I don’t know of clear precedent for all of this in other languages.) It also seems this will all have a strong bearing on the programmer model, but it’s hard to fully anticipate how.

I’m nervous about how the evolution process will handle a design problem of this kind. My trust in the core team’s good judgement tempers that nervousness.

> We were initially considering the use of "access underway" flags, but eventually decided that it made more sense to use a separate global tracking system.  Currently this table is thread-local

Fascinating! I’m surprised that a table you have to scan beats flags with known offsets. Maybe the small size of that table has less cache impact than larger objects? Or is it just a space vs. time tradeoff, and you chose space? Then again, maybe it’s better that I don’t know how the sausage is made!

> In your experiments, you've no doubt observed that it takes some effort to actually get an access to be non-instantaneous.

*smirk*

Cheers, P

> On May 3, 2017, at 10:43 AM, John McCall <rjmccall at apple.com> wrote:
> 
>> On May 3, 2017, at 12:05 AM, Paul Cantrell via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>> I like the spirit of the proposal. It leaves me a bit uneasy because the practical ramifications are difficult to fully anticipate. Here follow some questions / concerns for the proposal authors.
>> 
>> (Apologies if these were answered somewhere deep in a discussion thread; it’s the end of the semester and I’ve not kept up with the list.)
>> 
>> • • •
>> 
>> First, have you done any systematic exploration of how impactful this rule is? The proposal states in several places that it hopes/expects violations will be rare, but is that empirical or speculative?
> 
> Initially, it was just well-informed speculation, but we're gathering more and more evidence that it's borne out in practice.  Even large code bases have at most a couple of static violations, and that's before we've implemented some of the refinements around struct properties.
> 
>> I tried playing around with the implications, thought I’d found a problematic example by writing an in-place Array.shuffle() method — and then was pleased to realize that the proposal already covered my example with swapAt(). In half an hour, I couldn’t come up with something the proposal disallows that it clearly should allow.
>> 
>> Still, the implications of a rule like this are difficult to anticipate. It would be interesting to know how many errors it flags in the Swift source compatibility suite, for example. Is that something we could do before going all-in on a fully robust implementation?
> 
> That is definitely an experiment we're running.  Unfortunately, we can only test for static violations with this, because checking for dynamic violations requires execution tests.
> 
>> Second, what are the performance implications of dynamic exclusivity enforcement? Introducing overhead to something as basic as variable access seems dicey. Will it kill “C-like” performance for object properties? For closure vars?
> 
> It will definitely add overhead to class member access, global variable access, and closure access.  Optimization may be able to reduce those overheads; for example, we can look at every use of a private property and often quickly prove that it is impossible to have an access violation on it.  (That can be extended to internal properties in WMO builds; it is impossible for open properties.)
> 
>> Will “accesses underway” flags add to the memory footprint of every object & every closure?
> 
> We were initially considering the use of "access underway" flags, but eventually decided that it made more sense to use a separate global tracking system.  Currently this table is thread-local, and we expect it to be small enough at any given time that a linear scan will perform acceptably.  (In your experiments, you've no doubt observed that it takes some effort to actually get an access to be non-instantaneous.)
> 
>> You write that your “hope is that…dynamic access-tracking [will be] cheap enough to enable by default;” have you proved this out with a prototype implementation yet?
> 
> This is underway.
> 
>> Or maybe this check is disabled in optimized builds, and I’m fussing over nothing?
> 
> That is something we're considering.  To quote from a different reply (that I just made, you haven't missed anything):
> 
> One direction we're exploring for dynamic enforcement is enabling it only in certain builds by default (e.g. debug configurations).  Unlike array bounds checks and integer overflows, exclusivity violations are generally not data-dependent:  it's usually true that just executing the code in any configuration will adequately test for dynamic exclusivity violations.  You can get them with races, of course, but the dynamic enforcement mechanisms we're looking at will probably not detect cross-thread violations anyway.  Of course, you can make a reasonable argument that not enabling dynamic enforcement checks in production builds would contradict Swift's general policy of "safe by default", so this is not a certain thing.  We would very much like to hear swift-evolution's thoughts about this.
> 
> John.
> 
>> 
>> Cheers,
>> 
>> Paul
>> 
>> 
>>> On May 2, 2017, at 3:07 PM, Ben Cohen <ben_cohen at apple.com <mailto:ben_cohen at apple.com>> wrote:
>>> 
>>> Hello Swift community,
>>> 
>>> The review of SE-0176: "Enforce Exclusive Access to Memory" begins now and runs through May 8, 2017.
>>> 
>>> The proposal is available here:
>>> 
>>> https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md <https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md>
>>> Reviews are an important part of the Swift evolution process. All reviews should be sent to the swift-evolution mailing list at:
>>> 
>>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
>>> or, if you would like to keep your feedback private, directly to the review manager. 
>>> 
>>> When replying, please try to keep the proposal link at the top of the message:
>>> 
>>> Proposal link:
>>> 
>>> https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md <https://github.com/apple/swift-evolution/blob/master/proposals/0176-enforce-exclusive-access-to-memory.md>
>>> Reply text
>>> 
>>> Other replies
>>> 
>>>  <https://github.com/apple/swift-evolution#what-goes-into-a-review-1>
>>> What goes into a review?
>>> 
>>> The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:
>>> 
>>> What is your evaluation of the proposal?
>>> Is the problem being addressed significant enough to warrant a change to Swift?
>>> Does this proposal fit well with the feel and direction of Swift?
>>> If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
>>> How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
>>> More information about the Swift evolution process is available at:
>>> 
>>> https://github.com/apple/swift-evolution/blob/master/process.md <https://github.com/apple/swift-evolution/blob/master/process.md>
>>> 
>>> Thanks,
>>> Ben Cohen
>>> Review Manager
>>> 
>>> _______________________________________________
>>> swift-evolution-announce mailing list
>>> swift-evolution-announce at swift.org <mailto:swift-evolution-announce at swift.org>
>>> https://lists.swift.org/mailman/listinfo/swift-evolution-announce <https://lists.swift.org/mailman/listinfo/swift-evolution-announce>
>> 
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170505/140d447b/attachment.html>