[swift-evolution] [Proposal] Add an API to Unmanaged to get the instance it holds "@guaranteed"

Mon Mar 14 17:08:26 CDT 2016

> On Mar 14, 2016, at 1:29 PM, John McCall <rjmccall at apple.com> wrote:
> 
>> On Mar 14, 2016, at 12:51 PM, Arnold Schwaighofer <aschwaighofer at apple.com> wrote:
>>> On Mar 14, 2016, at 11:53 AM, John McCall <rjmccall at apple.com> wrote:
>>> 
>>>> On Mar 14, 2016, at 8:00 AM, Arnold Schwaighofer <aschwaighofer at apple.com> wrote:
>>>> I would like to propose adding an additional method to the standard library’s ‘Unmanaged’ wrapper to mark values as having a guaranteed lifetime by another reference to it.
>>>> 
>>>> Currently, there is no way to opt out of ARC even if we know the lifetime of a value is guaranteed by another reference to it. We can pass around values in the wrapper without incurring reference counting but as soon as we take the instance out of the wrapper we will incur reference counting.
>>>> 
>>>> func doSomething(u: Unmanaged<Owned>
>>>> ) {
>>>> 
>>>> // Incurs refcount increment before the call and decrement after for self.
>>>> 
>>>>   u
>>>> .takeUnretainedValue().doSomeWork()
>>>> 
>>>> The addition of this API will allow selectively disabling ARC for the scope of the lifetime of the value returned by this API call.
>>> 
>>> Calling this method makes a semantic guarantee that the returned reference is being kept alive somehow for a certain period of time.  So, the obvious questions here are:
>>> 
>>> 1. What are the bounds of that period of time?
>> 
>> I mention in the API doc that this is for the duration of lifetime the returned value.
>> 
>> var x = takeUnretainedValue()
>> var y = x
>> 
>> In this example this would be for the lifetime for the value in x which extends to the lifetime of y.
> 
> What if I return it or assign it to non-local memory?

See my comment about the l-value.

> 
> I feel like you’re trying to define this by optimizer behavior.  That’s not a workable language rule; programmers are not going to reason about SSA values.

I did mentioned that this applies to storing the value in an l-value. 

But I think you are saying to define this in terms of a “precise lifetime language”. I will try to talk in person to flesh out the language.

> 
>> I will try to address your comments on needing more precise language here (precise lifetime), i.e for locals we need to fix the lifetime, classes, globals.
>> 
>> 
>>> 2. How can the user actually satisfy this guarantee?
>> 
>> By “fixing the lifetime” of the value is one way or by relying on lifetime relation ships, see the class example I give. The fact that we are calling a method on "self: Owner" guarantees that the reference in “ref: Ownee” was not ultimately released (Admittedly, this is relying on self being passed @guaranteed). The user must know that nobody will store to “ref” in-between.
>> 
>> Your point is that this needs clear language.
>> 
>> "
>> // Get the value of the unmanaged referenced as a managed reference without
>> // consuming an unbalanced retain of it. Asserts that some other owner 
>> // guarantees the lifetime of the value for the lifetime of the return managed
>> // reference. You are responsible for making sure that this assertion holds
>> // true.
>> //

Comment about the l-value:

>> // NOTE:
>> // Be aware that storing the returned managed reference in an L-Value extends
>> // the lifetime this assertion must hold true for by the lifetime of the  
>> // L-Value.
>> //   var owningReference = Instance()  
>> //   var lValue : Instance
>> //   withFixedLifetime(owningReference) { 
>> //     lValue = Unmanaged.passUnretained(owningReference).takeGuaranteedValue()
>> //   }
>> //   lValue.doSomething() // Bug: owningReference lifetime has ended earlier.
>> "
>> 
>>> 3. What happens if they’re wrong?
>> 
>> Undefined behavior. This is an unsafe extension. Users can shoot themselves in the foot today using Unmanaged by calling release() on it.
>> 
>>> 
>>> #1 is totally unclear in your proposal.  Usually we do this sort of scoping thing with a callback, rather than a property, for exactly the reason that it lets us naturally bound the extent of the guarantee.  The library’s implementation of that function would then use some builtin functions to bound how long the guarantee lasts.  If we decide that that’s too awkward to actually use, we could burn this into the language as a custom result convention and invent some language rules for the guaranteed extents of such a result based on how the expression is used; that would permit more APIs to opt in to a +0 guaranteed result, but the language rules would probably be pretty ad hoc.  
>> 
>> 
>> I am not sure you can bound it this that way. I think we want this assertion to be propagated through l-values.
>> 
>> 
>> var outlive: Ownee
>> var owningReference: Ownee
>> 
>> withFixedLifetime(owningReference) {
>> Unamanged.passUnretained(owningReference).withGuaranteedValue {
>>    escape($0, to: &outlive)
>> }
>> 
>> I think we should just define this as undefined behavior.
> 
> Why?  It doesn’t seem harmful.  It’s just that the guarantee doesn’t apply.

Yes the guarantee does not apply, the user has escaped “the returned value to an l-value that outlives the “owningReference”.
> 
> …to be clear, I certainly hope that the intended optimization strategy here isn’t just to unconditionally remove any retains and releases visible within the guaranteed block.  You can still at best remove them in pairs.  The effect on the optimizer here is just that uses that are known to occur within a guaranteed block can be ignored for the purposes of eliminating imprecise retain/release pairs.

Yes, only paired removal. Just like we can do with @guaranteed parameters today.

But imagine the “escape” function has been inlined. The optimizer will see:

Unamanged.passUnretained(owningReference).withGuaranteedValue {
   outlive = $0
}

This is not different to

Unamanged.passUnretained(owningReference).withGuaranteedValue {
   let x = $0
   x.doSomething()
}

And we want to optimize the latter.

The closure syntax does not add any guarantee.

> 
>>> #2 is also uncertain.  In ObjC ARC, we have a concept of an “object with precise lifetime semantics”.  (Here, “object” is used in the C/C++ sense of “variable”, not the reference-type-OO sense of “class instance”.)  Globals always have precise lifetime semantics, locals can opt in with an attribute, and members depend on their container.  So one possible answer is to say that the value must be precisely referenced somehow, perhaps by the system. That does require doing some leg work in the proposal (to import the concept of precise lifetime), the language (to allow locals to be made precise), and and the compiler/optimizer (to honor precise guarantees).
>> 
>> We have the withFixedLifetime API in the standard library today.
>> 
>> withFixedLifetime(ownningReference) {
>> }
>> 
>> Users can rely on that a stored reference in a class is not ultimately released during a method call on that class instance if they can guaranteed that nobody stores to that reference.
>> 
>> The compiler honor withFixedLifetime today (if not that is a bug).
>> 
>> Your point is that this needs to be made clear.
> 
> Yeah, I think withFixedLifetime is probably not a good enough tool; at the very least, you need to reason about indirect references.  (e.g. if the object is held by an array element or a class property, then the array/instance itself needs to be held precisely, etc.)  It’s probably better for local values than a precise-lifetime attribute on local variables, though, assuming withFixedLifetime works for value types, and assuming it doesn’t actually cause reference-counting traffic itself.

withFixedLifetime just inserts the SIL builtin fix_lifetime (assuming optimization happens).

fix_lifetime
```````````` 

Acts as a use of a value operand, or of the value in memory referenced by an                                                     
address operand. Optimizations may not move operations that would destroy the                                                    
value, such as ``release_value``, ``strong_release``, ``copy_addr [take]``, or                                                   
``destroy_addr``, past this instruction.

This guarantee is strong enough to guarantee the “owningReference” lifetime.

> 
>>> It sounds like the only reasonable answer for #3 is “it’s undefined behavior”.  That argues for putting “unsafe” somewhere in the name, although perhaps that’s already covered by the proposal to rename Unmanaged to UnsafeReference.
>> 
>> I believe, Unmanaged is already unsafe. I have put unsafe in the name of the SIL builtin, I am happy to propagate this to the API call.
> 
> Unmanaged is already unsafe.  Like I said, there’s an extant proposal to rename it to something like UnsafeReference.  I was raising this issue to see whether swift-evolution has a sense of whether that’s good enough or whether we should call attention to the additional unsafely of this API.

Ok.