<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Nov 18, 2016, at 7:40 AM, Karl <<a href="mailto:razielim@gmail.com" class="">razielim@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On 18 Nov 2016, at 13:05, Adrian Zubarev via swift-users <<a href="mailto:swift-users@swift.org" class="">swift-users@swift.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="bloop_markdown" style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(254, 254, 254);"><p style="margin: 15px 0px; -webkit-margin-before: 0px;" class="">Hi there,</p><p style="margin: 15px 0px;" class="">I just can’t get my head around mutable views and COW.</p><p style="margin: 15px 0px;" class="">Here is a small example:</p><pre style="margin: 15px 0px; font-family: Menlo, Consolas, 'Liberation Mono', Courier, monospace; font-size: 10pt; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgb(248, 248, 248); color: inherit; border: 1px solid rgb(204, 204, 204); overflow: auto; padding: 4px 8px; word-break: normal; word-wrap: normal;" class=""><code class="swift" style="font-family: Menlo, Consolas, 'Liberation Mono', Courier, monospace; font-size: 10pt; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgb(248, 248, 248); color: inherit; border: 0px; margin: 0px; padding: 0px; word-break: normal; word-wrap: normal; -webkit-margin-before: 0px;">final class Storage {
var keys: [String] = []
var values: [Int] = []
}
public struct Document {
var _storageReference: Storage
public init() {
self._storageReference = Storage()
}
public init(_ values: DocumentValues) {
self._storageReference = values._storageReference
}
public var values: DocumentValues {
get { return DocumentValues(self) }
set { self = Document(newValue) }
}
}
public struct DocumentValues : MutableCollection {
unowned var _storageReference: Storage
init(_ document: Document) {
self._storageReference = document._storageReference
}
public var startIndex: Int {
return self._storageReference.keys.startIndex
}
public var endIndex: Int {
return self._storageReference.keys.endIndex
}
public func index(after i: Int) -> Int {
return self._storageReference.keys.index(after: i)
}
public subscript(position: Int) -> Int {
get { return _storageReference.values[position] }
set { self._storageReference.values[position] = newValue } // That will break COW
}
}
</code></pre><p style="margin: 15px 0px;" class="">First of all the<span class="Apple-converted-space"> </span><code style="font-family: Menlo, Consolas, 'Liberation Mono', Courier, monospace; font-size: 10pt; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgb(248, 248, 248); color: inherit; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; word-break: normal; word-wrap: normal; -webkit-margin-before: 0px;" class="">_storageReference</code><span class="Apple-converted-space"> </span>property is<span class="Apple-converted-space"> </span><code style="font-family: Menlo, Consolas, 'Liberation Mono', Courier, monospace; font-size: 10pt; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgb(248, 248, 248); color: inherit; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; word-break: normal; word-wrap: normal;" class="">unowned</code><span class="Apple-converted-space"> </span>because I wanted to check the following:</p><pre style="margin: 15px 0px; font-family: Menlo, Consolas, 'Liberation Mono', Courier, monospace; font-size: 10pt; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgb(248, 248, 248); color: inherit; border: 1px solid rgb(204, 204, 204); overflow: auto; padding: 4px 8px; word-break: normal; word-wrap: normal;" class=""><code class="swift" style="font-family: Menlo, Consolas, 'Liberation Mono', Courier, monospace; font-size: 10pt; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgb(248, 248, 248); color: inherit; border: 0px; margin: 0px; padding: 0px; word-break: normal; word-wrap: normal; -webkit-margin-before: 0px;">var document = Document()
print(CFGetRetainCount(document._storageReference)) //=> 2
print(isKnownUniquelyReferenced(&document._storageReference)) // true
var values = document.values
print(CFGetRetainCount(values._storageReference)) //=> 2
print(isKnownUniquelyReferenced(&values._storageReference)) // false
</code></pre><p style="margin: 15px 0px;" class="">Why is the second check false, even if the property is marked as<span class="Apple-converted-space"> </span><code style="font-family: Menlo, Consolas, 'Liberation Mono', Courier, monospace; font-size: 10pt; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgb(248, 248, 248); color: inherit; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; word-break: normal; word-wrap: normal; -webkit-margin-before: 0px;" class="">unowned</code><span class="Apple-converted-space"> </span>for the view?</p><p style="margin: 15px 0px;" class="">Next up, I don’t have an idea how to correctly COW optimize this view. Assume the following scenario:</p><p style="margin: 15px 0px;" class="">Scenario A:</p><pre style="margin: 15px 0px; font-family: Menlo, Consolas, 'Liberation Mono', Courier, monospace; font-size: 10pt; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgb(248, 248, 248); color: inherit; border: 1px solid rgb(204, 204, 204); overflow: auto; padding: 4px 8px; word-break: normal; word-wrap: normal;" class=""><code class="swift" style="font-family: Menlo, Consolas, 'Liberation Mono', Courier, monospace; font-size: 10pt; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgb(248, 248, 248); color: inherit; border: 0px; margin: 0px; padding: 0px; word-break: normal; word-wrap: normal; -webkit-margin-before: 0px;">var document = Document()
// just assume we already added some values and can mutate safely on a given index
// mutation in place
document.values[0] = 10
</code></pre><p style="margin: 15px 0px;" class="">VS:</p><p style="margin: 15px 0px;" class="">Scenario B:</p><pre style="margin: 15px 0px; font-family: Menlo, Consolas, 'Liberation Mono', Courier, monospace; font-size: 10pt; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgb(248, 248, 248); color: inherit; border: 1px solid rgb(204, 204, 204); overflow: auto; padding: 4px 8px; word-break: normal; word-wrap: normal;" class=""><code class="swift" style="font-family: Menlo, Consolas, 'Liberation Mono', Courier, monospace; font-size: 10pt; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgb(248, 248, 248); color: inherit; border: 0px; margin: 0px; padding: 0px; word-break: normal; word-wrap: normal; -webkit-margin-before: 0px;">var document = Document()
let copy = document
// just assume we already added some values and can mutate safely on a given index
// mutation in place
document.values[0] = 10 // <--- this should only mutate `document` but not `copy`
</code></pre><p style="margin: 15px 0px;" class="">We could change the subscript setter on the mutable view like this:</p><pre style="margin: 15px 0px; font-family: Menlo, Consolas, 'Liberation Mono', Courier, monospace; font-size: 10pt; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgb(248, 248, 248); color: inherit; border: 1px solid rgb(204, 204, 204); overflow: auto; padding: 4px 8px; word-break: normal; word-wrap: normal;" class=""><code class="swift" style="font-family: Menlo, Consolas, 'Liberation Mono', Courier, monospace; font-size: 10pt; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgb(248, 248, 248); color: inherit; border: 0px; margin: 0px; padding: 0px; word-break: normal; word-wrap: normal; -webkit-margin-before: 0px;">set {
if !isKnownUniquelyReferenced(&self._storageReference) {
self._storageReference = ... // clone
}
self._storageReference.values[position] = newValue
}
</code></pre><p style="margin: 15px 0px;" class="">There is only one problem here. We’d end up cloning the storage every time, because as shown in the very first example, even with<span class="Apple-converted-space"> </span><code style="font-family: Menlo, Consolas, 'Liberation Mono', Courier, monospace; font-size: 10pt; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgb(248, 248, 248); color: inherit; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; word-break: normal; word-wrap: normal; -webkit-margin-before: 0px;" class="">unowned</code><span class="Apple-converted-space"> </span>the function<span class="Apple-converted-space"> </span><code style="font-family: Menlo, Consolas, 'Liberation Mono', Courier, monospace; font-size: 10pt; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgb(248, 248, 248); color: inherit; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; word-break: normal; word-wrap: normal;" class="">isKnownUniquelyReferenced</code><span class="Apple-converted-space"> </span>will return false for scenario A.</p><p style="margin: 15px 0px;" class="">Any suggestions?<span class="Apple-converted-space"> </span></p><p style="margin: 15px 0px;" class="">PS: In general I also wouldn’t want to use<span class="Apple-converted-space"> </span><code style="font-family: Menlo, Consolas, 'Liberation Mono', Courier, monospace; font-size: 10pt; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgb(248, 248, 248); color: inherit; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; word-break: normal; word-wrap: normal; -webkit-margin-before: 0px;" class="">unowned</code><span class="Apple-converted-space"> </span>because the view should be able to outlive it’s parent.</p><div style="margin: 15px 0px;" class=""><br class="webkit-block-placeholder"></div></div><div class="bloop_original_html" style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(254, 254, 254);"><div id="bloop_customfont" style="font-family: Helvetica, Arial; font-size: 13px; margin: 0px;" class=""><br class=""></div><br class=""><div id="bloop_sign_1479469826202990080" class="bloop_sign"><div style="font-family: helvetica, arial; font-size: 13px;" class="">-- <br class="">Adrian Zubarev<br class="">Sent with Airmail</div></div></div><div class="bloop_markdown" style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(254, 254, 254);"><div style="margin: 15px 0px; -webkit-margin-before: 0px;" class=""><br class="webkit-block-placeholder"></div></div><span style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(254, 254, 254); float: none; display: inline !important;" class="">_______________________________________________</span><br style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(254, 254, 254);" class=""><span style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(254, 254, 254); float: none; display: inline !important;" class="">swift-users mailing list</span><br style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(254, 254, 254);" class=""><a href="mailto:swift-users@swift.org" style="color: rgb(65, 131, 196); background-color: rgb(254, 254, 254); text-decoration: none; font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">swift-users@swift.org</a><br style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(254, 254, 254);" class=""><a href="https://lists.swift.org/mailman/listinfo/swift-users" style="color: rgb(65, 131, 196); background-color: rgb(254, 254, 254); text-decoration: none; font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">https://lists.swift.org/mailman/listinfo/swift-users</a><br style="font-family: Helvetica, Arial; font-size: 13px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(254, 254, 254);" class=""></div></blockquote></div><br class=""><div class=""><br class=""></div><div class="">This is kind of an invalid/unsafe design IMO; DocumentValues may escape the scope of the Document and the underlying storage may be deallocated.</div><div class=""><br class=""></div><div class="">Instead, I’d recommend a function:</div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class=""><font face="Courier" class="">func withDocumentValues<T>(_ invoke: (inout DocumentValues)->T) -> T {</font></div><div class=""><font face="Courier" class=""><span class="Apple-tab-span" style="white-space:pre"> </span>var view = DocumentValues(self)</font></div><div class=""><font face="Courier" class=""> defer { _fixLifetime(view) }</font></div><div class=""><font face="Courier" class=""> return invoke(&view)</font></div><div class=""><font face="Courier" class="">}</font></div><div class=""><font face="Courier" class=""><br class=""></font></div></blockquote>(unfortunately, this isn’t completely safe because somebody could still copy the <font face="Courier" class="">DocumentValues</font> from their closure, the same way you can copy the pointer from String’s <font face="Courier" class="">withCString</font>, but that’s a limitation of Swift right now)</div></div></blockquote><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><br class=""></div><div class="">CC: John McCall, because I read his suggestion in the thread about contiguous memory/borrowing that we could have a generalised <font face="Courier" class="">@noescape</font>. In this example, you would want the <font face="Courier" class="">DocumentValues</font> parameter in the closure to be<font face="Courier" class=""> @noescape</font>.</div></div></div></blockquote><div><br class=""></div><div>I think you guys understand this stuff, but let me talk through it, and I hope it will be illuminating about where we're thinking of taking the language.</div><div><br class=""></div>In value semantics, you expect something like:</div><div> let values = document.values</div><div>to produce an independent value, and mutations of it shouldn't affect the original document value.</div><div><br class=""></div><div>But there <i class="">is</i> a situation where values aren't independent, which is when one value is just a projected component of another. In Swift, this is (currently, at least) always expressed with properties and subscripts. So when you write:</div><div> document.values.mutateInSomeWay()</div><div>this is expected to actually change the document. So it makes language sense for views like "values" to be expressed in this way; the only question is whether that can be done efficiently while still providing a satisfactory level of safety etc.</div><div><br class=""></div><div>When a property is actually stored directly in a value, Swift allows direct access to it (although for subscripts this mechanism is not currently documented + exposed, intentionally). This sort of direct access is optimal, but it's not general enough for use cases like views and slices because the slice value doesn't actually exist anywhere; it needs to be created. We do allow properties to be defined with get / set, but there are problems with that, which are exactly what you're seeing: slice values need to assert ownership of the underlying data if they're going to be used as independent values, but they also need to not assert ownership so that they don't interfere with copy-on-write. get / set isn't good enough for this because get is used to both derive an independent value (which should assert ownership) and initiate a mutation (which should not). The obvious solution is to allow a third accessor to be provided which is used when a value is mutated, as opposed to just copied (get) or overwritten whole-sale (set). We're still working out various ideas for how this will look at the language level.</div><div><br class=""></div><div>So that would be sufficient to allow DocumentValues to store either a stong or an unowned reference to the storage, depending on how the property is being used. However, that creates the problem that, like with Karl's solution, the value can be copied during the mutation, and the user would expect that to create an independent value, i.e. to promote an unowned reference to strong. The most general solution for this is to provide some sort of "copy constructor" feature which would be used to create an independent value. But that's a pretty large hammer to pull out for this nail.</div><div><br class=""></div><div>A third problem is that the original document can be copied and/or mutated during the projection of the DocumentValues, leaving the copy / the view in a potentially inconsistent state. But this is a problem that we expect to thoroughly solve with the ownership system, which will statically (or dynamically when necessary) prevent simultaneous conflicting accesses to a value.</div><div><br class=""></div><div>In the meantime, I think the best alternative is to</div><div> - allow the view to hold either an unowned or owned reference and</div><div> - create a callback-based accessor like Karl's and document that copies from the value are not permitted</div><div><br class=""></div><div>On a purely technical level:</div><div><blockquote type="cite" class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class=""><blockquote type="cite" class=""><div class=""><div class="bloop_markdown" style="font-family: Helvetica, Arial; font-size: 13px; background-color: rgb(254, 254, 254);"><pre class="" style="margin-top: 15px; margin-bottom: 15px; font-family: Menlo, Consolas, 'Liberation Mono', Courier, monospace; font-size: 10pt; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgb(248, 248, 248); color: inherit; border: 1px solid rgb(204, 204, 204); overflow: auto; padding: 4px 8px; word-break: normal; word-wrap: normal;"><code class="swift" style="font-family: Menlo, Consolas, 'Liberation Mono', Courier, monospace; font-size: 10pt; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; color: inherit; border: 0px; margin: 0px; padding: 0px; word-break: normal; word-wrap: normal; -webkit-margin-before: 0px;">print(isKnownUniquelyReferenced(&values._storageReference)) // false
</code></pre><p class="" style="margin: 15px 0px;">Why is the second check false, even if the property is marked as <code class="" style="font-family: Menlo, Consolas, 'Liberation Mono', Courier, monospace; font-size: 10pt; border-top-left-radius: 3px; border-top-right-radius: 3px; border-bottom-right-radius: 3px; border-bottom-left-radius: 3px; background-color: rgb(248, 248, 248); color: inherit; border: 1px solid rgb(234, 234, 234); margin: 0px 2px; padding: 0px 5px; word-break: normal; word-wrap: normal; -webkit-margin-before: 0px;">unowned</code> for the view?</p></div></div></blockquote></div></div></blockquote></div><div>A function taking an "inout T" expects to be passed an l-value for an ordinary (strong) reference. Swift makes this work when passing an unowned or weak reference by passing a temporary variable holding a temporarily-promoted strong reference. That's usually good, but it's wrong for isKnownUniquelyReferenced, and even more unfortunately, I don't think there's any supported way to make this work in the current compiler; you need language support.</div><div><br class=""></div><div>John.</div></body></html>