<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class="">Ah, it's a Swift bug in the SynchronizedArray / MyClass cases, and your bug in the very first example with the global (since access to the global itself is not synchronized). The case I actually tested myself was with your most recent example ("Yet, data race can occur here").</div><div class=""><br class=""></div><div class="">Jordan</div><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Dec 5, 2017, at 15:53, Romain Jacquinot <<a href="mailto:rjacquinot@me.com" class="">rjacquinot@me.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html; charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hi Jordan,<div class=""><br class=""></div><div class="">For which specific code sample(s) do you think it’s a bug? In the previous code samples, are there cases where you think a data race is to be expected?</div><div class=""><br class=""></div><div class="">Thanks.</div><div class=""><br class=""><blockquote type="cite" class=""><div class="">On Dec 6, 2017, at 12:05 AM, Jordan Rose <<a href="mailto:jordan_rose@apple.com" class="">jordan_rose@apple.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html; charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">I'm seeing the race too when compiling with -O (and TSan enabled). I'm 95% sure this should be valid Swift code without any further synchronization, so please file a bug at <a href="https://bugs.swift.org/" class="">https://bugs.swift.org</a>. (But I'm only 95% sure because concurrency is hard.)<div class=""><br class=""></div><div class="">Looking at the backtraces, it looks like one thread thinks it has exclusive ownership of the buffer while the other thread is still copying things out of it. <b class="">This is a bug on Swift's side</b>; this code should work. I'm pretty sure this is actually a situation I was just talking about with Michael I from the stdlib team a few days ago, so it's good to have a test case for it.</div><div class=""><br class=""></div><div class=""><div class="">Jordan</div><div class=""><div class=""><br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On Dec 5, 2017, at 14:23, Romain Jacquinot via swift-users <<a href="mailto:swift-users@swift.org" class="">swift-users@swift.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html; charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Also, on the official Swift blog (<a href="https://developer.apple.com/swift/blog/?id=10" class="">https://developer.apple.com/swift/blog/?id=10</a>), it is stated that:<div class=""><br class=""></div><div class="">"One of the primary reasons to choose value types over reference types is the ability to more easily reason about your code. If you always get a unique, copied instance, you can trust that no other part of your app is changing the data under the covers. This is especially helpful in multi-threaded environments where a different thread could alter your data out from under you.</div><div class="">[…]</div><div class="">In Swift, Array, String, and Dictionary are all value types. They behave much like a simple int value in C, acting as a unique instance of that data. You don’t need to do anything special — such as making an explicit copy — to prevent other code from modifying that data behind your back. Importantly, you can safely pass copies of values across threads without synchronization. In the spirit of improving safety, this model will help you write more predictable code in Swift.”</div><div class=""><br class=""></div><div class="">Yet, data race can occur here:</div><div class=""><br class=""></div><div class=""><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; color: rgb(186, 45, 162); background-color: rgb(255, 255, 255);" class="">public<span style="" class=""> </span>class<span style="" class=""> MyClass {</span></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255); min-height: 14px;" class=""><br class=""></div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="color: #ba2da2" class="">private</span> <span style="color: #ba2da2" class="">var</span> _myArray: <span style="color: #703daa" class="">Array</span><<span style="color: #703daa" class="">Int</span>> = []</div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="color: #ba2da2" class="">private</span> <span style="color: #ba2da2" class="">var</span> _lock = <span style="color: #703daa" class="">NSLock</span>()</div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255); min-height: 14px;" class=""><br class=""></div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="color: #ba2da2" class="">public</span> <span style="color: #ba2da2" class="">var</span> myArray: <span style="color: #703daa" class="">Array</span><<span style="color: #703daa" class="">Int</span>> {</div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""> <span style="color: #4f8187" class="">lock</span>.<span style="color: #31595d" class="">lock</span>()</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""> <span style="color: #ba2da2" class="">defer</span> {</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""> <span style="color: #4f8187" class="">lock</span>.<span style="color: #31595d" class="">unlock</span>()</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""> }</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""> <span style="color: #ba2da2" class="">let</span> copy = <span style="color: #4f8187" class="">_myArray</span></div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""> <span style="color: #ba2da2" class="">return</span> copy</div></div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> }</div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255); min-height: 14px;" class=""><br class=""></div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="color: #ba2da2" class="">public</span> <span style="color: #ba2da2" class="">func</span> doSomethingThatMutatesArray() {</div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="color: #4f8187" class="">_lock</span>.<span style="color: #3e1e81" class="">lock</span>()</div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; color: rgb(0, 132, 0); background-color: rgb(255, 255, 255);" class=""><span style="" class=""> </span><span style="color: #4f8187" class="">_myArray</span><span style="" class="">.</span><span style="color: #3e1e81" class="">append</span><span style="" class="">(</span><span style="color: #272ad8" class="">1</span><span style="" class="">) </span>// data race here: write of size 8 by thread 1</div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="color: #4f8187" class="">_lock</span>.<span style="color: #3e1e81" class="">unlock</span>()</div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> }</div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">}</div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255); min-height: 14px;" class=""><br class=""></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255); min-height: 14px;" class=""><br class=""></div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""><span style="color: #ba2da2" class="">let</span> myClass = <span style="color: #4f8187" class="">MyClass</span>()</div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255); min-height: 14px;" class=""><br class=""></div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""><span style="color: #ba2da2" class="">func</span> mutateArray() {</div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; color: rgb(49, 89, 93); background-color: rgb(255, 255, 255);" class=""><span style="" class=""> </span><span style="color: #4f8187" class="">myClass</span><span style="" class="">.</span>doSomethingThatMutatesArray<span style="" class="">()</span></div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="color: #ba2da2" class="">var</span> arrayCopy = <span style="color: #4f8187" class="">myClass</span>.<span style="color: #4f8187" class="">myArray</span></div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; color: rgb(0, 132, 0); background-color: rgb(255, 255, 255);" class=""><span style="" class=""> arrayCopy.</span><span style="color: #3e1e81" class="">append</span><span style="" class="">(</span><span style="color: #272ad8" class="">2</span><span style="" class="">) </span>// data race here: read of size 8 by thread 10</div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">}</div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255); min-height: 14px;" class=""><br class=""></div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""><span style="color: rgb(186, 45, 162);" class="">let</span> q1 = <span style="color: rgb(112, 61, 170);" class="">DispatchQueue</span>(label: <span style="color: rgb(209, 47, 27);" class="">"testQ1"</span>)</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""><span style="color: rgb(186, 45, 162);" class="">let</span> q2 = <span style="color: rgb(112, 61, 170);" class="">DispatchQueue</span>(label: <span style="color: rgb(209, 47, 27);" class="">"testQ2"</span>)</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""><span style="color: rgb(186, 45, 162);" class="">let</span> q3 = <span style="color: rgb(112, 61, 170);" class="">DispatchQueue</span>(label: <span style="color: rgb(209, 47, 27);" class="">"testQ3"</span>)</div><div style="font-family: Helvetica; margin: 0px; font-stretch: normal; line-height: normal; min-height: 14px;" class=""><br class=""></div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""><span style="color: rgb(186, 45, 162);" class="">let</span> iterations = <span style="color: rgb(39, 42, 216);" class="">100_000</span></div><div style="font-family: Helvetica; margin: 0px; font-stretch: normal; line-height: normal; min-height: 14px;" class=""><br class=""></div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class="">q1.<span style="color: rgb(62, 30, 129);" class="">async</span> {</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""> <span style="color: rgb(186, 45, 162);" class="">for</span> <span style="color: rgb(186, 45, 162);" class="">_</span> <span style="color: rgb(186, 45, 162);" class="">in</span> <span style="color: rgb(39, 42, 216);" class="">0</span>..<iterations {</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""> <span style="color: rgb(49, 89, 93);" class="">mutateArray</span>()</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""> }</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class="">}</div><div style="font-family: Helvetica; margin: 0px; font-stretch: normal; line-height: normal; min-height: 14px;" class=""><br class=""></div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class="">q2.<span style="color: rgb(62, 30, 129);" class="">async</span> {</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""> <span style="color: rgb(186, 45, 162);" class="">for</span> <span style="color: rgb(186, 45, 162);" class="">_</span> <span style="color: rgb(186, 45, 162);" class="">in</span> <span style="color: rgb(39, 42, 216);" class="">0</span>..<iterations {</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""> <span style="color: rgb(49, 89, 93);" class="">mutateArray</span>()</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""> }</div></div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class="">}</div><div style="font-family: Helvetica; margin: 0px; font-stretch: normal; line-height: normal; min-height: 14px;" class=""><br class=""></div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class="">q3.<span style="color: rgb(62, 30, 129);" class="">async</span> {</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""> <span style="color: rgb(186, 45, 162);" class="">for</span> <span style="color: rgb(186, 45, 162);" class="">_</span> <span style="color: rgb(186, 45, 162);" class="">in</span> <span style="color: rgb(39, 42, 216);" class="">0</span>..<iterations {</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""> <span style="color: rgb(49, 89, 93);" class="">mutateArray</span>()</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""> }</div></div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class="">}</div><div style="font-family: Helvetica; margin: 0px; font-stretch: normal; line-height: normal; min-height: 14px;" class=""><br class=""></div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""><span style="color: rgb(186, 45, 162);" class="">for</span> <span style="color: rgb(186, 45, 162);" class="">_</span> <span style="color: rgb(186, 45, 162);" class="">in</span> <span style="color: rgb(39, 42, 216);" class="">0</span>..<iterations {</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""> <span style="color: rgb(49, 89, 93);" class="">mutateArray</span>()</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class="">}</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""><br class=""></div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class="">q1.<span style="color: rgb(62, 30, 129);" class="">sync</span> {}</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class="">q2.<span style="color: rgb(62, 30, 129);" class="">sync</span> {}</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class="">q3.<span style="color: rgb(62, 30, 129);" class="">sync</span> {}</div><div style="margin: 0px; font-stretch: normal; line-height: normal;" class=""><span style="color: rgb(62, 30, 129);" class="">NSLog</span>(<span style="color: rgb(209, 47, 27);" class="">"done"</span>)</div></div><div class=""><br class=""></div><div class="">It also can occur for instance if I replace the mutateArray() function with the following method:</div><div class=""><br class=""></div><div class=""><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""><span style="color: rgb(186, 45, 162);" class="">func</span> enumerateArray() {</div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; color: rgb(49, 89, 93); background-color: rgb(255, 255, 255);" class=""><span style="" class=""> </span><span style="color: rgb(79, 129, 135);" class="">myClass</span><span style="" class="">.</span>doSomethingThatMutatesArray<span style="" class="">()</span></div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="color: rgb(186, 45, 162);" class="">for</span> element <span style="color: rgb(186, 45, 162);" class="">in</span> <span style="color: rgb(79, 129, 135);" class="">myClass</span>.<span style="color: rgb(79, 129, 135);" class="">myArray</span> { <span style="color: rgb(0, 132, 0);" class="">// data race here: read of size 8 by thread 5</span></div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="color: rgb(186, 45, 162);" class="">let</span> <span style="color: rgb(186, 45, 162);" class="">_</span> = element</div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> }</div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">}</div><div style="margin: 0px; font-stretch: normal; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""><br class=""></div><div style="margin: 0px; font-stretch: normal; line-height: normal; background-color: rgb(255, 255, 255);" class="">How can I return a “predictable” copy from the MyClass.myArray getter, so that I can later mutate the copy without synchronization like in the <span style="font-family: Menlo;" class="">mutateArray()</span> function?</div><div class=""><br class=""><blockquote type="cite" class=""><div class="">On Dec 5, 2017, at 9:23 PM, Romain Jacquinot via swift-users <<a href="mailto:swift-users@swift.org" class="">swift-users@swift.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html; charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hi Jens,<div class=""><br class=""></div><div class="">In the SynchronizedArray class, I use a lock to perform mutating operations on the array. However, my questions concern the case where an array is exposed as a public property of a thread-safe class, like this:</div><div class=""><br class=""></div><div class="">public class MyClass {</div><div class=""><br class=""></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>private var _myArray: Array<Int> = []</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>private var _lock = NSLock()</div><div class=""><br class=""></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>public var myArray: Array<Int> {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>_lock.lock()</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>defer { _lock.unlock() }</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>return _myArray</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class=""><br class=""></div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>public func doSomethingThatMutatesArray() {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>_lock.lock()</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>_myArray.append(1)</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>_lock.unlock()</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>}</div><div class="">}</div><div class=""><br class=""></div><div class="">Arrays are implemented as structs in Swift. This is a value type, but because Array<T> implements copy-on-write, there is an issue if you do something like this from multiple threads:</div><div class=""><br class=""></div><div class="">let myClass = MyClass()</div><div class=""><br class=""></div><div class="">func callFromMultipleThreads() {</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>let arrayCopy = myClass.myArray</div><div class=""><span class="Apple-tab-span" style="white-space:pre"> </span>arrayCopy.append(2) // race condition here</div><div class="">}</div><div class=""><br class=""></div><div class="">This is quite problematic, because the user of MyClass shouldn’t have to worry about side-effects when using a copy of the value from myArray.<br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On Dec 5, 2017, at 8:22 PM, Jens Alfke <<a href="mailto:jens@mooseyard.com" class="">jens@mooseyard.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html; charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On Dec 5, 2017, at 6:24 AM, Michel Fortin via swift-users <<a href="mailto:swift-users@swift.org" class="">swift-users@swift.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><span style="font-family: Alegreya-Regular; font-size: 15px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">The array *storage* is copy on write. The array variable (which essentially contains a pointer to the storage) is not copy on write. If you refer to the same array variable from multiple threads, you have a race. Rather, use a different copy of the variable to each thread. Copied variables will share the same storage but will make a copy of the storage when writing to it.</span><br class="Apple-interchange-newline"></div></blockquote></div><br class=""><div class="">I think you’ve misunderstood. The race condition Romain is referring to is when the two threads both access the shared storage, through separate array variables.</div><div class=""><br class=""></div><div class="">Romain:</div><div class="">From the thread sanitizer warnings, it sounds like the implementation of Array is not using synchronization around its call(s) to isKnownUniquelyReferenced … which would mean the class is not thread-safe.</div><div class=""><br class=""></div><div class="">It’s pretty common for the regular (mutable) collection classes supplied by a framework to be non-thread-safe; for example consider Foundation and Java. The reason is that the overhead of taking a lock every time you access an array element is pretty high. Generally it’s preferable to use larger-granularity locks, i.e. grab an external lock before performing a number of array operations.</div><div class=""><br class=""></div><div class="">—Jens</div></div></div></blockquote></div><br class=""></div></div>_______________________________________________<br class="">swift-users mailing list<br class=""><a href="mailto:swift-users@swift.org" class="">swift-users@swift.org</a><br class=""><a href="https://lists.swift.org/mailman/listinfo/swift-users" class="">https://lists.swift.org/mailman/listinfo/swift-users</a><br class=""></div></blockquote></div><br class=""></div></div></div>_______________________________________________<br class="">swift-users mailing list<br class=""><a href="mailto:swift-users@swift.org" class="">swift-users@swift.org</a><br class=""><a href="https://lists.swift.org/mailman/listinfo/swift-users" class="">https://lists.swift.org/mailman/listinfo/swift-users</a><br class=""></div></blockquote></div><br class=""></div></div></div></div></div></blockquote></div><br class=""></div></div></blockquote></div><br class=""></body></html>