<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On 30 Oct 2016, at 19:23, Dave Abrahams via swift-users <<a href="mailto:swift-users@swift.org" class="">swift-users@swift.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class=""><br class="">on Sun Oct 30 2016, Karl <<a href="http://swift-users-AT-swift.org" class="">swift-users-AT-swift.org</a>> wrote:<br class=""><br class=""><blockquote type="cite" class=""><blockquote type="cite" class="">On 30 Oct 2016, at 09:15, Karl <<a href="mailto:raziel.im+swift-users@gmail.com" class="">raziel.im+swift-users@gmail.com</a>> wrote:<br class=""><br class="">I had the need for a concurrent map recently. I had a part of a<br class="">program which needed to read chunks of data and concurrently process<br class="">them and assemble the results in an array. This isn’t necessarily as<br class="">obvious as it sounds, because of arrays being value types. I came up<br class="">with the following snippet which I’d like to check for correctness;<br class="">it could also be helpful to others.<br class=""><br class="">Perhaps this is something Dispatch should provide out-of-the-box?<br class=""><br class="">- Karl<br class=""></blockquote><br class="">Ah one second, I was refactoring this and forgot to test it. Here’s the actual code:<br class=""></blockquote><br class="">A map presumably requires an input <br class=""></div></div></blockquote><div><br class=""></div>DispatchQueue.concurrentMap maps a Range<Int> -> T, but since the range is always 0..<n, we only ask for the value of n. It could also be written quite naturally as an extension on Range and build everything on top of it.</div><div><br class=""></div><div><blockquote type="cite" class=""><div class=""><div class=""><br class=""><blockquote type="cite" class="">extension DispatchQueue {<br class=""><br class=""> static func concurrentMap<T>(iterations: Int, execute block: (Int) -> T) -> [T] {<br class=""><br class=""> let __result = UnsafeMutableRawBufferPointer.allocate(count: iterations * MemoryLayout<T>.stride)<br class=""> defer { __result.deallocate() }<br class=""> let _result = __result.baseAddress?.assumingMemoryBound(to: T.self)<br class=""></blockquote><br class="">You never bound the memory to T, so this will be undefined behavior. <br class=""><br class=""><blockquote type="cite" class=""> let result = UnsafeMutableBufferPointer<T>(start: _result, count: iterations)<br class=""> concurrentPerform(iterations: iterations) { idx in<br class=""> result[idx] = block(idx)<br class=""></blockquote><br class="">You also never initialized the Ts in that memory region, so assigning<br class="">into them will also be undefined behavior.<br class=""><br class=""><blockquote type="cite" class=""> }<br class=""> return Array(result)<br class=""> }<br class="">}<br class=""><br class="">extension Array {<br class=""> func concurrentMap<T>(execute block: (Element)->T) -> [T] {<br class=""> return DispatchQueue.concurrentMap(iterations: count) { block(self[$0]) }<br class=""> }<br class="">}<br class=""><br class="">Unfortunately I don’t think there’s a way to get an array to take over a +1<br class="">UnsafeMutableBufferPointer without copying.<br class=""></blockquote><br class="">The only correct way to do this without creating intermediate storage is<br class="">to have a way to initialize your result elements, e.g.:<br class=""><br class=""> import Dispatch<br class=""><br class=""> protocol DefaultInitializable {<br class=""> init()<br class=""> }<br class=""><br class=""> extension RandomAccessCollection {<br class=""> func concurrentMap<T>(_ transform: (Iterator.Element)->T) -> [T]<br class=""> where T : DefaultInitializable {<br class=""> var result = Array(<br class=""> repeating: T(), count: numericCast(self.count))<br class=""><br class=""> DispatchQueue.concurrentPerform(iterations: result.count) {<br class=""> offset in <br class=""> result[offset] = transform(<br class=""> self[index(startIndex, offsetBy: numericCast(offset))])<br class=""> }<br class=""> return result<br class=""> }<br class=""> }<br class=""><br class=""> extension Int : DefaultInitializable { }<br class=""><br class=""> print((3..<20).concurrentMap { $0 * 2 })<br class=""><br class=""></div></div></blockquote><div><br class=""></div><div>I had a go at doing that before, using Optional<T> and unwrapping at the end — but it occurred to me that it would be very inefficient for things like Optional<Int>, and introduces more allocations.</div><div><br class=""></div><br class=""><blockquote type="cite" class=""><div class=""><div class="">If you don't want the DefaultInitializable requirement (or some other<br class="">way to prepare initialized elements), you'll need to manage memory<br class="">yourself:<br class=""><br class=""> extension RandomAccessCollection {<br class=""> func concurrentMap<T>(_ transform: (Iterator.Element)->T) -> [T] {<br class=""> let n = numericCast(self.count) as Int<br class=""> let p = UnsafeMutablePointer<T>.allocate(capacity: n)<br class=""> defer { p.deallocate(capacity: n) }<br class=""><br class=""> DispatchQueue.concurrentPerform(iterations: n) {<br class=""> offset in<br class=""> (p + offset).initialize(<br class=""> to: transform(<br class=""> self[index(startIndex, offsetBy: numericCast(offset))]))<br class=""> }<br class=""><br class=""> return Array(UnsafeMutableBufferPointer(start: p, count: n))<br class=""> }<br class=""> }<br class=""><br class="">This posting highlights a couple of weaknesses in the standard library<br class="">for which I'd appreciate bug reports:<br class=""><br class="">1. No way to arbitrarily initialize an Array's storage.<br class="">2. UnsafeMutableBufferPointer doesn't have an allocating init<br class=""><br class="">Thanks!<br class=""><br class="">-- <br class="">-Dave<br class=""><br class="">_______________________________________________<br class="">swift-users mailing list<br class=""><a href="mailto:swift-users@swift.org" class="">swift-users@swift.org</a><br class="">https://lists.swift.org/mailman/listinfo/swift-users<br class=""></div></div></blockquote></div><br class=""><div class=""><br class=""></div><div class="">Filed:</div><div class=""><br class=""></div><div class="">1. <a href="https://bugs.swift.org/browse/SR-3087" class="">https://bugs.swift.org/browse/SR-3087</a></div><div class="">2. <a href="https://bugs.swift.org/browse/SR-3088" class="">https://bugs.swift.org/browse/SR-3088</a></div><div class=""><br class=""></div><div class="">What is your opinion on the corelibs extending the standard library types? Foundation does it to provide APIs from NSString, but it’s kind of a special case. Would it be reasonable for Dispatch (which is not _such_ a special case) to also extend types like Range and Collection?</div><div class=""><br class=""></div><div class="">I quite like the API as an extension on Range. I think it would be a nice addition to Dispatch (once we start allowing additive proposals):</div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class="">extension Range where Bound : Strideable, Bound.Stride : SignedInteger {</div><div class=""><br class=""></div><div class=""> func concurrentMap<T>(_ transform: (Bound) -> T) -> [T] {</div><div class=""> let n = numericCast(count) as Int</div><div class=""> let buffer = UnsafeMutablePointer<T>.allocate(capacity: n)</div><div class=""><br class=""></div><div class=""> DispatchQueue.concurrentPerform(iterations: n) {</div><div class=""> (buffer + $0).initialize(to: transform(lowerBound + numericCast($0)))</div><div class=""> }</div><div class=""><br class=""></div><div class=""> // Unfortunately, the buffer is copied when making it an Array<T>.</div><div class=""> defer { buffer.deallocate(capacity: n) }</div><div class=""> return Array(UnsafeMutableBufferPointer<T>(start: buffer, count: n))</div><div class=""> }</div><div class="">}</div><div class=""><br class=""></div><div class="">extension Collection {</div><div class=""> func concurrentMap<T>(_ transform: (Iterator.Element)->T) -> [T] {</div><div class=""><br class=""></div><div class=""> // ‘as Range’ because CountableRange is a collection, causing the function to be recursive.</div><div class=""> return ((0..<numericCast(count)) as Range).concurrentMap {</div><div class=""> transform(self[index(startIndex, offsetBy: numericCast($0))])</div><div class=""> }</div><div class=""> }</div><div class="">}</div><div class=""><br class=""></div></blockquote>Thanks<div class=""><br class=""></div><div class="">- Karl<br class=""><div class=""><br class=""></div></div></body></html>