<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Dec 5, 2015, at 8:10 PM, Joe Groff <<a href="mailto:jgroff@apple.com" class="">jgroff@apple.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On Dec 5, 2015, at 11:34 AM, plx via swift-evolution <<a href="mailto:swift-evolution@swift.org" class="">swift-evolution@swift.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class=""><p class="">Swift’s standard library's core collections' value-like, copy-on-write behavior is handy in the small, but they compose rather poorly with each other: if you have e.g. [K:Set<V>], and plan to mutate the “inner" sets, it seems at present as if there is no way to avoid an unnecessary transient copy of each set for each mutation you perform upon it.</p><p class="">Various workarounds/solutions exist but all are full of unfortunate tradeoffs and/or limited further composability; in concrete cases you can do alright, but it’s hard to craft a generic solution that doesn’t inadvertently reintroduce the problem you started-out trying to solve.</p><p class="">Is anything currently on the roadmap to address this specifically — whether optimizer improvements, extended APIs, implementation adjustments, a `borrow` construct, or some other approach?</p></div></div></blockquote><br class=""></div><div class="">Do you have specific examples of problems you're trying to solve?</div></div></div></blockquote><div><br class=""></div><div><div>I'm not entirely sure if you're asking:</div><div><br class=""></div><div>1. what are some specific problems with the standard library collections' composability?</div><div>2. what are some specific problems with available solutions to the problems in (1)?</div><div>3. what are some specific problems for which *performant* nested collections would be useful?</div><div><br class=""></div><div>...so I'll answer (1) and (2) quickly, then provide some context for (3).</div><div><br class=""></div><div>Before I'll begin, I'll preface this by re-stating what I wanted to know:</div><div><br class=""></div><div>- whether or not the problems I see for (1) and (2) are seen as such</div><div>- whether or not anything is already on the roadmap to address (1) and (2)</div><div><br class=""></div><div>...and in case, thanks for your time and consideration.</div><div><br class=""></div><div>For (1), the problem is simply that when you nest Swift's collections -- at least within a `Dictionary`, e.g. `[K:Set<V>]` -- there's currently no direct way to do efficient, in-place mutations of the inner `Set<V>` values; performance here is essentially quadratic (see Appendix A).</div><div> </div><div>That you get ~quadratic performance in this context isn't surprising, given the collections' design; the aspect I see as problematic is the lack of any straightforward way to do any better.</div><div><br class=""></div><div>For (2), if you want e.g. all of the following:</div><div><br class=""></div><div>- a collection `DictionaryOfSets` behaving ~ [K:Set<V>]</div><div>- re-using the basic collection types</div><div>- with value-like semantics (e.g. to fit in with the rest of the language)</div><div>- with better-than-quadratic performance on updates</div><div><br class=""></div><div>...then the following outline seems to be the minimal amount of work to achieve all of the above:</div><div><br class=""></div><div>- a class like `class Box<T> { var storage: T }` (etc.)</div><div>- a struct like `struct _DictionaryOfSets<K,V> { var storage: [K:Box<Set<V>>] }`</div><div> - best if this implements all-or-most of the public API on `DictionaryOfSets`</div><div> - knows how to perform a deep copy of `storage`</div><div>- a struct like `struct DictionaryOfSets<K,V> { var buffer: ManagedBuffer<_DictionaryOfSets<K,V>,Void> }` </div><div> - takes care of checking `isUniquelyReferenced` and making deep-copies when needed</div><div> - with ownership now assured, forwards all calls to the buffer's `_DictionaryOfSets` </div><div> </div><div>...which is certainly tractable, but also feels rather disproportionate: to make it work we've now added 2 additional levels of indirection -- one due to the outer `ManagedBuffer`, and one more due to `Box` -- which we only need because we have no other way to manipulate the essentially-equivalent ownership-and-copying-(...etc.) behavior of `Dictionary` and `Set`.</div><div><br class=""></div><div>This all feels like it could -- and should! -- be unnecessary, with e.g. some way of indicating an intent to do an in-place update if possible.</div><div><br class=""></div><div>As for (3), a surprising amount of application-infrastructure code amounts to babysitting a nested collection of some kind.</div><div><br class=""></div><div>As one example, if you want a generic solution to the "pool-and-reuse a bunch of expensive-to-create elements" (consider e.g. `UITableView` or `UICollectionView`), a `[Key:Set<V>]`-like collection is extremely handy:</div><div><br class=""></div><div> protocol ReusableComponentType: Equatable, Hashable {</div><div> func prepareForPooling()</div><div> func prepareForReuse()</div><div> var reuseIdentifier: String { get }</div><div> }</div><div> </div><div> protocol ComponentFactoryType {</div><div> typealias Component: ReusableComponentType</div><div> func instantiateComponent(reuseIdentifier: String) -> Component</div><div> }</div><div> </div><div> class ReusableComponentPool<Component:ReusableComponentType> {</div><div> private var factories: [String:AnyComponentFactory<Component>]</div><div> private var poolSizeLimits: [String:Int]</div><div> private var reusePool: DictionaryOfSets<String,Component></div><div> // ^ NESTED COLLECTION HERE, ~ [String:Set<Component>]</div><div> </div><div> func dequeueComponentForReuseIdentifier(identifier: String) -> Component {</div><div> if let existing = self.reusePool.popFirstElementForKey(identifier) {</div><div> existing.prepareForReuse()</div><div> } else if let created = self.factories[identifier]?.instantiateComponent(reuseIdentifier: identifier) {</div><div> return created</div><div> } else {</div><div> fatalError("No registered factory for identifier '\(identifier)'!")</div><div> }</div><div> }</div><div> </div><div> func poolComponentForPotentialReuse(component: Component) {</div><div> self.reusePool.insertElement(</div><div> component,</div><div> forKey: component.identifier</div><div> )</div><div> }</div><div> </div><div> func drainOverfullPoolsIfNecessary() {</div><div> for (identifier,sizeLimit) in self.poolSizeLimits {</div><div> self.reusePool.shrinkSetForKey(</div><div> identifier,</div><div> ifAboveCount: sizeLimit</div><div> )</div><div> }</div><div> }</div><div> </div><div> }</div><div><br class=""></div><div>...but it's also a handy thing in other places (tracking active-transfers-by-host as ~`[String:Set<NSURL>]`, tracking discovered characteristics-of-services in `CoreBluetooth` as ~`[CBUUID:Set<CBUUID>]`, tracking invalidated layout attributes by supplementary-element-kind as ~`[String:Set<NSIndexPath>]`, organizing equivalence-classes for certain types of unit tests, and so on).</div><div><br class=""></div><div>Note that it's not just `[K:Set<V>]`, either; sometimes `[K:[V]]` is handy, sometimes `[K:[Q:V]]` is handy (and ~ `[(K,Q):V]` isn't right); if Swift's stdlib gets an `OrderedSet` or `OrderedDictionary` there would be uses for those, as well.</div><div><br class=""></div><div>Note that although in any specific use-case, you are arguably better off with a specific solution, e.g.:</div><div><br class=""></div><div> private class SingleComponentPool<Component:ReusableComponentType> {</div><div> let factory: AnyComponentFactory<Component></div><div> let sizeLimit: Int?</div><div> var pool: Set<Component></div><div> }</div><div> </div><div> class ReusableComponentPool<Component:ReusableComponentType> {</div><div> private var pools: [String:SingleComponentPool<Component>]</div><div> }</div><div><br class=""></div><div>...instead of the earlier above, there's still arguably a utility to having a fancy collection that can be used in various contexts, with a rich supporting API.</div><div><br class=""></div><div>In closing, thanks again for taking the time to read and thanks in advance for any consideration or response.</div><div><br class=""></div><div>## Appendix: Concrete Benchmarks</div><div><br class=""></div><div>To make the sure the performance implications are clear:</div><div><br class=""></div><div> func test10BareInserts() {</div><div> self.measureBlock() {</div><div> var target = Set<Int>()</div><div> for index in 0..<10 {</div><div> target.insert(index)</div><div> }</div><div> }</div><div> }</div><div> </div><div> func test10NestedInserts() {</div><div> self.measureBlock() {</div><div> var target = [Int:Set<Int>]()</div><div> var target[0] = Set<Int>()</div><div> for index in 0..<10 {</div><div> target[0]?.insert(index)</div><div> }</div><div> }</div><div> }</div><div><br class=""></div><div>...and so on, for e.g. `test100*`, `test1000*`, ...etc. Here's how this performs:</div><div><br class=""></div><div>- 10 inserts: </div><div> - bare: *0.000003</div><div> - nested: *0.000008</div><div>- 100 inserts:</div><div> - bare: *0.000012</div><div> - nested: *0.000073</div><div>- 1000 inserts:</div><div> - bare: *0.000081</div><div> - nested: 0.003</div><div>- 10000 inserts:</div><div> - bare: 0.001</div><div> - nested: 0.431</div><div>- 100000 inserts:</div><div> - bare: 0.009</div><div> - nested: 41.937</div><div>- 1000000 inserts:</div><div> - bare: 0.112</div><div> - nested: ????? (didn't run, but > 1 hr if trend continues...)</div><div> </div><div>...(where the un-marked values are average times, and those marked `*` are eyeballed modes over the runs, since XCTest seemingly rounds averages < 0.001 to 0.000). Tested under a Release build, FWIW.</div><div><br class=""></div><div>As always whether this matters depends on problem-size and context.</div></div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">-Joe</div><br class=""></div></div></blockquote></div><br class=""></body></html>