<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div><blockquote type="cite" class=""><div class="">On Jul 14, 2017, at 1:12 PM, Charles Srstka via swift-evolution <<a href="mailto:swift-evolution@swift.org" class="">swift-evolution@swift.org</a>> wrote:</div><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">MOTIVATION:<div class=""><br class=""></div><div class="">Meet Bob. Bob is a developer with mostly C++ and Java experience, but who has been learning Swift. Bob needs to write an app to parse some proprietary binary data format that his company requires. Bob’s written this app, and it’s worked pretty well on Linux:</div><div class=""><br class=""></div><div class=""><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""><span style="color: #ba2da2" class="">import</span> Foundation</div><div style="margin: 0px; line-height: normal; background-color: rgb(255, 255, 255); min-height: 14px;" class=""><br class=""></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; color: rgb(186, 45, 162); background-color: rgb(255, 255, 255);" class="">do<span style="" class=""> {</span></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""><span style="" class=""> </span><span style="color: rgb(186, 45, 162);" class="">let</span><span style="" class=""> url = </span><font color="#703daa" class="">...</font></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255); min-height: 16px;" class=""> <br class="webkit-block-placeholder"></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="color: #ba2da2" class="">let</span> handle = <span style="color: #ba2da2" class="">try</span> <span style="color: #703daa" class="">FileHandle</span>(forReadingFrom: url)</div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; color: rgb(0, 132, 0); background-color: rgb(255, 255, 255);" class=""><span style="" class=""> </span><span style="color: #ba2da2" class="">let</span><span style="" class=""> bufsize = </span><span style="color: #272ad8" class="">1024</span><span style="" class=""> * </span><span style="color: #272ad8" class="">1024</span><span style="" class=""> </span>// read 1 MiB at a time</div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255); min-height: 16px;" class=""> <br class="webkit-block-placeholder"></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; color: rgb(186, 45, 162); background-color: rgb(255, 255, 255);" class=""><span style="" class=""> </span>while<span style="" class=""> </span>true<span style="" class=""> {</span></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="color: #ba2da2" class="">let</span> data = handle.<span style="color: #3e1e81" class="">readData</span>(ofLength: bufsize)</div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255); min-height: 16px;" class=""> <br class="webkit-block-placeholder"></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="color: #ba2da2" class="">if</span> data.<span style="color: #703daa" class="">isEmpty</span> {</div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> <span style="color: #ba2da2" class="">break</span></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> }</div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255); min-height: 16px;" class=""> <br class="webkit-block-placeholder"></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> data.<span style="color: #3e1e81" class="">withUnsafeBytes</span> { (bytes: <span style="color: #703daa" class="">UnsafePointer</span><<span style="color: #703daa" class="">UInt8</span>>) <span style="color: #ba2da2" class="">in</span></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; color: rgb(0, 132, 0); background-color: rgb(255, 255, 255);" class=""><span style="" class=""> </span>// do something with bytes</div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> }</div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""> }</div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; color: rgb(186, 45, 162); background-color: rgb(255, 255, 255);" class=""><span style="" class="">} </span>catch<span style="" class=""> {</span></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; color: rgb(112, 61, 170); background-color: rgb(255, 255, 255);" class=""><span style="" class=""> </span><span style="color: #3e1e81" class="">print</span><span style="" class="">(</span><span style="color: #d12f1b" class="">"Error occurred: </span><span style="" class="">\</span><span style="color: #d12f1b" class="">(</span><span style="" class="">error.</span>localizedDescription<span style="color: #d12f1b" class="">)"</span><span style="" class="">)</span></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class="">}</div></div><div class=""><br class=""></div><div class="">Later, Bob needs to port this same app to macOS. All seems to work well, until Bob tries opening a large file of many gigabytes in size. Suddenly, the simple act of running the app causes Bob’s Mac to completely lock up, beachball, and finally pop up with the dreaded “This computer is out of system memory” message. If Bob’s particularly unlucky, things will locked up tight enough that he can’t even recover from there, and may have to hard-reboot the machine.</div><div class=""><br class=""></div><div class="">What happened?</div><div class=""><br class=""></div><div class="">Experienced Objective-C developers will spot the problem right away; the Foundation APIs that Bob used generated autoreleased objects, which would never be released until Bob’s loop finished. However, Bob’s never programmed in Objective-C, and to him, this behavior is completely undecipherable.</div><div class=""><br class=""></div><div class="">After a copious amount of time spent Googling for answers and asking for help on various mailing lists and message boards, Bob finally gets the recommendation from someone to try wrapping the file handle read in an autorelease pool. So he does:</div><div class=""><br class=""></div><div class=""><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""><div style="margin: 0px; line-height: normal;" class=""><span style="color: #ba2da2" class="">import</span> Foundation</div><div style="margin: 0px; font-size: 12px; line-height: normal; font-family: Helvetica; min-height: 14px;" class=""><br class=""></div><div style="margin: 0px; line-height: normal; color: rgb(186, 45, 162);" class="">do<span style="" class=""> {</span></div><div style="margin: 0px; line-height: normal;" class=""><span style="" class=""> </span><span style="color: rgb(186, 45, 162);" class="">let</span><span style="" class=""> url = </span><font color="#703daa" class="">...</font></div><div style="margin: 0px; line-height: normal; min-height: 16px;" class=""> <br class="webkit-block-placeholder"></div><div style="margin: 0px; line-height: normal;" class=""> <span style="color: #ba2da2" class="">let</span> handle = <span style="color: #ba2da2" class="">try</span> <span style="color: #703daa" class="">FileHandle</span>(forReadingFrom: url)</div><div style="margin: 0px; line-height: normal; color: rgb(0, 132, 0);" class=""><span style="" class=""> </span><span style="color: #ba2da2" class="">let</span><span style="" class=""> bufsize = </span><span style="color: #272ad8" class="">1024</span><span style="" class=""> * </span><span style="color: #272ad8" class="">1024</span><span style="" class=""> </span>// read 1 MiB at a time</div><div style="margin: 0px; line-height: normal; min-height: 16px;" class=""> <br class="webkit-block-placeholder"></div><div style="margin: 0px; line-height: normal; color: rgb(186, 45, 162);" class=""><span style="" class=""> </span>while<span style="" class=""> </span>true<span style="" class=""> {</span></div><div style="margin: 0px; line-height: normal;" class=""> <span style="color: #ba2da2" class="">let</span> data = <span style="color: #3e1e81" class="">autoreleasepool</span> { handle.<span style="color: #3e1e81" class="">readData</span>(ofLength: bufsize) }</div><div style="margin: 0px; line-height: normal; min-height: 16px;" class=""> <br class="webkit-block-placeholder"></div><div style="margin: 0px; line-height: normal;" class=""> <span style="color: #ba2da2" class="">if</span> data.<span style="color: #703daa" class="">isEmpty</span> {</div><div style="margin: 0px; line-height: normal;" class=""> <span style="color: #ba2da2" class="">break</span></div><div style="margin: 0px; line-height: normal;" class=""> }</div><div style="margin: 0px; line-height: normal; min-height: 16px;" class=""> <br class="webkit-block-placeholder"></div><div style="margin: 0px; line-height: normal;" class=""> data.<span style="color: #3e1e81" class="">withUnsafeBytes</span> { (bytes: <span style="color: #703daa" class="">UnsafePointer</span><<span style="color: #703daa" class="">UInt8</span>>) <span style="color: #ba2da2" class="">in</span></div><div style="margin: 0px; line-height: normal; color: rgb(0, 132, 0);" class=""><span style="" class=""> </span>// do something with bytes</div><div style="margin: 0px; line-height: normal;" class=""> }</div><div style="margin: 0px; line-height: normal;" class=""> }</div><div style="margin: 0px; line-height: normal; color: rgb(186, 45, 162);" class=""><span style="" class="">} </span>catch<span style="" class=""> {</span></div><div style="margin: 0px; line-height: normal; color: rgb(112, 61, 170);" class=""><span style="" class=""> </span><span style="color: #3e1e81" class="">print</span><span style="" class="">(</span><span style="color: #d12f1b" class="">"Error occurred: </span><span style="" class="">\</span><span style="color: #d12f1b" class="">(</span><span style="" class="">error.</span>localizedDescription<span style="color: #d12f1b" class="">)"</span><span style="" class="">)</span></div><div style="margin: 0px; line-height: normal;" class="">}</div></div></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);" class=""><br class=""></div><div style="margin: 0px; line-height: normal; background-color: rgb(255, 255, 255);" class="">Unfortunately, Bob’s program still eats RAM like Homer Simpson in an all-you-can-eat buffet. Turns out the data.withUnsafeBytes call *also* causes the data to be autoreleased.</div></div></div></blockquote><div><br class=""></div><div>This seems like a bug that should be fixed. I don't know why the other one would cause an unreclaimable autorelease.</div><div><br class=""></div><div>John.</div><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="margin: 0px; line-height: normal; background-color: rgb(255, 255, 255);" class=""> What Bob really needs to do is to wrap the whole thing in an autorelease pool, creating a Pyramid of Doom:</div><div style="margin: 0px; line-height: normal; background-color: rgb(255, 255, 255);" class=""><br class=""></div><div style="margin: 0px; line-height: normal; background-color: rgb(255, 255, 255);" class=""><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo;" class=""><span style="color: #ba2da2" class="">import</span> Foundation</div><div style="margin: 0px; line-height: normal; min-height: 14px;" class=""><br class=""></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; color: rgb(186, 45, 162);" class="">do<span style="" class=""> {</span></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo;" class=""><span style="" class=""> </span><span style="color: rgb(186, 45, 162);" class="">let</span><span style="" class=""> url = </span><font color="#703daa" class="">...</font></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; min-height: 16px;" class=""> <br class="webkit-block-placeholder"></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo;" class=""> <span style="color: #ba2da2" class="">let</span> handle = <span style="color: #ba2da2" class="">try</span> <span style="color: #703daa" class="">FileHandle</span>(forReadingFrom: url)</div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; color: rgb(0, 132, 0);" class=""><span style="" class=""> </span><span style="color: #ba2da2" class="">let</span><span style="" class=""> bufsize = </span><span style="color: #272ad8" class="">1024</span><span style="" class=""> * </span><span style="color: #272ad8" class="">1024</span><span style="" class=""> </span>// read 1 MiB at a time</div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; min-height: 16px;" class=""> <br class="webkit-block-placeholder"></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; color: rgb(186, 45, 162);" class=""><span style="" class=""> </span>while<span style="" class=""> </span>true<span style="" class=""> {</span></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; color: rgb(62, 30, 129);" class=""><span style="" class=""> </span>autoreleasepool<span style="" class=""> {</span></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo;" class=""> <span style="color: #ba2da2" class="">let</span> data = handle.<span style="color: #3e1e81" class="">readData</span>(ofLength: bufsize)</div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; min-height: 16px;" class=""> <br class="webkit-block-placeholder"></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo;" class=""> <span style="color: #ba2da2" class="">if</span> data.<span style="color: #703daa" class="">isEmpty</span> {</div><div style="margin: 0px; line-height: normal;" class=""><font face="Menlo" class=""><span style="font-size: 14px;" class=""> </span></font><span style="font-family: Menlo; font-size: 14px; text-decoration: underline; color: rgb(186, 45, 162);" class="">b</span><span style="font-family: Menlo; font-size: 14px; color: rgb(186, 45, 162);" class="">reak </span><font color="#008400" face="Menlo" class=""><span style="font-size: 14px;" class="">// error: ‘break’ is allowed only inside a loop, if, do, or switch</span></font></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo;" class=""> }</div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; min-height: 16px;" class=""> <br class="webkit-block-placeholder"></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo;" class=""> data.<span style="color: #3e1e81" class="">withUnsafeBytes</span> { (bytes: <span style="color: #703daa" class="">UnsafePointer</span><<span style="color: #703daa" class="">UInt8</span>>) <span style="color: #ba2da2" class="">in</span></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; color: rgb(0, 132, 0);" class=""><span style="" class=""> </span>// do something with bytes</div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo;" class=""> }</div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo;" class=""> }</div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo;" class=""> }</div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; color: rgb(186, 45, 162);" class=""><span style="" class="">} </span>catch<span style="" class=""> {</span></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; color: rgb(112, 61, 170);" class=""><span style="" class=""> </span><span style="color: #3e1e81" class="">print</span><span style="" class="">(</span><span style="color: #d12f1b" class="">"Error occurred: </span><span style="" class="">\</span><span style="color: #d12f1b" class="">(</span><span style="" class="">error.</span>localizedDescription<span style="color: #d12f1b" class="">)"</span><span style="" class="">)</span></div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo;" class="">}</div><div style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo;" class=""><br class=""></div><div style="margin: 0px; line-height: normal;" class="">However, when Bob tries to run this, he now gets a compile error on the ‘break’ statement; it’s no longer possible to break out of the loop, since everything inside the autorelease block is in a closure.</div><div style="margin: 0px; line-height: normal;" class=""><br class=""></div><div style="margin: 0px; line-height: normal;" class="">Bob is now regretting his decision not to become an insurance adjuster instead.</div><div style="margin: 0px; line-height: normal;" class=""><br class=""></div><div style="margin: 0px; line-height: normal;" class="">Bob’s problem, of course, can be solved by using *two* autorelease pools, one when getting the data, and the next when working with it. But this situation is confusing to newcomers to the language, since autorelease pools are not really part of Swift’s idiom, and aren’t mentioned anywhere in the usual Swift documentation. Thus, without Objective-C experience, autorelease-related issues are completely mysterious and baffling, particularly since, as a struct, it isn’t obvious that Objective-C will be involved at all when using the Data type. Even to experienced Objective-C developers, autorelease pools in Swift can become awkward since, unlike with Objective-C, they can’t simply be tacked onto a loop without losing flow control via break and continue.</div><div style="margin: 0px; line-height: normal;" class=""><br class=""></div><div style="margin: 0px; line-height: normal;" class="">PROPOSED SOLUTION:</div><div style="margin: 0px; line-height: normal;" class=""><br class=""></div><div style="margin: 0px; line-height: normal;" class="">In the Foundation overlay, wrap calls to Objective-C NSFileHandle and NSData APIs that generate autoreleased objects in an autorelease pool, so that they behave the way a user new to the language would expect, and in a manner consistent with how they likely behave on other platforms which lack the Objective-C bridge.</div><div style="margin: 0px; line-height: normal;" class=""><br class=""></div><div style="margin: 0px; line-height: normal;" class="">This would likely add a small performance overhead, but this should be negligible compared to the overhead involved in reading from the disk which will occur when using a FileHandle. In addition, if Data objects are being accessed frequently enough for performance to be an issue, it’s likely that enough of them to be generated to make memory overhead an issue if an autorelease pool is not used.</div><div style="margin: 0px; line-height: normal;" class=""><br class=""></div><div style="margin: 0px; line-height: normal;" class="">IMPACT ON EXISTING CODE:</div><div style="margin: 0px; line-height: normal;" class=""><br class=""></div><div style="margin: 0px; line-height: normal;" class="">Code that currently works around these issues with an autorelease pool may end up double-wrapping until these manual workarounds are removed.</div><div style="margin: 0px; line-height: normal;" class=""><span class="Apple-tab-span" style="white-space:pre"> </span></div><div style="margin: 0px; line-height: normal;" class="">Charles</div><div style="margin: 0px; line-height: normal;" class=""><br class=""></div></div></div>_______________________________________________<br class="">swift-evolution mailing list<br class=""><a href="mailto:swift-evolution@swift.org" class="">swift-evolution@swift.org</a><br class="">https://lists.swift.org/mailman/listinfo/swift-evolution<br class=""></div></blockquote></div><br class=""></body></html>