<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><blockquote type="cite" class="">On Jul 14, 2017, at 2:35 PM, John McCall <<a href="mailto:rjmccall@apple.com" class="">rjmccall@apple.com</a>> wrote:<br class=""></blockquote><div><blockquote type="cite" class=""><br class="Apple-interchange-newline"><div class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class=""><blockquote type="cite" class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;"><div class="">On Jul 14, 2017, at 1:12 PM, Charles Srstka via swift-evolution <<a href="mailto:swift-evolution@swift.org" class="">swift-evolution@swift.org</a>> wrote:</div><div class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">MOTIVATION:<div class=""><br class=""></div><div class="">Meet Bob. Bob is a developer with mostly C++ and Java experience, but who has been learning Swift. Bob needs to write an app to parse some proprietary binary data format that his company requires. Bob’s written this app, and it’s worked pretty well on Linux:</div><div class=""><br class=""></div><div class=""><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);"><span class="" style="color: rgb(186, 45, 162);">import</span><span class="Apple-converted-space"> </span>Foundation</div><div class="" style="margin: 0px; line-height: normal; background-color: rgb(255, 255, 255); min-height: 14px;"><br class=""></div><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; color: rgb(186, 45, 162); background-color: rgb(255, 255, 255);">do<span class=""><span class="Apple-converted-space"> </span>{</span></div><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);"><span class=""> <span class="Apple-converted-space"> </span></span><span class="" style="color: rgb(186, 45, 162);">let</span><span class=""><span class="Apple-converted-space"> </span>url =<span class="Apple-converted-space"> </span></span><font color="#703daa" class="">...</font></div><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255); min-height: 16px;"> <br class="webkit-block-placeholder"></div><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);"> <span class="Apple-converted-space"> </span><span class="" style="color: rgb(186, 45, 162);">let</span><span class="Apple-converted-space"> </span>handle =<span class="Apple-converted-space"> </span><span class="" style="color: rgb(186, 45, 162);">try</span><span class="Apple-converted-space"> </span><span class="" style="color: rgb(112, 61, 170);">FileHandle</span>(forReadingFrom: url)</div><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; color: rgb(0, 132, 0); background-color: rgb(255, 255, 255);"><span class=""> <span class="Apple-converted-space"> </span></span><span class="" style="color: rgb(186, 45, 162);">let</span><span class=""><span class="Apple-converted-space"> </span>bufsize =<span class="Apple-converted-space"> </span></span><span class="" style="color: rgb(39, 42, 216);">1024</span><span class=""><span class="Apple-converted-space"> </span>*<span class="Apple-converted-space"> </span></span><span class="" style="color: rgb(39, 42, 216);">1024</span><span class=""><span class="Apple-converted-space"> </span></span>// read 1 MiB at a time</div><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255); min-height: 16px;"> <br class="webkit-block-placeholder"></div><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; color: rgb(186, 45, 162); background-color: rgb(255, 255, 255);"><span class=""> <span class="Apple-converted-space"> </span></span>while<span class=""><span class="Apple-converted-space"> </span></span>true<span class=""><span class="Apple-converted-space"> </span>{</span></div><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);"> <span class="Apple-converted-space"> </span><span class="" style="color: rgb(186, 45, 162);">let</span><span class="Apple-converted-space"> </span>data = handle.<span class="" style="color: rgb(62, 30, 129);">readData</span>(ofLength: bufsize)</div><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255); min-height: 16px;"> <br class="webkit-block-placeholder"></div><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);"> <span class="Apple-converted-space"> </span><span class="" style="color: rgb(186, 45, 162);">if</span><span class="Apple-converted-space"> </span>data.<span class="" style="color: rgb(112, 61, 170);">isEmpty</span><span class="Apple-converted-space"> </span>{</div><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);"> <span class="Apple-converted-space"> </span><span class="" style="color: rgb(186, 45, 162);">break</span></div><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);"> }</div><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255); min-height: 16px;"> <br class="webkit-block-placeholder"></div><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);"> data.<span class="" style="color: rgb(62, 30, 129);">withUnsafeBytes</span><span class="Apple-converted-space"> </span>{ (bytes:<span class="Apple-converted-space"> </span><span class="" style="color: rgb(112, 61, 170);">UnsafePointer</span><<span class="" style="color: rgb(112, 61, 170);">UInt8</span>>)<span class="Apple-converted-space"> </span><span class="" style="color: rgb(186, 45, 162);">in</span></div><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; color: rgb(0, 132, 0); background-color: rgb(255, 255, 255);"><span class=""> <span class="Apple-converted-space"> </span></span>// do something with bytes</div><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);"> }</div><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);"> }</div><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; color: rgb(186, 45, 162); background-color: rgb(255, 255, 255);"><span class="">}<span class="Apple-converted-space"> </span></span>catch<span class=""><span class="Apple-converted-space"> </span>{</span></div><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; color: rgb(112, 61, 170); background-color: rgb(255, 255, 255);"><span class=""> <span class="Apple-converted-space"> </span></span><span class="" style="color: rgb(62, 30, 129);">print</span><span class="">(</span><span class="" style="color: rgb(209, 47, 27);">"Error occurred:<span class="Apple-converted-space"> </span></span><span class="">\</span><span class="" style="color: rgb(209, 47, 27);">(</span><span class="">error.</span>localizedDescription<span class="" style="color: rgb(209, 47, 27);">)"</span><span class="">)</span></div><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);">}</div></div><div class=""><br class=""></div><div class="">Later, Bob needs to port this same app to macOS. All seems to work well, until Bob tries opening a large file of many gigabytes in size. Suddenly, the simple act of running the app causes Bob’s Mac to completely lock up, beachball, and finally pop up with the dreaded “This computer is out of system memory” message. If Bob’s particularly unlucky, things will locked up tight enough that he can’t even recover from there, and may have to hard-reboot the machine.</div><div class=""><br class=""></div><div class="">What happened?</div><div class=""><br class=""></div><div class="">Experienced Objective-C developers will spot the problem right away; the Foundation APIs that Bob used generated autoreleased objects, which would never be released until Bob’s loop finished. However, Bob’s never programmed in Objective-C, and to him, this behavior is completely undecipherable.</div><div class=""><br class=""></div><div class="">After a copious amount of time spent Googling for answers and asking for help on various mailing lists and message boards, Bob finally gets the recommendation from someone to try wrapping the file handle read in an autorelease pool. So he does:</div><div class=""><br class=""></div><div class=""><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);"><div class="" style="margin: 0px; line-height: normal;"><span class="" style="color: rgb(186, 45, 162);">import</span><span class="Apple-converted-space"> </span>Foundation</div><div class="" style="margin: 0px; font-size: 12px; line-height: normal; font-family: Helvetica; min-height: 14px;"><br class=""></div><div class="" style="margin: 0px; line-height: normal; color: rgb(186, 45, 162);">do<span class=""><span class="Apple-converted-space"> </span>{</span></div><div class="" style="margin: 0px; line-height: normal;"><span class=""> <span class="Apple-converted-space"> </span></span><span class="" style="color: rgb(186, 45, 162);">let</span><span class=""><span class="Apple-converted-space"> </span>url =<span class="Apple-converted-space"> </span></span><font color="#703daa" class="">...</font></div><div class="" style="margin: 0px; line-height: normal; min-height: 16px;"> <br class="webkit-block-placeholder"></div><div class="" style="margin: 0px; line-height: normal;"> <span class="Apple-converted-space"> </span><span class="" style="color: rgb(186, 45, 162);">let</span><span class="Apple-converted-space"> </span>handle =<span class="Apple-converted-space"> </span><span class="" style="color: rgb(186, 45, 162);">try</span><span class="Apple-converted-space"> </span><span class="" style="color: rgb(112, 61, 170);">FileHandle</span>(forReadingFrom: url)</div><div class="" style="margin: 0px; line-height: normal; color: rgb(0, 132, 0);"><span class=""> <span class="Apple-converted-space"> </span></span><span class="" style="color: rgb(186, 45, 162);">let</span><span class=""><span class="Apple-converted-space"> </span>bufsize =<span class="Apple-converted-space"> </span></span><span class="" style="color: rgb(39, 42, 216);">1024</span><span class=""><span class="Apple-converted-space"> </span>*<span class="Apple-converted-space"> </span></span><span class="" style="color: rgb(39, 42, 216);">1024</span><span class=""><span class="Apple-converted-space"> </span></span>// read 1 MiB at a time</div><div class="" style="margin: 0px; line-height: normal; min-height: 16px;"> <br class="webkit-block-placeholder"></div><div class="" style="margin: 0px; line-height: normal; color: rgb(186, 45, 162);"><span class=""> <span class="Apple-converted-space"> </span></span>while<span class=""><span class="Apple-converted-space"> </span></span>true<span class=""><span class="Apple-converted-space"> </span>{</span></div><div class="" style="margin: 0px; line-height: normal;"> <span class="Apple-converted-space"> </span><span class="" style="color: rgb(186, 45, 162);">let</span><span class="Apple-converted-space"> </span>data =<span class="Apple-converted-space"> </span><span class="" style="color: rgb(62, 30, 129);">autoreleasepool</span><span class="Apple-converted-space"> </span>{ handle.<span class="" style="color: rgb(62, 30, 129);">readData</span>(ofLength: bufsize) }</div><div class="" style="margin: 0px; line-height: normal; min-height: 16px;"> <br class="webkit-block-placeholder"></div><div class="" style="margin: 0px; line-height: normal;"> <span class="Apple-converted-space"> </span><span class="" style="color: rgb(186, 45, 162);">if</span><span class="Apple-converted-space"> </span>data.<span class="" style="color: rgb(112, 61, 170);">isEmpty</span><span class="Apple-converted-space"> </span>{</div><div class="" style="margin: 0px; line-height: normal;"> <span class="Apple-converted-space"> </span><span class="" style="color: rgb(186, 45, 162);">break</span></div><div class="" style="margin: 0px; line-height: normal;"> }</div><div class="" style="margin: 0px; line-height: normal; min-height: 16px;"> <br class="webkit-block-placeholder"></div><div class="" style="margin: 0px; line-height: normal;"> data.<span class="" style="color: rgb(62, 30, 129);">withUnsafeBytes</span><span class="Apple-converted-space"> </span>{ (bytes:<span class="Apple-converted-space"> </span><span class="" style="color: rgb(112, 61, 170);">UnsafePointer</span><<span class="" style="color: rgb(112, 61, 170);">UInt8</span>>)<span class="Apple-converted-space"> </span><span class="" style="color: rgb(186, 45, 162);">in</span></div><div class="" style="margin: 0px; line-height: normal; color: rgb(0, 132, 0);"><span class=""> <span class="Apple-converted-space"> </span></span>// do something with bytes</div><div class="" style="margin: 0px; line-height: normal;"> }</div><div class="" style="margin: 0px; line-height: normal;"> }</div><div class="" style="margin: 0px; line-height: normal; color: rgb(186, 45, 162);"><span class="">}<span class="Apple-converted-space"> </span></span>catch<span class=""><span class="Apple-converted-space"> </span>{</span></div><div class="" style="margin: 0px; line-height: normal; color: rgb(112, 61, 170);"><span class=""> <span class="Apple-converted-space"> </span></span><span class="" style="color: rgb(62, 30, 129);">print</span><span class="">(</span><span class="" style="color: rgb(209, 47, 27);">"Error occurred:<span class="Apple-converted-space"> </span></span><span class="">\</span><span class="" style="color: rgb(209, 47, 27);">(</span><span class="">error.</span>localizedDescription<span class="" style="color: rgb(209, 47, 27);">)"</span><span class="">)</span></div><div class="" style="margin: 0px; line-height: normal;">}</div></div></div><div class="" style="margin: 0px; font-size: 14px; line-height: normal; font-family: Menlo; background-color: rgb(255, 255, 255);"><br class=""></div><div class="" style="margin: 0px; line-height: normal; background-color: rgb(255, 255, 255);">Unfortunately, Bob’s program still eats RAM like Homer Simpson in an all-you-can-eat buffet. Turns out the data.withUnsafeBytes call *also* causes the data to be autoreleased.</div></div></div></blockquote><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">This seems like a bug that should be fixed. I don't know why the other one would cause an unreclaimable autorelease.</div></div></div></div></blockquote><br class=""></div>Sticking a break at the end of my original code snippet so the loop runs only once and then running it through Instruments, it seems the NSConcreteData instance gets autoreleased… 32,769 times. o_O<div class=""><br class=""></div><div class="">First autorelease occurs inside -[NSConcreteFileHandle readDataOfLength:], the next 32,768 occur in Data.Iterator.next(), which is called by specialized RangeReplaceableCollection.init<A>.</div><div class=""><br class=""></div><div class="">I can send you the trace off-list if you’d like.</div><div class=""><br class=""></div><div class="">Charles</div><div class=""><br class=""></div></body></html>