<div dir="ltr"><div><div><div><div>Kitura does not mandate any particular distribution,
but 14.04 is one of the distributions that we expect to support for
some time.<br><br></div>I did investigate what's required to build a
Swift snapshot with a newer version of ICU on Linux. I was pleased to
discover that of the support for doing so is already there, however the
Foundation build needs tweaking to point to the newly built version. I
opened a couple of PRs to fix it, but then someone pointed out an
existing (unmerged) PR that fixes it in a simpler way:<br><a href="https://github.com/apple/swift-corelibs-foundation/pull/722" target="_blank">https://github.com/apple/<wbr>swift-corelibs-foundation/<wbr>pull/722</a><br><br></div>With #722, I can build a new snapshot on Ubuntu 14.04 or 16.04 using ICU 57.1 from <a href="http://site.icu-project.org/download/57#TOC-ICU4C-Download" target="_blank">http://site.icu-project.org/<wbr>download/57#TOC-ICU4C-Download</a><br></div>I'm using the buildbot_linux preset, so I added libicu=true and install-libicu to the preset in utils/build-presets.ini<br><br></div>With
this build, Kitura's 14.04 performance is much closer to the memcmp
version (leaving around 8% gap, similar to 16.04, which is what I'd
expect):<br><br><font face="monospace,monospace"><span class="gmail-im">Language (Runtime) Framework (Middleware) Max [sec] Min [sec] Ave [sec]<br>------------------------- ------------------------- --------------- --------------- ---------------<br></span>swift kitura_40 <wbr> 8.131616 7.634630 7.959684<br>swift kitura_40_memcmp <wbr> 5.072511 4.784608 4.963862</font><br><font face="monospace,monospace">swift kitura_40_icu <wbr> 5.523411 5.297158 5.367100</font></div><div class="gmail_extra"><br><div class="gmail_quote">On 31 July 2017 at 16:57, David Jones <span dir="ltr"><<a href="mailto:djones6dev@gmail.com" target="_blank">djones6dev@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div>Kitura does not mandate any particular distribution, but 14.04 is one of the distributions that we expect to support for some time.<br><br></div>I did investigate what's required to build a Swift snapshot with a newer version of ICU on Linux. I was pleased to discover that of the support for doing so is already there, however the Foundation build needs tweaking to point to the newly built version. I opened a couple of PRs to fix it, but then someone pointed out an existing (unmerged) PR that fixes it in a simpler way:<br><a href="https://github.com/apple/swift-corelibs-foundation/pull/722" target="_blank">https://github.com/apple/<wbr>swift-corelibs-foundation/<wbr>pull/722</a><br><br></div>With #722, I can build a new snapshot on Ubuntu 14.04 or 16.04 using ICU 57.1 from <a href="http://site.icu-project.org/download/57#TOC-ICU4C-Download" target="_blank">http://site.icu-project.org/<wbr>download/57#TOC-ICU4C-Download</a><br></div>I'm using the buildbot_linux preset, so I added libicu=true and install-libicu to the preset in utils/build-presets.ini<br><br></div><div>With this build, Kitura's 14.04 performance is much closer to the memcmp version (leaving around 8% gap, similar to 16.04, which is what I'd expect):<br><br><font face="monospace,monospace"><span class="">Language (Runtime) Framework (Middleware) Max [sec] Min [sec] Ave [sec]<br>------------------------- ------------------------- --------------- --------------- ---------------<br></span>swift kitura_40 <wbr> 8.131616 7.634630 7.959684<br>swift kitura_40_memcmp <wbr> 5.072511 4.784608 4.963862</font><br><font face="monospace,monospace">swift kitura_40_icu <wbr> 5.523411 5.297158 5.367100<br><br></font></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On 27 July 2017 at 18:48, Michael Ilseman <span dir="ltr"><<a href="mailto:milseman@apple.com" target="_blank">milseman@apple.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div>The performance issues in Ubuntu 14.04 is not surprising. 14.04’s ICU version is ancient, relatively speaking (e.g. it predates both Swift and Unicode 7.0). Performance issues is not the only problem there, as Unicode semantics such as grapheme breaking has evolved dramatically since then. Out of curiosity, is there a reason why Kitura is stuck on 14.04? Can your platform bundle a newer ICU so that your users can have things like modern emoji support?</div><div><div class="m_-8706833233740895112h5"><div><br></div><div><br></div><br><div><blockquote type="cite"><div>On Jul 27, 2017, at 7:28 AM, David Jones via swift-dev <<a href="mailto:swift-dev@swift.org" target="_blank">swift-dev@swift.org</a>> wrote:</div><br class="m_-8706833233740895112m_2463072520094240643Apple-interchange-newline"><div><div dir="ltr"><div><div><div>FWIW, here is an illustration of how much an optimization in this area would be worth to Kitura, running on Ubuntu 14.04 (where ICU 52 is particularly expensive when comparing ASCII strings [1]).<br></div>The workload I used here is <a href="https://github.com/tbrand/which_is_the_fastest" target="_blank">https://github.com/tbrand/whic<wbr>h_is_the_fastest</a><br></div>I have compiled Kitura 1.7.6 with the latest 4.0 toolchain (07-26-a), and again with the same toolchain modified to remove the #if in StringCompare's ==.<br></div><div><div><div><br><span style="font-family:monospace,monospace">djones6@needletail:~/which_is_<wbr>the_fastest$ numactl --physcpubind=0-3,16-19 --membind=0 bin/benchmarker kitura_40 kitura_40_memcmp<br>Language (Runtime) Framework (Middleware) Max [sec] Min [sec] Ave [sec]<br>------------------------- ------------------------- --------------- --------------- ---------------<br>swift kitura_40 <wbr> 7.824673 7.682657 7.740933<br>swift kitura_40_memcmp <wbr> 5.163788 4.811082 4.955571<br></span><br></div><div>The difference here is around 35%. Other Kitura workloads I've performed this comparison on in the past (such as involving JSON serialization) have showed a difference in the 15 - 20% region. <br></div><div>The difference is far smaller on Ubuntu 16.04 (around 8% for this workload), due to improvements in the newer level of ICU:<br><br><span style="font-family:monospace,monospace">djones6@gruffalo:~/which_is_th<wbr>e_fastest$ numactl --physcpubind=0-3,16-19 --membind=0 bin/benchmarker kitura_40 kitura_40_memcmp<br>Language (Runtime) Framework (Middleware) Max [sec] Min [sec] Ave [sec]<br>------------------------- ------------------------- --------------- --------------- ---------------<br>swift kitura_40 <wbr> 4.691993 4.531465 4.580086<br>swift kitura_40_memcmp <wbr> 4.349387 4.015061 4.201105<br></span><br></div><div>- David.<br>---<br></div><div>David Jones, Swift@IBM<br></div><div><br></div><div>[1] <a href="https://github.com/apple/swift/pull/7339" target="_blank">https://github.com/apple/swift<wbr>/pull/7339</a><br><div><div class="gmail_extra"><br><div class="gmail_quote">On 26 July 2017 at 03:59, Michael Ilseman via swift-dev <span dir="ltr"><<a href="mailto:swift-dev@swift.org" target="_blank">swift-dev@swift.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div>Unfortunately after some investigations and discussion, the situation seems to be more murky. This approach would break transitivity of String comparison on Linux, at least with any implementation of UCA using the normal collation weights. A < B, B < C should imply A < C. But, if both A and B are known-ASCII while C is UTF16, transitivity can be broken for any character that UCA yields a different sort order for (e.g. “#” vs “&”). On Darwin, the comparison implementation happens to preserve transitivity as the platform (in effect) relatively weights ASCII by code unit values.</div><div><br></div><div>While I would like to get some performance improvements in time for Linux, I don’t think this approach is viable for Swift 4.0. Unless anyone has any ideas about another minimally invasive approach, my recommendation is to do the long-term solution (lexicographical order of normalized code units) immediately after Swift 4.0.</div><br><div><br><blockquote type="cite"><div><div class="m_-8706833233740895112m_2463072520094240643gmail-h5"><div>On Jul 25, 2017, at 2:01 PM, Michael Ilseman via swift-dev <<a href="mailto:swift-dev@swift.org" target="_blank">swift-dev@swift.org</a>> wrote:</div><br class="m_-8706833233740895112m_2463072520094240643gmail-m_-5574471918370430643Apple-interchange-newline"></div></div><div><div><div class="m_-8706833233740895112m_2463072520094240643gmail-h5"><div><p style="margin:0px 0px 15px;font-family:Helvetica,arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">On Darwin, known-ASCII strings are sorted according to the lexicographical ordering of their code units. All non-known-ASCII strings are otherwise ordered based on the UCA[1]. On Linux, however, even known-ASCII strings are ordered based on UCA. I propose to unify these by changing Linux’s string sort order to match Darwin’s in Swift 4.0.</p><h4 id="m_-8706833233740895112m_2463072520094240643gmail-m_-5574471918370430643toc_0" style="margin:20px 0px 10px;padding:0px;font-size:16px;font-family:Helvetica,arial,sans-serif;background-color:rgb(255,255,255)">Background</h4><p style="margin:15px 0px;font-family:Helvetica,arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">Swift’s default ordering for strings is appropriate for machine consumption (e.g. implementing sorted collections). It obeys Unicode canonical equivalence[2], that is strings compare the same modulo normalization. However, it is not meant to be sufficient for presenting a meaningful ordering to human consumers, as that requires incorporating reader-specific information (e.g. [3]). </p><p style="margin:15px 0px;font-family:Helvetica,arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">Known-ASCII strings are a trivial case for the described sort order semantics: pure ASCII is unaffected by normalization. Thus, lexicographical ordering of code units is a valid machine ordering for ASCII strings. On Darwin, this is used to order known-ASCII strings while Linux uses UCA even for known-ASCII strings.</p><p style="margin:15px 0px;font-family:Helvetica,arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">Long term, the plan is to switch String’s sort order to be the lexicographical ordering of normalized code units (or perhaps scalar values), as mentioned in the String Manifesto[4]. This is a more efficient ordering than that provided by UCA. However, this will not make it in time for Swift 4.0. </p><h4 id="m_-8706833233740895112m_2463072520094240643gmail-m_-5574471918370430643toc_1" style="margin:20px 0px 10px;padding:0px;font-size:16px;font-family:Helvetica,arial,sans-serif;background-color:rgb(255,255,255)">Changes</h4><p style="margin:15px 0px;font-family:Helvetica,arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">I propose to change Linux’s sort order for known-ASCII strings to be the same as it is on Darwin. This will be accomplished by dropping the relevant <code style="margin:0px 2px;padding:0px 5px;white-space:nowrap;border-width:1px;border-style:solid;border-color:rgb(234,234,234);background-color:rgb(248,248,248);border-radius:3px">#if</code> guards in StringCompare.swift. An example implementation can be found at [5].</p><p style="margin:15px 0px;font-family:Helvetica,arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">In addition to unifying sort order semantics across platforms, this will also deliver significant performance boosts to pure ASCII strings on Linux.</p><h2 id="m_-8706833233740895112m_2463072520094240643gmail-m_-5574471918370430643toc_2" style="margin:20px 0px 10px;padding:0px;font-size:24px;border-bottom:1px solid rgb(204,204,204);font-family:Helvetica,arial,sans-serif;background-color:rgb(255,255,255)"></h2><p style="margin:15px 0px;font-family:Helvetica,arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">[1] <a href="http://unicode.org/reports/tr10/" style="color:rgb(65,131,196)" target="_blank">UTS #10: Unicode Collation Algorithm</a></p><p style="margin:15px 0px;font-family:Helvetica,arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">[2] <a href="http://unicode.org/notes/tn5/" style="color:rgb(65,131,196)" target="_blank">Canonical Equivalence in Applications</a></p><p style="margin:15px 0px;font-family:Helvetica,arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">[3] <a href="http://unicode.org/reports/tr10/#Contextual_Sensitivity" style="color:rgb(65,131,196)" target="_blank">UCA: Contextual Sensitivity</a></p><p style="margin:15px 0px;font-family:Helvetica,arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">[4] <a href="https://github.com/apple/swift/blob/master/docs/StringManifesto.md#comparing-and-hashing-strings" style="color:rgb(65,131,196)" target="_blank">String Manifesto: Comparing and Hashing Strings</a></p><p style="margin:15px 0px 0px;font-family:Helvetica,arial,sans-serif;font-size:14px;background-color:rgb(255,255,255)">[5] <a href="https://github.com/milseman/swift/commit/5560e13198d5cc284f46bf190f59a2edf7ed747b" style="color:rgb(65,131,196)" target="_blank">Unifying Linux/Darwin ASCII sort order semantics - github</a></p></div></div></div>______________________________<wbr>_________________<br>swift-dev mailing list<br><a href="mailto:swift-dev@swift.org" target="_blank">swift-dev@swift.org</a><br><a href="https://lists.swift.org/mailman/listinfo/swift-dev" target="_blank">https://lists.swift.org/mailma<wbr>n/listinfo/swift-dev</a><br></div></blockquote></div><br></div><br>______________________________<wbr>_________________<br>
swift-dev mailing list<br>
<a href="mailto:swift-dev@swift.org" target="_blank">swift-dev@swift.org</a><br>
<a href="https://lists.swift.org/mailman/listinfo/swift-dev" rel="noreferrer" target="_blank">https://lists.swift.org/mailma<wbr>n/listinfo/swift-dev</a><br>
<br></blockquote></div><br></div></div></div></div></div></div>
______________________________<wbr>_________________<br>swift-dev mailing list<br><a href="mailto:swift-dev@swift.org" target="_blank">swift-dev@swift.org</a><br><a href="https://lists.swift.org/mailman/listinfo/swift-dev" target="_blank">https://lists.swift.org/mailma<wbr>n/listinfo/swift-dev</a><br></div></blockquote></div><br></div></div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>