<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Nov 28, 2017, at 5:01 PM, Mark Lacey via swift-dev <<a href="mailto:swift-dev@swift.org" class="">swift-dev@swift.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class="Apple-interchange-newline"><br class=""><blockquote type="cite" class=""><div class="">On Nov 19, 2017, at 9:02 PM, Brian Gesiak via swift-dev <<a href="mailto:swift-dev@swift.org" class="">swift-dev@swift.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">Thanks for the reply, Graydon, and for<span class="Apple-converted-space"> </span><a href="https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20171113/006001.html" target="_blank" class="">your other email on the topic</a>!<div class=""><br class=""></div><div class="">I need to take more time to look into some of the things you mentioned, but I won't be able to do so in earnest for another few days. In the meantime, I'll just reply with a few uninformed opinions -- feel free not to respond to these :)<br class=""><div class="gmail_extra"><br class=""><div class="gmail_quote">On Fri, Nov 17, 2017 at 12:54 AM, Graydon Hoare<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:ghoare@apple.com" target="_blank" class="">ghoare@apple.com</a>></span><span class="Apple-converted-space"> </span>wrote:<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div class="" style="word-wrap: break-word; line-break: after-white-space;"><div class=""><div class="">Sadly, there's not always a 1:1 mapping between source entities and time like that. Certainly _some_ cases can be so egregious (say, typechecking time on an expression that's triggering an exponential-time inference) that they dominate compile time, and can be identified-and-fixed in isolation; but often the total amount of work attributable to a given source entity is spread around the compilation, occurs in multiple phases, emerges out of interaction between entities, overlaps with others, etc.</div></div></div></blockquote><div class=""><br class=""></div><div class=""><div class="">It's these particularly egregious cases that I had in mind when I wrote my email. Many blog posts on "how to reduce your Swift project compile times" suggest one of the following approaches:</div><div class=""><br class=""></div><div class="">1. Use `-debug-time-function-bodies` and `-debug-time-expression-<wbr class="">checking` to print a list of times, sort those lists in descending order of time spent, and add explicit types to, or otherwise simplify, the slowest functions and expressions.</div><div class="">2. Use `-warn-long-function-bodies=` and `-warn-long-expression-type-<wbr class="">checking=`. Compile the project several times, gradually lowering the thresholds passed to these options, in order to surface the slowest functions and expressions. Add explicit types to or otherwise simplify the slowest functions and expressions.</div></div></div></div></div></div></div></blockquote><div class=""><br class=""></div>One of the drawbacks with the type checker timers is that they do not exclude the time spent in client code called by the type checker, e.g. in deserialization code. The result is that you can e.g. have a simple expression involving a stdlib type like String, and have it be reported as taking 500ms to type check, when in fact it’s nearly instantaneous to type check, but we spent 500ms deserializing things while type checking that expression. What gets really confusing about this is some of this deserializing only happens if you don’t use explicit types, so adding an explicit type to one of these expressions can mean that this expression is now fast, but some other expression which was previously reported as being fast now suddenly “regresses” and takes 500ms due to that deserialization happening later.</div></div></blockquote><div><br class=""></div>On the flip side, I would argue that if referencing a single stdlib declaration like String results in 500ms spent in deserialization, we have a problem, even if it s a one-time cost. In multiple-frontend mode, every source file will end up paying the 500ms penalty.</div><div><br class=""><blockquote type="cite" class=""><div class=""><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">I haven’t looked closely, but offhand I don’t know of a clean, robust, maintainable way of excluding this time.</div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">I’m a bit concerned about productizing these without solving that problem.</div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">It’s also not necessarily reasonable that we’re doing this deserialization when an explicit type is not used so there might be something useful to investigate and fix there as well.</div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">Mark</div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""><br class=""></div><div class="">My original idea was to expand or improve these options, since I think they'll continue to be useful for large Swift projects. But I'm certainly open to working on something else instead.</div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div class="" style="word-wrap: break-word; line-break: after-white-space;"><div class=""><div class="">I don't have an especially strong feeling about the degree-of-support / stability of such features; I'm going to have to leave that part of your question to others.</div></div></div></blockquote><div class=""><br class=""></div><div class="">Yeah, this is definitely still an open question of mine. In the meantime I'll try looking at some of the other approaches you've suggested (thanks!).</div><div class=""><br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div class="" style="word-wrap: break-word; line-break: after-white-space;"><div class=""> 1. See if you can leverage the existing counters and stats-gathering / reporting machinery in the compiler; for example the -trace-stats-events infrastructure lets you bundle together work done on behalf of a given source range and any changes to compiler counters during that work, as a single virtual "stats-event" in processing, and trace those events to a .csv file. Maybe something related to that would be helpful for the task you're interested in?<br class=""></div></div></blockquote><div class=""><br class=""></div><div class="">I wasn't aware of `-trace-stats-events`, thanks! When using `-stats-output-dir` with a primary-file compilation mode, I can see that the stats include the amount of time spent on each Swift module being produced. I'll need to take a closer look at how `-trace-stats-events` works, though -- I get an empty .csv file when I use that option in conjunction with `-stats-output-dir`, and I'm not sure yet how to use it with source ranges. I'll look into this further.</div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div class="" style="word-wrap: break-word; line-break: after-white-space;"><div class=""> 2. Consider going up a level from declarations or functions to _files_, and see if there's a useful way to visualize hot-spots in the inter-file dependency graph that the driver interprets, during incremental compilation. The units of work at this level are likely to be large (especially if they involve cascading dependencies, that invalidate "downstream" files) and often cutting or changing an edge in that graph can be a simpler matter of moving a declaration, or changing its visibility: reasonably easy changes that don't cost the user much to experiment with.</div><div class=""><br class=""></div><div class="">Hope that helps! Happy to discuss any of this further.</div></div></blockquote><div class=""><br class=""></div><div class="">Yes, thank you! I'm also interested in the "improving incremental mode" section of your other email, so thanks for writing all that down.</div><div class=""><br class=""></div><div class="">- Brian Gesiak</div></div><br class=""></div></div></div>_______________________________________________<br class="">swift-dev mailing list<br class=""><a href="mailto:swift-dev@swift.org" class="">swift-dev@swift.org</a><br class=""><a href="https://lists.swift.org/mailman/listinfo/swift-dev" class="">https://lists.swift.org/mailman/listinfo/swift-dev</a><br class=""></div></blockquote></div><br class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">_______________________________________________</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">swift-dev mailing list</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="mailto:swift-dev@swift.org" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">swift-dev@swift.org</a><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><a href="https://lists.swift.org/mailman/listinfo/swift-dev" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">https://lists.swift.org/mailman/listinfo/swift-dev</a></div></blockquote></div><br class=""></body></html>