[swift-dev] Questions about Swift-CI
atrick at apple.com
Tue May 16 18:26:46 CDT 2017
> On May 11, 2017, at 1:33 PM, Pavol Vaskovic via swift-dev <swift-dev at swift.org> wrote:
> Douglas Gregor suggested in SR-4814 (that was merged as dupe into SR-4669) that I get in contact with you because I’m working on issue SR-4669 Add a Benchmark_Driver --rerun N option <https://bugs.swift.org/browse/SR-4669> and I should make sure that I don’t break the Swift-CI.
> The point of that is to deal with false changes (regressions and improvements) that are often seen in benchmark reports from CI.
> I have run Benchmark_O with --num-iters=100 on my machine for the the whole performance test suite, to get a feeling for the distribution of benchmark samples, because I also want to move the Benchmark_Driver to use MEAN instead of MIN in the analysis.
Using MEAN wasn’t part of the aforementioned SR-4669. The purpose of that task is to reduce the time CI takes to get useful results (e.g. by using 3 runs as a baseline). MEAN isn’t useful if you’re only gathering 3 data points.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the swift-dev