[swift-dev] Questions about Swift-CI

[Andrew Trick]

> On May 11, 2017, at 1:33 PM, Pavol Vaskovic via swift-dev <swift-dev at swift.org> wrote:
> 
> Douglas Gregor suggested in SR-4814 (that was merged as dupe into SR-4669) that I get in contact with you because I’m working on issue SR-4669 Add a Benchmark_Driver --rerun N option <https://bugs.swift.org/browse/SR-4669> and I should make sure that I don’t break the Swift-CI.
> 
> The point of that is to deal with false changes (regressions and improvements) that are often seen in benchmark reports from CI.
> 
> I have run Benchmark_O with --num-iters=100 on my machine for the the whole performance test suite, to get a feeling for the distribution of benchmark samples, because I also want to move the Benchmark_Driver to use MEAN instead of MIN in the analysis.

Using MEAN wasn’t part of the aforementioned SR-4669. The purpose of that task is to reduce the time CI takes to get useful results (e.g. by using 3 runs as a baseline). MEAN isn’t useful if you’re only gathering 3 data points.

-Andy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-dev/attachments/20170516/7fda5b85/attachment.html>