[swift-dev] Measuring MEAN Performance (was: Questions about Swift-CI)

[Andrew Trick]

> On Jun 12, 2017, at 4:45 PM, Pavol Vaskovic <pali at pali.sk> wrote:
> 
> We really have two problems:
> 1. spurious results 
> 2. the turnaround time for the entire benchmark suite
> 
> 
> I don't think we can get more consistent test results just from re-running tests that were detected as changes in the first pass, as described inSR-4669 <https://bugs.swift.org/browse/SR-4669>, because that improves accuracy only for one side of the comparison - the branch. When the measurement error is with the baseline from the master, re-running the branch would not help.

My understanding of this feature is that it would rerun both branches (or possibly whichever is slower or more jittery, but that’s probably over complicating it).

-Andy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-dev/attachments/20170612/4ebb8bb4/attachment.html>