[swift-corelibs-dev] [xctest] Removing outliers from performance tests

[Drew Crawford]

> Unfortunately our corelibs implementation of XCTest isn’t ready yet for performance testing.

That's why I'm here; I'm taking the temperature on implementing it.  I'm at the pain level where I need a solution in the next several months, even if the solution is to code it up myself.  My tests have failed 10x over this so far today.

I think the real question is, if I did implement basic performance testing, and I did implement a variable-sized window of runs, would that departure from the Old XCTest behavior (which uses 10 runs) disqualify the PR?  It's a basic compatibility question about how close we need to follow the Old XCTest behavior.

e.g. if XCS wanted to migrate to corelibs-xctest and it used variable #s of runs, presumably that would be an undertaking for the XCS team.  But I don't know whether those concerns (if they exist) play a role in what this project decides to do.

I'm going to do something on this problem eventually, unless someone else solves it first.  I'm just trying to work out whether I can do something upstream or whether this is a better candidate for an independent effort.