[swift-dev] Testing, validation, and many-core machines

[Michael Gottesman]

> On Jan 17, 2018, at 8:23 AM, David Zarzycki via swift-dev <swift-dev at swift.org> wrote:
> 
> Hello,
> 
> In docs/Testing.md, “long test” is acknowledged but never defined. When should tests be marked “long_test”? I’m asking because there are arguably a number of latent long_test tests that are exposed on many-core machines.

I think about it this way. A long test is a test that we want to run, but would bog down PR testing or are enough of a validation test that the trade-off in between signal/PR testing delays is found to be lacking. That being said I think of it as an approximation.

> 
> Speaking of many-core machines, there really isn’t a practical difference between smoke/primary testing and secondary/validation testing. Both test suites rush through 95-99% of the tests in very short order, and then wait a relatively long time for the remaining tests to finish. Can we move some of the exhaustive/stress tests in the “primary” suite to the validation suite? Likewise, can we move some of the short/functional tests in the validation directory to the primary test directory? That would match people’s expectations better, would it not?

Do you have specific test? Again, it isn't just about test time. It is about the trade-off in between signal and length of time.

> 
> Dave
> _______________________________________________
> swift-dev mailing list
> swift-dev at swift.org
> https://lists.swift.org/mailman/listinfo/swift-dev