[swift-dev] smoke test not very smoky?
mishal_shah at apple.com
Thu Oct 20 19:47:27 CDT 2016
Linux smoke test builds with new preset (buildbot_linux,smoketest) and it takes 39mins compare to 1hr 4min.
> On Oct 20, 2016, at 9:33 AM, Jordan Rose via swift-dev <swift-dev at swift.org> wrote:
>> On Oct 19, 2016, at 10:17, Dave Abrahams via swift-dev <swift-dev at swift.org> wrote:
>> on Wed Oct 19 2016, Jordan Rose <swift-dev-AT-swift.org> wrote:
>>>> On Oct 19, 2016, at 9:44, Dave Abrahams via swift-dev <swift-dev at swift.org> wrote:
>>>> on Wed Oct 19 2016, Dave Abrahams <swift-dev-AT-swift.org <http://swift-dev-at-swift.org/>> wrote:
>>>>> It still seems like, for a smoke test, we're doing way too much work.
>>>>> This appears to be much more than what I get from build-script -t when
>>>>> I run tests locally. Maybe I'm misunderstanding the intended role of
>>>>> our smoke tests, but since nobody is correcting me, I'm betting not.
>>> Even smoke tests should run the validation tests…
>> What's the point of distinguishing validation from other tests if even
>> the smoke tests run them?
> I think the validation tests are tests that compiler developers don’t bother running locally before they move to a PR. I’d be very concerned about landing changes in master without having run the validation tests—that would get us back to the days of consistent failures because someone forgot to update compiler_crashers.
>>>> Someone wrote to me privately:
>>>> "buildbot_linux_1404" preset used in Linux smoke test contains
>>>> This seems wrong to me. Can we fix it?
>>> …but I could see "long tests" going either way.
>> Well, what is “smoke” supposed to mean? If it requires running
>> validation tests and maybe even long tests, what is the “smoke test”
>> distinction *for*?
> It only builds and runs one platform per builder (no iOS/watchOS/tvOS), and doesn’t necessarily build and test all the downstream projects.
> I agree that if we are to have something called “smoke test”, 40 minutes is too long.
> swift-dev mailing list
> swift-dev at swift.org <mailto:swift-dev at swift.org>
> https://lists.swift.org/mailman/listinfo/swift-dev <https://lists.swift.org/mailman/listinfo/swift-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the swift-dev