[swift-dev] smoke test not very smoky?

[mishal_shah]

Linux smoke test builds with new preset (buildbot_linux,smoketest) and it takes 39mins compare to 1hr 4min.

https://github.com/apple/swift/pull/5378

Thanks, 
Mishal Shah
> On Oct 20, 2016, at 9:33 AM, Jordan Rose via swift-dev <swift-dev at swift.org> wrote:
> 
>> 
>> On Oct 19, 2016, at 10:17, Dave Abrahams via swift-dev <swift-dev at swift.org> wrote:
>> 
>> 
>> on Wed Oct 19 2016, Jordan Rose <swift-dev-AT-swift.org> wrote:
>> 
>>>> On Oct 19, 2016, at 9:44, Dave Abrahams via swift-dev <swift-dev at swift.org> wrote:
>>>> 
>>>> 
>>>> on Wed Oct 19 2016, Dave Abrahams <swift-dev-AT-swift.org <http://swift-dev-at-swift.org/>> wrote:
>>>> 
>>>>> It still seems like, for a smoke test, we're doing way too much work.
>>>>> This appears to be much more than what I get from build-script -t when
>>>>> I run tests locally.  Maybe I'm misunderstanding the intended role of
>>>>> our smoke tests, but since nobody is correcting me, I'm betting not.
>>> 
>>> Even smoke tests should run the validation tests…
>> 
>> What's the point of distinguishing validation from other tests if even
>> the smoke tests run them?
> 
> I think the validation tests are tests that compiler developers don’t bother running locally before they move to a PR. I’d be very concerned about landing changes in master without having run the validation tests—that would get us back to the days of consistent failures because someone forgot to update compiler_crashers.
> 
>> 
>>>> Someone wrote to me privately:
>>>> 
>>>> "buildbot_linux_1404" preset used in Linux smoke test contains
>>>> "--long-test".
>>>> 
>>>> This seems wrong to me.  Can we fix it?
>>> 
>>> …but I could see "long tests" going either way.
>> 
>> Well, what is “smoke” supposed to mean?  If it requires running
>> validation tests and maybe even long tests, what is the “smoke test”
>> distinction *for*?
> 
> It only builds and runs one platform per builder (no iOS/watchOS/tvOS), and doesn’t necessarily build and test all the downstream projects.
> 
> I agree that if we are to have something called “smoke test”, 40 minutes is too long.
> 
> Jordan
> 
> 
> _______________________________________________
> swift-dev mailing list
> swift-dev at swift.org <mailto:swift-dev at swift.org>
> https://lists.swift.org/mailman/listinfo/swift-dev <https://lists.swift.org/mailman/listinfo/swift-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-dev/attachments/20161020/21c133b4/attachment.html>