[swift-dev] [swift-lldb-dev] [Swift CI] Build Failure: OSS - Swift Package - Ubuntu 15.10 (master) #1930

Kate Stone k8stone at apple.com
Fri Aug 19 16:21:04 CDT 2016 

I don’t disagree.  There’s no clear signal to be had from a flaky test, so we’ve already made strides with some of the worst offenders.  The existing timeout strategy 

We’re actively working on strategies to make the test suite more robust (moving pexcept tests to a different strategy, for one, as we have for one of the most troublesome examples.)  I expect this to be part of our quality push for the next year.  We have existing radars tracking some of the expected work on the suite, but just in case this isn’t being tracked I’ve filed:

<rdar://problem/27930082> Replace the timeout strategy used in our test suite with something more robust

Kate Stone k8stone at apple.com <mailto:k8stone at apple.com>
 Xcode Low Level Tools

> On Aug 19, 2016, at 2:10 PM, Daniel Dunbar via swift-lldb-dev <swift-lldb-dev at swift.org> wrote:
> 
> The amount of flakiness in the LLDB tests is disturbing... is there a high-level bug tracking improving this? It seems like it might be worth running a stress test of the tests on a loaded machine to try and shake out such problems.
> 
>  - Daniel
> 
>> On Aug 19, 2016, at 1:58 PM, Todd Fiala via swift-lldb-dev <swift-lldb-dev at swift.org <mailto:swift-lldb-dev at swift.org>> wrote:
>> 
>>> 
>>> On Aug 19, 2016, at 1:42 PM, Enrico Granata <egranata at apple.com <mailto:egranata at apple.com>> wrote:
>>> 
>>>> 
>>>> On Aug 19, 2016, at 11:20 AM, Todd Fiala via swift-lldb-dev <swift-lldb-dev at swift.org <mailto:swift-lldb-dev at swift.org>> wrote:
>>>> 
>>>> It looks like this may be the packaging build:
>>>> 
>>>>>> oss-swift-package-linux-ubuntu-15_10
>>>> 
>>>> It was this build:
>>>> 
>>>> https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7 <https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7>
>>>> 
>>>> It has since gone blue.
>>>> 
>>>> I don’t think I recall ever seeing that test fail before in an intermittent way, but that might be what happened here.
>>>> 
>>> 
>>> Given the failure mode, it does sound likely:
>>> 
>>>   File "/usr/lib/python2.7/socket.py", line 206, in accept
>>>     sock, addr = self._sock.accept()
>>> timeout: timed out
>>> 
>>> 
>> 
>> If we see that again, we may be able to tweak the socket timeout to allow for heavily loaded test CI.
>> 
>>>> -Todd
>>>> 
>>>>> On Aug 19, 2016, at 11:15 AM, Kate Stone via swift-lldb-dev <swift-lldb-dev at swift.org <mailto:swift-lldb-dev at swift.org>> wrote:
>>>>> 
>>>>> Where is this failure being reported?  I’m not seeing anything on swift-3.0-branch or master on ci.swift.org <http://ci.swift.org/>.
>>>>> 
>>>>> Kate Stone k8stone at apple.com <mailto:k8stone at apple.com>
>>>>>  Xcode Low Level Tools
>>>>> 
>>>>>> On Aug 19, 2016, at 12:15 AM, mishal_shah via swift-lldb-dev <swift-lldb-dev at swift.org <mailto:swift-lldb-dev at swift.org>> wrote:
>>>>>> 
>>>>>> + swift-lldb-dev group.  
>>>>>> 
>>>>>>> On Aug 18, 2016, at 9:47 PM, Daniel Dunbar <daniel_dunbar at apple.com <mailto:daniel_dunbar at apple.com>> wrote:
>>>>>>> 
>>>>>>> Failing in lldb:
>>>>>>> Command invoked: /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py --executable /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb --rerun-all-issues -C /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang -s 2016-08-18-15_15_07 --results-port 41187 --inferior -p TestStubReverseConnect.py /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=14:int
>>>>>>> 
>>>>>>> Configuration: arch=x86_64 compiler=/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang
>>>>>>> ----------------------------------------------------------------------
>>>>>>> Collected 2 tests
>>>>>>> 
>>>>>>> ======================================================================
>>>>>>> ERROR: test_reverse_connect_works_llgs (TestStubReverseConnect.TestStubReverseConnect)
>>>>>>> ----------------------------------------------------------------------
>>>>>>> Traceback (most recent call last):
>>>>>>>   File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py", line 121, in wrapper
>>>>>>>     func(*args, **kwargs)
>>>>>>>   File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/decorators.py", line 121, in wrapper
>>>>>>>     func(*args, **kwargs)
>>>>>>>   File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py", line 89, in test_reverse_connect_works_llgs
>>>>>>>     self.reverse_connect_works()
>>>>>>>   File "/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test/tools/lldb-server/commandline/TestStubReverseConnect.py", line 67, in reverse_connect_works
>>>>>>>     (stub_socket, address) = self.listener_socket.accept()
>>>>>>>   File "/usr/lib/python2.7/socket.py", line 206, in accept
>>>>>>>     sock, addr = self._sock.accept()
>>>>>>> timeout: timed out
>>>>>>> Config=x86_64-/home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang-3.9
>>>>>>> ----------------------------------------------------------------------
>>>>>>> Ran 2 tests in 22.006s
>>>>>>> 
>>>>>>> RESULT: FAILED (0 passes, 0 failures, 1 errors, 1 skipped, 0 expected failures, 0 unexpected successes)
>>>>>>> Session logs for test failures/errors/unexpected successes can be found in directory '2016-08-18-15_15_07'
>>>>>>> 
>>>>>>>  <>[TestStubReverseConnect.py FAILED]
>>>>>>> Command invoked: /usr/bin/python /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/test/dotest.py --executable /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/lldb-linux-x86_64/bin/lldb --rerun-all-issues -C /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/build/buildbot_linux/llvm-linux-x86_64/bin/clang -s 2016-08-18-15_15_07 --results-port 41187 --inferior -p TestStubReverseConnect.py /home/buildnode/jenkins/workspace/oss-swift-package-linux-ubuntu-15_10/lldb/packages/Python/lldbsuite/test --event-add-entries worker_index=14:int
>>>>>>> 
>>>>>>>  - Daniel
>>>>>>> 
>>>>>>>> On Aug 18, 2016, at 9:18 PM, no-reply at swift.org <mailto:no-reply at swift.org> wrote:
>>>>>>>> 
>>>>>>>> [FAILURE] oss-swift-package-linux-ubuntu-15_10 [#1930]
>>>>>>>> 
>>>>>>>> Build URL:	https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/ <https://ci.swift.org/job/oss-swift-package-linux-ubuntu-15_10/1930/>
>>>>>>>> Project:	oss-swift-package-linux-ubuntu-15_10
>>>>>>>> Date of build:	Thu, 18 Aug 2016 20:26:52 -0700
>>>>>>>> Build duration:	51 min
>>>>>>>> Identified problems:
>>>>>>>> 
>>>>>>>> Regression test failed: This build failed because a regression test in the test suite FAILed. Below is a list of all errors:
>>>>>>>> Indication 1 <https://ci.swift.org//job/oss-swift-package-linux-ubuntu-15_10/1930/consoleFull#465392916fca400bf-2f4a-462e-b517-e058d770b2d7>
>>>>>>>> Changes
>>>>>>>> 
>>>>>>>> Commit 2ebab526bfd7a5ce6ef460bebee0399fe669612d by xi_ge:
>>>>>>>> [FixCode] Apply coercion fixits on return statement and initialization
>>>>>>>> 
>>>>>>>> edit: test/FixCode/fixits-apply-objc.swift
>>>>>>>> edit: test/FixCode/fixits-apply-objc.swift.result
>>>>>>>> edit: lib/Sema/CSDiag.cpp
>>>>>>>> edit: test/FixCode/fixits-apply.swift.result
>>>>>>>> 
>>>>>>>> Commit 6447a2d16edcfbceab940d58a321b3ef70525f1a by daniel_dunbar:
>>>>>>>> [Basic] Update Thread shim to match changes on Linux side.
>>>>>>>> 
>>>>>>>> edit: Sources/Basic/Thread.swift
>>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> swift-lldb-dev mailing list
>>>>>> swift-lldb-dev at swift.org <mailto:swift-lldb-dev at swift.org>
>>>>>> https://lists.swift.org/mailman/listinfo/swift-lldb-dev <https://lists.swift.org/mailman/listinfo/swift-lldb-dev>
>>>>> 
>>>>> _______________________________________________
>>>>> swift-lldb-dev mailing list
>>>>> swift-lldb-dev at swift.org <mailto:swift-lldb-dev at swift.org>
>>>>> https://lists.swift.org/mailman/listinfo/swift-lldb-dev <https://lists.swift.org/mailman/listinfo/swift-lldb-dev>
>>>> 
>>>> _______________________________________________
>>>> swift-lldb-dev mailing list
>>>> swift-lldb-dev at swift.org <mailto:swift-lldb-dev at swift.org>
>>>> https://lists.swift.org/mailman/listinfo/swift-lldb-dev <https://lists.swift.org/mailman/listinfo/swift-lldb-dev>
>>> 
>>> 
>>> Thanks,
>>> - Enrico
>>> 📩 egranata@ <about:blank> <about:blank>.com <about:blank> ☎️ 27683
>> 
>> _______________________________________________
>> swift-lldb-dev mailing list
>> swift-lldb-dev at swift.org <mailto:swift-lldb-dev at swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-lldb-dev <https://lists.swift.org/mailman/listinfo/swift-lldb-dev>
> _______________________________________________
> swift-lldb-dev mailing list
> swift-lldb-dev at swift.org
> https://lists.swift.org/mailman/listinfo/swift-lldb-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-dev/attachments/20160819/f2988d3e/attachment.html>

More information about the swift-dev mailing list