[swift-lldb-dev] [swift-dev] repl/test-repl-glibc.py Failure on Ubuntu

Jim Ingham jingham at apple.com
Fri Apr 15 17:01:32 CDT 2016 

The REPL makes a swift compiler, in which it runs your code (obvious...)  So when you do "import <some_module>" in the REPL, we need to load that module into our swift compiler. In the lldb REPL that is managed in SwiftExpressionParser::PerformAutoImport.  This will call lldb's SwiftASTContext::FindAndLoadModule -> SwiftASTContext::LoadModule, which will then do three things: find & tell the swift compiler about the .swiftmodule, dlopen any underlying libraries the .swiftmodule needs.  Finally, if the module imported C/ObjC code, the swift module doesn't describe the types coming from C but relies on types being in a clang module that can be imported into Swift.  That means that the REPL must rebuild a clang module which matches the one originally used to build the swift module.  

It's that final step that tends to be tricky, since Swift has to be convinced that the Clang module that the ClangImporter builds is compatible with the one that was originally used to build the swift module, and it is somewhat picky about this.  All that checking goes on in swift, not lldb, and I'm not very familiar with how this is done.

One difficulty that comes up often is figuring out where the module map & headers that were used to build the ClangImporter when the Swift module was originally constructed are on the current system.  On OS X the options that were used to build the ClangImporter are serialized in the swift module.  So we just grab that blob and use it to initialize the ClangImporter.  Construction this environment happens in SwiftASTContext::CreateInstance - for instance the call to loadFromSerializedAST is where we load the serialized options.  

That wasn't the way it was done in the early Swift days, instead the DWARF for the swift module would have the compiler options used to create the swift compiler (including some that were for the importer) and lldb would try to parse them and extract whatever goodies it needed.  That was fragile, and we switched to the serialization approach, but the code to parse up the DWARF is still there, as well as a bunch of assist functions so that we could add any extra information we might know (like what SDK's are being used.)

If you poke around in this code you'll see that we log pretty extensively the options we are using to make up both our Swift compiler and the ClangImporter.  To see this in action in the REPL, turn on the "types" log:

 > :log enable -f /tmp/lldb-types-log.txt lldb type

Then do your import.  If this is getting auto-imported too early for you to turn on the log by hand, then just put the log command (without the ":") in your .lldbinit file, the REPL, being just a particular invocation of lldb, will read that file.

Hope this helps,

Jim




> On Apr 15, 2016, at 8:19 AM, Ryan Lovelett via swift-lldb-dev <swift-lldb-dev at swift.org> wrote:
> 
> I've tried reverting c6121d56b19305cf59148d46af54c06b771f3180 just to
> see if doing that will restore functionality to lldb/repl.
> 
> Unfortunately too many things have changed in the build system since
> that commit for a revert to make sense anymore
> 
> Can anyone provide documentation/explain the mechanics of what happens
> when I type "import Glibc" into the REPL/lldb?
> 
> On Thu, Apr 14, 2016, at 05:47 PM, Dmitri Gribenko wrote:
>> +Brian
>> 
>> On Thu, Apr 14, 2016 at 2:46 PM, Ryan Lovelett via swift-dev
>> <swift-dev at swift.org> wrote:
>>> I've played around with `git bisect` and I think I've tracked it down to
>>> this commit [1]. Which came from PR #1704 [2]. I've also updated the issue,
>>> SR-1109, to include this information.
>>> 
>>> c6121d56b19305cf59148d46af54c06b771f3180 is the first bad commit
>>> commit c6121d56b19305cf59148d46af54c06b771f3180
>>> Author: Brian Gesiak <bgesiak at fb.com>
>>> Date:   Wed Mar 16 13:29:42 2016 -0400
>>> 
>>>    [Un-revert][Glibc] Configure modulemap for target, not host
>>> 
>>>    This reverts commit f2154ee94d, which reverted 04e1cd5bda. The original
>>>    commit needed to be reverted because of an issue in which install
>>>    targets were added to OS X builds that did not target Linux. This
>>>    addresses that issue by guarding all the Linux-only CMake logic with a
>>>    check for the SDK being built.
>>> 
>>> :040000 040000 e92829c16aa22f20edfdf95f3bb18bb15a3fa226
>>> 90062ad44050a19fc0d5bc846409945e83619b01 M    lib
>>> :040000 040000 bbe94bf4af832e154065bc0bcafffab2dacb839e
>>> 1a8be73f86884bda848cde22885bcd72a17660b2 M    stdlib
>>> :040000 040000 abf55068f67ee44e2bd52169aa8b988fb8aead28
>>> 41db0f8ddec3281f51c6798dd47c86675b7118b3 M    tools
>>> 
>>> [1]
>>> https://github.com/apple/swift/commit/c6121d56b19305cf59148d46af54c06b771f3180
>>> [2] https://github.com/apple/swift/pull/1704
>>> 
>>> 
>>> On Thu, Apr 14, 2016, at 12:36 PM, Jordan Rose wrote:
>>> 
>>> +swift-lldb-dev
>>> 
>>> 
>>> On Apr 14, 2016, at 7:20 , Joseph Bell via swift-dev <swift-dev at swift.org>
>>> wrote:
>>> 
>>> Ryan, thanks.  I've voted on SR-1109 and will add the steps I use to
>>> reproduce as well.
>>> 
>>> I think now its clear why the 14.04 and 15.10 packaging tests are passing,
>>> and that's because they aren't running the tests that leverage pexpect, if
>>> you look at the console log for the 14.04 test:
>>> https://ci.swift.org/view/Packages/job/oss-swift-package-linux-ubuntu-14_04/993/consoleText
>>> 
>>> lit.py: lit.cfg:101: note: 'pexpect' module unavailable, skipping related
>>> tests
>>> 
>>> Perhaps pexpect should be added to the CI server so these tests can begin
>>> failing properly.
>>> 
>>> 
>>> On Thu, Apr 14, 2016 at 8:47 AM, Ryan Lovelett via swift-dev
>>> <swift-dev at swift.org> wrote:
>>> 
>>> 
>>> 
>>> On Thu, Apr 14, 2016, at 09:17 AM, Joseph Bell via swift-dev wrote:
>>> 
>>> Howdy,
>>> 
>>> I've mentioned this once before and didn't get any feedback; I thought I'd
>>> give it one more shot.
>>> 
>>> Has anyone out there tried building, from scratch, the Swift 3.0 package on
>>> Ubuntu?  The compile, link, packaging steps all complete successfully, but
>>> then the repl/test-repl-glibc.py fails.  The failure is that the REPL
>>> doesn't interact (the underlying script is using pexpect to send/expect)
>>> properly:
>>> 
>>>  2> import Glibc
>>> warning: <REPL>:1:1: warning: #line directive is deprecated, please use
>>> #sourceLocation instead
>>> #line 2 "repl.swift"
>>> ^~~~~
>>> #sourceLocation
>>> 
>>> warning: repl.swift:3:1: warning: #line directive is deprecated, please use
>>> #sourceLocation instead
>>> #line
>>> ^~~~~
>>> #sourceLocation
>>> 
>>> error: repl.swift:2:8: error: missing required module 'SwiftGlibc'
>>> import Glibc
>>> 
>>> This is occurring on two separate Ubuntu 14.04 systems, one of which is a
>>> greenfield VM with all of the prerequisites/clang-3.6 installed.
>>> 
>>> Stumped on this one and was just curious if anyone can reproduce.
>>> 
>>> 
>>> 
>>> I can also reproduce. I actually broght this up yesterday too (just on a
>>> different list) [1]. I suggest you go vote for SR-1109 [2] which is the bug
>>> report for this issue.
>>> 
>>> I think this is show stopper. Not for the REPL break but because it also
>>> breaks the debugger on Linux as well.
>>> 
>>> Right now I'm trying to bisect the repos to see which commit(s?) might have
>>> introduced this regression. Kate Stone mentioned that she thinks this issue
>>> was introduced sometime after the 3-16 snapshot. I'm trying to corroborate
>>> that.
>>> 
>>> [1]
>>> https://lists.swift.org/pipermail/swift-lldb-dev/Week-of-Mon-20160411/000106.html
>>> [2] https://bugs.swift.org/browse/SR-1109
>>> 
>>> 
>>> 
>>> 
>>> Thanks,
>>> Joe
>>> 
>>> 
>>> 
>>> ---
>>> http://dev.iachieved.it/iachievedit/
>>> @iachievedit
>>> 
>>> _______________________________________________
>>> swift-dev mailing list
>>> swift-dev at swift.org
>>> https://lists.swift.org/mailman/listinfo/swift-dev
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> swift-dev mailing list
>>> swift-dev at swift.org
>>> https://lists.swift.org/mailman/listinfo/swift-dev
>>> 
>>> 
>>> 
>>> 
>>> 
>>> --
>>> ---
>>> http://dev.iachieved.it/iachievedit/
>>> @iachievedit
>>> _______________________________________________
>>> swift-dev mailing list
>>> swift-dev at swift.org
>>> https://lists.swift.org/mailman/listinfo/swift-dev
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> swift-dev mailing list
>>> swift-dev at swift.org
>>> https://lists.swift.org/mailman/listinfo/swift-dev
>>> 
>> 
>> 
>> 
>> -- 
>> main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
>> (j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr at gmail.com>*/
> _______________________________________________
> swift-lldb-dev mailing list
> swift-lldb-dev at swift.org
> https://lists.swift.org/mailman/listinfo/swift-lldb-dev

More information about the swift-lldb-dev mailing list