[swift-dev] New to building Swift toolchain

Tue Sep 19 14:46:27 CDT 2017

Hey Maksym. Your email was too big b/c of the attachment. Can you put the attachment into a pastebin or something like that?

> On Sep 19, 2017, at 12:35 AM, Maksym Grebenets <mgrebenets at gmail.com> wrote:
> 
> Hi Michael
> 
> Thanks for the feedback!
> 
> I gave it another go and this time made sure I have a proper setup.
> 
> ./swift/utils/update-checkout --clone
> ./swift/utils/update-checkout --tag swift-DEVELOPMENT-SNAPSHOT-2017-09-18-a
> 
> I've double checked all the submodules directories to make sure the tag was properly checked out.
> 
> Next I've configured Xcode with this command:
> 
> 
> sudo xcode-select -s /Applications/Xcode-beta.app/Contents/Developer
> 
> The xcodebuild -version output is:
> 
> Xcode 9.0
> Build version 9M214v
> 
> And the Host OS is 10.12.6, which matches requirements on CI: https://ci.swift.org/ <https://ci.swift.org/>.
> 
> With all that configured I finally kick off the build:
> 
> 
> ./swift/utils/build-toolchain swift.local | tee build.log
> 
> After about 2+ hours it ends up with same 3 failures:
> 
> Failing Tests (3):
> 
>     Swift(iphonesimulator-i386) :: stdlib/Character.swift
>     Swift(iphonesimulator-i386) :: stdlib/CodableTests.swift
>     Swift(iphonesimulator-i386) :: stdlib/MapKit.swift
> 
>   Expected Passes    : 9957
>   Expected Failures  : 27
>   Unsupported Tests  : 446
>   Unexpected Failures: 3
> 
> Here's log for one of the failing tests:
> 
> 
> 
> --
> 
> 
> 
> ********************
> 
> Testing: 0
> 
> FAIL: Swift(iphonesimulator-i386) :: stdlib/MapKit.swift (566 of 10433)
> 
> ******************** TEST 'Swift(iphonesimulator-i386) :: stdlib/MapKit.swift' FAILED ********************
> 
> Script:
> 
> --
> 
> rm -rf /Users/grebenma/Projects/oss/swift/build/buildbot_osx/swift-macosx-x86_64/test-iphonesimulator-i386/stdlib/Output/MapKit.swift.tmp && mkdir -p /Users/grebenma/Projects/oss/swift/build/buildbot_osx/swift-macosx-x86_64/test-iphonesimulator-i386/stdlib/Output/MapKit.swift.tmp && xcrun --toolchain default --sdk /Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator11.0.sdk /Users/grebenma/Projects/oss/swift/build/buildbot_osx/swift-macosx-x86_64/bin/swiftc -target i386-apple-ios7.0  -module-cache-path '/var/folders/vs/50_hfsmx4m1cpwbn7j3x6r6c0000gp/T/swift-testsuite-clang-module-cachefYcCzs' -F /Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator11.0.sdk/../../../Developer/Library/Frameworks  -swift-version 3  -module-cache-path '/var/folders/vs/50_hfsmx4m1cpwbn7j3x6r6c0000gp/T/swift-testsuite-clang-module-cachefYcCzs' /Users/grebenma/Projects/oss/swift/swift/test/stdlib/MapKit.swift -o /Users/grebenma/Projects/oss/swift/build/buildbot_osx/swift-macosx-x86_64/test-iphonesimulator-i386/stdlib/Output/MapKit.swift.tmp/a.out -module-name main && xcrun --toolchain default --sdk /Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator11.0.sdk simctl spawn 'iPhone 5'  /Users/grebenma/Projects/oss/swift/build/buildbot_osx/swift-macosx-x86_64/test-iphonesimulator-i386/stdlib/Output/MapKit.swift.tmp/a.out
> 
> --
> 
> Exit Code: 134
> 
> 
> 
> Command Output (stdout):
> 
> --
> 
> [ RUN      ] MapKit.NSValue bridging
> 
> stdout>>> check failed at /Users/grebenma/Projects/oss/swift/swift/stdlib/private/StdlibUnittestFoundationExtras/StdlibUnittestFoundationExtras.swift, line 130
> 
> stdout>>> expected: <00000000 00003140 00000000 00004340> (of type NSConcreteValue)
> 
> stdout>>> actual: <00000000 00003140 00000000 00004340> (of type NSConcreteValue)
> 
> [     FAIL ] MapKit.NSValue bridging
> 
> MapKit: Some tests failed, aborting
> 
> UXPASS: []
> 
> FAIL: ["NSValue bridging"]
> 
> SKIP: []
> 
> To debug, run:
> 
> $ /Users/grebenma/Projects/oss/swift/build/buildbot_osx/swift-macosx-x86_64/test-iphonesimulator-i386/stdlib/Output/MapKit.swift.tmp/a.out --stdlib-unittest-in-process --stdlib-unittest-filter "NSValue bridging"
> 
> 
> 
> --
> 
> Command Output (stderr):
> 
> --
> 
> Child process terminated with signal 6: Abort trap
> 
> 
> 
> --
> 
> The logs don't make much sense to me, especially since actual and expected values look the same :shrugs:.
> 
> I'm not sure if mailing lists accept attachment, but I attached full build log just in case.
> 
> I've also downloaded full console log from build #303 (which builds same snapshot as I have) and I find that that build log is somewhat different but overall has same contents.
> 
> So far this is where I got stuck with building toolchain.
> There wasn't much issues with running stuff like
> swift/utils/build-script --release-debuginfo --debug-swift 
> and then running incremental builds, e.g. 
> ninja -C build/Ninja-RelWithDebInfoAssert+swift-DebugAssert/swift-macosx-x86_64 swift-refactor
> 
> But when it comes to building the toolchain, this is what I arrived at. 
> 
> Thanks again for your help!
> 
> Max
> 
> 
> On Tue, Sep 19, 2017 at 4:58 AM, Michael Gottesman <mgottesman at apple.com <mailto:mgottesman at apple.com>> wrote:
> 
>> On Sep 18, 2017, at 8:54 AM, Maksym Grebenets via swift-dev <swift-dev at swift.org <mailto:swift-dev at swift.org>> wrote:
>> 
>> Hi all
>> 
>> I guess like many other iOS devs with c/c++ background I really wanted to get into Swift development one day, at least a tiny bit, at least to understand how things work.
>> 
>> So new refactoring tools and starter tasks finally pulled me in.
>> 
>> With a lot of struggle and a lot of help from community I was able to understand how lit works, how to walk the AST and write changes back to the editor. I've got code and tests for new refactoring kind and new refactoring action, so I was like "yay, time to build the keychain and just try it in Xcode!!!".
>> 
>> That's where I was thrown back to the "what do I do?" state again...
>> 
>> I pulled latest master branch and ran build-toolchain and all of that just to be told after almost an hour or so that there are 3 failing tests...
> 
> What were the failing tests?
> 
>> 
>> So I thought:
>> 
>> cd swift
>> git checkout swift-DEVELOPMENT-SNAPSHOT-2017-09-15-a
>> cd ..
>> ./swift/utils/update-checkout --tag swift-DEVELOPMENT-SNAPSHOT-2017-09-15-a
>> 
>> then build and fail again :(...
> 
> So you are saying that you took the last "known good snapshot" and that also failed. How did it fail? On the same 3 tests?
> 
>> 
>> OK then, so I just disable the tests and give it another go:
>> 
>> # build-presets.ini
>> #test
>> #validation-test
>> #long-test
>> 
>> Yay! I got a toolchain!
>> I copy it over to /Library/Developer/Toolchains/ 
>> Select it in Xcode (latest Xcode 9 GM) and... no luck
>> The source code highlighting disappears, 
>> I get "An internal error occurred. Source editor functionality is limited..." error.
>> All of the refactoring actions are disabled.
>> 
>> I guess those 3 failing tests did matter in the end...
>> 
>> 
>> I may be frustrated, because I don't understand what's going on, but I don't feel like giving up.
>> 
>> I'd appreciate any help or advice I can get!
>> 
>> Should I choose an older tag from before to get all tests passing?
> 
> Do you mean before the last known good snapshot? If a snapshot was published on swift.org <http://swift.org/>, it /should/ pass all of the tests.
> 
>> Should I build using Xcode beta rather then GM?
> 
> You should be using the Xcode beta. This can be seen either via the README.md under System Requirements for macOS [and don't worry I run into this issue sometimes as well = )]. It is also documented on the CI: https://ci.swift.org <https://ci.swift.org/> with a nice link.
> 
>> Is there any way to build/re-build the OS X toolchain any faster? 
>> Why does it take so long even if I just re-run same command again?
> 
> When you rerun is it not incremental? (I forgot TBH). In terms of the amount of work performed, build-toolchain command is intended to be used for producing snapshots and thus it has to be complete and include code for all platforms/etc. We could potentially have a preset that builds a toolchain only for 1 platform. I am not 100% sure if such a toolchain would work, but it should take significantly less time to build. Would that help?
> 
>> Where did I get it wrong so the toolchain doesn't really work with Xcode?
>> Can I connect lldb to process that runs my lit tests?
>> 
>> I'm sorry for having so many questions, but you can imagine there isn't a lot of information for newbies like me trying to get into Swift compiler development. Well, let's just say I couldn't find much :) I've just realized that this mail list is better option that StackOverflow about an hour ago :) Though, of course, I have read Brian Gesiak's posts and the swift.org <http://swift.org/> blog post on refactoring.
> 
> There are no bad questions or too many questions. Welcome to the dev-list = ).
> 
>> 
>> Just to sum up all of the above, how to I get to build local Xcode toolchain which I can plug in to Xcode and test?
> 
> Answer my questions above... I'll help you get across the finish line.
> 
> On another note, I noticed you mention starter tasks: keep in mind there are more starter tasks than have been documented. If you are interested in a specific area of the compiler and can not find starter tasks in this area, please feel free to send a nice email to this list. I am sure we can find /something/ ; ).
> 
> Michael
> 
>> 
>> Thank you!
>> _______________________________________________
>> swift-dev mailing list
>> swift-dev at swift.org <mailto:swift-dev at swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-dev <https://lists.swift.org/mailman/listinfo/swift-dev>
> 
> 
> <build.log.zip>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-dev/attachments/20170919/af73ac21/attachment.html>