[swift-dev] How to run the tests out-of-tree?

[Karl Wagner]

> On 13 May 2016, at 02:36, Greg Parker <gparker at apple.com> wrote:
> 
> 
>> On May 12, 2016, at 7:45 AM, Karl via swift-dev <swift-dev at swift.org <mailto:swift-dev at swift.org>> wrote:
>> 
>> So I’ve been working on cross-compiling for ARM, and it seems pretty good - only a bit of refactoring of the build script really needed. It’s a popular request, the lack of which is hindering lots of people and businesses who would like to experiment with swift on their ARM-based devices.
>> 
>> The thing that’s really missing before I can propose merging these changes is running the in-tree tests (swift/tests/ folder) to validate the products. I basically want to copy that folder in to a package and run it on the target device, but I’m not really sure how to do that with lit. I’ve built the unit test binaries (the ones which statically link against the runtime), and those obviously are easy enough to package and script for out-of-tree use.
>> 
>> Can anybody help with this?
> 
> You should look at the iOS device and iOS simulator machinery in swift/test/lit.cfg for one solution. Both cases use lit substitutions like %target-run to execute a helper tool that can run the test in the appropriate environment and collect the output. Our iOS device testing tool is not open-source, but it uploads a test's source and build directories to the device, runs the test executable on the device, and collects stdout and stderr and any output files. Any other tools used by the test script such as FileCheck still run on the test hosting machine, not on the target device being tested.
> 
> Some of the test scripts are complicated, interleaving compile commands and execution commands, so they can't be cleanly separated into a cross-compilation step and an on-device execution step. The %target-run mechanism allows on-device execution without forcing test authors to separate the device-side content.
> 
> 
> -- 
> Greg Parker     gparker at apple.com <mailto:gparker at apple.com>     Runtime Wrangler
> 
> 

My original idea was a lot simpler - create a script grabbing the specific executables we need (FileCheck, llvm-link, etc) and the tests, package it up, ship it out via ssh and run it on the intended target, piping the output back. I just couldn’t really figure out what the specific dependencies where - the site config seemed to want the entire source and build tree of swift and LLVM, and I don’t want to package all of that stuff up.

Karl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-dev/attachments/20160513/d5df1f27/attachment.html>