<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">Thanks for the feedback, Dmitri!<br><br>I think I misstated options #2 and #3: where I typed "m<span style="font-size:12.8px">ove business logic like </span><code style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wrap;border:1px solid rgb(234,234,234);border-radius:3px;display:inline;background-color:rgb(248,248,248)">IsTestCandidate</code><span style="font-size:12.8px"> to libIDE", I meant "move a lot of the non-XCTest-specific logic from SourceKit to libIDE, such that implementing </span><span style="font-family:Consolas,Inconsolata,Courier,monospace;font-size:11.05px;white-space:pre-wrap;background-color:rgb(248,248,248)">IsTestCandidate</span><span style="font-size:12.8px"> would be trivial in SourceKit and our test generation tool". The last thing I'd want to do is couple libIDE and XCTest!</span></div><div class="gmail_quote"><br>On Sun, Apr 3, 2016 at 6:36 PM, Dmitri Gribenko <span dir="ltr"><<a href="mailto:gribozavr@gmail.com" target="_blank">gribozavr@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span class="">On Sun, Apr 3, 2016 at 2:11 PM, Brian Gesiak <<a href="mailto:modocache@gmail.com">modocache@gmail.com</a>> wrote:<br>
> I think #2 is the best option. It’s less work than both #1 and #3. I believe<br>
> logic like IsTestCandidate belongs in libIDE anyway—SourceKit should stick<br>
> to XPC and asynchronous communication with libIDE.<br>
<br>
</span>I like #3 better (an option to swiftc), because that would decouple<br>
the test discovery tool from the Swift compiler. That would allow you<br>
to use the discovery tool with different compilers. And, because we<br>
would avoid statically linking libIDE, it would mean one less copy of<br>
LLVM, Clang and Swift in the toolchain.<br>
<span class=""><br></span></blockquote><div><br></div><div>My concern with adding a swiftc option would be deciding upon an interface to it. Would we want something like clang-query--a generic method of finding methods that match a set of criteria? That seems like it would be a lot of work to implement.<br><br>Perhaps I'm misunderstanding what you have in mind for swiftc. Could you elaborate?</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span class="">
> Not being an expert in many of these components, I have several questions:<br>
><br>
> I’m assuming the reflection API to return a list of instance methods on a<br>
> XCTestCase subclass is not ready yet, and won’t be for some time. Is this<br>
> accurate?<br>
<br>
</span>I think so.<br>
<span class=""><br>
> I’m assuming that SourceKit is intended to be an asynchronous wrapper over<br>
> libIDE, and that logic like IsTestCandidate should be moved to libIDE. Is<br>
> this accurate?<br>
<br>
</span>SourceKit has a lot of functionality of its own, but moving this<br>
particular piece of logic to libIDE sounds reasonable.<br>
<span class=""><br>
> I’m assuming that SourceKit is coupled with XPC, and that it would be more<br>
> work to port it to Linux than it would be to move its logic to libIDE. Is<br>
> this accurate?<br>
<br>
</span>It is not tightly coupled with XPC, there is a portability layer that<br>
you could implement for Linux. You would need to decide on an IPC<br>
mechanism and serialization format though.<br>
<span class=""><br>
> If you have thoughts/feedback, please reply to this email or comment on<br>
> SR-710. Your input would be greatly appreciated!!<br>
<br>
</span>I'm wondering how feasible is it to change the XCTest API to<br>
accommodate better the Swift language that we have, rather than trying<br>
to add custom tooling to make the existing API work. Adding magic<br>
tooling that adds behavior not present in the language seems unnatural<br>
to me.<br>
<br>
Compare with StdlibUnittest -- by using an API to build tests we get<br>
the following advantages:<br>
<br>
- We completely avoid having the issue of test discovery, executing<br>
the code discovers the tests. No reflection needed!<br>
<br>
- We can add attributes to tests (for example, skip, xfail). In the<br>
current XCTest API this would require adding some kind of user-defined<br>
attributes, which is another language which is a long way from being<br>
designed and implemented.<br>
<br>
- We can define data-parameterized tests.<br>
<br>
- Tests can be dynamically synthesized by control flow. In the<br>
current XCTest API, dynamically generating tests would mean<br>
dynamically generating methods, which is even more far off than<br>
read-only method reflection.<br></blockquote><div><br></div><div>I definitely agree that all of these are fantastic features. However, the corelibs-xctest README spells out the following goal for the project: "This version of XCTest uses the same API as the XCTest you are familiar with from Xcode. Our goal is to enable your project's tests to run on all Swift platforms without having to rewrite them."</div><div><br></div><div>This goal often conflicts with what would be the best API for a Swift testing framework--we are forced to use mutable reference types, extensive subclassing, include a ton of optional properties--all in order to mirror exactly the Apple XCTest API.</div><div><br></div><div>I suppose we *could* modify the SDK overlay for XCTest to provide an API more suitable for Swift, but I think that would involve a lot of coordination with Apple XCTest and Xcode teams, some swift-evolution proposals, and a potentially harsh migration path for developers writing their tests in Apple XCTest. So while I think corelibs-xctest and Apple XCTest can (and should) evolve to better take advantage of Swift features over time, I don't think we can migrate both before Swift 3's release.<br><br>(+CC Mike Ferris, who works on Apple XCTest and corelibs-xctest.)</div><div><br></div><div>- Brian Gesiak</div><div><br></div></div></div></div>