<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Jan 8, 2016, at 12:51 PM, Brian Pratt <<a href="mailto:brian@pratt.io" class="">brian@pratt.io</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><span style="font-family: HelveticaNeue; font-size: 13px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class=""><span class="Apple-converted-space"> </span>A simple function that can throw an error (and *maybe* returns a simple status of pass/fail?) is probably enough to facilitate the vast majority of testing frameworks, and it *doesn't* come with lots of baggage of protocols and implementation details that we can't assume will be required or even useful for said framework. We should consider what we can leave out instead.</span></div></blockquote></div><br class=""><div class="">This is not the case, for reasons you can find in the third-party test frameworks proposal; but shortly: it is not innovation to have a protocol which merely allows 10 competing implementations of an XCTest-like framework.</div><div class=""><br class=""></div><div class="">Testing is not a one-size-fits-all concept. The role of SwiftPM is IMO:</div><div class=""><br class=""></div><div class=""><ul class="MailOutline"><li class="">To have a simple way to run XCTest suites, which is the way most people (including SwiftPM & Foundation & many many codebases) write their tests now, and</li><li class="">To push innovation into third-party test frameworks that are free to explore the problem space in a more unrestricted way that is not bound by the constraints of XCTest, source compatibility, or the swift-evolution proposal process</li></ul><div class=""><br class=""></div></div><div class=""><b class="">This proposal is mostly targeted at the first situation.</b> I do believe it is the "minimum viable proposal" to address that first situation. Perhaps the assumption that we are studying (only) the first situation needs to be more clearly stated.</div><div class=""><br class=""></div><div class="">XCTest compatibility is a feature many people require. I understand folks who find it heavyweight for new code. But I think the best way forward is to kick the design problem for other models out to third parties, because this mailing list will not scale to designing a novel testing framework one API at a time. Let's let the ecosystem go to work on this problem.</div><div class=""><br class=""></div><div class="">We need a proposal to do that, and it is currently being drafted on swift-build-dev. Please help.</div><div class=""><br class=""></div><div class=""><blockquote type="cite" class=""><span style="font-family: HelveticaNeue;" class="">than continue to debate a proposal that itself admits that it amounts to a temporary solution</span></blockquote><br class=""></div><div class="">I don't think this is the case at all; the proposal admits to being one solution among many, and one that will need further extensions. But people have XCTest test suites, they want to test them, we should support it. There's nothing temporary about that.</div><div class=""><br class=""></div><div class=""><blockquote type="cite" class="">On Jan 8, 2016, at 11:13 AM, David Owens II <<a href="mailto:david@owensd.io" class="">david@owensd.io</a>> wrote:</blockquote></div><div class=""><blockquote type="cite" class=""><span style="font-family: HelveticaNeue;" class="">For example, before any meaningful work can be done to support different types of test runners, the work is going to need to done to break the build & run tests coupling. In this way, I think the proposal does a disservice because it’s not laying a good foundation to build on, it’s laying a foundation that we already know needs to be broken up.</span></blockquote></div><div class=""><br class=""></div><div class="">I don't believe this is the case either. Under this proposal, building and running is coupled at the CLI, but introducing new CLI arguments is not any great hardship. </div><div class=""><br class=""></div><div class="">It may help to clearly identify the coupling you mean, and how this proposal very specifically establishes it. Everybody agrees that "coupling is bad, kids" but not everyone understands how this proposal brings it into being.</div><div class=""><br class=""></div><div class="">I wonder if your objection is simply that the XCTest proposal hit review first, rather than the third-party frameworks proposal. That is only because the XCTest folks are simply better motivated to draft it: there is tons of existing code that can benefit, the test framework actually exists, etc. The third-party frameworks are not as well-motivated.</div></body></html>