[swift-build-dev] [swift-evolution] [Review] SE-0019 Swift Testing (Package Manager)

[David Owens II]

> On Jan 8, 2016, at 8:28 PM, Drew Crawford <drew at sealedabstract.com> wrote:
> 
> 
>> On Jan 8, 2016, at 9:20 PM, David Owens II <david at owensd.io <mailto:david at owensd.io>> wrote:
>> 
>> At some later date, maybe we’ll get “swift build --run-tests” or “swift tests --no-build” that simply runs the tests without attempting to do a build. Ok, maybe we will. If it’s a week later, ok. If it’s a month later, ok. If it’s going to be a year away, that becomes a real issue. However, there is no compelling reason to couple this from the start, especially if there is no architectural considerations to worry about.
> 
> 
> I think the common case (perhaps it is uncommon for you) is a test-driven implementation loop, where somebody has a failing test, they are working in a text editor or IDE, and they want to know if it the thing they changed caused the tests to pass.  This is my workflow when I PR something to Foundation or SwiftPM, and my workflow on other projects.

I don’t run all of my tests when I do this. I re-run the subset that actually apply to what I’m working on. Then before performing a check-in, I re-run all of the tests.

> Certainly that could be accomplished via chaining commands instead (e.g. "swift build && swift test") which better supports your workflow although that is slightly more awkward for the common situation.

This is only the “common situation” if what I was talking about in earlier posts is true: the testing that is being support here is primary for unit tests only or integration tests that can run as quickly as unit tests. But yes, I’m saying that the proposal should enable that exact scenario as the building block. 

> But in any case this whole argument feels to me like a bikeshed–we have infinite freedom to support any number of CLI commands, with builds & tests together, separated, in a box, with a fox.   This one was elected to support a common workflow, not to the exclusion of all the other things we could do additionally.

Again, if it so easy, then why not simply come out of the gate with “build”, “test”, and “run-tests”. Surely we’ve already discussed this with more time then it would be to implement if SPM’s architecture is actually going to allow this and that shortcuts are not taken to get the “test” portion working more quickly. Call me jaded by experience with other tools, but whenever I’ve seen shortcuts like these taken, it’s always more work than a simple add of a new command to break what is being treated by the current implementation as an atomic command into two.


> But there is language being used on this point (from several people) like:
> 
>> there is no compelling reason to couple this from the start
> 
>> My biggest concern with this proposal, as it stands, is that it isn't considering what I think are some very important details that will shape the way the build-tool works longer-term.
> 
>> For example, before any meaningful work can be done to support different types of test runners, the work is going to need to done to break the build & run tests coupling. In this way, I think the proposal does a disservice because it’s not laying a good foundation to build on, it’s laying a foundation that we already know needs to be broken up.
> 
> I fail to understand how the election of one (first) command line argument has any of these follow-on consequences at all.  We are discussing a CLI frontend, not an architecture diagram.
> 
> If someone believes we are in danger of laying a bad foundation then that is an important objection, but to understand it we need an argument of the form
> 
> 1.  "swift build -t"
> 2. ???
> 3.  Therefore, [specific irreversible harm]
> 
> Until that is laid out in a way I can follow, I can't make sense of the objection.

I don’t know how else to explain my objection; I’ve done so in at least three different ways, including examples of how a similar design affects my previous team with still unsupported behavior in other Apple tools that have the same design as being illustrated here (Rick, I’m reaching out to my old counterparts to see if they have any radars they want followed up with). It’s not irreversible harm, it’s starting the process with unnecessary technical debt. 

Fortunately, this time around we can at least fork the tools to get the support we need, if it doesn’t come (and potentially send the fix back or submit proposals), instead of do what Facebook had to do with xctool.

-David

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-build-dev/attachments/20160108/16e18839/attachment.html>