[swift-evolution] [swift-build-dev] [Review] SE-0019 Swift Testing (Package Manager)

[Brian Pratt]

> The caveat is have is this: don’t couple the the atomic steps together
when incrementally building it up. When the steps aren’t coupled, it’s
significantly easier to iteratively make changes to each of the components.
It’s also much easier to review and develop additions because you’ve
reduced the surface area for the integration points so there is less to
consider, and consequently, it’s easier to validate breaking changes.
>
> For example, before any meaningful work can be done to support different
types of test runners, the work is going to need to done to break the build
& run tests coupling. In this way, I think the proposal does a disservice
because it’s not laying a good foundation to build on, it’s laying a
foundation that we already know needs to be broken up.
>
> There’s a difference between “get something done quick” and “get the most
minimal viable product” (MVP) out the door. My objection to the proposal is
that I don’t see it setting up the MVP.


Can't agree with this more. The point of integration between all these
tools (and atomic steps) should be as minimal as possible, and so far the
proposals that have come through regarding these components of the building
and testing project have either felt very heavy-weight or very coupled (for
example, to XCTest or similar).  As an example: A simple function that can
throw an error (and *maybe* returns a simple status of pass/fail?) is
probably enough to facilitate the vast majority of testing frameworks, and
it *doesn't* come with lots of baggage of protocols and implementation
details that we can't assume will be required or even useful for said
framework. We should consider what we can leave out instead.

Swift facilitates all sorts of practical wins in code to promote loose
coupling in an extremely lightweight way, without having to resort to the
relatively heavy OO patterns we see in Java (and, at times, in C#) -- many
of which largely exist because the languages didn't have support for
closures until recently. As someone who has authored a third-party testing
framework (and made it work with the XCode-provided XCTest API, which was a
tremendous headache) I would much prefer to see small, atomic, decoupled
integration points. The way to do that is omitting details and working at a
nice, high level of abstraction.

So thank you, David, for mentioning this, and I'm 100% in agreement with
you.

My biggest concern with this proposal, as it stands, is that it isn't
considering what I think are some very important details that will shape
the way the build-tool works longer-term. If there's a generic testing
protocol provided by the build tool, how do I import my testing framework
within Package.swift *before* Package.swift has been able to download all
my dependencies?

Is there a need for a multi-step, multi-file process here?

I personally would much rather see us focus our discussion on those
questions to create a longer-term proposal than continue to debate a
proposal that itself admits that it amounts to a temporary solution. Maybe
that's worth opening another topic on swift-build-dev for, but for now, I'm
made really uncomfortable by the uncertainty left in this proposal as
someone who wants to use and/or create third-party testing tools -- this
proposal feels very one-size-fits-all, and that really concerns me.


On Fri, Jan 8, 2016 at 11:13 AM, David Owens II via swift-build-dev <
swift-build-dev at swift.org> wrote:

>
> > On Jan 8, 2016, at 6:08 AM, Drew Crawford <drew at sealedabstract.com>
> wrote:
> >
> > <all the good stuff>
>
> I agree with pretty much all you’ve said. The caveat is have is this:
> don’t couple the the atomic steps together when incrementally building it
> up. When the steps aren’t coupled, it’s significantly easier to iteratively
> make changes to each of the components. It’s also much easier to review and
> develop additions because you’ve reduced the surface area for the
> integration points so there is less to consider, and consequently, it’s
> easier to validate breaking changes.
>
> For example, before any meaningful work can be done to support different
> types of test runners, the work is going to need to done to break the build
> & run tests coupling. In this way, I think the proposal does a disservice
> because it’s not laying a good foundation to build on, it’s laying a
> foundation that we already know needs to be broken up.
>
> There’s a difference between “get something done quick” and “get the most
> minimal viable product” (MVP) out the door. My objection to the proposal is
> that I don’t see it setting up the MVP.
>
> -David
>
> _______________________________________________
> swift-build-dev mailing list
> swift-build-dev at swift.org
> https://lists.swift.org/mailman/listinfo/swift-build-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20160108/5fc4261b/attachment.html>