[swift-evolution] [Draft] Test-Only Package Dependencies and Targets

[Ankit Agarwal]

On Thu, Jan 26, 2017 at 6:37 AM, thislooksfun via swift-evolution <
swift-evolution at swift.org> wrote:

> I still much prefer 'testDependencies/Targets'. You seem to be confusing
> this proposal with the (already accepted) Package Manager Product
> Definitions
> <https://github.com/apple/swift-evolution/blob/master/proposals/0146-package-manager-product-definitions.md>
>  proposal.
> This proposal is strictly for running under `swift test`. The issue
> currently is that running any swift command (specifically anything that
> ends up calling `swift build`), will attempt to build testing libraries /
> targets, which then fail because they rely on XCTest, which is only
> available for import when running `swift test`. The idea here is that the
> libraries / targets will *only* be compiled when running in a testing
> environment, without having to manually switch out Package.swift files.
>
>
The test targets are *not* compiled when you run swift build. They're only
complied on invocation of `swift test`. If a test target depends on a
testing library (as in my example), they will *not* be built on invocation
of `swift build`.

Also, you can import XCTest from your regular targets but that code will
only run from a test target. (see TestSupport module in SwiftPM).



> -thislooksfun (tlf)
>
> On Jan 25, 2017, at 2:07 AM, Ankit Aggarwal via swift-evolution <
> swift-evolution at swift.org> wrote:
>
>
> On 25-Jan-2017, at 4:02 AM, Robert Widmann via swift-evolution <
> swift-evolution at swift.org> wrote:
>
> Hello Swift Community,
>
> Harlan Haskins and I have been working on libraries
> <https://github.com/trill-lang> to make interacting with LLVM and Clang’s
> APIs more elegant with native Swift interfaces.  While writing up the
> packages we realized the package manager wouldn’t allow us to specify
> testing targets and test-only dependencies.  To rectify that, I have
> attached a draft proposal for adding test-only targets and dependency
> fields to the Swift Package manager.  This proposal can also be read in
> gist form
> <https://gist.github.com/CodaFi/6bd83e5315c7d30aeaf4154ed3b03a38>.
>
> Cheers,
>
> ~Robert Widmann
>
>
> Thanks for driving this! It is a very desirable feature which needs
> proposal work. Comments inline.
>
>
>
> Test-Only Package Dependencies and Targets
>
>    - Proposal: SE-NNNN <https://gist.github.com/CodaFi/NNNN-filename.md>
>    - Authors: Harlan Haskins <https://github.com/harlanhaskins>, Robert
>    Widmann <https://github.com/codafi>
>    - Review Manager: TBD
>    - Status: Awaiting review
>
>
> <https://gist.github.com/CodaFi/6bd83e5315c7d30aeaf4154ed3b03a38#introduction>
> Introduction
>
> This proposal reinstates Swift package manager’s ability to fetch
> dependencies and build targets scoped exclusively to the testing module(s)
> of a given package.
>
> Swift-evolution thread: Discussion thread topic for that proposal
> <https://lists.swift.org/pipermail/swift-evolution/>
>
> <https://gist.github.com/CodaFi/6bd83e5315c7d30aeaf4154ed3b03a38#motivation>
> Motivation
>
> Soon after SE-0019
> <https://github.com/apple/swift-evolution/blob/master/proposals/0019-package-manager-testing.md#test-only-dependencies> identified
> the need for richer test-only dependencies and targets, a decision was made
> to remove the package manager’s fledgling ability to treat certain
> dependencies as test-only. This has led to a myriad of
> clever-but-needlessly-complex workarounds ([1]
> <https://github.com/ReactiveCocoa/ReactiveSwift/blob/master/.travis.yml#L85>
> , [2] <https://github.com/ReactiveX/RxSwift/blob/master/Package.swift#L3>
> , [3] <https://github.com/Quick/Quick/blob/master/.Package.test.swift>)
> on the part of 3rd parties to recover the feature themselves. In addition,
> the Swift community has come up with a number of their own frameworks to
> augment functionality in XCTest but depending on these external testing
> frameworks is brittle and difficult to get right.
>
> <https://gist.github.com/CodaFi/6bd83e5315c7d30aeaf4154ed3b03a38#proposed-solution>Proposed
> solution
>
> We propose the re-introduction of the testDependencies parameter in
> Package Manifests to support external test-only dependencies. To support
> local test-only targets we also propose the introduction of the
> testTargets parameter and an extension of the existing swift test command
> to support individual invocation of these targets.
>
> <https://gist.github.com/CodaFi/6bd83e5315c7d30aeaf4154ed3b03a38#detailed-design>Detailed
> design
>
> The behavior of the new testDependencies parameter mirrors that of the
> existing dependencies parameter with one important difference: fetched
> dependencies are only built to support package-defined test targets as part
> of an invocation of swift test.
>
> import PackageDescription
> let package = Package(
>     name: "Foo",
>     targets: [
>         Target(name: "Foo")
>     ],
>     dependencies: [
>         .Package(url: "https://github.com/org/ana.git", versions: Version(1,0,0)...Version(1,9,9)),
>     ],
>     testDependencies: [
>         .Package(url: "https://github.com/org/anism.git", versions: Version(1,0,0)...Version(1,9,9)),
>     ]
> )
>
>
> I think this feature should be called local dependencies (or maybe dev
> dependencies) because it can be used for tests as well as regular targets.
> As an example say you have a networking library package and you want to
> create an example CLI target which uses a JSON mapper package. You wouldn't
> want to vend the CLI tool when you act as a dependency to other packages,
> or include the JSON mapper in your dependencies. Test dependency doesn't
> sound right in that context.
>
> After the product proposal
> <https://github.com/apple/swift-evolution/blob/master/proposals/0146-package-manager-product-definitions.md> is
> implemented, you will be able to control what you vend to your clients, and
> we thought about adding ability to define dependencies in-line with target
> dependencies but left it out of the proposal to keep it simpler. Maybe this
> proposal can add that instead of a separate `testDependencies` property.
> Consider this manifest:
>
> let package = Package(
>     name: "FooNetworking",
>     targets: [
>         Target(
>             name: "FooNetworking"),
>         Target(
>             name: "FooNetworkingExample",
>             dependencies: [
>                 "FooNetworking",
>                 .package(url: "/path/to/BarJSON.git", version: "1.0.0", product: "BarJSON"), // Note: This doesn't actually exists right now.
>             ]),
>         Target(
>             name: "FooNetworkingTests",
>             dependencies: [
>                 "FooNetworking",
>                 .package(url: "/path/to/Quick.git", version: "1.0.0", product: "Quick"), // Note: This doesn't actually exists right now.
>             ]),
>    ],
>    products: [
>        .Library(name: "FooNetworking", targets: ["FooNetworking"]),
>    ])
>
>
> This manifest defines three targets and one product.
>
> *FooNetworking*: The base target and the actual library.
> *FooNetworkingExample*: The example cli tool. It depends on
> *FooNetworking* target and *BarJSON* product from an external package.
> *FooNetworkingTests*: The test target depends on an external package
> *Quick*.
>
> Both *BarJSON* and *Quick* are local dependencies to this package. If
> *FooNetworkingExample* was also vended as a product, *BarJSON* would
> automatically become a regular external dependency.
>
> Similarly, the behavior of the testTargets field mirrors that of the
> existing targets field but defines a set of targets that are only built
> during an invocation of swift test. Importantly, a target defined in
> testTargets may reference a target defined in targets but not vice-versa.
> Should that behavior be needed, the test target should be promoted to a
> “full” target.
>
> import PackageDescription
> let package = Package(
>     name: "SwiftPM",
>     targets: [
>         Target(
>             name: "PackageDescription",
>             dependencies: []),
>
>         // MARK: Support libraries
>         Target(
>             /** Cross-platform access to bare `libc` functionality. */
>             name: "libc",
>             dependencies: []),
>         Target(
>             /** “Swifty” POSIX functions from libc */
>             name: "POSIX",
>             dependencies: ["libc"]),
>         Target(
>             /** Basic support library */
>             name: "Basic",
>             dependencies: ["libc", "POSIX"]),
>
>         /* Omitted for Brevity */
>      ],
>      testTargets: [
>         Target(
>             name: "BasicPerformanceTests",
>             dependencies: ["Basic"]),
>        /* Omitted for Brevity */
>      ]
>  )
>
> Finally, with well-defined test targets in hand, we propose swift test be
> amended to support individual test execution.
>
> We propose the following syntax to execute all tests of all known test
> targets.
>
> $ swift test
>
> To run a set of specific test cases, reference the module-defining test
> target and the specific name of a subclass of XCTestCase:
>
> $ swift test TestModule.TestCase
>
> To run an individual test case, reference the module-defining test target,
> the name of the test case subclass, and the name of the test:
>
> $ swift test TestModule.TestCase.exampleTest
>
>
> We already have all these features. A target is a test target if it has a
> suffix *Tests *and is placed under *Tests/ *directory — we might loosen
> these restrictions with custom target conventions feature. This proposal
> <https://github.com/apple/swift-evolution/blob/master/proposals/0129-package-manager-test-naming-conventions.md> could
> clarify further.
>
> Here is a summary of what `swift test` currently supports:
> * Test targets are only built (and run) when you run `swift test`.
> * `swift test --skip-build` skips building of test targets and runs
> whatever was last built.
> * `swift test -s <test-module>.<test-case>` will run a test case class.
> * `swift test -s <test-module>.<test-case>/<test>` will run an individual
> test.
> * `swift test --list-tests` will list all tests in above format.
> * `swift test --parallel` will run tests in parallel.
>
> You can view these options and their help text using `swift test --help`.
>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
>
>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
>


-- 
Ankit
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170126/09e3749d/attachment.html>