[swift-corelibs-dev] Add support for gathering test coverage data on XCTest

[William Dillon]

I would love for there to be access to test coverage data (especially on linux). One thing that concerns me about the way tests work on linux is that it’s very easy to miss tests in the allTests method. I’ve always got a nagging worry that there are missing tests I don’t know about floating around that aren’t running on Linux. This is to say nothing about coverage differences due to conditional compilation arising from "#if os(OSX) || os(iOS) || os(watchOS) || os(tvOS)” constructs.

- Will


On December 14, 2016 at 1:16:56 PM, Jacopo Andrea Giola via swift-corelibs-dev (swift-corelibs-dev at swift.org(mailto:swift-corelibs-dev at swift.org)) wrote:

> Hi all,  
>  
> I was wondering if it can be a great idea to build the capability of gathering test coverage data inside the XCTest framework to be able to use it for gathering the data on the various platform without having to relay on the Xcode capability.  
>  
> Today the only way a swift project can be gather coverage data during its ci build is to relay on building at least for the macOS platform, and only in that case, generate the xcproj file, running the tests with xcbuild, and after that we can have data to look at.  
>  
> I’m trying to find on GitHub some swift project that has integrated a ci flow, and all the one that at the end wants to have the tests coverage data use this flow, on linux they use the swift cli commands for building and testing the project, and on macOS they use Xcode to gathering the data. Having two different flow for me is kinda crazy, and more crazy is to relay on the fact that if the tests coverage on macOS is 100% it will be even on Linux (or any other platform that Swift will support in the future) counting that in the code there can be different path used based on the platform.  
>  
> Asking on the SwiftPM Slack channel, they point me on this list and here I am, it can be done? There is interest in this from the community?  
> The top will be that the swift test command will automatically generate the coverage data, and we can then pass it some kind of flag for choosing between two or three different output (like JSON, XML or what is the main file format this day to integrate nicely with things around the web and using services like codecov.io(http://codecov.io)).
>  
> - Jacopo _______________________________________________
> swift-corelibs-dev mailing list
> swift-corelibs-dev at swift.org
> https://lists.swift.org/mailman/listinfo/swift-corelibs-dev