[swift-evolution] adding automated-testing of uncompilable features in XCTest

[Jean-Daniel]

An interesting reading about testing private members:

https://cocoacasts.com/how-to-unit-test-private-methods-in-swift/ <https://cocoacasts.com/how-to-unit-test-private-methods-in-swift/>


> Le 12 déc. 2016 à 06:10, Derrick Ho via swift-evolution <swift-evolution at swift.org> a écrit :
> 
> It bugs me as well that we can not test private components using XCTest. Objective-c never had this problem since privacy doesn't exist. I would like to see a version of XCTest that allows us to test every component.
> 
> 
> On Mon, Dec 12, 2016 at 12:02 AM Benjamin Spratling via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
> Howdy,
>         I’d like to see how much interest there is for adding these to the XCTest module.  If I have missed some pro or con, or missed a technical point, your feedback is welcome before I go to the lengths to draw up a formal proposal.
> 
> 
> There are several features of Swift which cannot be effectively automated-tested.
> For instance, when a type or one of its members is private or file private, an outside test suite cannot access it, and the information that the type is private is not included in a Mirror.  There are similar concerns for testability with internal, and public access levels.  Tests can be written ensuring these access levels are >= some level, but not == or < some level.  In other words the very usefulness of these features cannot be tested.
> 
> Other attributes to be tested in this way are:
> 
> - Mutability of a member.  No automated test can be written which verifies that a function cannot be called on a let struct, or that a property cannot be set at a specific access level.
> 
> - That a stored property is weak or unowned.
> 
> - That a class or class member is “final”
> 
> These are concepts which need to be unit-tested to ensure good design is not broken, but do not need to be included in a release build, so including them in the XCTest framework seems like an appropriate destination.
> Moreover, the information for all of these features exists in the .swiftmodule files, which are included in test builds, but sometimes stripped for release.
> 
> Examples:
> Since these features inherently have to do with testing features which cannot be stated in compiled code, I recommend specifying names with Strings.  Here are some examples of what I would like to write in my test code:
> 
> XCTAssertEqual( Module(named:”SingMusicLayout”)?.type(named:”NoteSetter”)?.property(named:”session”)?.accessLevel, .private)
> 
> XCTAssertEqual( Module(named:”SingMusicLayout”)?.type(named:”ScaleLayout”)?.method(named:”baselineOffset(for:PitchInterval)->CGFloat”)?.mutable, false)
> 
> 
> Alternatives:
> 1.      Building an independent .swiftmodule parser in a single Swift module, which can be included in test builds.
>                 + Can be distributed independently from Swift sources, requiring 0 buy-in from Swift community
>                 + requires a single additional module for the test.
>                 - Depends on ever-changing binary interface.
>                 : Intractable, not DRY
> 
> 2.      Use existing sourcekitd.
>                 + harnesses changes in the compiler’s source code with SourceKit
>                 - cannot be run in a test suite without extensive work by user to configure bundles explicitly.
>                         Exceptionally poor user experience
>                 : sourcekitd XPC architecture only works on macOS
> 
> 3.      Use a standalone tool for tests
>                 + harnesses changes in the compiler’s source code with SourceKit
>                 + no installation in user’s source code necessary
>                 : cannot be effectively run by SPM test infrastructure
> 
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20161213/07575246/attachment.html>