It bugs me as well that we can not test private components using XCTest. Objective-c never had this problem since privacy doesn't exist. I would like to see a version of XCTest that allows us to test every component.<br><br><br><div class="gmail_quote"><div dir="ltr">On Mon, Dec 12, 2016 at 12:02 AM Benjamin Spratling via swift-evolution <<a href="mailto:swift-evolution@swift.org">swift-evolution@swift.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Howdy,<br class="gmail_msg">
I’d like to see how much interest there is for adding these to the XCTest module. If I have missed some pro or con, or missed a technical point, your feedback is welcome before I go to the lengths to draw up a formal proposal.<br class="gmail_msg">
<br class="gmail_msg">
<br class="gmail_msg">
There are several features of Swift which cannot be effectively automated-tested.<br class="gmail_msg">
For instance, when a type or one of its members is private or file private, an outside test suite cannot access it, and the information that the type is private is not included in a Mirror. There are similar concerns for testability with internal, and public access levels. Tests can be written ensuring these access levels are >= some level, but not == or < some level. In other words the very usefulness of these features cannot be tested.<br class="gmail_msg">
<br class="gmail_msg">
Other attributes to be tested in this way are:<br class="gmail_msg">
<br class="gmail_msg">
- Mutability of a member. No automated test can be written which verifies that a function cannot be called on a let struct, or that a property cannot be set at a specific access level.<br class="gmail_msg">
<br class="gmail_msg">
- That a stored property is weak or unowned.<br class="gmail_msg">
<br class="gmail_msg">
- That a class or class member is “final”<br class="gmail_msg">
<br class="gmail_msg">
These are concepts which need to be unit-tested to ensure good design is not broken, but do not need to be included in a release build, so including them in the XCTest framework seems like an appropriate destination.<br class="gmail_msg">
Moreover, the information for all of these features exists in the .swiftmodule files, which are included in test builds, but sometimes stripped for release.<br class="gmail_msg">
<br class="gmail_msg">
Examples:<br class="gmail_msg">
Since these features inherently have to do with testing features which cannot be stated in compiled code, I recommend specifying names with Strings. Here are some examples of what I would like to write in my test code:<br class="gmail_msg">
<br class="gmail_msg">
XCTAssertEqual( Module(named:”SingMusicLayout”)?.type(named:”NoteSetter”)?.property(named:”session”)?.accessLevel, .private)<br class="gmail_msg">
<br class="gmail_msg">
XCTAssertEqual( Module(named:”SingMusicLayout”)?.type(named:”ScaleLayout”)?.method(named:”baselineOffset(for:PitchInterval)->CGFloat”)?.mutable, false)<br class="gmail_msg">
<br class="gmail_msg">
<br class="gmail_msg">
Alternatives:<br class="gmail_msg">
1. Building an independent .swiftmodule parser in a single Swift module, which can be included in test builds.<br class="gmail_msg">
+ Can be distributed independently from Swift sources, requiring 0 buy-in from Swift community<br class="gmail_msg">
+ requires a single additional module for the test.<br class="gmail_msg">
- Depends on ever-changing binary interface.<br class="gmail_msg">
: Intractable, not DRY<br class="gmail_msg">
<br class="gmail_msg">
2. Use existing sourcekitd.<br class="gmail_msg">
+ harnesses changes in the compiler’s source code with SourceKit<br class="gmail_msg">
- cannot be run in a test suite without extensive work by user to configure bundles explicitly.<br class="gmail_msg">
Exceptionally poor user experience<br class="gmail_msg">
: sourcekitd XPC architecture only works on macOS<br class="gmail_msg">
<br class="gmail_msg">
3. Use a standalone tool for tests<br class="gmail_msg">
+ harnesses changes in the compiler’s source code with SourceKit<br class="gmail_msg">
+ no installation in user’s source code necessary<br class="gmail_msg">
: cannot be effectively run by SPM test infrastructure<br class="gmail_msg">
<br class="gmail_msg">
_______________________________________________<br class="gmail_msg">
swift-evolution mailing list<br class="gmail_msg">
<a href="mailto:swift-evolution@swift.org" class="gmail_msg" target="_blank">swift-evolution@swift.org</a><br class="gmail_msg">
<a href="https://lists.swift.org/mailman/listinfo/swift-evolution" rel="noreferrer" class="gmail_msg" target="_blank">https://lists.swift.org/mailman/listinfo/swift-evolution</a><br class="gmail_msg">
</blockquote></div>