[swift-evolution] adding automated-testing of uncompilable features in XCTest

Wed Dec 14 23:41:43 CST 2016

You might be able to ensure access modifiers by using protocols.

If you want to ensure that a class has a property called foo that is
private you can make a private protocol that specifies a private property
called foo.

This isn't a XCTest but it is a compile time check. If a programmer tries
to change a private property to public then the the compiler will complain.

The same can be done with final. Make a protocol with a static method.

On Wed, Dec 14, 2016 at 8:42 AM Jeremy Pereira via swift-evolution <
swift-evolution at swift.org> wrote:

> The Swift compiler can give you the information that you want.
>
> Here is what you can do:
>
> Write a Swift source file that creates an instance of the type you want to
> test and then tries to access each of the private members.
>
> Write a shell script to compile this source file i a module with the file
> the type is defined in. Have it capture all the error messages by
> redirecting stderr and then count them. If it doesn’t have the right
> number, have the shell script emit a message that looks like a Swift error
> message.
>
> Install the script in a run script build phase. Now you will get an error
> every time one of your private properties or methods loses its access
> modifier.
>
> Personally, I wouldn’t bother. Any test to make sure that private members
> are private requires a separately maintained list of the private members.
> If somebody isn’t disciplined enough to add the word “private” to the
> beginning of a definition, they almost certainly aren’t going to bother
> updating a separate list. And the consequences of omitting “private” are
> only that the module has visibility of it. That’s not a huge deal.
>
>
> > On 13 Dec 2016, at 15:57, Benjamin Spratling via swift-evolution <
> swift-evolution at swift.org> wrote:
> >
> > Howdy,
> >
> > If my immediate goal were to verify private member values at
> intermediate steps in some process, which is the concept under
> consideration in the article you provided, I could access their values with
> a Mirror.
> >
> > That's not my immediate goal.  My immediate goal is to falsify that the
> property is private.  The logic in the article asserts that a member is
> private, then asserts that because the member is private we do not need to
> test it.  But it fails to provide an automated test that the member is
> indeed private and remains so.  To accomplish that goal without an
> automated unit test is painstakingly tedious for a human.
> >
> > And as to whether the logic in the article is correct, assuming its
> assumptions are true, it is still only one kind of testing.  Depending on
> the field it might be called "interface" or "black box" testing, and they
> rightly call that out.  But it would not be considered to be the only kind
> of testing which needs to be done.
> >
> > -Ben
> >
> > Sent from my iPhone.
> >
> > On Dec 13, 2016, at 1:35 AM, Jean-Daniel <mailing at xenonium.com> wrote:
> >
> >> An interesting reading about testing private members:
> >>
> >> https://cocoacasts.com/how-to-unit-test-private-methods-in-swift/
> >>
> >>
> >>> Le 12 déc. 2016 à 06:10, Derrick Ho via swift-evolution <
> swift-evolution at swift.org> a écrit :
> >>>
> >>> It bugs me as well that we can not test private components using
> XCTest. Objective-c never had this problem since privacy doesn't exist. I
> would like to see a version of XCTest that allows us to test every
> component.
> >>>
> >>>
> >>> On Mon, Dec 12, 2016 at 12:02 AM Benjamin Spratling via
> swift-evolution <swift-evolution at swift.org> wrote:
> >>> Howdy,
> >>>         I’d like to see how much interest there is for adding these to
> the XCTest module.  If I have missed some pro or con, or missed a technical
> point, your feedback is welcome before I go to the lengths to draw up a
> formal proposal.
> >>>
> >>>
> >>> There are several features of Swift which cannot be effectively
> automated-tested.
> >>> For instance, when a type or one of its members is private or file
> private, an outside test suite cannot access it, and the information that
> the type is private is not included in a Mirror.  There are similar
> concerns for testability with internal, and public access levels.  Tests
> can be written ensuring these access levels are >= some level, but not ==
> or < some level.  In other words the very usefulness of these features
> cannot be tested.
> >>>
> >>> Other attributes to be tested in this way are:
> >>>
> >>> - Mutability of a member.  No automated test can be written which
> verifies that a function cannot be called on a let struct, or that a
> property cannot be set at a specific access level.
> >>>
> >>> - That a stored property is weak or unowned.
> >>>
> >>> - That a class or class member is “final”
> >>>
> >>> These are concepts which need to be unit-tested to ensure good design
> is not broken, but do not need to be included in a release build, so
> including them in the XCTest framework seems like an appropriate
> destination.
> >>> Moreover, the information for all of these features exists in the
> .swiftmodule files, which are included in test builds, but sometimes
> stripped for release.
> >>>
> >>> Examples:
> >>> Since these features inherently have to do with testing features which
> cannot be stated in compiled code, I recommend specifying names with
> Strings.  Here are some examples of what I would like to write in my test
> code:
> >>>
> >>> XCTAssertEqual(
> Module(named:”SingMusicLayout”)?.type(named:”NoteSetter”)?.property(named:”session”)?.accessLevel,
> .private)
> >>>
> >>> XCTAssertEqual(
> Module(named:”SingMusicLayout”)?.type(named:”ScaleLayout”)?.method(named:”baselineOffset(for:PitchInterval)->CGFloat”)?.mutable,
> false)
> >>>
> >>>
> >>> Alternatives:
> >>> 1.      Building an independent .swiftmodule parser in a single Swift
> module, which can be included in test builds.
> >>>                 + Can be distributed independently from Swift sources,
> requiring 0 buy-in from Swift community
> >>>                 + requires a single additional module for the test.
> >>>                 - Depends on ever-changing binary interface.
> >>>                 : Intractable, not DRY
> >>>
> >>> 2.      Use existing sourcekitd.
> >>>                 + harnesses changes in the compiler’s source code with
> SourceKit
> >>>                 - cannot be run in a test suite without extensive work
> by user to configure bundles explicitly.
> >>>                         Exceptionally poor user experience
> >>>                 : sourcekitd XPC architecture only works on macOS
> >>>
> >>> 3.      Use a standalone tool for tests
> >>>                 + harnesses changes in the compiler’s source code with
> SourceKit
> >>>                 + no installation in user’s source code necessary
> >>>                 : cannot be effectively run by SPM test infrastructure
> >>>
> >>> _______________________________________________
> >>> swift-evolution mailing list
> >>> swift-evolution at swift.org
> >>> https://lists.swift.org/mailman/listinfo/swift-evolution
> >>> _______________________________________________
> >>> swift-evolution mailing list
> >>> swift-evolution at swift.org
> >>> https://lists.swift.org/mailman/listinfo/swift-evolution
> >>
> > _______________________________________________
> > swift-evolution mailing list
> > swift-evolution at swift.org
> > https://lists.swift.org/mailman/listinfo/swift-evolution
>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20161215/083e76eb/attachment.html>