[swift-evolution] [swift-evolution-announce] [Review] SE-0159: Fix Private Access Levels
Xiaodi Wu
xiaodi.wu at gmail.com
Tue Mar 21 17:26:21 CDT 2017
On Tue, Mar 21, 2017 at 3:59 PM, Drew Crawford via swift-evolution <
swift-evolution at swift.org> wrote:
> The arguments for the revert are in the proposal and in the discussions
> in this thread.
>
>
> What is in the proposal and this thread are a set of opinions, e.g.
>
> * "the access level change of SE-0025 was met with dissatisfaction by a
> substantial proportion of the general Swift community."
> * "Those changes can be viewed as actively harmful,”
> * "it is extremely common to use several extensions within a file.”
> * “[existing regime] encourages overuse of scoped access control”
> * “[fileprivate is the] more reasonable default”
>
> I am asking for the arguments or evidence that led you or others to these
> opinions. I understand that you and others believe them, but you
> understand that they are now a matter of debate. We are not going to reach
> any mutual understanding just asserting things we believe. The underlying
> evidence is what would be necessary to convince others to our point of
> view. I am trying to ascertain what that evidence is.
>
> > It has pointed quite a few times by core team members that comparison to
> languages is not a very strong arguments,
>
>
> The context here is that you presented an argument that Swift’s “private”
> was confusing to users of other languages:
>
> > In most languages, that keyword is “private” so its valid to say that
> newcomers to the language will “default” to using that one.
>
> If you no longer think that argument is strong because it relies on a
> comparison to other languages, then we agree :-) But in that case we lose
> an argument for this proposal. The reason we are talking about other
> languages is because that was an argument advanced to support the proposal.
>
>
I am confused by this response. An argument in _support_ of new `private`
was that it is more in line with expectations of users coming from other
languages. In other some cases, such an argument might have its place.
However, as others have pointed out on this list, those other languages
don't have facilities like Swift's extensions, and therefore it was not a
very strong argument for new `private` to say that it better aligns with
user expectations coming from other languages.
> > Some people used the for(;;) loop, the ++ operator, var parameters.
>
>
> In those cases, there was extensive discussion of how to achieve the
> things people were achieving with those features with alternative patterns.
> var parameters for example have a 1LOC workaround to achieve the previous
> semantics, ++ had two characters, for(;;) was slightly trickier but we had
> a clear concept of the scope of the impact. So far as I’m aware, the only
> suggestion to people who currently use a scoped access modifier is to stop
> having their problems.
>
You've given one example of code that takes advantage of new `private`. As
far as I can see, in that example, you can rename `private func prune` to
`private func _prune` and have no further problems. That is to say, the
workaround is a single character at each use site, which by your metric
makes this proposal half as onerous as removing `++`.
So the circumstances of earlier removals and this one are very different.
>
> > • either a programmer ... will use private as often as possible
> > • or a programmer will … use fileprivate all the time
>
> There is also the possibility that a programmer considers which modifier
> is appropriate for the code they are writing. You "argue that... is very
> rare” but I still am not sure what this argument is or what led you or
> others to this conclusion.
>
I'm not enamored of the argumentation in the proposal set out above. But
since you are saying that you're not sure how anybody came to support this
proposal, here's how I arrived at supporting it:
A core team member (I'm blanking on who) has pointed out that, in the end,
the only necessary access modifiers are public and not public (spelled
"internal" in Swift). It has been my understanding that supporting all
possible ways of hiding a member from arbitrary other code within the same
module is a _non-goal_. There have been numerous attempts at designing
elaborate access modifier schemes that make this possible with friends,
protected, shared, what-have-you, and I try to bring up this point each
time.
Beyond the two absolutely essential access modifiers, anything else does
not add to safety as the term is understood in Swift, but rather improves
developer experience. But what is gained in terms of developer experience
with each additional access modifier (due to increased expressiveness) is
offset by what is lost due to complexity. So then the question is: what is
the ideal number of access modifiers? is it two? three? four? five? six?
eight? ten? twelve?
When new `private` and `fileprivate` were shipped, there were numerous
questions asked by users on forums such as Stack Overflow (easily googled;
just as a single example:
http://stackoverflow.com/questions/39027250/what-is-a-good-example-to-differentiate-between-fileprivate-and-private-in-swift),
on blogs (again, easily googled; as a single example:
https://useyourloaf.com/blog/swift-3-access-controls/), and messages to
this list (including from people not part of the original discussion but
who, on using the feature, found it irksome enough to join the list and
make their voice heard).
You see the new access modifiers described, for example, as something that
"take[s] some getting used to." Rather damning as a euphemism, don't you
think? Certainly not a ringing endorsement--
"How do you like my new haircut?"
"It, um, takes some getting used to."
Others also wrote to say that they were finding it difficult to *teach*
this new scheme to students; you can find such messages in the list
archives. I take this to mean that having four/five access modifiers
(depending on whether you consider `open` to be an access modifier) to be
one too many.
When the Swift core team approved the proposal, they expressed the belief
that `fileprivate` would be rarely used. This has turned out plainly and
indisputably not to be true. It is clear that they expected Swift to have,
after SE-0025 (and before `open`), three commonly used access modifiers and
one vestigial one. Instead we have four/five commonly used access
modifiers. Thus, the complexity of the current system is _not_ what was
envisioned at the time of approval for SE-0025, and the resulting system
should be re-evaluated in light of this real-world data.
So, if four/five access modifiers are too many, which one is carrying the
least weight? Which one could be removed to simplify the scheme while
maintaining the most expressiveness? Which one doesn't fulfill even its own
stated goals? Well, one of the key goals of `private` was to allow members
to be encapsulated within an extension, hidden even from the type being
extended (and vice versa for members defined in the type). It says so in
the first sentence of SE-0025. As seen above in my discussion with Charles
Srstka, even supporters of `private` disagree with that motivation to begin
with. The kicker is, _it also doesn't work_. Try, for instance:
```
struct Foo {
private var bar: Int { return 42 }
}
extension Foo {
private var bar: Int { return 43 }
}
```
The code above should compile and does not. If I understood correctly the
explanation from a core team member on this list, it's unclear if it can be
made to work without changing how mangling works, which I believe impacts
ABI and is not trivial at all. Thus, (a) even proponents of new `private`
disagree on one of two key goals stated for new `private`; (b) that goal
was never accomplished, and making it work is not trivial; (c) no one even
complained about it, suggesting that it was a low-yield goal in the first
place.
So, I conclude that new `private` needs re-evaluation in light of
real-world implementation and usage experience. I haven't even mentioned
the nuances of `private` in one scope being different from `private` in the
containing scope, requiring distinguishing of effective access levels from
nominal ones--just one more wrinkle in this whole scheme. I feel that
allowing someone to write `private var foo` instead of `private var _foo //
don't call this outside the type`, when that member isn't vended and can't
be seen outside the file anyway, is a tiny win in comparison to the huge
complexity both of implementation (it's still not correctly implemented and
may not be in time for Swift 4!!!) and of usage.
On March 21, 2017 at 1:41:48 PM, David Hart (david at hartbit.com) wrote:
>
>
>
>
>
> Sent from my iPhone
> On 21 Mar 2017, at 16:57, Drew Crawford <drew at sealedabstract.com> wrote:
>
>
>
> > I’m not arguing that it is less or more than a majority. I’m just saying
> that we’ve seen a lot of talk against the original change.
>
> This proposal asks us to balance the convenience of one group
> (extension-writers) against the existence of another (scoped-access
> users). To do that, we need a clear idea of the composition of both groups.
>
> “A lot of talk” is not the evidentiary standard to remove a feature. It
> was not good enough when we introduced the feature, that required argument
> and clear use-cases.
>
> "A lot of talk" is not the evidence supporting the proposal: it's just a
> warning that something may be very controversial among the community. The
> arguments for the revert are in the proposal and in the discussions in this
> thread.
>
> > By default, I did not mean the syntactic default of the language but the
> access modifier users will use “by default” when trying to restrict
> visibility. In most languages, that keyword is “private” so its valid to
> say that newcomers to the language will “default” to using that one.
>
> Apologies, but I do not understand the argument:
>
> 1. A user wants to restrict visibility (e.g. they are dissatisfied
> with “internal”)
> 2. The user *chooses* private because of familiarity from another
> language
> 3. The user is then surprised that their choice of private indeed
> restricted the visibility, thus achieving their goal?
>
> What language does the user come from in which “private” is file-visible?
> It isn’t Java, C++, or PHP. C#’s “partial” is the closest I can think of,
> and it isn’t at all close.
>
>
> It has pointed quite a few times by core team members that comparison to
> languages is not a very strong arguments, especially when Swift does things
> differently for a good reason. I can't stop from quoting Xiaodi from a
> month back:
>
> «The beauty of Swift 2's access modifiers was that they were based around
> files and modules, explicitly rejecting types and scopes as units for
> determining visibility.» -- Xiaodi
>
> A user who wants a middle-ground visibility would “default” to
> “protected”, “friend”, “partial”, or similar. After that does not compile,
> they will use google to find a middle-road visibility keyword, for which
> the only candidate is “fileprivate”. But they will not choose “private”,
> it’s just not a reasonable expectation of what the keyword means to a new
> Swift developer.
>
> The popularity of private “as a default” is simply because many users
> prefer to hide their implementation details as a matter of routine code
> hygiene. Redefining private in order to thwart their code hygiene goal
> seems extreme.
>
>
> The point is that keeping both private and fileprivate feels like an
> un-necessary complication:
>
> • either a programmer falls on your side of the fence and will use private
> as often as possible and relegate to fileprivate when the design leaves no
> other choice. At that point it feels like a language wart.
> • or a programmer will fall on my side of the fence and use fileprivate
> all the time and the language feels like it has an unnecessary access
> modifier.
>
> I'd argue that the cases when a programmer will use both meaningfully is
> very rare. As a consequence, we should try to only keep one. Removing
> fileprivate is a no-go with extensions so that leaves us with removing
> private.
>
> I agree with several here (as I did in SE-0025) that our access modifiers
> are not well-named. However, that’s not the proposal in front of us.
>
> > My own statistics in my projects show the contrary. At best, this shows
> how divisive this feature is.
>
> This *may* show that, if contrary statistics were presented, but that
> hasn’t occurred.
>
> I can generate statistics from my projects if you want. But it's
> unnecessary: I haven't used private once since it's introduction in Swift
> 3. I don't see the advantages it brings worth the trouble.
>
>
> - In old code, statistics could be biased by the migrator having
> replaced all previous instances of private by fileprivate.
>
> If the migrator migrated code to private, and it *worked* (e.g. did not
> introduce visibility errors) this is not bias, this is a correct use of the
> feature.
>
> The migrator migrated to fileprivate everywhere, not private, disagreeing
> with your use of fileprivate.
>
> > I'm just arguing that the additional scope-based access modifier does
> not provide enough differentiation to be worth that complexity.
>
> The only argument I have seen so far around “complexity” boils down to:
> “some people do not use it”. But some people *do* use it, and anyway if we
> are going to remove all the features “not enough people” use then we are in
> for a ride.
>
> Some people used the for(;;) loop, the ++ operator, var parameters. Many
> other features were removed from Swift to simplify he language, make it
> more consistent. Those are worthwhile goals. Yes, we are past Swift 3 now,
> but that doesn't mean we shouldn't be able to propose a few rare breaking
> proposals. The implementation of access modifiers came so late in the Swift
> 3 timeframe that we had little time to play around with them before Swift 3
> was released. Now that we have, we have a short window of time to fix
> mistakes that were made. I'm just arguing that the proposal was one of
> those mistakes. But you have a right to disagree.
>
> Swift 3 shipped, so what we are discussing now is yanking a keyword
> without replacement. There is code written that uses private to enforce
> its threading or security invariants. There is code written that uses
> private in order to shadow another declaration. There is code that will
> not compile after migration. We need more than a vague fear of complexity
> generally to throw a brick through all those windows. That brick will
> introduce quite a bit of complexity itself.
>
> Concerning the one-class-per-file argument, I would suggest this
> counter-argument: when working in large projects, I believe it's a good
> thing if the language encourages (forces is too strong a word for my taste)
> a one class per file structure, it's good practice.
>
>
> The form of the argument is invalid. Suppose I argued: "it’s a good thing
> for the language to encourage one definition per class (no extensions),
> it’s good practice. So we do not need fileprivate.” That would be very
> silly (although it has already been advanced as a straw-man position
> elsewhere in this thread). The argument that we do not need private because
> nobody should put multiple classes in a file is equally silly. There are
> reasons to do so, in fact one motivation was given in SE-0025:
>
> > Putting related APIs and/or related implementations in the same file
> helps ensure consistency and reduces the time to find a particular API or
> implementation.
>
>
> These concerns are not resolved by arguments of the form “just don’t do
> that”.
>
> I empathize with the Swift2 programmer who got through two releases
> without a scoped access modifier and is annoyed by change. However,
> removing the feature now is more change, not less, so it makes their
> problem worse, not better.
>
>
> On March 21, 2017 at 2:17:40 AM, David Hart (david at hartbit.com) wrote:
>
> Perhaps it was a mistake, but I purposefully did not go into too much
> detail in the proposal because I think this debate is purely a question of
> philosophy on Swift and its language features. I did not want to add
> un-necessary bloat that would have added little rationalisation. Let me try
> to explain the holes in the proposal by answering your review:
>
> On 21 Mar 2017, at 02:26, Drew Crawford via swift-evolution <
> swift-evolution at swift.org> wrote:
>
> I disagree quite strongly with the proposal.
>
> First, the document draws conclusions without apparent supporting
> evidence, e.g.
>
> > Since the release of Swift 3, the access level change of SE–0025 was met
> with dissatisfaction by a substantial proportion of the general Swift
> community. Those changes can be viewed as actively harmful, the new
> requirement for syntax/API changes.
>
>
> - What is “dissatisfaction by a substantial proportion of the general
> Swift community”? How was this measured/determined?
>
> It’s not feasible to measure precisely the feeling of a whole community.
> But we get a feeling for it by following the mailing-list, by talking to
> colleagues, by reading twitter, etc… And it think we all agree that the
> debate is highly divisive and that a “substantial proportion” of the
> community was dissatisfied: I’m not arguing that it is less or more than a
> majority. I’m just saying that we’ve seen a lot of talk against the
> original change.
>
>
> - What was done to control for the population happy with SE-0025 who
> would e.g. not be likely to take up pitchforks?
>
> That’s why its important we have this debate now.
>
>
> - Who argues these changes are “actively harmful” and where were they
> during SE-0025?
>
> The proposal makes the argument that the changes are actively harmful.
> It’s now up to debate. By the way, even if several people (including me)
> were already against this proposal during the review, I don’t see why
> anybody would not have the right to change his mind, especially after
> several months of production usage and argue differently now.
>
> > subtly encourages overuse of scoped access control and discourages the
> more reasonable default
>
>
> - Who claims that scoped access is “overused” and what is their
> argument for doing so?
> - Why is “fileprivate” the “more reasonable default”? In fact neither
> fileprivate **nor** private are default (reasonable or not!). Internal
> is the default. Nor does this proposal suggest we change that. So this
> seems a very strange statement.
>
> By default, I did not mean the syntactic default of the language but the
> access modifier users will use “by default” when trying to restrict
> visibility. In most languages, that keyword is “private” so its valid to
> say that newcomers to the language will “default” to using that one. If the
> proposal is accepted, file-scoped private will regain that status.
>
> > But is that distinction between private and fileprivate actively used by
> the larger community of Swift developers?
>
> Yes. To cite some evidence, here are codebases I actively maintain:
>
> | codebase | private # |
> fileprivate # | ratio |
>
> |--------------------------------------------------------|--
> ---------|---------------|-------|
>
> | "M" (proprietary) | 486 | 249
> | 2x |
>
> | "N"(proprietary) | 179 | 59
> | 3x |
>
> | NaOH https://code.sealedabstract.com/drewcrawford/NaOH | 15 | 1
> | 15x |
>
> | atbuild https://github.com/AnarchyTools/atbuild | 54 | 5
> | 11x |
>
> So from my chair, not only is the distinction useful, but scoped access
> control (private) is overwhelmingly (2-15x) more useful than fileprivate.
>
> My own statistics in my projects show the contrary. At best, this shows
> how divisive this feature is. During the discussion of this proposal, it
> was argued that making decisions based upon project statistics would be
> dangerous:
>
>
> - In old code, statistics could be biased by the migrator having
> replaced all previous instances of private by fileprivate.
> - In new code, satistics could be biased by people using private
> because of it being the “soft-default”, regardless of proper semantics.
>
> > And if it were used pervasively, would it be worth the cognitive load
> and complexity of keeping two very similar access levels in the language?
> This proposal argues that answer to both questions is no
>
> This proposal does not make any later argument about “cognitive load” or
> “complexity” I can identify. Did the proposal get truncated?
>
> Sorry if I did not state it explicitly, but I see any feature/keyword
> added to the language as “additional complexity”. And that complexity is
> completely worth it when the feature adds significant expressivity. I'm
> just arguing that the additional scope-based access modifier does not
> provide enough differentiation to be worth that complexity.
>
> What is stated (without evidence) is that "it is extremely common to use
> several extensions within a file” and that use of “private” is annoying in
> that case. I now extend the above table
>
> | codebase | private # |
> fileprivate # | ratio | # of extensions (>=3 extensions in file) |
>
> |--------------------------------------------------------|--
> ---------|---------------|-------|--------------------------
> ----------------|
>
> | "M" (proprietary) | 486 | 249
> | 2x | 48 |
>
> | "N"(proprietary) | 179 | 59
> | 3x | 84 |
>
> | NaOH https://code.sealedabstract.com/drewcrawford/NaOH | 15 | 1
> | 15x | 3 |
>
> | atbuild https://github.com/AnarchyTools/atbuild | 54 | 5
> | 11x | 6 |
>
> in order to demonstrate in my corner of Swift this is not “extremely
> common”, and is actually less popular than language features the proposal
> alleges aren’t used.
>
> My point here is that ***different people in different corners of the
> community program Swift differently and use different styles***. I can
> definitely empathize with folks like the author who use extensions to group
> functions and are annoyed that their favorite visibility modifier grew four
> extra characters. Perhaps we can come up with a keyword that is more
> succint.
>
> I agree that different people in different corners use different styles.
> But you could use that argument to validate many features which would make
> a group of users happy; but all those feature together would just add bloat
> to the language. Swift has been known to be a very opinionated language, to
> keep the language simple yet expressive.
>
> However, that is no reason to take away features from working codebases.
> A scoped access modifier is perhaps my favorite feature in Swift 3. Let’s
> not throw stuff away because it adds extra characters to one programming
> style.
>
> Finally, SE-0025 establishes clear motivation for the scoped access
> modifier:
>
> > Currently, the only reliable way to hide implementation details of a
> class is to put the code in a separate file and mark it as private. This is
> not ideal for the following reasons:
>
> > It is not clear whether the implementation details are meant to be
> completely hidden or can be shared with some related code without the
> danger of misusing the APIs marked as private. If a file already has
> multiple classes, it is not clear if a particular API is meant to be hidden
> completely or can be shared with the other classes.
>
> > It forces a one class per file structure, which is very limiting.
> Putting related APIs and/or related implementations in the same file helps
> ensure consistency and reduces the time to find a particular API or
> implementation. This does not mean that the classes in the same file need
> to share otherwise hidden APIs, but there is no way to express such
> sharability with the current access levels.
>
> As far as I can see, the proposal does not actually address or acknowledge
> these problems at all, but cheerfully returns us to them. It would be a
> mistake to deprecate this feature without examining at all why we
> introduced it. And realistically we need new solutions to those problems
> before removing the existing one.
>
> Drew
>
> On March 20, 2017 at 6:54:55 PM, Douglas Gregor (dgregor at apple.com) wrote:
>
> Hello Swift community,
>
> The review of SE–0159 “Fix Private Access Levels” begins now and runs
> through March 27, 2017. The proposal is available here:
>
> https://github.com/apple/swift-evolution/blob/master/
> proposals/0159-fix-private-access-levels.md Reviews are an important part
> of the Swift evolution process. All reviews should be sent to the
> swift-evolution mailing list at
>
> https://lists.swift.org/mailman/listinfo/swift-evolution or, if you would
> like to keep your feedback private, directly to the review manager. When
> replying, please try to keep the proposal link at the top of the message:
>
> Proposal link:
>
> https://github.com/apple/swift-evolution/blob/master/
> proposals/0159-fix-private-access-levels.md Reply text Other replies What
> goes into a review?
>
> The goal of the review process is to improve the proposal under review
> through constructive criticism and, eventually, determine the direction of
> Swift. When writing your review, here are some questions you might want to
> answer in your review:
>
> What is your evaluation of the proposal? Is the problem being addressed
> significant enough to warrant a change to Swift? Does this proposal fit
> well with the feel and direction of Swift? If you have used other languages
> or libraries with a similar feature, how do you feel that this proposal
> compares to those? How much effort did you put into your review? A glance,
> a quick reading, or an in-depth study? More information about the Swift
> evolution process is available at
>
> https://github.com/apple/swift-evolution/blob/master/process.md Thank you,
>
> -Doug
>
> Review Manager
> ------------------------------
>
> swift-evolution-announce mailing list swift-evolution-announce at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution-announce
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
>
>
> -
>
>
> -
> -
>
>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170321/f54cdf5c/attachment.html>
More information about the swift-evolution
mailing list