[swift-evolution] [Pitch] consistent public access modifiers

[Matthew Johnson]

> On Feb 13, 2017, at 8:24 AM, Brent Royal-Gordon <brent at architechies.com> wrote:
> 
> Sorry, I meant to jump in a lot earlier than this, but lost track of this thread on my mental to-do list. I've read most of the thread, but I really can't keep all the replies in my head at once, so I apologize if some of this is duplicative.

Hi Brent, no problem!  Thanks for taking time to read it and offer feedback.

> 
> I agree with Jordan Rose that "closed enums" and "closed protocols" are separate things and they should be discussed separately, so I'll be doing that here. But first, a criticism of both:

I agree that they are separate things.  But there is also important semantic overlap.  One of the major motivations behind my proposal is that I think we’re ignoring this semantic overlap and therefore being sloppy with our terminology.  I think it would be wise to consider carefully whether it is a good idea to continue ignoring the overlap.  I think one can make an argument that it ignoring it is a pragmatic choice, but one can also make an argument that it is hand-wavy.  

If you look closely, when most people say “closed enum” they mean a fixed, complete set of cases that are all public.  But when people say “closed protocol” they don’t actually mean a fixed, complete set of conformances that are all public.  They simply mean clients cannot add conformances.  This is the semantic contract of resilient enums, not closed enums.

> 
>> On Feb 8, 2017, at 3:05 PM, Matthew Johnson via swift-evolution <swift-evolution at swift.org> wrote:
>> 
>> This proposal introduces the new access modifier `closed` as well as clarifying the meaning of `public` and expanding the use of `open`.
> 
> If the `closed` keyword is to stand alone as an access level, I think `closed` is a bad choice. `open` is acceptable because it sounds as visible or even *more* visible than `public`. (AppKit is "public", but GTK is "open". Which exposes more of itself to a programmer?) But `closed` sounds like some form of privacy. I think that, unless it's paired with a word like `public`, it will not be understood correctly.

This certainly is a fair criticism.  `closed` is the term that has been heavily used by the community and is an inverse of `open`, which makes sense because it is in many respects a semantic inverse.  That said, I would embrace a healthy round of bike shedding to try and find a better keyword.

> 
> Closed enums:
> 
>> A recent thread (https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20170206/031566.html) discussed a similar tradeoff regarding whether public enums should commit to a fixed set of cases by default or not.  The current behavior is that they *do* commit to a fixed set of cases and there is no option (afaik) to modify that behavior.  The Library Evolution document (https://github.com/apple/swift/blob/master/docs/LibraryEvolution.rst#enums) suggests a desire to change this before locking down ABI such that public enums *do not* make this commitment by default, and are required to opt-in to this behavior using an `@closed` annotation.
>> 
>> In the previous discussion I stated a strong preference that closed enums *not* be penalized with an additional annotation.  This is because I feel pretty strongly that it is a design smell to: 1) expose cases publicly if consumers of the API are not expected to switch on them and 2) require users to handle unknown future cases if they are likely to switch over the cases in correct use of the API.
> 
> The thing is, you do *lots* of things with enums other than exhaustively switching on them. One extremely common example is `Error` enums. But even leaving that aside, there are lots of cases where you construct an enum and hand it off to a library to work or interpret it; adding a case won't break these.
> 
> Basically, when an enum is an input to a library, adding cases is safe. When it's an output from a library, adding cases is potentially unsafe, unless the code using the enum is designed to permit additional cases. 

Yes, I agree with this.  I was overreaching a bit in that paragraph.  My point is that you can achieve the same client syntax for library inputs using designs that don’t expose cases publicly (i.e. discourage users from writing a switch).  Maybe, at least in some cases, that is what a library should do.  Saying it *is* a design smell was an exaggeration, but saying it is a contract worthy of close consideration is accurate.

> 
>> The conclusion I came to in that thread is that we should adopt the same strategy as we did with classes: there should not be a default.
> 
> But classes *do* have a default: closed, i.e., `public`. It is an extremely soft default, and we've made it as easy as possible to switch to `open`, but `public` is absolutely the default—both because `public` is the keyword people will know from other languages, and because `public` is the term used with non-class-related symbols.

Ok, I can accept this.  With that in mind, I am arguing for the same “soft default” for enums and protocols.  This means the same semantics and it also means that adopting a different contract is not penalized syntactically and that an “error of omission" mistake cannot be made.

> 
> And remember, it's the default for a good reason: `public`'s semantic is the more forgiving one. If you make a class `public` when it should be `open`, fixing it is not a breaking change, but the opposite is.

Yes, of course.  This was the right decision.

> 
> In the case of enums, public-but-nonexhaustive is the more forgiving semantic. If you make something public-but-nonexhaustive when it should be public-but-exhaustive, fixing it is not a breaking change, but the opposite is.

Agree.

> 
> As I mentioned earlier, I don't think `closed` is a good keyword standing alone. And I also think that, given that we have `open`, `closed` also won't pair well with `public`—they sound like antonyms when they aren’t.

The semantics I am proposing do have an inverse relationship.  That said, it may not be an intuitive or immediately obvious inverse.  I am certainly not wedded to the idea of using `closed` as the keyword.

> 
> What I instead suggest is that we think of a closed enum as being like a fragile (non-resilient) struct. In both cases, you are committing to a particular design for the type. So I think we should give them both the same keyword—something like:
> 
> 	@fixed struct Person {
> 		var name: String
> 		var birthDate: Date
> 	}
> 	@fixed enum Edge {
> 		case start
> 		case end
> 	}
> 

You omitted public here.  Does that mean you intend for `@fixed` to imply public visibility?  If so, I could get behind this.  But I am curious why you made it an attribute rather than a keyword.

> As I mentioned in another post, inheriting from an enum is not really a sensible thing to do. It is perhaps possible that we could eventually introduce `open enum`s, which would permit you to add cases in outside extensions. (That might be useful for error enums, but I honestly can't think of many other use cases.) But enums would need to gain new features—like member overrides attached to cases—to make that useful. All in all, I'm not entirely convinced about open enums.

I agree.  `open` enums are a possibly interesting idea, but not necessarily a useful one.  That’s why included a note that specifically excluded them from the pitch.

> 
> Closed protocols:
> 
>> There have also been several discussions both on the list and via Twitter regarding whether or not we should allow closed protocols.  In a recent Twitter discussion Joe Groff suggested that we don’t need them because we should use an enum when there is a fixed set of conforming types.  There are at least two  reasons why I still think we *should* add support for closed protocols.
>> 
>> As noted above (and in the previous thread in more detail), if the set of types (cases) isn’t intended to be fixed (i.e. the library may add new types in the future) an enum is likely not a good choice.  Using a closed protocol discourages the user from switching and prevents the user from adding conformances that are not desired.
>> 
>> Another use case supported by closed protocols is a design where users are not allowed to conform directly to a protocol, but instead are required to conform to one of several protocols which refine the closed protocol.  Enums are not a substitute for this use case.  The only option is to resort to documentation and runtime checks.
> 
> I'm definitely on board with having closed protocols. I've seen several use cases where they'd be helpful.
> 
> I don't see it mentioned here (maybe I just missed it), but even though we *could* do exhaustiveness checking on non-open protocols, I'm not convinced that's a good idea. Usually when you have several types conforming to a protocol, you should access type-specific behavior through polymorphism, not by switching on the protocol. A protocol is supposed to represent a behavior, not just mark a type in some arbitrary way.

I agree that you should usually be adding polymorphism, but preventing exhaustive switch on what is effectively a style argument seems like an unnecessary restriction to me.  There will be times when it could be used to good effect.  I think the community has done a pretty good job of figuring out how to use Swift’s many features well and don’t believe it would be frequently abused.

> 
>> Finally, a protocol that refines a `closed` protocol need not be `closed`.  It may also be `open`.
> 
> I support this, and I'd like to demonstrate a use case for it. (Here I will use `open` for the current semantics of `public`, and `public` for visible-but-not-conformable.)
> 
> Suppose that you're writing a SQLite wrapper and want to support binding parameters. There are a number of types SQLite supports natively:
> 
> 	public protocol SQLiteValue {
> 		init(statement: SQLiteStatement, columnAt index: Int) throws
> 		func bind(to statement: SQLiteStatement, at index: Int) throws
> 	}
> 	extension Int: SQLiteValue {
> 		public init(statement: SQLiteStatement, columnAt index: Int) throws {
> 			self = sqlite3_column_int(statement.stmt, index)
> 		}
> 		public func bind(to statement: SQLiteStatement, at index: Int) throws {
> 			try throwIfNotOK(
> 				sqlite3_bind_int64(statement.stmt, index, self)
> 			)
> 		}
> 	}
> 	extension Double: SQLiteValue {…}
> 	extension Data: SQLiteValue {…}
> 	extension String: SQLiteValue {…}
> 	extension Optional: SQLiteValue where Wrapped: SQLiteValue {…}
> 
> But you also want people to be able to conform their own types to a protocol which adapts itself to this:
> 
> 	open protocol SQLiteValueConvertible: SQLiteValue {
> 		associatedtype PrimitiveSQLiteValue: SQLiteValue
> 		
> 		init(primitiveSQLiteValue: PrimitiveSQLiteValue) throws
> 		var primitiveSQLiteValue: PrimitiveSQLiteValue { get }
> 	}
> 	extension SQLiteValueConvertible {
> 		public init(statement: SQLiteStatement, columnAt index: Int) throws {
> 			let primitive = try PrimitiveSQLiteValue(statement: statement, columnAt: index)
> 			try self.init(primitiveSQLiteValue: primitive)
> 		}
> 		public func bind(to statement: SQLiteStatement, at index: Int) throws {
> 			let value = primitiveSQLiteValue
> 			try value.bind(to: statement, at: index)
> 		}
> 	}
> 	
> 	// Usage:
> 	extension Bool: SQLiteValueConvertible {
> 		public init(primitiveSQLiteValue: Int) {
> 			self = primitiveSQLiteValue != 0
> 		}
> 		public var primitiveSQLiteValue: Int {
> 			return self ? 1 : 0
> 		}
> 	}
> 
> In this case, there is no good reason to make `SQLiteValue` open—all the functions you'd need are encapsulated anyway. You could even make its requirements non-public, since users don't need to use them directly. But `SQLiteValueConvertible` should be open—it has a narrow interface that's easy to conform to, with all the implementation details neatly encapsulated.
> 
> I also think you should be able to refine a `closed` protocol from outside the module, but to conform to the sub-protocol, you'd either need to conform to an `open` sub-protocol of the original, or add a retroactive conformance to a type that already conforms.

Yes, I agree with this.  I didn’t call out protocol refinement in the initial pitch but it came up earlier in the discussion.  Refining closed protocols has no semantic impact on the library itself, including it’s ability to evolve resiliently so there is no reason to prevent users from doing it.

> For instance, suppose I want to write a SQLite logging system in a separate module:
> 
> 	protocol SQLiteLoggable: SQLiteValue {
> 		var logDescription: String { get }
> 	}
> 	
> 	struct Money: SQLiteLoggable, SQLiteValueConvertible {
> 		// OK, conforms to SQLiteValue through SQLiteValueConvertible
> 	}
> 	
> 	extension Int: SQLiteLoggable {
> 		// OK, retroactive conformance on type original module conformed to SQLiteValue
> 	}
> 	
> 	struct ID: SQLiteLoggable {
> 		// Error: Cannot conform directly to SQLValue from this module
> 	}
> 
> This is a somewhat more niche feature, and could perhaps be delayed.
> 
>> This proposal affects both public enums and public protocols.  The current behavior of enums is equivalent to a `closed` enum under this proposal and the current behavior of protocols is equivalent to an `open` protocol under this proposal.  Both changes allow for a simple mechanical migration, but that may not be sufficient given the source compatibility promise made for Swift 4.  We may need to identify a multi-release strategy for adopting this proposal.
>> 
>> Brent Royal-Gordon suggested such a strategy in a discussion regarding closed protocols on Twitter:
>> 
>> * In Swift 4: all unannotated public protocols receive a warning, possibly with a fix-it to change the annotation to `open`.
>> * Also in Swift 4: an annotation is introduced to opt-in to the new `public` behavior.  Brent suggested `@closed`, but as this proposal distinguishes `public` and `closed` we would need to identify something else.  I will use `@annotation` as a placeholder.
>> * Also In Swift 4: the `closed` modifier is introduced.
>> 
>> * In Swift 5 the warning becomes a compiler error.  `public protocol` is not allowed.  Users must use `@annotation public protocol`.
>> * In Swift 6 `public protocol` is allowed again, now with the new semantics.  `@annotation public protocol` is also allowed, now with a warning and a fix-it to remove the warning.
>> * In Swift 7 `@annotation public protocol` is no longer allowed.
> 
> I still support this general approach. One spelling could simply be `@nonopen`. Although if we don't use `closed`, we could simply use `@closed` like I suggested—here it really *would* be an antonym to `open`.

I like the idea of using `@nonopen` for the transitional attribute.  Both because it “removes the openness” that `public protocol` currently implies.  In that sense it is probably the most accurate term we could find and it’s also pretty concise.

> 
>> A similar mult-release strategy would work for migrating public enums.
> 
> What is it that needs migrating here? Lack of exhaustiveness checking? It sounds like we were planning to break that anyway in some fashion.

Public enums are not currently resilient.  Clients are allowed to switch over them without a `default` clause.  This means that client code will fail to compile in a version of Swift where `public enum` has the resilient contract unless the library changes to adopt closed semantics or the client adds a default case.

It sounds like we’re mostly on the same page here, but you have identified an alternative name for what I have called `closed` that might be a better one: `fixed`.  It would require taking a little bit different approach to defining the semantics than I have taken with `closed`, but I think it could work in a way that still solves the problem of inconsistency that I have pointed out.

This exactly a reply to you Brent, but I wanted to add to this thread the step-by-step line of reasoning which led to my current pitch in case it might be helpful.

1. We should acknowledge that there is a meaningful semantic overlap between enum cases, subclasses, and protocol conformances.
2. Having acknowledged the overlap, we should strive to keep the language consistent syntactically and semantically where applicable.
3. When we introduced `open` we expanded the scope of access modifiers such that they talk about not just visibility, but also who can subclass a class.
4. Taking 1-3 together, it follows that speaking about who can add cases to an enum or conformance to a protocol is also in scope for access modifiers.
5. `public` currently has a different semantic contract in regards to who can add cases, subclass or conformance (i.e. the relevant semantic overlap).
6. Because we have `open` as a keyword that means *clients* can add subclasses, we should fix the current inconsistency with protocols by making them use the same keywords as classes (after a multi-release transition).
7.  We make `public enum` be the resilient variety which would match the semantics of `public class` and `public protocol` (post-6).
8. We need a new way to spell the current behavior we get with `public enum` (i.e. closed).
9. In the `open` discussion we concluded that we don't want a library author to accidentally publish semantics they didn't intend through an "error of omission" (i.e. forgetting an annotation).  It would also be good to avoid errors of omission in regards to publishing closed and resilient enums.
10. We also didn't want to subtly favor `open` or `public` by making one syntactically lighter weight than the other.  It would also be good to avoid syntactic preference of either closed or resilient enums.
11. We want to avoid using annotations when an as-good (or better) solution is available that does not require them.
12. From the above it follows that the logical thing to do is to introduce an access modifier that specifies that the complete set of cases is public and won't change (module breaking changes).  This modifier would speak about cases in a way that is effectively an inverse of what `open` means for classes.
13. Because of the inverse relationship with `open` and because it is a term already in common use, `closed` is the most obvious keyword to use, but bikeshedding is expected, as always.
14. Under this system, `open enum`, `closed class` and `closed protocol` all have a well defined meaning.  We won't add them right away, but we could do so in the future if we find interesting use cases.

> 
> -- 
> Brent Royal-Gordon
> Architechies
>