[swift-evolution] Yet Another Take on Swift Sub-modules

[David Hart]

I strongly agree with all your points. The other submodule proposals are too complex for me so I'm very happy to see a different solution which more closely fits my design priorities.

Comments inline:

> On 3 Mar 2017, at 16:24, Karim Nassar via swift-evolution <swift-evolution at swift.org> wrote:
> 
> 
> I’ve read through the last couple of Swift (sub)Module proposals put forward, and since my particular use-cases for a sub-module solution seemed to be under-served by them, I’ve decided to write up my thoughts on the matter to prompt discussion. 
> 
> Perhaps my use-cases are outliers, and my approach will be deemed naive by the community… I’m happy to learn better ways of doing things in Swift, and welcome any thoughts, criticism, or illumination related to these ideas.

I don't think your use cases are outliers. I think your solution really feels more Swifty by starting simple.

> I’m including the write-up below, but it’s also available as a gist: https://gist.github.com/anonymous/9806f4274f1e13860670d6e059be5dce
> 
> —
> 
> # Sub-modules
> 
> A sub-module solution in Swift should have the following properties:
> 
> * Extremely light-weight
> * Low API surface area
> * Adopt progressive disclosure
> * Integrate with Access Control features to enable a level of encapsulation & hiding between the Module and File level
> * Be permeable when desired

Yes, yes, yes. Very good goals :)

> ## Discussion
> 
> As we get deeper into building real applications & frameworks with Swift, we begin to realize that having a way to express relationships between types is desireable.  Currently, Swift only allows us to express these relationships at two levels, the Module and the File. 
> 
> The Module boundary is acceptable for small, focused frameworks, while the File boundary is acceptable for small, focused Types, but both levels can be unweildy when dealing with certain cases where a cluster of internally related types needs to know about each other but may only want to publish a narrow set of APIs to the surrounding code, or in large complex applications which are necessarily structured as a single Module. In these cases, we wind up with large monolithic Modules or (even worse) large monolithic Files.
> 
> I have seen this proliferation of Huge Sprawling Files (HSFs) in my own code, and seek a way to combat this rising tide.
> 
> ## Goals 
> 
> It is a goal of this proposal to:
> 
> * Suggest a mechanism for organizing code between the Module and File levels that is as lightweight and low-friction as possible
> * Provide mechanisms for authors to create both "hard" and "soft" API boundaries between the Module and File levels of their code
> 
> ## Anti-Goals
> 
> It is not a goal of this proposal to:
> 
> * Move Swift away from filesystem-based organization
> * Significantly alter the current Access Control philosophy of Swift
> 
> ## Proposal Notes
> 
> Please take the following proposal wholely as a Straw-Man... I would be equally satisfied with any solution which meets the critera described at the top of this document.
> 
> Unless specified otherwise, all spellings proposed below are to be considered straw-men, and merely illustrative of the concepts.
> 
> ## Proposed Solution
> 
> Two things are clear to me after using Swift and following the Swift Evolution list since their respective publications:
> 
> 1. Swift has a preference for file-based organization
> 2. Vocal Swift Users dislike `fileprivate` and want to revert to Swift2-style `private`
> 
> Because of #1, this proposal does not seek to change Swift's inherent file-system organization, and instead will expand on it.
> 
> Since I personally fall into the camp described by #2, and most of the community response to this has been "Lets wait to deal with that until sub-modules", I'm making this proposal assuming that solving that quagmire is in-scope for this propsoal.

I have posted a proposal which proposes #2. It should hopefully get merged and reviewed soon.

> ### Changes to Access Control Modifiers
> 
> As part of this proposal, I suggest the following changes to Swift 3's Access Control modifiers:
> 
> * Revert `private` to Swift 2's meaning: "hidden outside the file"
> * Remove `fileprivate` as redundant
> 
> This is potentially a source-breaking change. However, it is interesting to note that this change is **not** required for the following proposal to function.
> 
> Changes that *are* necessary are:
> 
> * Change the spelling of `internal` to `module` (making `module` the new default)
> * Introduce a new modifier `internal` to mean "Internal to the current sub-module and its child-sub-modules"

I like that you introduce only one access level and maintain source compatibility. This is also what makes the proposal fit the "progressive disclosure" goal: you don't need to learn or worry about the `module` access level until you start working with submodules.

> These changes are *not* source-breaking because the new `internal` modifier acts exactly as the old `internal` modifier unless it is used within a sub-module. The specific spelling of this new `internal` modifier is necessary to maintain backwards source compatibility.
> 
> The new `module` modifier allows authors to make APIs permeable between sub-modules while still hidden outside the owning Module if desired.
> 
> All other Access Control modifiers behave the same as they currently do irrespective of sub-module boundaries, so:
> 
> * `public` => Visible outside the Module
> * `open` => Sub-classable outside the Module
> 
> ### Making a Sub-module
> 
> To create a sub-module within a Module (or sub-module) is simple: The author creates a directory, and places a "sub-module declaration file" within the directory:
> 
> ```
> //  __submodule.swift
> //  MyModule
> 
> submodule SubA
> 
> ```

Like the previous response, I think this potentially the weakest part of he proposal. I like the idea of a file-system based submodule system but I'm not sure I like this version of it. People are going to be very vocal about this too. Perhaps you could rewrite the proposal with several solutions for this part? Let people discuss the different solutions.

> Then any files within that directory are part of the sub-module:
> 
> ```
> //  Foo.swift
> //  MyModule.SubA
> 
> struct Foo {
>    private var mine: Bool
>    internal var sub: Bool
>    module var mod: Bool
> }
> 
> public struct Bar {
>    module var mod: Bool
>    public var pub: Bool
> }
> 
> ```
> 
> This creates a sub-module called "SubA" within the module "MyModule". All files within the directory in which this file appears are understood to be contained by this sub-module.
> 
> If in the future we choose to add additional complexity (versioning, #availability, etc) to the sub-module syntax, the sub-module declaration gives a natural home for this configuration.
> 
> It's important to note some benefits of this approach:
> 
> * Using the "special file" means that not all Directories are automatically submodules
> * Any given source file may only be a member of 1 submodule at a time
> * Use of filesystem structure to denote sub-modules plays nicely with source control
> * The sub-module structure is instantly clear whether using an IDE (which can be taught to parse the `__submodule.swift` files to decorate the UI), or simple text-editor (assuming a convention of naming the Directory the same as the sub-module, which is a linter problem)
> 
> ### Using Sub-modules
> 
> Referencing a sub-module should be natural and clear at this point:
> 
> #### From Within the Parent Module/Sub-module
> 
> Sub-modules are simply code-organization & namespacing tools within modules. As such, when referenced from within their parent Module, there is no need for `import`s
> 
> ```
> //  in MyModule
> 
> let foo = SubA.Foo()
> foo.mine = true // Compiler error because it's private
> foo.sub = true  // Compiler error because it's internal to the sub-module
> foo.mod = true  // OK
> 
> ```
> 
> #### From Outside the Parent Module/Sub-module
> 
> When referenced from outside their parent Module, one imports the whole module in the standard way:
> 
> ```
> import MyModule
> 
> let foo = SubA.Foo() // Compiler error because it's internal to the Module
> 
> let bar = SubA.Bar() // OK
> bar.mod = true  // Compiler error because it's internal to the Module
> bar.pub = true  // OK
> 
> ```
> 
> ## What this Proposal Deliberately Omits
> 
> This proposal deliberately omits several concepts which may be integral to various use-cases for sub-modules, primarily because they can be treated as purely additive concepts and I don't wish to weigh down the consideration of the overall approach with a larger API surface area that might be debated separately. I.e: Keep it as small as possible for now, then if it's any good, iterate on the design.
> 
> ### Inter-Sub-Module Access Control
> 
> One might ask given a sub-module structure like:
> 
> ```
> MyModule
>  |
>  +--- SubA
>        |
>        +--- SubB
> 
> ```
> 
> "How can SubB hide properties from MyModule without hiding them from SubA?"
> 
> This is a valid question, and not answered by this proposal for two reasons:
> 
> * This trivial case could be solved by simply adding a new modifier `submodule` if we so desired, but:
> * In the absence of any direct response, the status-quo provides a work-around: Omit the sub-sub-module structure and use the file-access constraints of `private`
> * This overall problem probably should be solved by addressing larger questions in the Access Control scheme of Swift, irrespective of the sub-module mechanism
> 
> ### Expressiveness of Sub-module Imports
> 
> One might ask: "Why can't I import only a specific sub-module or alias a sub-module?"
> 
> I have ignored this aspect of submodules because the question of `import` expressiveness is a separate issue in my mind. The fact that we cannot say:
> 
> ```
> import MyModule as Foo
> ```
> 
> Has no relationship to the lack of sub-modules in Swift. 
> 
> If the community deems it an important enough use-case to warrant altering import behavior, so be it, but that can be treated as purely additive to this proposal.
> 
> But it should be understood that this approach to sub-modules is not designed to provide an expressive "exports" capability. It is primarily interested in organizing code *within* a Module
> 

I think it's good that the proposal does not try to find solutions to these problems. We ca. More easy discuss solutions when we have been using ah modules in production for a while.

> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution