[swift-evolution] Yet Another Take on Swift Sub-modules
Karim Nassar
karim at karimnassar.com
Sat Mar 4 10:55:11 CST 2017
Here is a new gist with updates based on all the great feedback I’ve received (accidentally created the first one anonymously):
https://gist.github.com/knassar/5c9933207663dad7eeb29ff7c526af62
And inline:
# Sub-modules
A sub-module solution in Swift should have the following properties:
* Extremely light-weight
* Low API surface area
* Adopt progressive disclosure
* Integrate with Access Control features to enable a level of encapsulation & hiding between the Module and File level
* Be permeable when desired
## Discussion
As we get deeper into building real applications & frameworks with Swift, we begin to realize that having a way to express relationships between types is desireable. Currently, Swift only allows us to express these relationships at two levels, the Module and the File.
The Module boundary is acceptable for small, focused frameworks, while the File boundary is acceptable for small, focused Types, but both levels can be unweildy when dealing with certain cases where a cluster of internally related types needs to know about each other but may only want to publish a narrow set of APIs to the surrounding code, or in large complex applications which are necessarily structured as a single Module. In these cases, we wind up with large monolithic Modules or (even worse) large monolithic Files.
I have seen this proliferation of Huge Sprawling Files (HSFs) in my own code, and seek a way to combat this rising tide.
## Goals
It is a goal of this proposal to:
* Suggest a mechanism for organizing code between the Module and File levels that is as lightweight and low-friction as possible
* Provide mechanisms for authors to create both "hard" and "soft" API boundaries between the Module and File levels of their code
## Anti-Goals
It is not a goal of this proposal to:
* Move Swift away from filesystem-based organization
* Significantly alter the current Access Control philosophy of Swift
## Proposal Notes
Please take the following proposal wholely as a Straw-Man... I would be equally satisfied with any solution which meets the critera described at the top of this document.
Unless specified otherwise, all spellings proposed below are to be considered straw-men, and merely illustrative of the concepts.
# Proposed Solution
Two things are clear to me after using Swift and following the Swift Evolution list since their respective publications:
1. Swift has a preference for file-based organization
2. Vocal Swift Users dislike `fileprivate` and want to revert to Swift2-style `private`
Because of #1, this proposal does not seek to change Swift's inherent file-system organization, and instead will expand on it.
Since I personally fall into the camp described by #2, and most of the community response to this has been "Lets wait to deal with that until sub-modules", I'm making this proposal assuming that solving that quagmire is in-scope for this propsoal.
## Changes to Access Control Modifiers
As part of this proposal (or as a pre-requisite assumption), I suggest the following changes to Swift 3's Access Control modifiers:
* Revert `private` to Swift 2's meaning: "hidden outside the file"
* Remove `fileprivate` as redundant
This is potentially a source-breaking change. However, it is interesting to note that this change is **not** required for the following proposal to function.
Changes that *are* necessary are:
* Change the spelling of `internal` to `module` (making `module` the new default)
* Introduce a new modifier `internal` to mean "Internal to the current sub-module (or if at the 'top' level of a module, Internal to the top-level, ie: hidden from it's sub-modules)"
These changes are *not* source-breaking because the new `internal` modifier acts exactly as the old `internal` modifier when used without sub-modules. The specific spelling of this new `internal` modifier is necessary to maintain backwards source compatibility.
The new `module` modifier allows authors to make APIs permeable between sub-modules while still hidden outside the owning Module if desired.
All other Access Control modifiers behave the same as they currently do irrespective of sub-module boundaries, so:
* `public` => Visible outside the Module
* `open` => Sub-classable outside the Module
## Making a Sub-module
There are various alternative mechanisms that might be chosen for declaring a sub-module. I describe 3 of them here, each with some observations, pros, and cons. The intent of this proposal is to select **one** mechanism that best balances the proposal goals and the mechanism's compromises. Other alternatives are welome.
### Alternative 1: Using the Filesystem
To create a sub-module within a Module (or sub-module) is simple: The author creates a directory, and places a "sub-module declaration file" within the directory:
```
// __submodule.swift
// MyModule
submodule SubA
```
Then any files within that directory are part of the sub-module:
```
// Foo.swift
// MyModule.SubA
struct Foo {
private var mine: Bool
internal var sub: Bool
module var mod: Bool
}
public struct Bar {
module var mod: Bool
public var pub: Bool
}
```
This creates a sub-module called "SubA" within the module "MyModule". All files within the directory in which this file appears are understood to be contained by this sub-module.
Nested sub-modules are created by the same structure in nested directories
```
// __submodule.swift
// MyModule.SubA
submodule SubB
```
```
// Bast.swift
// MyModule.SubA.SubB
module struct Bast {
}
```
If in the future we choose to add additional complexity (versioning, #availability, etc) to the sub-module syntax, the sub-module declaration gives a natural home for this configuration:
```
submodule SubA {
// configuration goes here
}
```
An alternative to the proposed `__submodule.swift` might be to introduce a new extension `.swiftsubmodule` or somesuch. This choice may have additional benefits or drawbacks.
#### Benefits
* Any given source file may only be a member of 1 submodule at a time
* Use of filesystem structure to denote sub-modules plays nicely with source control
* The sub-module structure is instantly clear whether using an IDE (which can be taught to parse the `__submodule.swift` files to decorate the UI), or simple text-editor (assuming a convention of naming the Directory the same as the sub-module, which is a linter problem)
#### Drawbacks
* Code-meaning becomes dependent on the code file's position within the filesystem
* Requires special compiler rules to parse & validate the new declaration file
* `submodule` may only appear in a sub-module declaration file
* `submodule` may only appear once in a file
#### Observations
* Using the "special file" means that not all Directories are automatically submodules
* The naming of this file should attempt to prevent accidental collision with source files the author might choose
* If adopted, it is my hope that IDE's (Xcode) would provide first-class support for this mechanism, so that authors would not have to manually manage file-system representation of sub-module memberships
### Alternative 2: In Situ Declaration & Adoption
A sub-module is declared within the module in any source file:
```
submodule SubA
```
Files which are to be members of the sub-module declare their membership with a new declaration `in_submodule` which is only allowed to appear **once** in any source file to prevent the complexity of multi-sub-module memberships.
```
in_submodule SubA
struct Foo {
private var mine: Bool
internal var sub: Bool
module var mod: Bool
}
public struct Bar {
module var mod: Bool
public var pub: Bool
}
```
Nested sub-modules are declared & adopted with dot-syntax
```
submodule SubA.SubB
in_submodule SubA.SubB
module struct Bast {
}
```
#### Benefits
* Pure in-code sub-module definition, with no dependencies on filesystem location
#### Drawbacks
* Increases the surface area— we need to introduce a new declaration for announcing sub-module inclusion
* Requires special compiler rules to parse & validate the new adoption declaration
* `in_submodule` may only appear once in a file
#### Observations
* Future iteration might choose to relax the "one sub-module membership only" rule (albeit at significant authoring complexity)
### Alternative 3: Scope-like Sub-modules
In this alternative, declaration looks similar to Alternative 2, but its semantic meaning has changed:
The sub-module declaration represents a symbolic entity, following the established rules for type declarations:
```
submodule SubA {
struct Foo { }
}
```
Memberships are expressed either by declaring types within the submodule declaration itself, or by leveraging the `extension` syntax:
```
extension SubA {
public struct Bar { }
submodule SubB {
module struct Bast { }
}
}
```
#### Benefits
* Extends an existing pattern in a somewhat reasonable-seeming way
* easily discoverable
#### Drawbacks
* Will require changes to how top-level declarations are parsed & evaluated (i.e.: Protocols, global statements, operator funcs, etc)
* Visually uglier due to indentation
#### Observations
* Though the pattern *seems* reasonable, it’s actually an odd-duck; a sub-module does not behave the same as a type in many important ways, and the use of the Type-like syntax mechanisms may lead (esp. novice) authors to expect behaviors which cannot be met (ie: "Why can't sub-modules have properties?")
* The mental model required for this approach is **much** more complicated, as submodule declarations can now be mixed within a file, and the (file)private Access Controls will span sub-modules
## Using Sub-modules
Referencing a sub-module should be natural and clear:
### From Within the Parent Module/Sub-module
Sub-modules are simply code-organization & namespacing tools within modules. As such, when referenced from within their parent Module, there is no need for `import`s
```
// in MyModule
let foo = SubA.Foo()
foo.mine = true // Compiler error because it's private
foo.sub = true // Compiler error because it's internal to the sub-module
foo.mod = true // OK
```
### From Outside the Parent Module/Sub-module
When referenced from outside their parent Module, one imports the whole module in the standard way:
```
import MyModule
let foo = SubA.Foo() // Compiler error because it's internal to the Module
let bar = SubA.Bar() // OK
bar.mod = true // Compiler error because it's internal to the Module
bar.pub = true // OK
```
## What this Proposal Deliberately Omits
This proposal deliberately omits several concepts which may be integral to various use-cases for sub-modules, primarily because they can be treated as purely additive concepts and I don't wish to weigh down the consideration of the overall approach with a larger API surface area that might be debated separately. I.e: Keep it as small as possible for now, then if it's any good, iterate on the design.
### Inter-Sub-Module Access Control
One might ask given a sub-module structure like:
```
MyModule
|
+--- SubA
|
+--- SubB
```
"How can SubB hide properties from MyModule without hiding them from SubA?"
This is a valid question, and not answered by this proposal for two reasons:
* This trivial case could be solved by simply adding a new modifier `submodule` if we so desired, but:
* In the absence of any direct response, the status-quo provides a work-around: Omit the sub-sub-module structure and use the file-access constraints of `private`
* This overall problem probably should be solved by addressing larger questions in the Access Control scheme of Swift, irrespective of the sub-module mechanism
### Expressiveness of Sub-module Imports
One might ask: "Why can't I import only a specific sub-module or alias a sub-module?"
I have ignored this aspect of submodules because the question of `import` expressiveness is a separate issue in my mind. The fact that we cannot say:
```
import MyModule as Foo
```
Has no relationship to the lack of sub-modules in Swift.
If the community deems it an important enough use-case to warrant altering import behavior, so be it, but that can be treated as purely additive to this proposal.
But it should be understood that this approach to sub-modules is not designed to provide an expressive "exports" capability. It is primarily interested in organizing code *within* a Module
More information about the swift-evolution
mailing list