[swift-evolution] [Proposal][Discussion] Modular Swift

Robert Widmann devteam.codafi at gmail.com
Tue Feb 21 22:59:45 CST 2017 

Then let’s discuss that.

You claim Swift has some emphasis on file-oriented programming constructs, I argue this is the case locally (fileprivate), but not the case in the language as a whole.  Looking more broadly, it is possible to restrict yourself to a particular view where one public type resides in one file, but then what to make of extensions?  What becomes of “internal” access, which permeates the entire module crossing over any file boundaries that might exist?  Today, a Swift module can depend on the kinds of files given to the compiler, but there is particular emphasis on a module not necessarily being a collection of files, but a collection of interfaces all living under the same shared namespace (the module name).  A particular facet of that interface set may be freely extended by the presence, or contracted by the absence, of a file or files when the module is built but one could very well imagine alternative abstractions that would serve this purpose just as well - e.g. an in-memory AST.

Maybe what you notice is that you are encouraged (or rather, required) by your tools to organize your code a particular way, and this is no mistake.  They exist apart from the language itself for a reason, because they themselves often depend on the state of the filesystem or the outside world and that concern spills over (correctly or not) into your organizational style.  But organizational style is just that: we can enable a broader class of programs to be designed with the system given here than with one that restricts us to a subset of it.  Because let’s not forget, it is actually a subset of the functionality presented herein.  If the proposal’s were implemented today, you would absolutely be able to design these kinds of modules and tie them to filesystem locations.  But you would also be free to group functionality and tie a set of files together into the same (sub)module.

> On Feb 21, 2017, at 10:46 PM, Xiaodi Wu <xiaodi.wu at gmail.com> wrote:
> 
> Well put. As for me, it is not the syntactic delta with which I am concerned. It is just a canary for the organizational delta.
> 
> There are certain freedoms enabled not by the _lack_ of constraints but rather their judicious application. Swift has in the past--and it is your burden to justify why it shouldn't continue to in the future--judged the file-based approach to be one such judicious constraint.
> 
> 
> On Tue, Feb 21, 2017 at 21:42 Robert Widmann <devteam.codafi at gmail.com <mailto:devteam.codafi at gmail.com>> wrote:
>> On Feb 21, 2017, at 10:36 PM, Matthew Johnson <matthew at anandabits.com <mailto:matthew at anandabits.com>> wrote:
>> 
>>> 
>>> On Feb 21, 2017, at 9:28 PM, Robert Widmann via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>> 
>>>> 
>>>> On Feb 21, 2017, at 10:03 PM, Xiaodi Wu <xiaodi.wu at gmail.com <mailto:xiaodi.wu at gmail.com>> wrote:
>>>> 
>>>> On Tue, Feb 21, 2017 at 8:41 PM, Robert Widmann <devteam.codafi at gmail.com <mailto:devteam.codafi at gmail.com>>wrote:
>>>> 
>>>>> On Feb 21, 2017, at 9:37 PM, Xiaodi Wu <xiaodi.wu at gmail.com <mailto:xiaodi.wu at gmail.com>> wrote:
>>>>> 
>>>>> On Tue, Feb 21, 2017 at 8:22 PM, Robert Widmann <devteam.codafi at gmail.com <mailto:devteam.codafi at gmail.com>>wrote:
>>>>> 
>>>>>> On Feb 21, 2017, at 9:13 PM, Xiaodi Wu <xiaodi.wu at gmail.com <mailto:xiaodi.wu at gmail.com>> wrote:
>>>>>> 
>>>>>> On Tue, Feb 21, 2017 at 7:59 PM, Robert Widmann via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>>>>> 
>>>>>>> On Feb 21, 2017, at 7:36 PM, Nevin Brackett-Rozinsky via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>>>>>> 
>>>>>>> To my mind, any submodule system for Swift should be designed to relieve the pressure for long files, and make it easy to group tightly related files into a single unit with shared visibility. That way developers can easily organize their code into smaller files while utilizing Swift’s pattern of providing protocol conformances in extensions and keeping implementation details hidden from the rest of the module at large.
>>>>>>> 
>>>>>> 
>>>>>> Wonderful, because that’s absolutely supported by this proposal.  To group tightly related files into a single unit, simply declare a submodule for them and extend it in each of your related files.
>>>>>> 
>>>>>> It's supported, but it isn't first-class. By this I mean: there are two distinguishable uses supported by your proposal, lumped together by the fact that they are both about grouping units of code together. Put crudely, one use case is grouping lines of code, while the other is about grouping files of code. The merits of supporting both have already been debated in this discussion. The issue I'll touch on is supporting both with the same syntax. The chief drawbacks here are:
>>>>>> 
>>>>> 
>>>>> What exactly would be required to make it first class?  Referencing file names in the module declaration?
>>>>> 
>>>>>  See below.
>>>>>> - It makes sense to use braces to group lines of code, but it makes no sense to use braces to group files of code; this just causes entire files to be indented.
>>>>>> 
>>>>> 
>>>>> If braces aren’t used to demarcate scopes, nesting modules becomes ambiguous.
>>>>> 
>>>>> Again, let's observe the distinction about grouping files vs. grouping lines.
>>>>> 
>>>>> Grouping files does not require braces: if the intended use of your feature were to label files X, Y, and Z as belonging to one submodule and A, B, and C to another, it would not matter if X, Y, and Z belonged to Foo.Bar and A, B, and C to Foo.Bar.Baz: your syntax would not require braces.
>>>>>  
>>>>> It’s important to note that indentation is one particular style.  LLVM code style, in particular, chooses not to indent after namespace declarations.  This issue also crops up when dealing with nested type declarations, and I distinctly remember it not being a big enough deal to "fix this" at the time when a proposal to “flatten” these declaration was brought up.
>>>>>  
>>>>> Mine is not a critique of the syntax itself; I don't particularly care about indents, nor do I mind not indenting namespaces.
>>>>> 
>>>>> What I'm saying is, you would not have chosen to require braces if your proposed feature were aimed at making the grouping of files into submodules as simple as possible. You chose to accommodate grouping lines using the same syntax as grouping files over the simplest design for grouping files. Make no mistake, this promotes one use over another.
>>>> 
>>>> Ah, I see.  Yes, one of the stated goals is to become filesystem-independent.  We certainly cannot do that by encouraging the alternative.
>>>>  
>>>> Swift's current design is deliberately not file system-independent. A submodule design built on top of Swift could preserve that. Your draft proposal makes two changes: it introduces a design for submodules; and, it eliminates files as a unit of code by default (not least by declaring `fileprivate` redundant). To my mind, you have presented no justification for the second change other than to say that it is a stated goal--but why?
>>> 
>>> 
>>> 
>>> fileprivate access can be recreated by creating a private "utility submodule" containing declarations of at most internal access.
>>> 
>>> 
>>>>> 
>>>>>> - Because some lines of code necessarily precede some other lines of code, it makes sense to declare the first group using `module` and to extend that with the second group using `extension`. However, because a file of code does not necessarily precede another file of code, it is arbitrary which file is surrounded with a `module` declaration and which one is surrounded with an `extension` declaration.
>>>>> 
>>>>> Absolutely.  But it is similarly arbitrary which public APIs are exposed in a type declaration and which are exposed in an extension declaration.
>>>>> 
>>>>> Not entirely, no. Stored properties must be in the type declaration. Enum cases must be in the type declaration. Perhaps you regard these as temporary inconveniences of the current grammar; I see them as quite reasonable ways to give some consistency as to what's written where in a language where types can be retroactively extended. In a very real sense, you must read the type declaration before you read the extensions in order to understand the latter. By comparison, there is nothing that must be in your proposed module declaration.
>>>>>  
>>>>> My hope is that the module declaration itself will become the one-stop-shop for re-exports and general public bookkeeping just as aggregate declarations are today.  Module extensions exist to accommodate users that wish to break related functionality across files or into separate independent regions within the same file for the same reasons type extensions exist.
>>>>> 
>>>>> Indeed, that you phrase it this way supports Nevin's argument. _Module extensions_ exist to accommodate his use case; however, his use case (which, mind you, is what I think most people are thinking of when it comes to submodules, given previous threads on this topic) isn't the raison d'etre for your submodule proposal. Quite simply, a syntax that accommodates both grouping lines and grouping files cannot make the latter first class, because the former necessarily requires more ceremony.
>>>>> 
>>>> 
>>>> Okay, but the question still stands: what do we need to make Nevin's use-case first class?
>>>> 
>>>> We need to have either *two* spellings for a submodule feature that supports both grouping files and grouping lines of code, or we need to have *two* features, one for grouping files and another for grouping lines of code. In any case, the spelling for grouping files should:
>>>> - Not require braces
>>>> - Not require one file to be a `module` and another to be an `extension`
>>>> 
>>>> The simplest such spelling would be:
>>>> 
>>>> File A:
>>>> ```
>>>> module Foo
>>>> // rest of file
>>>> ```
>>>> 
>>>> File B:
>>>> ```
>>>> module Foo
>>>> // rest of file
>>>> ```
>>> 
>>> This was mentioned earlier, so I will quote my response to the last poster:
>>> 
>>> That is a valid spelling (Rust, IIRC, allows that spelling), but one that is easy to miss sitting in a file and makes it confusing to introduce submodules.  If you include the annotation then define a submodule later down in the file, suddenly you have to remember whether you annotated the file or whether the submodule you’ve just written is going into the top-level module.  See:
>>> 
>>> // -module-name=Foo
>>> // module Foo {
>>> module Bar; // Shorthand for “This file defines Foo.Bar”
>>> 
>>> /* Code */
>>> 
>>> // This defines “Foo.Bar.Baz”, but would you know that if it appeared below the fold?
>>> module Baz {}
>>> //}
>>> 
>>> But that reply was oriented towards a “why can’t we have nested modules and a shorthand too?”  Here, you’re referring more to the one-module-one-file language restrictions, so I will quote another response:
>>> 
>>> For one-file-per-module, that kind of restriction represents a particular way of organizing code and is a design pattern that is supported under this proposal.  We just happen to not enforce that particular pattern, and feel that it is the job of a linter to do so.  Really, this kind of restriction is to ease the mental burden on compiler writers who use it to build compilation unit dependency graphs.  Swift already considers all files when building its modules because you can extend any type (and now, any module) from any one of them so it doesn’t buy us anything other than an arbitrary restriction.
>>> 
>>> Realistically, the only difference between the proposal’s syntax and this one is two characters (the braces) and maybe some tabs if you decide to enforce that style.  There are few objective organizational benefits from your style, and it creates a bureaucratic rule where none need exist.  Subjectively, of course, people may prefer this way, or they may prefer a more ad-hoc approach.  But we designed for both cases and showed our hand by favoring non-physical organization because it is a subset of possible organizational styles available to users.
>> 
>> Your earlier post argues that your syntax is better because with the alternative approach it’s too easy to forget whether there is a module statement at the top of the file or not.  Now you’re arguing that the difference is just two braces and that they’re relatively insignificant.  You can’t have it both ways.
> 
> Now I’m context switching… I will clarify: Syntactically the delta between the proposal and Xiaodi’s syntax is two braces.  Organizationally it means the difference between Java-style packages that divide named modules into files and approaches that allow for freedom by ignoring this constraint.
> 
>> 
>> 
>>> 
>>>> 
>>>> To my mind, we’ve offered a syntax and semantics internal to the language that supports file-only aggregation because file-only aggregation enables a subset of the actual use cases of this module system.  We aren’t enforcing this by compiler-fiat because it is a stylistic choice that can be enforced by a linter.
>>>> 
>>>> It's not enough to offer a syntax and semantics that supports it; if it is the intended major use case, the design ought to reflect that by making that use case no less cumbersome than necessary.
>>> 
>>> See above for aesthetic concerns.
>>> 
>>>> 
>>>>>> Any variables defined with `internal` access will be visible across those files to those extensions and only those extensions (see the section on access control and modules).  Any variables declared fileprivate or private will, obviously, not be visible across these files.  As an example:
>>>>>> 
>>>>>> // FooUtilities.swift
>>>>>> //
>>>>>> // -module-name=Foo
>>>>>> // module Foo {
>>>>>> // Defines Foo.Utilities
>>>>>> module Utilities {
>>>>>>   public func exportableOutsideThisSubmodule() {}
>>>>>>   func visibleInThisSubmodule() {}
>>>>>>   private func invisibleToOtherFiles() {}
>>>>>> }
>>>>>> //}
>>>>>> 
>>>>>> // FooUtilities+MoreUtilities.swift
>>>>>> extension Utilities {
>>>>>>   private func privateHelper() {
>>>>>>     visibleInThisSubmodule()
>>>>>>   }
>>>>>> }
>>>>>> 
>>>>>> I’m not sure where you got the impression that we were just trying to make another fileprivate happen.
>>>>>> 
>>>>>>> 
>>>>>>> Nevin
>>>>>>> _______________________________________________
>>>>>>> swift-evolution mailing list
>>>>>>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>>>>>>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> swift-evolution mailing list
>>>>>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>>>>>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
>>> _______________________________________________
>>> swift-evolution mailing list
>>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170221/7869661c/attachment-0001.html>

More information about the swift-evolution mailing list