[swift-evolution] Yet Another Take on Swift Sub-modules

[Rien]

> On 06 Mar 2017, at 15:10, Karim Nassar <karim at karimnassar.com> wrote:
> 
>> 
>> On Mar 6, 2017, at 3:04 AM, Rien <Rien at Balancingrock.nl> wrote:
>> 
>>>>> 
>>>>> I’m not dead-set on this approach, but as you say, it solves a *lot* of problems that other approaches introduce. I do recognize the reasonableness of the main argument against, that file location shouldn’t have such a dramatic affect on behavior… *but* the fact is (whether by convention or expediency) we already *do* have filesystem location dependencies on our code projects… 
>>>>> 
>>>>> * One can’t willy-nilly move files around the disk and expect a project to build… if you move it somewhere the compiler doesn’t know to look for it, you’re going to break things
>>>>> * One can’t just move a file out of your SCM repo root, or you’ll “lose” the file
>>>> 
>>>> True, but if other files do not refer to the lost file, you don’t even know for which file to look.
>>>> (imo this is already a weak point in swift, but this approach to submodules would make it -much?- worse)
>>>> 
>>>> If you were to include filenames in the “submodule declaration file” at least this omission in swift would be partly covered, and you would not need to move the other files into special places. (You could though)
>>>> 
>>> 
>>> But isn’t this exactly the same case if you’re using Swift without an IDE? It’s true that Xcode keeps track of files in your project, and there’s no reason it couldn’t also keep track of files in a submodule (in fact it would… it just might not know they belong in the sub-module without some work from the Xcode dev team).
>>> 
>>> But as a thought-experiment, here’s how I’m thinking of this problem:
>>> 
>>> Swift (as a language) has established that a file *is* a unit of code. Not the only one of course, but it is one, as evidenced by access controls (fileprivate, Swift2 private) which bound on the file.
>> 
>> I disagree with this: Swift is just a set of -more or less- arbitrary rules to allow the development of applications.
>> The criteria for those rules are chosen by the core team and will fluctuate with time.
>> As such there is no “Swift way” of doing things. Any such remark is by its nature a personal preference. (not that there is anything wrong with that, its just not an argument)
>> 
>> (Besides, a computer only knows files, thus of course a file is a unit of code. That does not mean we should extend “code” to be something outside the files)
> 
> First, that’s simply not true. In Swift 1, a file was *not* a logical unit of code because file boundaries were meaningless to the way the code in the project behaved. In Swift 2 we got `private` which made it possible for file boundaries to affect code behavior. Swift 3 changed `private` but introduced `fileprivate` for the same purpose.
> 
> Second, yes, Swift is just a language, but as that language evolves, it is an expression of the values & preferences of its development team and more recently, this community. If the file was not *intended* to be a unit of code, we would not have `fileprivate`.

Take this not too personally, but this is a clear example of “cherrypicking”.

For example: Quite clearly swift currently defines modules as an end product of the compile/link process. So if we want multiple modules in a project, we can simply do it that way.

Yet you are clearly willing to throw this rule under the bus and replace it with something new. But if we should not adhere to the rule set in swift, then it is equally not possible to find justification in the current rules.

That is not to say that we should ignore the current rules, because it reduced the cognitive load of developers if new rules closely track exiting rules. But each change (proposal) should be examined against the current rules, and if it does not violate them, or if it does not introduce something that creates cognitive dissonance, then the proposal -at least- should not be rejected because of that.

I.e. when proposing a change, the current rule set -not directly affected by the change- can be used as a reason to reject the proposal, but never to accept it.


> 
> 
>>> I (and many others) want to be able to describe another unit of code larger than a file and smaller than a Module to help organize the file-units we have which go together within a Module, but which shouldn’t (for various reasons) describe a complete Module in and of themselves.
>> 
>> Sure, and I have nothing against that. However I an dead set against extending the scope of the language itself to the organisation of its files. It is traditional to leave the organisation of the files to the configuration of the compiler/linker. And I do not see how it would simplify the language itself by extending the language to cover file organisation.
> 
> Clearly adding any new feature will not simplify the language. The questions are 1) should we add this feature, and 2) how to add the feature. I *do* take your objections to heart and that’s why I added some alternatives not involving the filesystem to my proposal. These alternatives each have their own problems and complications which we can discuss… maybe they’re not as bad as they seem, maybe they’re worse, and *probably*—because I’m just one person—there are solutions to the problem which I haven’t thought of.
> 
>> I will predict that if this approach is taken you should brace for the storm that would want to reverse this feature. It would imo dwarf the private vs fileprivate debate.
> 
> And no one wants that. That’s why we are having this discussion, no? I’ll be honest, I saw the fileprivate/private discussion on this list, and didn’t voice my opinion then because I thought “oh, they’ll do the right thing”, then was unhappy with what we got. That’s the reason I’ve started this discussion, and it’s also the reason I *really* value your dissenting input. I want not just *a* solution, but *the right* solution, and I know I don’t know everything that needs knowing to come up with it alone.
> 
> I and many others want a unit between Module & File. Someone else may not want such a thing, but they don’t have to use it, just like they don’t actually have to use Access Control modifiers. Now, you might object, saying *no one* should be able to organize their code this way… Then I’ll disagree with you on that, and we can discuss that position (and again, maybe I’m wrong, but my experience says otherwise). If you accept that wanting that in-between unit is a reasonable thing for us to want (even if you don’t plan to use it), then lets discuss the second question: “How to do it”.
> 
> One of my goals with this is that it should add *zero* additional ceremony should an author choose not to use it, and have no impact on existing code.
> 
>> Besides, why create something new? (Yes there are systems out there that do some of this) There are many existing solutions out there that we could adopt and stay completely within the bounds of the language, and give the users something they are likely already familiar with.
> 
> I’ve laid out some alternatives, and *to me* the filesystem-based model *currently* looks the best (or at least, “least bad”). You disagree, and that’s fine. And as I’ve said more than once, I’m open to suggestions.

I’d be interested in your opinion on my counter proposal that I posted yesterday. Is there anything in there that would create a problem for you?

> 
> Many existing solutions that I’ve looked at from other languages face complications when overlaid on Swift due to Swift’s extensions, and the exact structural topology we’re working within (the intersection of: Access Control, type declarations, profligate type extensions, lack of headers, etc), means that a solution for the in-between unit we’re calling ‘sub-modules’ (for lack of a definitive name) *might* need to be somewhat unique to Swift. Or might not.
> 
> We also all want to do what is “right” for the Swift language. Many people were calling for a `protected` access control modifier. That it was common in other languages didn't mean it was right for Swift. 
> 
> At the end of the day, I don’t know what the right solution is. This “problem” came about due to the meaning of `private` (and then `fileprivate`), and the “simple” approach taken to access control, which caused us to organize code around the file, causing some *very* large files. Some time back, many people proposed solving this problem through additional Access Control modifiers to "fill in the gaps” between `internal` and `fileprivate`. There has been a lot of discussion on that score, and time and again, the consensus seemed to be “This smells like a sub-modules problem, lets solve it there”. Very well, lets. The problem hasn’t gone away (for those of us who have it).

Yes, there seems to be a need for “protected” on type level. With “protected” taken in the usual C++ parlour. (Sub) modules are a way to approach that, but it could be solved with an additional access level as well.

Personally, with the arrival of SPM I no longer have a _need_ for modules. But it is probably still a good idea to have some mechanism to emulate/implement the “protected” access level.

> 
>>> For the moment—though we actually propose to call this unit a sub-module—we'll call this new unit a Foo. Let's describe properties of a Foo:
>>> 
>>> * A Foo should serve to organize one or more File code units by:
>>>  - ensuring a File (the code-unit) may belong to one and only one Foo
>>>  - making clear (with or without an IDE) to which Foo a given file might belong
>>> * A Foo should have a unique name
>>> * A Foo should be able to nest under another Foo, and the name of that nesting should be clear & unique
>>> * A Foo should be able to be easily created, split, and merged as the codebase evolves
>>> * A Foo should be able to be moved, copied, and SCMed as a unit, without reference to anything outside the Foo
>>> 
>>> Having now described the critical properties of this new unit called “Foo” have we not also described a filesystem directory?
>> 
>> Many things in life share characteristics. And while we should certainly look to them for lessons, we should not conclude that they are the same.
> 
> And that’s also fair. As I said, it was a thought-experiment trying to explain how I got to the directories concept. If it’s wrong, then it’s wrong. 
> 
> —Karim