[swift-evolution] Yet Another Take on Swift Sub-modules

[Rien]

> On 05 Mar 2017, at 15:52, Karim Nassar via swift-evolution <swift-evolution at swift.org> wrote:
> 
> 
>> On Mar 4, 2017, at 2:54 PM, Matthew Johnson <matthew at anandabits.com> wrote:
>> 
>>> 
>>> On Mar 4, 2017, at 9:56 AM, Karim Nassar <karim at karimnassar.com> wrote:
>>> 
>>> 
>>>> On Mar 3, 2017, at 5:21 PM, Matthew Johnson <matthew at anandabits.com> wrote:
>>>> 
>>>>> 
>>>>> On Mar 3, 2017, at 9:24 AM, Karim Nassar via swift-evolution <swift-evolution at swift.org> wrote:
>>>>> 
>>>>> Changes that *are* necessary are:
>>>>> 
>>>>> * Change the spelling of `internal` to `module` (making `module` the new default)
>>>>> * Introduce a new modifier `internal` to mean "Internal to the current sub-module and its child-sub-modules”
>>>> 
>>>> Can you give concrete examples of use cases where a descendent submodule needs access to symbols declared by an ancestor?  I gave some thought to this when drafting my proposal and came to the conclusion that this runs against the grain of layering and is likely to be a bad idea in practice.  If there are use cases I didn’t consider I am very interested in learning about them.
>>> 
>>> On further reflection and examination of my notes, I think you’re right, and that the `internal` encapsulation should be purely horizontal. Will adjust to reflect that.
>>> 
>>>>> ### Making a Sub-module
>>>>> 
>>>>> To create a sub-module within a Module (or sub-module) is simple: The author creates a directory, and places a "sub-module declaration file" within the directory:
>>>>> 
>>>>> ```
>>>>> //  __submodule.swift
>>>> 
>>>> Why the double underscore prefix?  To make it sort to the top in a file browser?
>>>> 
>>>> Is this file allowed to have any Swift code?  Or is it limited to submodule-related declarations only?  If the latter, why not use an extension such as `.submodule` or `.swiftmodule` to differentiate it from ordinary Swift files and allow the submodule to be named by the name of this file?
>>> 
>>> So, my reasoning was that by requiring a specific standard name for the declaration file, we guarantee that any directory can only describe one submodule. Prefixing the proposed name with underscores was simply a way of preventing naming collision with potential “real code” files (and yes, as a side-effect alpha-floating it to the top). Since the `submodule` declaration might expand to include statements & configuration about the sub-module, I see no reason to prohibit Swift code from existing in that sub-module declaration file… Disallowing/controlling that seems to be a style/linter concern.
>>> 
>>> However, as I mentioned above, all the specific spellings (except `internal`)  for the different concepts in this proposal are straw-men awaiting input. I’d say the addition of a new type of file extension raises some concerns for me, but there’s already been a lot of push back on the general idea of using filesystem structures to organize sub-modules, so the whole idea may be moot. 
>> 
>> I’ve actually been starting to come around to the idea of using the file system.  Not so much because I really like it, but because I have been considering further some of the drawbacks of other approaches.  
>> 
>> One big reason is that a submodule should form a scope and scopes should consist of code that is physically adjacent.  In practice this means it should reside in close proximity in the file system.  Allowing any file in a project to be a part of any submodule partly defeats the purpose of using them to structure a project internally.  If we’re going to be organizing the files in a submodule physically anyway maybe we should just take advantage of that fact and prevent a stray file in a distant part of the file system from being part of the submodule.
>> 
>> The second reason is that there is a big problem with placing a simple `submodule Foo` declaration at the top of each file in a submodule.  We all make typos from time to time.  This method makes it too easy to accidentally type `submodule Fooo` and end up in a submodule you didn’t intend.  This mistake would likely be caught relatively quickly but it seems silly to have a system subject to this kind of mistake.  This means we would end arbitrarily placing the declaration in one file and saying `extends submodule Foo` in the rest of the files.  Your design avoids the need to arbitrarily choose where to place the declaration and avoids the need for a declaration in the rest of the files.
> 
> I’m not dead-set on this approach, but as you say, it solves a *lot* of problems that other approaches introduce. I do recognize the reasonableness of the main argument against, that file location shouldn’t have such a dramatic affect on behavior… *but* the fact is (whether by convention or expediency) we already *do* have filesystem location dependencies on our code projects… 
> 
> * One can’t willy-nilly move files around the disk and expect a project to build… if you move it somewhere the compiler doesn’t know to look for it, you’re going to break things
> * One can’t just move a file out of your SCM repo root, or you’ll “lose” the file

True, but if other files do not refer to the lost file, you don’t even know for which file to look.
(imo this is already a weak point in swift, but this approach to submodules would make it -much?- worse)

If you were to include filenames in the “submodule declaration file” at least this omission in swift would be partly covered, and you would not need to move the other files into special places. (You could though)

> 
> One might argue that these are pathological cases, but the fact remains that this approach only introduces a new constraint on project structure, not a new *kind* of constraint, and it is one that is purely optional. You don’t *have* to use sub-modules in a project, and if you don’t, there’s no additive burden on you.
> 
>>> 
>>> Shortly, I’m going to update the original proposal gist reflecting all the comments I’ve received and part of that will be expanding the proposal section to include multiple alternate strategies for discussion, along with pros/cons for each. I’ll include a discussion on naming options for this file in this section.
>> 
>> I’m going to look at your new draft this afternoon while I’m working on updating my proposal.  I think you’ll find it much more to your liking than the original draft.  Perhaps we will be able to converge if we keep the discussion going.
> 
> I’d be happy to cooperate-with/converge-with/withdraw-in-favor-of. I’m mostly interested in a solid, light-weight solution to my use-cases, however that looks. I look forward to seeing your updates!
> 
>>> 
>>>> If we’re going to use the file system to organize submodules this seems like a reasonable approach.  It allows larger submodules to have folder hierarchies within them and also creates a central location for submodule-related declarations.
>>>> 
>>>> A primary flaw I see in this approach is that Xcode is the dominant IDE for Swift and the way Xcode handles files is not conducive to file-system organization.  I really detest the way Xcode handles this and would vastly prefer that it simply reflected the physical file system hierarchy but I don’t think that will change any time soon.  On the other hand maybe a file system based submodule system in Swift would motivate the Xcode team to better reflect the physical file system organization.
>>> 
>>> My thoughts (hopes?) on this were that should Swift adopt such a mechanism for sub-modules, then part of Xcode’s integration of the feature would be to make it aware of sub-modules in a first-class way, so that the author wouldn’t have to manually manage the project files to take advantage of the feature. 
>>> 
>>> I’d envisioned either a new “Makes Sub-Module” property on Groups or a new “Sub-Module" mechanism alongside Groups & Folders in the project list. This may be wishful thinking on my part.
>> 
>> The problem with a "Makes Sub-Module” on a group in Xcode is that groups do not mirror the file system.  Maybe a file system based submodule feature would motivate the Xcode team to support group structures that do mirror the file system better.  I hope that would be the case!  Unfortunately I don’t think we’d have a guarantee when reviewing the feature and there are significant practical implications for using the feature in Xcode without that kind of support.  We would probably just have to trust Apple to get this right.
> 
> Too true. I don’t know how much the Swift team can influence or contribute to Xcode. On the other hand, using Xcode’s “Folder References” feature isn’t so onerous that it would prevent me from getting very large benefit from sub-modules in my projects.
> 
> —
> 
> As one concrete example, I have a large app that imports a Module (Framework) which represents a kind of domain-specific DOM graph. There is a Query type which must live within that module (both for WMO, and for more esoteric dependency reasons). This Query type has 11 related or sub- types including specialized graph nodes and a parser. Out of these 12 types, the API genuinely exposes 2 types (Query and QueryError) and a grand total of 5 methods/properties outside of the “Query” cluster of types. In order to achieve this encapsulation, I’ve had to lump all of these into a single 2k+ lines-long file with heavy use of fileprivate, which is a real pain to manage in terms of reading, editing, and SCM.
> 
> One might say, oh, that’s not so bad, except this is one example of many similar situations in a very large codebase consisting of the Application and 9 subordinate Modules.
> 
> With this or similar sub-module system, this code could easily be factored into a tight cluster of specialized files, with no one file being larger than a couple of hundred lines, and most being in the 10–50 line neighborhood—much more manageable.
> 
> In fact, I’d go so far as to say that if I had a real sub-module system, many of those Modules would get demoted. The cost of maintaining full frameworks for code that will never be used outside of this application just to achieve encapsulation isn’t trivial. 
> 
> And another neat thing about a directory-based approach as I’ve proposed is that if I ever do want to “promote” a sub-module to its own Module, it’s very easy to do because I know where *all* the files are, and all of the sub-module’s child sub-modules automatically get “promoted up” one level without any effort. 
> 
> —Karim
> 
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution