[swift-evolution] [Proposal][Discussion] Modular Swift
Matthew Johnson
matthew at anandabits.com
Fri Mar 3 15:10:21 CST 2017
> On Mar 2, 2017, at 10:52 PM, T.J. Usiyan <griotspeak at gmail.com> wrote:
>
> Submodules will, I hope, afford us the ability to share meaningfully arranged API. In an ideal scenario, I could have a module that has all of the smaller pieces of 'utilities and conveniences' that one builds up broken into submodules. Importing one of the submodules from its enclosing module shouldn't bring anything extra along. I don't think that we need to add nearly as many new concepts as `scope-based submodules` introduces. "ABI Boundaries", "Name Boundaries", "Scope Boundaries”?
These boundaries already exist. All the proposal does is give a name to them.
> What he have is built on LLVM Modules and holds together fairly well on its own.
> Our projects, at present, are a module. ABI doesn't have very much to do with the 'boundary' of our target. The wording around Module maps make it clear that submodules have been considered Module Maps <http://clang.llvm.org/docs/Modules.html#module-maps>
>
> Export statements are not a useful addition. Access modifiers capture this well enough via open|public etc. When paired with robust import syntax, we would have everything that we need.
Can you elaborate on what you mean by “robust import syntax”? The top-level export statement has similar semantics to the `public import` in Robert and Jaden’s proposal. They even used the word “export” in describing the semantics of `public import`.
It just uses the name `export` rather than `import` to avoid overloading the semantics of `import` and introduces additional features to avoid coupling the internal structure of a module to the view exposed to users of the module.
>
> The changes to scope and access modifiers in `scope-based submodules` are dramatic and don't seem to pay for themselves at all.
This aspect makes the proposal more powerful but is not essential to scope-based submodules. It has been broken out into an independent proposal which so far has received very positive feedback.
>
>
> "Top of the file" declaration to declare a submodule is not at all desirable to me. I would rather that we use the (not-entirely-pleasant-to-me) "curly brace without an indent" used for switches+cases.
If it’s not entirely pleasant why do you prefer that style?
>
> LLVM modules handle circularity, I think.
Do you believe circularity is important? What use cases do you have for that?
>
> The ability to import a single type or a list of specific types is of great value, in my opinion, especially when paired with renaming (we don't have it but one can dream, no?) Qualified imports are obviously the tact that I would like to make this possible. Allow the consumer of the API to get as specific as they need to when importing. I consider it an unfortunate concession that `import This.That.TheOther` imports as much as it does but it is, basically, the only major issue that I have with this proposal and it can't be helped without breaking compatibility.
Selective import is an additive feature that is orthogonal to submodules. I agree that it would be very nice to have, but it is a separate issue IMO.
>
> Overall, it is my opinion that we should solve the submodule and import problems with changes like those proposed here. Modify very little of the current syntax and semantics. What we have is fairly close and can actually be coherent even with the additional burden of exposing submodules.
It sounds like you don’t view encapsulation as an important feature of submodules. Is that correct? If so, why not? Do you believe we need encapsulation boundaries larger than a file but smaller than a module?
>
>
>
>
> On Thu, Mar 2, 2017 at 11:01 PM, Matthew Johnson <matthew at anandabits.com <mailto:matthew at anandabits.com>> wrote:
>
>
> Sent from my iPad
>
> On Mar 2, 2017, at 9:06 PM, T.J. Usiyan <griotspeak at gmail.com <mailto:griotspeak at gmail.com>> wrote:
>
>> +1 overall. prefer this approach over the "scope based" approach in the other proposal
>
> Can you elaborate? What problems are you hoping submodules will address?
>
>>
>> On Wed, Feb 22, 2017 at 10:10 AM, Matthew Johnson via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>
>>> On Feb 21, 2017, at 11:54 PM, Robert Widmann <devteam.codafi at gmail.com <mailto:devteam.codafi at gmail.com>> wrote:
>>>
>>>
>>>> On Feb 22, 2017, at 12:41 AM, Matthew Johnson <matthew at anandabits.com <mailto:matthew at anandabits.com>> wrote:
>>>>
>>>>
>>>>
>>>> Sent from my iPad
>>>>
>>>> On Feb 21, 2017, at 11:09 PM, Robert Widmann <devteam.codafi at gmail.com <mailto:devteam.codafi at gmail.com>> wrote:
>>>>
>>>>>
>>>>>> On Feb 21, 2017, at 11:59 PM, Matthew Johnson <matthew at anandabits.com <mailto:matthew at anandabits.com>> wrote:
>>>>>>
>>>>>>
>>>>>>> On Feb 21, 2017, at 10:41 PM, Robert Widmann <devteam.codafi at gmail.com <mailto:devteam.codafi at gmail.com>> wrote:
>>>>>>>
>>>>>>> By API boundaries I mean both the one internal to MyModule.Foo and the one defined by MyModule. Here “the API boundary” is explicitly about the submodule MyModule.Foo, whose internal state may have been “unsealed” in the top level by the extension, but has not been re-exported.
>>>>>>
>>>>>> I’m sorry, but I just don’t understand how modules form an API boundary in this system. To me a boundary means something that blocks access. In this system `internal` ranges over the entire module and all submodules. The only boundaries I can see besides the module itself are files and lexical scopes (with `fileprivate` and `private`).
>>>>>>
>>>>>
>>>>> A module is a named region that introduces a lexical scope into which declarations may be nested. The name of the module can be used to access these member declarations. A module, like other aggregate structures in Swift, may be extended with new declarations over one or more translation units (files).
>>>>>
>>>>>
>>>>> Your API boundary lives, as it does today, at the edges of each (sub)module declaration. APIs that are public or open in a module defines code that is free to move across this boundary and into the open. APIs that are internal are free to have their modules unsealed into other internal modules to enable modular composition. APIs that are private and fileprivate do not participate in the API boundary because they are not eligible for any kind of export.
>>>>>
>>>>> If any of that is unclear, please let me know.
>>>>
>>>> Yes, in fact parts are unclear.
>>>>
>>>> "APIs that are public or open in a module defines code that is free to move across this boundary and into the open"
>>>>
>>>> This is unclear because you're saying submodules form an API boundary and you're also saying we need to make APIs open or public to allow them to move across this boundary. But then you say we can unseal it (import or extend, right?) within the module and gain visibility to the internal symbols. Are you trying to say that it's a soft boundary within the module that can be permeated with an import or by extension?
>>>
>>> Of course. Soft implies more permeability than you are actually afforded, but if you want to think of it that way then it may help to put it in context.
>>>
>>> For what it’s worth, the bulk of the discussion around this feature is focused on author-side concerns like the behavior of internal modules because access control makes a mess of any reasonable semantics.
>>>
>>>>
>>>> If so, that's not the kind of boundary I think many of us are talking about. We're talking about a hard boundary within the module, but broader than a file.
>>>
>>> How then, does one go about accessing declarations contained in these kinds of impermeable modules? You must define points of exposure to be able to use the module. What you’re describing is as though you had can only build hierarchies of completely private types and then cherry-pick them one-by-one into the open - which, mind you, is not a pattern encouraged by any of the access control levels we have today and isn’t supported by any language I’m aware of.
>>
>> There are ways to do this without requiring cherry picking individual types. I’m not looking for impermeable modules. I’m looking for bounded visibility within the module in a way that is very similar to `fileprivate`, but at a larger granularity. I’m writing up my view of submodules so we have something more concrete to discuss.
>>
>>>
>>>>
>>>>>> means that it is trivial to put code anywhere within the module that extends the submodule and wraps a symbol in a new name and declares it `public`.
>>>>>
>>>>> Precisely. That’s the same pattern that good Swift code, arguably good code in any language that enables hiding, uses today.
>>>>>
>>>>>> They can also trivially add a `public import MyModule.Foo` anywhere at the top level of their file because every file is forced to include top level scope.
>>>>>
>>>>> Perhaps you misunderstand. Say the APIs in MyModule.Foo were all of internal or stricter access: The re-export is a no-op. You cannot change the access level of declarations, you can only do the modular thing and wrap them in a palatable interface for export by a module you want to be user-facing. You have to decide to make an API public, just as today you have to decide to make part of an interface public. I don’t see how this is distinct from the goals of this proposal.
>>>>
>>>> Yes, I understand this. But submodules aren't visible outside the module by default. It's possible for a submodule to have public and open symbols without the top level public import anywhere in the program.
>>>
>>>> What I'm saying here is that someone in a distant part of the code base could arbitrarily add it if they decided to. The system doesn't prevent it.
>>>>
>>>
>>> Because, by definition, you are not in a “distant part of the codebase” when you are extending a module. You’re introducing related functionality under the same namespace with more related functionality. Anything else is fundamentally anti-modular because it pollutes different concerns together into an interlocking directorate. Miles away implies cognitive and semantic distance when you’re probably physically in the same directory!
>>>
>>>> I understand that you consider that a non goal. I'm simply pointing out that the system has this property. I think it's reasonable to want a system with different properties. And I don't think it's clear yet exactly what kind of system might garner the support necessary to be accepted as Swift's submodule system. That's part of the reason we have these discussions! :)
>>>
>>> I so appreciate this, too. I genuinely enjoy discussions that try to poke holes and prod out better explanations. It’s how you iterate on proposals and just make good things happen in a community like this.
>>>
>>> ~Robert Widmann
>>>
>>>>
>>>>>
>>>>>>
>>>>>> In my opinion, we need to identify what goals we have for a submodule system - what problems are we trying to solve and what use cases do we intend to enable.
>>>>>>
>>>>>> There are quite a few of us who want the ability to form solid API boundaries inside a module and view this as one of the fundamental features of a submodule system. It’s reasonable to ask why we view this capability as essential.
>>>>>>
>>>>>> I can’t speak for anyone else, but here are a few reasons why it’s important to me:
>>>>>>
>>>>>> * Solid API boundaries are essential to good design.
>>>>>> * Having access to an entire code base does not reduce the benefits of #1. Some code bases are substantial in size and hard boundaries are important to keeping them manageable.
>>>>>> * Using full-fledged modules to do this is possible, but also involves a bit of ceremony that is incidental, not essential complexity in many cases. It would be better to have a lighter weight mechanism to do this.
>>>>>> * Swift currently only has whole module optimization, not whole program optimization. There is a performance penalty to using full-fledged modules.
>>>>>>
>>>>>>>
>>>>>>> ~Robert Widmann
>>>>>>>
>>>>>>>> On Feb 21, 2017, at 11:38 PM, Matthew Johnson <matthew at anandabits.com <mailto:matthew at anandabits.com>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Feb 21, 2017, at 10:29 PM, Robert Widmann <devteam.codafi at gmail.com <mailto:devteam.codafi at gmail.com>> wrote:
>>>>>>>>>
>>>>>>>>> This level of access, the “private to this submodule except to the select set of interfaces I want to see it” level, is the equivalent of friend classes in C++. I don’t consider leaving this out to be a hole, nor is it an "encapsulation-related problem” because at no point can you break the API boundary and re-export anything here with a higher level of access than it had previously.
>>>>>>>>
>>>>>>>> By API boundary you mean the top-level module, right?
>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Feb 21, 2017, at 11:13 PM, Matthew Johnson <matthew at anandabits.com <mailto:matthew at anandabits.com>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On Feb 21, 2017, at 10:11 PM, Matthew Johnson <matthew at anandabits.com <mailto:matthew at anandabits.com>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> On Feb 21, 2017, at 9:47 PM, Brent Royal-Gordon via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> On Feb 21, 2017, at 7:38 PM, Robert Widmann <devteam.codafi at gmail.com <mailto:devteam.codafi at gmail.com>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Correct. Because, in dividing the submodule across an extension, you have placed what should be a private API into a differently-scoped location.
>>>>>>>>>>>>
>>>>>>>>>>>> Okay. So is your submodule design not intended to address the "I want to encapsulate implementation details so they're only visible to several units of code in different files, but not the entire module" use case? Because if there's no way to scope a symbol to "everything inside this submodule, but nothing outside this submodule", I think it leaves that use case unserved.
>>>>>>>>>>>
>>>>>>>>>>> Unless I’m missing something there is also another encapsulation-related problem with the proposed design. Let’s suppose for the sake of discussion there was a `submoduleprivate` access modifier (intentionally ungainly and not realistic).
>>>>>>>>>>>
>>>>>>>>>>> // File 1
>>>>>>>>>>> module Foo {
>>>>>>>>>>> // internal, visible to the whole module
>>>>>>>>>>> class Bar { submoduleprivate var protectedState: Int = 0 }
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> // File 2 - Has nothing to do with Foo at all
>>>>>>>>>>> import MyModule.Foo
>>>>>>>>>>>
>>>>>>>>>>> module NotFoo {
>>>>>>>>>>> // Hey, I need to see Bar.protectedState!!!
>>>>>>>>>>> func totallyNotFoo() {
>>>>>>>>>>> var bar = Bar()
>>>>>>>>>>> bar.foosExposedPrivates = 42
>>>>>>>>>>> }
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> // ok, I’ll just add an extension to Foo so I can see submoduleprivate and wrap what I need
>>>>>>>>>>> module Foo {
>>>>>>>>>>
>>>>>>>>>> Oops, this should have been `extension Foo`, but otherwise I believe it is valid under this proposal.
>>>>>>>>>>
>>>>>>>>>>> // Hey, I’ll be nice and keep it fileprivate, but I could make it public if I wanted to.
>>>>>>>>>>> extension Foo {
>>>>>>>>>>> fileprivate var foosExposedPrivates: Int {
>>>>>>>>>>> // Yep, I’m inside Foo so I can see it’s submoduleprivate stuff
>>>>>>>>>>> get { return protectedState }
>>>>>>>>>>> set { protectedState = newValue }
>>>>>>>>>>> }
>>>>>>>>>>> }
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Brent Royal-Gordon
>>>>>>>>>>>> Architechies
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> swift-evolution mailing list
>>>>>>>>>>>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>>>>>>>>>>>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>
>>
>>
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170303/483e117b/attachment.html>
More information about the swift-evolution
mailing list