[swift-evolution] [Proposal][Discussion] Modular Swift

Thu Mar 2 22:52:04 CST 2017

Submodules will, I hope, afford us the ability to share meaningfully
arranged API. In an ideal scenario, I could have a module that has all of
the smaller pieces of 'utilities and conveniences' that one builds up
 broken into submodules. Importing one of the submodules from its enclosing
module shouldn't bring anything extra along. I don't think that we need to
add nearly as many new concepts as `scope-based submodules` introduces.
"ABI Boundaries", "Name Boundaries", "Scope Boundaries"? What he have is
built on LLVM Modules and holds together fairly well on its own.
Our projects, at present, are a module. ABI doesn't have very much to do
with the 'boundary' of our target. The wording around Module maps make it
clear that submodules have been considered Module Maps
<http://clang.llvm.org/docs/Modules.html#module-maps>

Export statements are not a useful addition. Access modifiers capture this
well enough via open|public etc. When paired with robust import syntax, we
would have everything that we need.

The changes to scope and access modifiers in `scope-based submodules` are
dramatic and don't seem to pay for themselves at all.

"Top of the file" declaration to declare a submodule is not at all
desirable to me. I would rather that we use the
(not-entirely-pleasant-to-me) "curly brace without an indent" used for
switches+cases.

LLVM modules handle circularity, I think.

The ability to import a single type or a list of specific types is of great
value, in my opinion, especially when paired with renaming (we don't have
it but one can dream, no?) Qualified imports are obviously the tact that I
would like to make this possible. Allow the consumer of the API to get as
specific as they need to when importing. I consider it an unfortunate
concession that `import This.That.TheOther` imports as much as it does but
it is, basically, the only major issue that I have with this proposal and
it can't be helped without breaking compatibility.

Overall, it is my opinion that we should solve the submodule and import
problems with changes like those proposed here. Modify very little of the
current syntax and semantics. What we have is fairly close and can actually
be coherent even with the additional burden of exposing submodules.

On Thu, Mar 2, 2017 at 11:01 PM, Matthew Johnson <matthew at anandabits.com>
wrote:

>
>
> Sent from my iPad
>
> On Mar 2, 2017, at 9:06 PM, T.J. Usiyan <griotspeak at gmail.com> wrote:
>
> +1 overall. prefer this approach over the "scope based" approach in the
> other proposal
>
>
> Can you elaborate?  What problems are you hoping submodules will address?
>
>
> On Wed, Feb 22, 2017 at 10:10 AM, Matthew Johnson via swift-evolution <
> swift-evolution at swift.org> wrote:
>
>>
>> On Feb 21, 2017, at 11:54 PM, Robert Widmann <devteam.codafi at gmail.com>
>> wrote:
>>
>>
>> On Feb 22, 2017, at 12:41 AM, Matthew Johnson <matthew at anandabits.com>
>> wrote:
>>
>>
>>
>> Sent from my iPad
>>
>> On Feb 21, 2017, at 11:09 PM, Robert Widmann <devteam.codafi at gmail.com>
>> wrote:
>>
>>
>> On Feb 21, 2017, at 11:59 PM, Matthew Johnson <matthew at anandabits.com>
>> wrote:
>>
>>
>> On Feb 21, 2017, at 10:41 PM, Robert Widmann <devteam.codafi at gmail.com>
>> wrote:
>>
>> By API boundaries I mean both the one internal to MyModule.Foo and the
>> one defined by MyModule.  Here “the API boundary” is explicitly about the
>> submodule MyModule.Foo, whose internal state may have been “unsealed” in
>> the top level by the extension, but has not been re-exported.
>>
>>
>> I’m sorry, but I just don’t understand how modules form an API boundary
>> in this system.  To me a boundary means something that blocks access.  In
>> this system `internal` ranges over the entire module and all submodules.
>> The only boundaries I can see besides the module itself are files and
>> lexical scopes (with `fileprivate` and `private`).
>>
>>
>> A module is a named region that introduces a lexical scope into which
>> declarations may be nested. The name of the module can be used to access
>> these member declarations. A module, like other aggregate structures in
>> Swift, may be extended with new declarations over one or more translation
>> units (files).
>>
>>
>> Your API boundary lives, as it does today, at the edges of each
>> (sub)module declaration.  APIs that are public or open in a module defines
>> code that is free to move across this boundary and into the open.  APIs
>> that are internal are free to have their modules unsealed into other
>> internal modules to enable modular composition.  APIs that are private and
>> fileprivate do not participate in the API boundary because they are not
>> eligible for any kind of export.
>>
>> If any of that is unclear, please let me know.
>>
>>
>> Yes, in fact parts are unclear.
>>
>> "APIs that are public or open in a module defines code that is free to
>> move across this boundary and into the open"
>>
>> This is unclear because you're saying submodules form an API boundary and
>> you're also saying we need to make APIs open or public to allow them to
>> move across this boundary.  But then you say we can unseal it (import or
>> extend, right?) within the module and gain visibility to the internal
>> symbols.  Are you trying to say that it's a soft boundary within the module
>> that can be permeated with an import or by extension?
>>
>>
>> Of course.  *Soft* implies more permeability than you are actually
>> afforded, but if you want to think of it that way then it may help to put
>> it in context.
>>
>> For what it’s worth, the bulk of the discussion around this feature is
>> focused on author-side concerns like the behavior of internal modules
>> because access control makes a mess of any reasonable semantics.
>>
>>
>> If so, that's not the kind of boundary I think many of us are talking
>> about.  We're talking about a hard boundary within the module, but broader
>> than a file.
>>
>>
>> How then, does one go about accessing declarations contained in these
>> kinds of impermeable modules?  You *must* define points of exposure to
>> be able to use the module.  What you’re describing is as though you had can
>> only build hierarchies of completely private types and then cherry-pick
>> them one-by-one into the open - which, mind you, is not a pattern
>> encouraged by any of the access control levels we have today and isn’t
>> supported by any language I’m aware of.
>>
>>
>> There are ways to do this without requiring cherry picking individual
>> types.  I’m not looking for impermeable modules.  I’m looking for bounded
>> visibility within the module in a way that is very similar to
>> `fileprivate`, but at a larger granularity.  I’m writing up my view of
>> submodules so we have something more concrete to discuss.
>>
>>
>>
>>  means that it is trivial to put code anywhere within the module that
>> extends the submodule and wraps a symbol in a new name and declares it
>> `public`.
>>
>>
>> Precisely.  That’s the same pattern that good Swift code, arguably good
>> code in any language that enables hiding, uses today.
>>
>> They can also trivially add a `public import MyModule.Foo` anywhere at
>> the top level of their file because *every* file is forced to include
>> top level scope.
>>
>>
>> Perhaps you misunderstand.  Say the APIs in MyModule.Foo were all of
>> internal or stricter access: The re-export is a no-op.  You *cannot* change
>> the access level of declarations, you can only do the modular thing and
>> wrap them in a palatable interface for export by a module you want to be
>> user-facing.  You have to *decide* to make an API public, just as today
>> you have to decide to make part of an interface public.  I don’t see how
>> this is distinct from the goals of this proposal.
>>
>>
>> Yes, I understand this.  But submodules aren't visible outside the module
>> by default.  It's possible for a submodule to have public and open symbols
>> without the top level public import anywhere in the program.
>>
>>
>> What I'm saying here is that someone in a distant part of the code base
>> could arbitrarily add it if they decided to.  The system doesn't prevent
>> it.
>>
>>
>> Because, by definition, you are not in a “distant part of the codebase”
>> when you are extending a module.  You’re introducing related functionality
>> under the same namespace with more related functionality.  Anything else is
>> fundamentally anti-modular because it pollutes different concerns together
>> into an interlocking directorate.  *Miles away* implies cognitive and
>> semantic distance when you’re probably physically in the same directory!
>>
>>
>> I understand that you consider that a non goal.  I'm simply pointing out
>> that the system has this property.  I think it's reasonable to want a
>> system with different properties.  And I don't think it's clear yet exactly
>> what kind of system might garner the support necessary to be accepted as
>> Swift's submodule system.  That's part of the reason we have these
>> discussions! :)
>>
>>
>> I so appreciate this, too.  I genuinely enjoy discussions that try to
>> poke holes and prod out better explanations. It’s how you iterate on
>> proposals and just make good things happen in a community like this.
>>
>> ~Robert Widmann
>>
>>
>>
>>
>> In my opinion, we need to identify what goals we have for a submodule
>> system - what problems are we trying to solve and what use cases do we
>> intend to enable.
>>
>> There are quite a few of us who want the ability to form solid API
>> boundaries *inside* a module and view this as one of the fundamental
>> features of a submodule system.  It’s reasonable to ask why we view this
>> capability as essential.
>>
>> I can’t speak for anyone else, but here are a few reasons why it’s
>> important to me:
>>
>> * Solid API boundaries are essential to good design.
>> * Having access to an entire code base does not reduce the benefits of
>> #1.  Some code bases are substantial in size and hard boundaries are
>> important to keeping them manageable.
>> * Using full-fledged modules to do this is possible, but also involves a
>> bit of ceremony that is incidental, not essential complexity in many
>> cases.  It would be better to have a lighter weight mechanism to do this.
>> * Swift currently only has whole module optimization, not whole program
>> optimization.  There is a performance penalty to using full-fledged modules.
>>
>>
>> ~Robert Widmann
>>
>> On Feb 21, 2017, at 11:38 PM, Matthew Johnson <matthew at anandabits.com>
>> wrote:
>>
>>
>> On Feb 21, 2017, at 10:29 PM, Robert Widmann <devteam.codafi at gmail.com>
>> wrote:
>>
>> This level of access, the “private to this submodule except to the select
>> set of interfaces I want to see it” level, is the equivalent of friend
>> classes in C++.  I don’t consider leaving this out to be a hole, nor is it
>> an "encapsulation-related problem” because at no point can you break the
>> API boundary and re-export anything here with a higher level of access than
>> it had previously.
>>
>>
>> By API boundary you mean the top-level module, right?
>>
>>
>> On Feb 21, 2017, at 11:13 PM, Matthew Johnson <matthew at anandabits.com>
>> wrote:
>>
>>
>> On Feb 21, 2017, at 10:11 PM, Matthew Johnson <matthew at anandabits.com>
>> wrote:
>>
>>
>> On Feb 21, 2017, at 9:47 PM, Brent Royal-Gordon via swift-evolution <
>> swift-evolution at swift.org> wrote:
>>
>> On Feb 21, 2017, at 7:38 PM, Robert Widmann <devteam.codafi at gmail.com>
>> wrote:
>>
>> Correct.  Because, in dividing the submodule across an extension, you
>> have placed what should be a private API into a differently-scoped location.
>>
>>
>> Okay. So is your submodule design not intended to address the "I want to
>> encapsulate implementation details so they're only visible to several units
>> of code in different files, but not the entire module" use case? Because if
>> there's no way to scope a symbol to "everything inside this submodule, but
>> nothing outside this submodule", I think it leaves that use case unserved.
>>
>>
>> Unless I’m missing something there is also another encapsulation-related
>> problem with the proposed design.  Let’s suppose for the sake of discussion
>> there was a `submoduleprivate` access modifier (intentionally ungainly and
>> not realistic).
>>
>> // File 1
>> module Foo {
>> // internal, visible to the whole module
>> class Bar { submoduleprivate var protectedState: Int = 0 }
>> }
>>
>> // File 2 - Has nothing to do with Foo at all
>> import MyModule.Foo
>>
>> module NotFoo {
>> // Hey, I need to see Bar.protectedState!!!
>> func totallyNotFoo() {
>>   var bar = Bar()
>>   bar.foosExposedPrivates = 42
>> }
>> }
>>
>> // ok, I’ll just add an extension to Foo so I can see submoduleprivate
>> and wrap what I need
>> module Foo {
>>
>>
>> Oops, this should have been `extension Foo`, but otherwise I believe it
>> is valid under this proposal.
>>
>> // Hey, I’ll be nice and keep it fileprivate, but I could make it public
>> if I wanted to.
>> extension Foo {
>>     fileprivate var foosExposedPrivates: Int {
>>        // Yep, I’m inside Foo so I can see it’s submoduleprivate stuff
>>        get { return protectedState }
>>        set  { protectedState = newValue }
>>     }
>> }
>> }
>>
>>
>> --
>> Brent Royal-Gordon
>> Architechies
>>
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170302/f2e8e389/attachment.html>