<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">I hate to be "that guy", but I believe the problem under study is a special case of a much more general paradox.<br class=""><div class=""><br class=""></div><div class="">Swift presently lacks any concept of "module encapsulation". A module's members are either available everywhere, or available nowhere. I refer to the following diagram:</div><div class=""><br class=""></div><div class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class=""><font face="Menlo" class="">- Sources</font></div><div class=""><font face="Menlo" class=""> - Country</font></div></div></blockquote><blockquote type="cite" class=""><font face="Menlo" class=""> Venezuela.swift</font></blockquote><div class=""><blockquote type="cite" class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"></div></blockquote></div><blockquote type="cite" class=""><font face="Menlo" class=""> - United_States</font></blockquote><blockquote type="cite" class=""><font face="Menlo" class=""> Rhode_island.swift </font></blockquote><blockquote type="cite" class=""><font face="Menlo" class=""> - Texas</font></blockquote><blockquote type="cite" class=""><font face="Menlo" class=""> Austin.swift</font></blockquote><br class=""></div><div class="">Consider the symbol Austin.SXSW. Either Austin.SXSW is public, in which case it is accessible to Texas, Rhode Island, United_States, etc, or it is private/internal, in which case it is only accessible to Austin. In no case it is accessible to Texas and not further, in no case is it "encapsulated" somehow by the Texas module. It is either invisible or global, Swift does not have another option. The namespace is flat. [0]</div><div class=""><br class=""></div><div class="">Meanwhile, Swift <b class="">does</b> have a concept of dependencies. We must compile *some* module first, and that module cannot depend on another module which is not yet compiled. Here, Austin has no dependencies and so it is compiled first. As a consequence it cannot depend on United States, and therefore it cannot access Rhode Island. However that has nothing to do with encapsulation, it has to do with a limitation of the Swift compiler model. A limitation which by the way, does not exist in other SwiftPM languages such as C. But in Swift, modules form a dependency tree [1].</div><div class=""><br class=""></div><div class="">The paradox is that, traditionally, a filesystem connotes both [0] and [1], while from a language point of view, we mean only one or the other. For example, in Unix, /usr/bin/git is an encapsulated path; if you try to run /usr/bin/git we first check that you are allowed to traverse /usr/, then /usr/bin/, and only finally /usr/bin/git. Whereas in Swift, to access Sources/Country/United_States/Texas/Austin.SXSW, we only check Austin; because Swift modules are not encapsulated. So a filesystem that promised meaning [0] did not deliver.</div><div class=""><br class=""></div><div class="">On the other hand, a folder depends on its contents; if you ask Unix to copy a folder you expect not just an empty folder, but a folder of the same contents, e.g. an identical folder. But if you copy a C module, you do not get all its dependencies, because there exists such a keyword as `extern`. So a filesystem that promised meaning [1] did not deliver it either.</div><div class=""><br class=""></div><div class=""><blockquote type="cite" class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class="">If we then extend the same rules to modules then there will be no surprising behavior.</div></div></blockquote><br class=""></div></div><div class="">I suppose "surprise" is subjective, but I don't believe there is a solution to the paradox that is not surprising. We could resolve the problem in the first diagram by reorganizing to solve [0], requiring</div><div class=""><br class=""></div><div class=""><div class=""><blockquote type="cite" class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class=""><font face="Menlo" class="">- Sources</font></div><div class=""><font face="Menlo" class=""> - Country</font></div></div></blockquote><blockquote type="cite" class=""><font face="Menlo" class=""> Venezuela.swift</font></blockquote><div class=""><blockquote type="cite" class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"></div></blockquote></div><blockquote type="cite" class=""><font face="Menlo" class=""> - United_States</font></blockquote><blockquote type="cite" class=""><font face="Menlo" class=""> Rhode_island.swift </font></blockquote><blockquote type="cite" class=""><font face="Menlo" class=""> - Texas</font></blockquote><blockquote type="cite" class=""><font face="Menlo" class=""> Austin.swift</font></blockquote></div></div><div class=""><br class=""></div><div class="">which I think is one of your proposals. But now we are wrong on [1]. That is, as a matter of compile-order, Rhode_island.Providence is not available in Austin while Austin.SXSW is available in Rhode_island, and this result is surprising given the flat filesystem layout shown.</div><div class=""><br class=""></div><div class="">Perhaps this sheds some light on my opposition to filesystem-defined build systems. The paradox above is not accidentally complex, that we could resolve it by "tightening the rules" to mandate some particular filesystem layout. It is essentially complex, because no filesystem layout can describe Swift's actual behavior. </div><div class=""><br class=""></div><div class="">We could of course redefine Swift to mean both [0] and [1], while that would break... pretty much all existing code, I would support it. However that does nothing to resolve the problem for C, so unless we plan to ban the extern keyword (which would no longer be C) that is no way out of the paradox.</div><div class=""><br class=""></div><div class=""><br class=""></div><div><blockquote type="cite" class=""><div class="">On Apr 19, 2016, at 2:42 PM, Max Howell via swift-build-dev <<a href="mailto:swift-build-dev@swift.org" class="">swift-build-dev@swift.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta http-equiv="Content-Type" content="text/html charset=utf-8" class=""><meta http-equiv="Content-Type" content="text/html charset=utf-8" class=""><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">SwiftPM is a pioneer for SwiftPM, and thus we have many modules. For some of these modules I’d like to split them out even more eg:<div class=""><br class=""></div><div class=""><div class="">- Sources</div><div class=""> - Build</div></div><div class=""><br class=""></div><div class="">Build is a module, but really I’d like:</div><div class=""><br class=""></div><div class="">- Sources</div><div class=""> - Build</div><div class=""> - BuildNode</div><div class=""> - BuildDescription</div><div class=""><br class=""></div><div class="">If I do this currently I still will end up with one module: `Build`, this is how the module layout rules that SwiftPM uses work. But instead I’d like Build to be purely an organizational folder and to get two modules *BuildNode* and *BuildDescription*.</div><div class=""><br class=""></div><div class="">I’d like to propose some method to allow further organization, but without ruining our existing rules.</div><div class=""><br class=""></div><div class="">I’d also like to propose tightening up our rules a little.</div><div class=""><br class=""></div><div class="">—————————</div><div class=""><br class=""></div><div class="">One clear way to accomplish the first goal is to extend the rules and say empty folders are not considered modules.</div><div class=""><br class=""></div><div class="">I am unhappy with this however, already there are easy ways to completely transform the build of a package, and the rules we have are somewhat confusing, one can see this in newcomer questions on StackOverflow.</div><div class=""><br class=""></div><div class="">For example:</div><div class=""><br class=""></div><div class="">- Foo</div><div class=""><div class=""> - Sources</div><div class=""> - Bar</div></div><div class=""> - main.swift</div><div class=""><br class=""></div><div class="">Will build a single executable called `Bar`. However if one accidentally creates a new swift file:</div><div class=""><br class=""></div><div class=""><div class="">- Foo</div><div class=""><div class=""> - Sources</div><div class=""> - baz.swift</div><div class=""> - Bar</div></div><div class=""> - main.swift</div></div><div class=""><br class=""></div><div class="">Then you will get a single executable called baz, or more likely a compile error. Diagnosing this is currently an exercise in frustration (though we could improve this with better diagnostics when compiles fail, eg. we could output the modules structure we inferred).</div><div class=""><br class=""></div><div class="">To prevent these transformations we could just tighten up our rules.</div><div class=""><br class=""></div><div class="">If one browses swift packages online there is a clear preference towards a root `Sources` directory.</div><div class=""><br class=""></div><div class="">Thus I propose we only allow</div><div class=""><br class=""></div><div class=""><ul class="MailOutline"><li class="">A Sources directory with sub folders for modules</li><li class="">A Sources directory with sources directly in the Sources directory and NO sub folders</li></ul><div class=""><br class=""></div></div><div class="">Any other options become errors.</div><div class=""><br class=""></div><div class="">If we then extend the same rules to modules then there will be no surprising behavior. At first I resisted this idea as folders are for organization, however in the new world of SwiftPM *modules* are for organization.</div><div class=""><br class=""></div><div class="">Modules have internal accessibility modifiers that allow powerful architectural designs. Modules have a defined public interface that maps onto folder organizational practices better than </div><div class=""><br class=""></div><div class="">However this prohibits eg:</div><div class=""><br class=""></div><div class="">- NetEngine/</div><div class=""> - CommonCode.swift</div><div class=""> - HTTP/</div><div class=""> - foo.swift</div><div class=""> - HTTPS/</div><div class=""> - bar.swift</div><div class=""><br class=""></div><div class="">And perhaps thus is overly restrictive. Though my argument is that HTTP and HTTPS would be better off being their own modules: it would encourage true encapsulation and a better code architecture.</div><div class=""><br class=""></div><div class="">I am unsure and thus am opening a discussion here. Certainly there are other ways to accomplish the goals here so please let us know what you think a better solution (or not) would be. Thanks for your time.</div><div class=""><br class=""></div><div class="">Max</div></div>_______________________________________________<br class="">swift-build-dev mailing list<br class=""><a href="mailto:swift-build-dev@swift.org" class="">swift-build-dev@swift.org</a><br class="">https://lists.swift.org/mailman/listinfo/swift-build-dev<br class=""></div></blockquote></div><br class=""></body></html>