[swift-evolution] [Pitch] Let's talk about submodules

[Brent Royal-Gordon]

> On Feb 20, 2017, at 7:59 PM, Matthew Johnson <matthew at anandabits.com> wrote:
> 
>> Any group of Swift files can be grouped together to form a submodule.
> 
> It isn't clear to me how you express that a particular group of files forms a submodule.  Can you elaborate?

I discuss this later, but maybe don't describe it that clearly. Basically, submodule membership is not directly expressed in the source code; rather, it's a function of how the build system invokes the compiler.

To understand what I'm suggesting, you must first understand how the Swift compiler is invoked to build code today. (The following is very simplified—there are some steps I've left out—but it's basically accurate.) Suppose your app, `PersonTalk`, has four files:

	AppDelegate.swift
	RootViewController.swift
	Person.swift
	Message.swift

When Xcode or SwiftPM or whatever goes to compile your app, the build system will invoke the compiler once for each file, passing *all* of the files but specifying only one as primary:

	swiftc -emit-object -module-name PersonTalk -primary-file AppDelegate.swift RootViewController.swift Person.swift Message.swift ...
	swiftc -emit-object -module-name PersonTalk AppDelegate.swift -primary-file RootViewController.swift Person.swift Message.swift ...
	swiftc -emit-object -module-name PersonTalk AppDelegate.swift RootViewController.swift -primary-file Person.swift Message.swift ...
	swiftc -emit-object -module-name PersonTalk AppDelegate.swift RootViewController.swift Person.swift -primary-file Message.swift ...

Each of these compiler invocations compiles its primary file into a .o file. The build system then asks Clang to link them all together, and you get a fully built module of some sort:

	clang -o PersonTalk AppDelegate.o RootViewController.o Person.o Message.o …

In my proposal, if you wanted Person.swift and Message.swift to be in a `PersonTalk.Model` submodule, you would do that by invoking the build tools in a slightly different way. First you would compile the two files in the submodule with one another:

	swiftc -emit-object -module-name PersonTalk.Model -primary-file Person.swift Message.swift ...
	swiftc -emit-object -module-name PersonTalk.Model Person.swift -primary-file Message.swift ...

Then you would emit a single PersonTalk.Model.o file encompassing both of them:

	swiftc -emit-submodule -module-name PersonTalk.Model Person.o Message.o ...

Then you would compile the other two files, providing the PersonTalk.Model.o file as context:

	swiftc -emit-object -module-name PersonTalk -primary-file AppDelegate.swift RootViewController.swift -with-submodule PersonTalk.Model.o ...
	swiftc -emit-object -module-name PersonTalk AppDelegate.swift -primary-file RootViewController.swift -with-submodule PersonTalk.Model.o ...

And finally, you would link everything together:

	clang -o PersonTalk AppDelegate.o RootViewController.o PersonTalk.Model.o …

So whether files belong to a submodule or the main module is not a property of the files themselves, but rather of how the build system chooses to combine them during compilation. That leaves the build system free to specify submodules in a way appropriate to its design.

I assume that Xcode would group files into submodules with per-file or per-target metadata. For instance, there might be a "Submodule" field in the File inspector of each Swift source file, or an additional tab in the Project editor. Upon building, it would examine all its metadata, create a list of submodules and files in each one, examine each submodule to determine its dependencies, order the submodules so that each one's dependencies will be available, and invoke the compiler dance properly to build everything.

SwiftPM, which relies mainly on the directory structure to organize your code, would probably have each submodule be a subdirectory of the Sources directory, using the presence of dots in the filename to match submodules:

	Package.swift
	Sources/
		PersonTalk/
			AppDelegate.swift
			RootViewController.swift
		PersonTalk.Model/
			Person.swift
			Message.swift
		TalkKit/
			Talker.swift
			TalkUser.swift
			TalkMessage.swift

Or perhaps it would use nested subdirectories to represent the submodules:

	Package.swift
	Sources/
		PersonTalk/
			AppDelegate.swift
			RootViewController.swift
			Model/
				Person.swift
				Message.swift
		TalkKit/
			Talker.swift
			TalkUser.swift
			TalkMessage.swift

In either case, I imagine that SwiftPM would try to build `PersonTalk`, detect the error indicating that the `PersonTalk.Model` submodule was not available, recurse in order to build `PersonTalk.Model`, and then try to build `PersonTalk` again.

Other build systems would do whatever made sense for them. If you were using `make`, for instance, you'd probably have either a target for each submodule, or a directory and `Makefile` for each submodule. If you used Ant, you'd write some sort of XML horror. The point is, specifying which files belong to which submodules is a build system responsibility, not a language responsibility. This keeps source code portable, avoids boilerplate, and keeps submodule information from being duplicated (and thus potentially becoming inconsistent) in file system structure or build system metadata.

Hope this helps,
-- 
Brent Royal-Gordon
Architechies