[swift-evolution] Draft Proposal SwiftPM System Module Search Paths

[Colin Barrett]

Hi Max,

Thanks for sending out this proposal. I admit I’m somewhat ignorant of the current state of SwiftPM so feel free to point me at existing resources if I’m missing basic things.

You don’t cover it in your proposal, but does this mean pkg-config is now a dependency for SwiftPM? I may be mistaken but it doesn’t seem as though pkg-config is installed on the base OS X system. (This leads me to assume it’s an optional dependency.)

Your proposal suggests the set of supported package mangers will be closed. What about folks using new, or non-standard package managers? What about packages that are installed system-wide without a package manager (e.g. with ./configure && make && make install)? 

This may be covered elsewhere, but about the case of a C package that is normally system-wide but is currently being hacked on by the user? (For example to while debugging it, or writing bindings for it.)

-Colin

> On Mar 23, 2016, at 2:10 PM, Max Howell via swift-evolution <swift-evolution at swift.org> wrote:
> 
> SwiftPM System Module Search Paths
> Proposal: SE-NNNN <https://github.com/apple/swift-evolution/blob/master/proposals/NNNN-swiftpm-system-module-search-paths.md>
> Author: Max Howell <https://github.com/mxcl>
> Status: Awaiting review
> Review manager: Anders Bertelrud
>  <>Introduction
> Swift is able to import C libraries in the same manner as Swift libraries.
> 
> For this to occur the library must be represented by a clang module-map file.
> 
> The current system for using these module-map files with SwiftPM works, but with a number of caveats that must be addressed.
> 
>  <>Motivation
> The current implementation of system module packages have a number of problems:
> 
> Install locations vary across platforms and modulemap files require absolute paths
> /usr/lib:/usr/local/lib is not always a sufficient -L search path
> /usr/include:/usr/local/include is not always a sufficient -I C compiler search path
> Installing the system library is left up to the end-user to figure out
> For example to import a module map representing the GTK library, the include search path must be supplemented with -I/usr/include/gtk so that a number of includes in the gtk.h header can be sourced for the complete modular definition of GTK.
> 
> For example to import a module map representing the GTK library a user must first have a copy of GTK and its headers installed. On Debian based systems the install name for this system package is libgtk-3-0-dev which is not entirely intuitive.
> 
> For example, Homebrew and MacPorts on OS X install to prefixes other than /usr..modulemap files must specify headers with absolute paths. The standard we encourage with modulemaps is for the headers to be specified with an assumed prefix of /usr, but you will not find eg. jpeglib.h at /usr/include/jpeglib.h if it is installed with Homebrew or MacPorts.
> 
>  <>Proposed Solution
> We propose that SwiftPM gains the ability to use the cross-platform pkg-config tool so that it can query pkg-config for the missing path and flag arguments.
> 
> We propose that SwiftPM gains the ability to use the cross-platform pkg-config tool to identify when the system package is not installed to a /usr and in such a case preprocess the modulemap changing the prefix it uses.
> 
> We propose that Package.swift is supplemented with metadata that provides the package-install-name for specific platforms.
> 
>  <>Detailed Design
>  <>Solving Path/Flags Issues
> Some of our problems can be solved by using the cross platform tool: pkg-config.
> 
> A C package can provide a pkg-config file (.pc) which describes:
> 
> Its install location
> Supplementary C-flags that should be used when building against this library
> If SwiftPM used the .pc file that comes with packages, this solves problems 1 through 3.
> 
> Of the tickets we currently have open describing issues using Swift-system-module-packages, reading the .pc file would fix all of them.
> 
> It is a convention to name the .pc file after the library link-name, so we can determine which .pc file to ask pkg-config for by parsing the .modulemap file in the Swift package.
> 
>  <>Providing Package Install Names
> Package.swift would be supplemented like so:
> 
> let package = Package(
>     name: "CFoo",
>     providers: .Brew(installName: "foo"),
>                 .Apt(installName: "libfoo-dev"),
> )
> Thus, in the event of build failure for modules that depend on this package we provide additional help to the user:
> 
> error: failed to build module `bar'
> note: you may need to install `foo' using your system-packager:
> 
>     apt-get install libfoo-dev
> Since the syntax to provide this information uses an explicit enum we can add code for each enum to detect which system packagers should be recommended. The community will need to write the code for their own platforms. It also means that if a specific packager requires additional parameters, they can be added on a per enum basis.
> 
>  <>Impact on Existing Code
> There will be no impact on existing code as this feature simply improves an existing feature making new code possible.
> 
>  <>Alternatives Considered
> A clear alternative is allowing additional flags to be specified in a system-module package’s Package.swift.
> 
> However since these paths and flags will vary by platform this would because a large matrix that is quite a maintenance burden. Really this information is recorded already, in the system package itself, and in fact almost all packages nowadays provide it in a .pc pkg-config file.
> 
> Also we do not want to allow arbitrary flags to be specified in Package.swift, this allows packages too much power to break a large dependency graph with bad compiles. The only entity that understands the whole graph and can manage the build without breakage is SwiftPM, and allowing packages themselves to add arbitrary flags prevents SwiftPM from being able to understand and control the build ensuring reliability and preventing “Dependency Hell”.
> 
>  <>Future Directions
> The build system could be made more reliable by having the specific packager provide the information that this proposal garners from pkg-config. For example, Homebrew installs everything into independent directories, using these directories instead of more general POSIX search paths means there is no danger of edge-case search path collisions and the wrong libraries being picked up.
> 
> If this was done pkg-config could become just one option for providing this data, and be used only as a fallback.
> 
> We could add an additional flag so that deployment of Swift Packages could be made simpler and system dependencies be installed automatically.
> 
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20160324/d3a4592d/attachment.html>