[swift-evolution] Draft Proposal SwiftPM System Module Search Paths

[Ankit Agarwal]

👍

On Friday 1 April 2016, Max Howell <max.howell at apple.com> wrote:

> I have updated the proposal with everyone’s feedback:
>
> SwiftPM System Module Search Paths
>
>    - Proposal: SE-NNNN
>    <https://github.com/apple/swift-evolution/blob/master/proposals/NNNN-swiftpm-system-module-search-paths.md>
>    - Author: Max Howell <https://github.com/mxcl>
>    - Status: *Awaiting review*
>    - Review manager: Anders Bertelrud
>
>
> <https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#introduction>
> Introduction
>
> Swift is able to import C libraries in the same manner as Swift libraries.
>
> For this to occur the library must be represented by a clang module-map
> file.
>
> The current system for using these module-map files with SwiftPM works,
> but with a number of caveats that must be addressed.
>
> <https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#motivation>
> Motivation
>
> The current implementation of system module packages have a number of
> problems:
>
>    1. Install locations vary across platforms and modulemap files require
>    absolute paths
>    2. /usr/lib:/usr/local/lib is not always a sufficient -L search path
>    3. /usr/include:/usr/local/include is not always a sufficient -I C
>    compiler search path
>    4. Installing the system library is left up to the end-user to figure
>    out
>
> For example to import a module map representing the GTK library, the
> include search path must be supplemented with -I/usr/include/gtk so that
> a number of includes in the gtk.h header can be sourced for the complete
> modular definition of GTK.
>
> For example to import a module map representing the GTK library a user
> must first have a copy of GTK and its headers installed. On Debian based
> systems the install name for this system package is libgtk-3-0-dev which
> is not entirely intuitive.
>
> For example, Homebrew and MacPorts on OS X install to prefixes other than
> /usr. .modulemap files must specify headers with absolute paths. The
> standard we encourage with modulemaps is for the headers to be specified
> with an assumed prefix of /usr, but you will not find eg. jpeglib.h at
> /usr/include/jpeglib.h if it is installed with Homebrew or MacPorts.
>
> <https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#proposed-solution>Proposed
> Solution
>
> We propose that SwiftPM gains the ability to use the cross-platform
> pkg-config tool so that it can query pkg-config for the missing path and
> flag arguments.
>
> We propose that SwiftPM gains the ability to use the cross-platform
> pkg-config tool to identify when the system package is not installed to a
> /usr and in such a case preprocess the modulemap changing the prefix it
> uses.
>
> We propose that Package.swift is supplemented with metadata that provides
> the package-install-name for specific platforms.
>
> <https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#detailed-design>Detailed
> Design
> <https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#solving-pathflags-issues>Solving
> Path/Flags Issues
>
> Some of our problems can be solved by using the cross platform tool:
> pkg-config.
>
> A C package can provide a pkg-config file (.pc) which describes:
>
>    1. Its install location
>    2. Supplementary C-flags that should be used when compiling against
>    this library
>    3. Supplementary C-flags that should be used when linking against this
>    library
>
> If SwiftPM used the .pc file that comes with packages, this solves
> problems 1 through 3.
>
> Of the tickets we currently have open describing issues using
> Swift-system-module-packages, reading the .pc file would fix all of them.
>
> It is a convention to name the .pc file after the library link-name, so
> we can determine which .pc file to ask pkg-configfor by parsing the
> .modulemap file in the Swift package. However sometimes this is not true,
> (eg. GTK-3 on Ubuntu), so we will make it possible to specify the .pc file
> name in Package.swift.
>
> pkg-config is not currently a dependency of the Swift toolchain, and thus
> to avoid depending on it we will schedule work to interpret .pc files
> without requiring pkg-config to be installed. The file format for .pc files
> is simple and standard so despite reinventing the wheel, this is a low risk
> choice.
>
> <https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#providing-package-install-names>Providing
> Package Install Names
>
> Package.swift would be supplemented like so:
>
> let package = Package(
>     name: "CFoo",
>     providers: .Brew(installName: "foo"),
>                 .Apt(installName: "libfoo-dev"),
>           .PkgConfig("foo.pc"),
> )
>
> Thus, in the event of build failure for modules that depend on this
> package we provide additional help to the user:
>
> error: failed to build module `bar'
> note: you may need to install `foo' using your system-packager:
>
>     apt-get install libfoo-dev
>
> Since the syntax to provide this information uses an explicit enum we can
> add code for each enum to detect which system packagers should be
> recommended. The community will need to write the code for their own
> platforms. It also means that if a specific packager requires additional
> parameters, they can be added on a per enum basis.
>
> <https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#install-names-are-not-standard>Install-names
> are not standard
>
> apt is used across multiple distirbutions and the install-names for tools
> vary. Even for the same distribution install-names may vary across releases
> (eg. from Ubuntu 15.04 to Ubuntu 15.10) or even on ocassion at finer
> granularity.
>
> We will not add explicit handling for this, but one can imagine the enums
> for different system packagers could be supplemented in a backwards
> compatible way to provide specific handling as real-world uses emerge, eg:
>
> case Apt(installName: String)
> // …could be adapted to:
> struct Debian: Linux {}struct Ubuntu: Debian {
>     enum Variant {
>         case Gubuntu
>         case Kubuntu(Version)
>     }
>     enum Version {
>         case v1510
>         case v1504
>     }
> }case Apt(installName: String, distribution: Linux? = nil)
>
>
> <https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#impact-on-existing-code>Impact
> on Existing Code
>
> There will be no impact on existing code as this feature simply improves
> an existing feature making new code possible.
>
> <https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#alternatives-considered>Alternatives
> Considered
>
> A clear alternative is allowing additional flags to be specified in a
> system-module package’s Package.swift.
>
> However since these paths and flags will vary by platform this would
> because a large matrix that is quite a maintenance burden. Really this
> information is recorded already, in the system package itself, and in fact
> almost all packages nowadays provide it in a .pc pkg-config file.
>
> Also we do not want to allow arbitrary flags to be specified in
> Package.swift, this allows packages too much power to break a large
> dependency graph with bad compiles. The only entity that understands the
> whole graph and can manage the build without breakage is SwiftPM, and
> allowing packages themselves to add arbitrary flags prevents SwiftPM from
> being able to understand and control the build ensuring reliability and
> preventing “Dependency Hell”.
>
> <https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#unsolved-problems>Unsolved
> Problems
>
> Some (usually more legacy) C libraries do not provide .pc files instead
> they may provide a tool named eg. foo-configthat can be queried for
> compile and link flags. We do not yet support these tools, and would prefer
> to take a wait and see approach to determine how important supporting them
> may be.
>
> Some libraries on OS X do not come with .pc files. Again we'd like to see
> which libraries are affected before potentially offering a solution here.
>
> <https://github.com/mxcl/swift-evolution/blob/system-module-search-paths/proposals/NNNN-swiftpm-system-module-search-paths.md#future-directions>Future
> Directions
>
> The build system could be made more reliable by having the specific
> packager provide the information that this proposal garners from
> pkg-config. For example, Homebrew installs everything into independent
> directories, using these directories instead of more general POSIX search
> paths means there is no danger of edge-case search path collisions and the
> wrong libraries being picked up.
>
> If this was done pkg-config could become just one option for providing
> this data, and be used only as a fallback.
> ------------------------------
>
> We do not wish to provide a flag to automatically install dependencies via
> the system packager. We feel this opens us up to security implications
> beyond the scope of this tool.
> Instead we can provide JSON output that can be parsed and executed by some
> other tooling developed outside of Apple.
>


-- 
Ankit
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20160401/1c5b8cb5/attachment.html>