[swift-build-dev] [swift-users] Importing C system libraries

Kelvin Ma kelvinsthirteen at gmail.com
Wed Mar 29 13:55:49 CDT 2017 

I agree that portability is valuable and that it’s something that’s lacking
in the c/makefile workflow. I just don’t think empty git repositories are
the right solution. Perhaps we can get the best of both worlds with
something like this in the SwiftPM:


    .Package(include: "cairo.h", link: "cairo" version: "1.1.4", remote: "
git://anongit.freedesktop.org/git/cairo")

It would search for the cairo lib on local machine in /usr/lib and
/usr/local/lib , and if it can’t find it, or if the version doesn’t match,
it will then download the C sources from the Cairo project’s *official* git
repository and build it for you. Options could be added to specify
alternative search paths or user prompts.

On Wed, Mar 29, 2017 at 12:42 PM, Ankit Aggarwal <ankit_aggarwal at apple.com>
wrote:

> I agree that libressl isn't a good example because of the security
> questions it raises, but libYAML is a good self contained package IMO. With
> custom targets layout proposal, the forks can be replaced by submodules.
> The problem with apt-get approach is, it takes away the "portability" from
> the package and it doesn't work in cases when you don't have full access to
> the system. That said, we do support this approach upto a certain degree.
> For e.g., swiftpm allows system package authors to declare hints for
> installing a system package dependency (via brew or apt-get). The
> /usr/include, /usr/lib etc are searched by default and non-standard paths
> are supported using pkg config. See: https://github.com/apple/
> swift-package-manager/blob/master/Documentation/Usage.md#
> require-system-libraries
>
> On 29-Mar-2017, at 10:55 PM, Kelvin Ma <kelvinsthirteen at gmail.com> wrote:
>
> I don’t think this is a good approach, the libressl
> <https://github.com/vapor/clibressl> repo is pretty much just the source
> code of the C library copied and pasted into a Swift module. That’s not a
> good thing™. The linux build paradigm is, the library maintains its own
> *official* source repository, the OS package distributors build it, users
> install the built dependencies with `sudo apt-get install libwhatever-dev`
> and the project source just builds and links to the library managed by the
> system.
>
> Here you have to keep the forked library source up to date with the real
> library source, download it from the internet and build the entire project
> plus the libraries of which there could be many.
>
> IMO the ideal way to import system libs would be for the user to install
> the dependency with `apt`, just as you would for any C program, have the
> `include` statements within the project source (like you would for any C
> program), and then have the paths to `/usr/include` and `/usr/lib` and
> `/usr/local/include` etc in a Makefile (or Package.swift). Usually it’s
> only “specialty” libraries like libspiro that people download and build
> manually, and even then, it’s downloaded from the Spiro project’s own
> official repository, not a third party fork. That’s the “accepted” way to
> do things, that linux ecosystems are designed around. Of course, this is
> very similar to the modulemap system that currently works in Swift. I just
> wish modulemaps could be streamlined a little, maybe combined with the top
> level Package.swift.
>
> On Wed, Mar 29, 2017 at 1:03 AM, Ankit Aggarwal <ankit_aggarwal at apple.com>
> wrote:
>
>>
>> On 29-Mar-2017, at 11:22 AM, kelvinsthirteen at gmail.com wrote:
>>
>> I figured that was the intention, but we can’t be too surprised that
>> everyone is maintaining personal modulemap repositories (and polluting
>> search results — just try googling for a Swift PNG library!), especially
>> when this central repo still doesn’t exist yet.
>>
>>
>> Yeah thats unfortunate, maybe this will improve once we have an index.
>>
>> If Swift ever comes on par with C in terms of being usage and the lingua
>> franca of the FOSS world, I can see linux distributions shipping standard
>> modulemaps the same way they ship C headers, but you have to admit this is
>> years (decades?) away at best.
>>
>> On the flip side of it, it does wonders in terms of motivating people (me
>> at least) to start writing pure Swift replacements for some of these C
>> libraries (like libpng)…
>>
>>
>> I think its better to reuse existing libraries than writing from scratch
>> (unless really needed). A good approach that works is "porting" these
>> libraries to build with SwiftPM, see: libYAML
>> <https://github.com/jpsim/Yams>, libressl
>> <https://github.com/vapor/clibressl>. However, this porting can be
>> difficult to do right but it should become much easier once we have build
>> settings support and custom targets layout
>> <https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20170320/034469.html>
>> .
>>
>>
>> On Mar 29, 2017, at 12:42 AM, Ankit Aggarwal <ankit_aggarwal at apple.com>
>> wrote:
>>
>> I think the idea was that there will be one such repository which other
>> packages can use, that too only until the system libraries start shipping
>> their standard modulemap. I thought we had this written down in our
>> documentation somewhere but couldn't find it.
>>
>> Maybe Daniel can expand on this.
>>
>> On Wed, Mar 29, 2017 at 10:48 AM, Kelvin Ma via swift-build-dev <
>> swift-build-dev at swift.org> wrote:
>>
>>> This worked! Thanks! But why is having empty git repositories strewn
>>> about the “correct” way? System libraries should be imported from within
>>> the project, as they are in C. You have to admit it’s getting quite silly
>>> that Swift devs keep repositories like these
>>> <https://github.com/kelvin13/swift-zlib> on our github accounts. That
>>> zlib repository contains exactly ten lines of code. I used to have 6 or 7
>>> repos like that one up there before I got rid of them and switched to local
>>> repos.
>>>
>>> On Wed, Mar 29, 2017 at 12:03 AM, Ankit Aggarwal <
>>> ankit_aggarwal at apple.com> wrote:
>>>
>>>> In this case, these are just umbrella headers. If your modulemap
>>>> contains absolute path to the header, then you don't need the header files,
>>>> but SwiftPM will probably warn about this. Note that this is a "hack" to
>>>> have system packages inside a single repository. The correct way is to have
>>>> system package as a separate published package which you only need to do
>>>> once.
>>>>
>>>> On 29-Mar-2017, at 10:26 AM, Kelvin Ma <kelvinsthirteen at gmail.com>
>>>> wrote:
>>>>
>>>> I will try this, but why are the header files inside the Sources
>>>> directory? System headers should live in /usr/include…
>>>>
>>>> On Tue, Mar 28, 2017 at 11:48 PM, Ankit Aggarwal <
>>>> ankit_aggarwal at apple.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Apologies for not replying to this earlier.
>>>>>
>>>>> You can have multiple targets in a single package. Each target can
>>>>> either be Swift or C-family. The type of target is determined by the
>>>>> sources contained in it (*.c/*.cpp etc means C target, *.swift means Swift
>>>>> target). So if you want to create multiple C targets, this layout should
>>>>> work:
>>>>>
>>>>> Package.swift
>>>>> Sources/
>>>>>     Bitmap
>>>>>     Cubify
>>>>>     Cairo/anchor.c <---- This is just an empty file to tell SwiftPM
>>>>> that this is a C target.
>>>>>     Cairo/include/Cairo.h
>>>>>     Cairo/include/module.modulemap
>>>>>     GLFW/anchor.c
>>>>>     GLFW/include/GLFW.h
>>>>>     GLFW/include/module.modulemap
>>>>>
>>>>> The modulemap is automatically generated, if not provided. This is a
>>>>> package which contains two targets (one C and one Swift):
>>>>> https://github.com/jpsim/Yams
>>>>>
>>>>> If you need to pass a bunch of compiler flags, you can use SwiftPM's
>>>>> pkgConfig feature but that will require you to have a separate repository
>>>>> for Cario and GLFW. You can experiment without creating tags using the edit
>>>>> feature
>>>>> <https://github.com/apple/swift-package-manager/blob/master/Documentation/Usage.md#editable-packages>
>>>>> .
>>>>>
>>>>> PS: You can join SwiftPM slack channel for quicker turn around time:
>>>>> https://lists.swift.org/pipermail/swift-build-dev/Week
>>>>> -of-Mon-20160530/000497.html
>>>>>
>>>>> Thanks,
>>>>> Ankit
>>>>>
>>>>>
>>>>> On Wed, Mar 29, 2017 at 6:06 AM, Michael Ilseman via swift-build-dev <
>>>>> swift-build-dev at swift.org> wrote:
>>>>>
>>>>>> This is into uncharted territory for me, but it seems you’re building
>>>>>> with SwiftPM. You’ll probably want to configure extra compiler flags if
>>>>>> that’s possible. You could also bite the bullet and build your C libraries
>>>>>> with SwiftPM as well. Hopefully someone on swift-build-dev can help you out.
>>>>>>
>>>>>> CC-ing Ankit
>>>>>>
>>>>>>
>>>>>> On Mar 28, 2017, at 5:09 PM, Kelvin Ma <kelvinsthirteen at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> How do I compile a project with many modules? My tree looks like this:
>>>>>>
>>>>>> <Selection_001.png>
>>>>>> ​
>>>>>>
>>>>>> On Tue, Mar 28, 2017 at 12:47 PM, Michael Ilseman <milseman at apple.com
>>>>>> > wrote:
>>>>>>
>>>>>>> Sure! In this example, I have built libgit2. I have a directory
>>>>>>> called Git, and inside that I have the following module map:
>>>>>>>
>>>>>>> module Git [system] {
>>>>>>>        header "<my path>/libgit2/include/git2.h"
>>>>>>>        export *
>>>>>>> }
>>>>>>>
>>>>>>> When I run, I use:
>>>>>>>
>>>>>>> swift -I <path-to-“Git”-directory> -L <path-to-built-libgit2> -lgit2
>>>>>>> foo.swift
>>>>>>>
>>>>>>> inside foo.swift I can:
>>>>>>>
>>>>>>> import Git
>>>>>>> // … use libGit2
>>>>>>>
>>>>>>>
>>>>>>> Read more about how to write a more appropriate module.map file for
>>>>>>> your purposes at https://clang.llvm.org/docs/Modules.html. For
>>>>>>> example, you might be able to define link flags inside the module.map, use
>>>>>>> umbrella directories, submodules, etc.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mar 28, 2017, at 6:27 AM, Kelvin Ma <kelvinsthirteen at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Can you give an example?
>>>>>>>
>>>>>>> On Mon, Mar 27, 2017 at 3:59 PM, Michael Ilseman <milseman at apple.com
>>>>>>> > wrote:
>>>>>>>
>>>>>>>> Sure. At a low level, you can create a module.map file and use
>>>>>>>> -L/-l flags in your invocation of Swift. If you want to do so at a higher
>>>>>>>> level, then perhaps SwiftPM can. CCing swift-build-dev for the SwiftPM part.
>>>>>>>>
>>>>>>>>
>>>>>>>> > On Mar 26, 2017, at 3:20 PM, Kelvin Ma via swift-users <
>>>>>>>> swift-users at swift.org> wrote:
>>>>>>>> >
>>>>>>>> > Idk if this has been asked before, but is there a way to import C
>>>>>>>> libraries into a Swift project without creating a local git repo?
>>>>>>>> Preferably something similar to C where you can just `#include` headers and
>>>>>>>> then specify the link flags (in Package.swift?)
>>>>>>>> >
>>>>>>>> > It’s getting very cumbersome to make a bunch of empty git repos
>>>>>>>> just to use libglfw or libcairo.
>>>>>>>> > _______________________________________________
>>>>>>>> > swift-users mailing list
>>>>>>>> > swift-users at swift.org
>>>>>>>> > https://lists.swift.org/mailman/listinfo/swift-users
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> swift-build-dev mailing list
>>>>>> swift-build-dev at swift.org
>>>>>> https://lists.swift.org/mailman/listinfo/swift-build-dev
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>> _______________________________________________
>>> swift-build-dev mailing list
>>> swift-build-dev at swift.org
>>> https://lists.swift.org/mailman/listinfo/swift-build-dev
>>>
>>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-build-dev/attachments/20170329/53aa51d8/attachment.html>

More information about the swift-build-dev mailing list