[swift-build-dev] Hard-Coded Paths in glibc.modulemap
Brian Croom
brian.s.croom at gmail.com
Thu Jun 9 11:09:58 CDT 2016
Note that the IBM buildpack works around the environmental restrictions by
configuring apt-get to install packages in subdirectories of the buildpack
sandbox, rather than in the standard system directories. This works great,
but can still cause issues with components that search for things with
absolute paths to the standard directories.
torsdag 9 juni 2016 skrev Daniel Dunbar via swift-build-dev <
swift-build-dev at swift.org>:
>
> On Jun 9, 2016, at 8:35 AM, Shao Miller via swift-build-dev <
> swift-build-dev at swift.org
> <javascript:_e(%7B%7D,'cvml','swift-build-dev at swift.org');>> wrote:
>
> Thanks, but let me try to explain for a third time: The build environment
> provided by CloudFoundry-style providers is locked-down. You have no
> choice where things go. They must all go inside a sandbox. That means you
> cannot put headers underneath the /usr/ directory. There is no way out of
> the sandbox, as there are no root-user privileges.
>
>
> You have stated that, but the IBM build pack works, uses CloudFoundry, and
> it clearly uses apt-get to install things in the system. You haven't
> explained how your environment is different yet.
>
> - Daniel
>
>
> This is why I'm asking about whether or not it's reasonable for Swift to
> avoid hard-coded paths, or to at least allow a given prefix to be specified.
>
> Shao Miller
> Synthetel Corporation
> T: +1.9053927729
> E: swift-build-dev at synthetel.com
> <javascript:_e(%7B%7D,'cvml','swift-build-dev at synthetel.com');>
> W: https://www.synthetel.com
>
> On 6/9/2016 11:31, Daniel Dunbar wrote:
>
> The IBM build pack installs a number of system dependencies which should
> include those headers, though. You should have the headers rooted at /usr,
> not try and have them in the app sandbox.
>
> - Daniel
>
> On Thu, Jun 9, 2016 at 5:32 AM, Brian Croom via swift-build-dev <
> <javascript:_e(%7B%7D,'cvml','swift-build-dev at swift.org');>
> swift-build-dev at swift.org
> <javascript:_e(%7B%7D,'cvml','swift-build-dev at swift.org');>> wrote:
>
>> What underlying OS is your build process running on? And where are you
>> getting your Swift toolchain?
>>
>> Within your toolchain is a `glibc.modulemap` which contains hard-coded
>> absolute paths to certain system headers. The failure you are seeing is
>> presumably due to the fact that system headers are not present at those
>> paths. You may have to manually modify the modulemap to point to the actual
>> location of the headers on the system that is performing the build.
>>
>> This old bug contains some tidbits that may also help you understand
>> what's going on here:
>> <https://bugs.swift.org/plugins/servlet/mobile#issue/SR-15>
>> https://bugs.swift.org/plugins/servlet/mobile#issue/SR-15
>>
>> Brian
>>
>>
>> torsdag 9 juni 2016 skrev Shao Miller <
>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at synthetel.com');>
>> swift-build-dev at synthetel.com
>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at synthetel.com');>>:
>>
>>> Thank you for your kind response.
>>>
>>> The command I execute is: swift build -Xcc -I/app/.delta/ -Xswiftc
>>> -I/app/.delta/ -v
>>>
>>> The /app/.delta/ directory is where Swift and most dependencies have
>>> been dumped. The file /app/.delta/usr/include/complex.h exists. The error
>>> is:
>>>
>>> /app/.delta/usr/lib/swift/linux/x86_64/glibc.modulemap:33:14: error:
>>>> header '///usr/include/complex.h' not found
>>>> header "///usr/include/complex.h"
>>>> ^
>>>> <unknown>:0: error: could not build Objective-C module 'SwiftGlibc'
>>>> error: exit(1): /app/.delta/usr/bin/swiftc --driver-mode=swift -I
>>>> /app/.delta/usr/lib/swift/pm -L /app/.delta/usr/lib/swift/pm
>>>> -lPackageDescription /app/PerfectTemplate/Package.swift -fileno 3
>>>>
>>>
>>> Thank you for pointing out that these build-packs are using 'swift
>>> build'. The two build-packs I'd looked at earlier today did not and I
>>> thought I'd recalled them having derived from the others you've mentioned.
>>> I was mistaken. My command somewhat resembles the IBM Bluemix build-pack
>>> command[1], which is:
>>> swift build --configuration release -Xcc -fblocks -Xcc
>>> -I$BUILD_DIR/.apt/usr/include ...and so on...
>>>
>>> I am using this[2] Swift. Running 'strace' on the BASh and all of its
>>> subprocesses during the compilation yields only these two instances of
>>> "complex":
>>>
>>> [pid 28678] stat("///usr/include/complex.h", 0x7ffe84061ef0) = -1 ENOENT
>>>> (No such file or directory)
>>>> [pid 28679] write(2, "header '///usr/include/complex.h' not found", 43)
>>>> = 43
>>>>
>>>
>>> Am I making an obvious mistake?
>>>
>>> [1]
>>> https://github.com/IBM-Swift/swift-buildpack/blob/bluemix-buildpack/bin/compile#L116
>>> [2]
>>> https://swift.org/builds/development/ubuntu1510/swift-DEVELOPMENT-SNAPSHOT-2016-05-31-a/swift-DEVELOPMENT-SNAPSHOT-2016-05-31-a-ubuntu15.10.tar.gz
>>>
>>> Shao Miller
>>> Synthetel Corporation
>>> T: +1.9053927729 <tel:+1.9053927729>
>>> E: swift-build-dev at synthetel.com
>>> W: https://www.synthetel.com
>>>
>>> On 6/9/2016 01:15, Brian Croom wrote:
>>>
>>>> IBM's buildpack as well as the ones from which it descends (
>>>> https://github.com/cloudfoundry-community/swift-buildpack and
>>>> https://github.com/kylef/heroku-buildpack-swift) are all based around
>>>> SwiftPM and the `swift build` command. I have not personally experienced
>>>> the problem you are describing either, although I have not tried pushing
>>>> any apps using more recent Swift toolchain snapshots.
>>>>
>>>> Can you provide more details about how the error presents itself? Have
>>>> you tried using any of these other buildpacks with your app code?
>>>>
>>>> Brian
>>>>
>>>> onsdag 8 juni 2016 skrev Shao Miller via swift-build-dev <
>>>> swift-build-dev at swift.org <mailto:swift-build-dev at swift.org>>:
>>>>
>>>> Thank you for your kind response.
>>>>
>>>> As mentioned, there is no choice: If the headers aren't present in
>>>> the base image that a particular Cloud provider provides, they can
>>>> only be present in the application sand-box by one's own hand.
>>>>
>>>> All Swift build-packs to date and to my knowledge use Swift 2.2
>>>> and do not use the Swift 3 'swift build' process. I'm trying to
>>>> develop the next generation.
>>>>
>>>> Shao Miller
>>>> Synthetel Corporation
>>>> T: +1.9053927729 <tel:+1.9053927729>
>>>> E: swift-build-dev at synthetel.com
>>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at synthetel.com
>>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at synthetel.com');>');>
>>>> W: https://www.synthetel.com
>>>>
>>>> On 6/8/2016 23:08, Daniel Dunbar wrote:
>>>>
>>>>> Why do you want the headers inside the app sandbox? Usually they
>>>>> would remain outside.
>>>>>
>>>>>
>>>>>
>>>>> Have you looked at IBM's CloudFoundry build pack (
>>>>> https://github.com/IBM-Swift/swift-buildpack)? How does it handle the
>>>>> problem you are trying to solve?
>>>>>
>>>>>
>>>>>
>>>>> - Daniel
>>>>>
>>>>>
>>>>>
>>>>> On Jun 8, 2016, at 8:03 PM, Shao Miller via swift-build-dev<
>>>>>> swift-build-dev at swift.org
>>>>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at swift.org');>>
>>>>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at swift.org
>>>>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at swift.org');>');>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> Good day, Swift package manager development folks.
>>>>>>
>>>>>>
>>>>>>
>>>>>> (There are at least two separate issues being inquired about, but
>>>>>> with the same introductory context.)
>>>>>>
>>>>>>
>>>>>>
>>>>>> "Cloudy" deployment options derived from or akin to CloudFoundry
>>>>>> are agonizingly locked-down environments. Essentially Swift and all of its
>>>>>> dependencies and one's project's dependencies must be stuffed into an
>>>>>> arbitrary directory (henceforth referred to as "the hole," but usually
>>>>>> /app/ ) and build processes performed without any root-user privileges.
>>>>>> One consequence is that one cannot use the OS' package-management system to
>>>>>> install dependencies, but one must obtain them and wrestle them into "the
>>>>>> hole," instead. The strategy seems rather silly.
>>>>>>
>>>>>>
>>>>>>
>>>>>> While developing a so-called "buildpack" for Swift 3 projects to
>>>>>> be deployed via CloudFoundryish options and utilizing the 'swift build'
>>>>>> command, I have come across a few issues.
>>>>>>
>>>>>>
>>>>>>
>>>>>> One issue is that 'swift build' wants to do something with the
>>>>>> /usr/lib/swift/linux/x86_64/glibc.modulemap file, but that file contains a
>>>>>> hard-coded path to a ///usr/include/complex.h header-file. As is usually
>>>>>> the case, this hard-coded path will only work in a narrow set of
>>>>>> environments, which excludes "the hole" that CloudFoundry provides. I have
>>>>>> attempted to use '-Xcc -I/the/hole' and '-Xswiftc -I/the/hole' command-line
>>>>>> arguments, but I do not observe these paths (nor sub-paths) being tried for
>>>>>> the complex.h header-file during complication. I used 'strace' to trace
>>>>>> the compilation process, including all subprocesses.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Is there some other mechanism to instruct the Swift 3 package
>>>>>> manager that its [unfortunately] hard-coded paths are relative to some
>>>>>> particular path? If not, would it be sensible to introduce some logic to
>>>>>> specify such a prefix path?
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thank you for your time and attention.
>>>>>>
>>>>>> --
>>>>>>
>>>>>>
>>>>>> Shao Miller
>>>>>>
>>>>>> Synthetel Corporation
>>>>>>
>>>>>> T: +1.9053927729 <tel:+1.9053927729>
>>>>>>
>>>>>> E:swift-build-dev at synthetel.com
>>>>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at synthetel.com
>>>>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at synthetel.com');>');>
>>>>>>
>>>>>> W:https://www.synthetel.com
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>>
>>>>>> swift-build-dev mailing list
>>>>>>
>>>>>> swift-build-dev at swift.org
>>>>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at swift.org
>>>>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at swift.org');>');>
>>>>>>
>>>>>> https://lists.swift.org/mailman/listinfo/swift-build-dev
>>>>>>
>>>>>>
>>>>
>>>
>> _______________________________________________
>> swift-build-dev mailing list
>> swift-build-dev at swift.org
>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at swift.org');>
>> https://lists.swift.org/mailman/listinfo/swift-build-dev
>>
>>
>
> _______________________________________________
> swift-build-dev mailing list
> swift-build-dev at swift.org
> <javascript:_e(%7B%7D,'cvml','swift-build-dev at swift.org');>
> https://lists.swift.org/mailman/listinfo/swift-build-dev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-build-dev/attachments/20160609/8cf6f0c0/attachment.html>
More information about the swift-build-dev
mailing list