[swift-build-dev] Hard-Coded Paths in glibc.modulemap
Shao Miller
swift-build-dev at synthetel.com
Thu Jun 9 14:25:12 CDT 2016
Thank you. The 'swiftc' command doesn't appear to care about the
'--sysroot' parameter with respect to header-paths. 'strace' yields
only examination of the absolute paths for the headers.
Shao Miller
Synthetel Corporation
T: +1.9053927729 <tel:+1.9053927729>
E: swift-build-dev at synthetel.com
W: https://www.synthetel.com
On 6/9/2016 12:13, Daniel Dunbar wrote:
> Ah, thanks! That was the missing piece of info and I missed that in
> the script.
>
> If you truly need to install a full set of headers in a sandboxed
> location, then the compiler arguments to use to point at that location
> are the `--sysroot` ones for Clang. I haven't tried this myself, so
> YMMV, but something like `-Xcc --sysroot -Xcc /app/.delta` *might* work.
>
> - Daniel
>
>> On Jun 9, 2016, at 9:09 AM, Brian Croom <brian.s.croom at gmail.com
>> <mailto:brian.s.croom at gmail.com>> wrote:
>>
>> Note that the IBM buildpack works around the environmental
>> restrictions by configuring apt-get to install packages in
>> subdirectories of the buildpack sandbox, rather than in the standard
>> system directories. This works great, but can still cause issues with
>> components that search for things with absolute paths to the standard
>> directories.
>>
>> torsdag 9 juni 2016 skrev Daniel Dunbar via swift-build-dev
>> <swift-build-dev at swift.org <mailto:swift-build-dev at swift.org>>:
>>
>>
>>> On Jun 9, 2016, at 8:35 AM, Shao Miller via swift-build-dev
>>> <swift-build-dev at swift.org
>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at swift.org');>> wrote:
>>>
>>> Thanks, but let me try to explain for a third time: The build
>>> environment provided by CloudFoundry-style providers is
>>> locked-down. You have no choice where things go. They must all
>>> go inside a sandbox. That means you cannot put headers
>>> underneath the /usr/ directory. There is no way out of the
>>> sandbox, as there are no root-user privileges.
>>
>> You have stated that, but the IBM build pack works, uses
>> CloudFoundry, and it clearly uses apt-get to install things in
>> the system. You haven't explained how your environment is
>> different yet.
>>
>> - Daniel
>>
>>>
>>> This is why I'm asking about whether or not it's reasonable for
>>> Swift to avoid hard-coded paths, or to at least allow a given
>>> prefix to be specified.
>>>
>>> Shao Miller
>>> Synthetel Corporation
>>> T: +1.9053927729 <tel:+1.9053927729>
>>> E: swift-build-dev at synthetel.com
>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at synthetel.com');>
>>> W: https://www.synthetel.com <https://www.synthetel.com/>
>>>
>>> On 6/9/2016 11:31, Daniel Dunbar wrote:
>>>> The IBM build pack installs a number of system dependencies
>>>> which should include those headers, though. You should have the
>>>> headers rooted at /usr, not try and have them in the app sandbox.
>>>>
>>>> - Daniel
>>>>
>>>> On Thu, Jun 9, 2016 at 5:32 AM, Brian Croom via swift-build-dev
>>>> <swift-build-dev at swift.org
>>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at swift.org');>> wrote:
>>>>
>>>> What underlying OS is your build process running on? And
>>>> where are you getting your Swift toolchain?
>>>>
>>>> Within your toolchain is a `glibc.modulemap` which contains
>>>> hard-coded absolute paths to certain system headers. The
>>>> failure you are seeing is presumably due to the fact that
>>>> system headers are not present at those paths. You may have
>>>> to manually modify the modulemap to point to the actual
>>>> location of the headers on the system that is performing
>>>> the build.
>>>>
>>>> This old bug contains some tidbits that may also help you
>>>> understand what's going on here:
>>>> https://bugs.swift.org/plugins/servlet/mobile#issue/SR-15
>>>>
>>>> Brian
>>>>
>>>>
>>>> torsdag 9 juni 2016 skrev Shao Miller
>>>> <swift-build-dev at synthetel.com
>>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at synthetel.com');>>:
>>>>
>>>> Thank you for your kind response.
>>>>
>>>> The command I execute is: swift build -Xcc
>>>> -I/app/.delta/ -Xswiftc -I/app/.delta/ -v
>>>>
>>>> The /app/.delta/ directory is where Swift and most
>>>> dependencies have been dumped. The file
>>>> /app/.delta/usr/include/complex.h exists. The error is:
>>>>
>>>> /app/.delta/usr/lib/swift/linux/x86_64/glibc.modulemap:33:14:
>>>> error: header '///usr/include/complex.h' not found
>>>> header "///usr/include/complex.h"
>>>> ^
>>>> <unknown>:0: error: could not build Objective-C
>>>> module 'SwiftGlibc'
>>>> error: exit(1): /app/.delta/usr/bin/swiftc
>>>> --driver-mode=swift -I /app/.delta/usr/lib/swift/pm
>>>> -L /app/.delta/usr/lib/swift/pm
>>>> -lPackageDescription
>>>> /app/PerfectTemplate/Package.swift -fileno 3
>>>>
>>>>
>>>> Thank you for pointing out that these build-packs are
>>>> using 'swift build'. The two build-packs I'd looked at
>>>> earlier today did not and I thought I'd recalled them
>>>> having derived from the others you've mentioned. I was
>>>> mistaken. My command somewhat resembles the IBM
>>>> Bluemix build-pack command[1], which is:
>>>> swift build --configuration release -Xcc -fblocks -Xcc
>>>> -I$BUILD_DIR/.apt/usr/include ...and so on...
>>>>
>>>> I am using this[2] Swift. Running 'strace' on the BASh
>>>> and all of its subprocesses during the compilation
>>>> yields only these two instances of "complex":
>>>>
>>>> [pid 28678] stat("///usr/include/complex.h",
>>>> 0x7ffe84061ef0) = -1 ENOENT (No such file or directory)
>>>> [pid 28679] write(2, "header
>>>> '///usr/include/complex.h' not found", 43) = 43
>>>>
>>>>
>>>> Am I making an obvious mistake?
>>>>
>>>> [1]
>>>> https://github.com/IBM-Swift/swift-buildpack/blob/bluemix-buildpack/bin/compile#L116
>>>> [2]
>>>> https://swift.org/builds/development/ubuntu1510/swift-DEVELOPMENT-SNAPSHOT-2016-05-31-a/swift-DEVELOPMENT-SNAPSHOT-2016-05-31-a-ubuntu15.10.tar.gz
>>>>
>>>> Shao Miller
>>>> Synthetel Corporation
>>>> T: +1.9053927729 <tel:%2B1.9053927729>
>>>> <tel:+1.9053927729 <tel:%2B1.9053927729>>
>>>> E: swift-build-dev at synthetel.com
>>>> W: https://www.synthetel.com <https://www.synthetel.com/>
>>>>
>>>> On 6/9/2016 01:15, Brian Croom wrote:
>>>>
>>>> IBM's buildpack as well as the ones from which it
>>>> descends
>>>> (https://github.com/cloudfoundry-community/swift-buildpack
>>>> and
>>>> https://github.com/kylef/heroku-buildpack-swift)
>>>> are all based around SwiftPM and the `swift build`
>>>> command. I have not personally experienced the
>>>> problem you are describing either, although I have
>>>> not tried pushing any apps using more recent Swift
>>>> toolchain snapshots.
>>>>
>>>> Can you provide more details about how the error
>>>> presents itself? Have you tried using any of these
>>>> other buildpacks with your app code?
>>>>
>>>> Brian
>>>>
>>>> onsdag 8 juni 2016 skrev Shao Miller via
>>>> swift-build-dev <swift-build-dev at swift.org
>>>> <mailto:swift-build-dev at swift.org>>:
>>>>
>>>> Thank you for your kind response.
>>>>
>>>> As mentioned, there is no choice: If the
>>>> headers aren't present in
>>>> the base image that a particular Cloud provider
>>>> provides, they can
>>>> only be present in the application sand-box by
>>>> one's own hand.
>>>>
>>>> All Swift build-packs to date and to my
>>>> knowledge use Swift 2.2
>>>> and do not use the Swift 3 'swift build'
>>>> process. I'm trying to
>>>> develop the next generation.
>>>>
>>>> Shao Miller
>>>> Synthetel Corporation
>>>> T: +1.9053927729 <tel:%2B1.9053927729>
>>>> <tel:+1.9053927729 <tel:%2B1.9053927729>>
>>>> E: swift-build-dev at synthetel.com
>>>>
>>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at synthetel.com
>>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at synthetel.com');>');>
>>>> W: https://www.synthetel.com
>>>> <https://www.synthetel.com/>
>>>>
>>>> On 6/8/2016 23:08, Daniel Dunbar wrote:
>>>>
>>>> Why do you want the headers inside the app
>>>> sandbox? Usually they would remain outside.
>>>>
>>>>
>>>>
>>>> Have you looked at IBM's CloudFoundry build
>>>> pack
>>>> (https://github.com/IBM-Swift/swift-buildpack)?
>>>> How does it handle the problem you are trying
>>>> to solve?
>>>>
>>>>
>>>>
>>>> - Daniel
>>>>
>>>>
>>>>
>>>> On Jun 8, 2016, at 8:03 PM, Shao Miller
>>>> via
>>>> swift-build-dev<swift-build-dev at swift.org
>>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at swift.org');>>
>>>>
>>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at swift.org
>>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at swift.org');>');>
>>>> wrote:
>>>>
>>>>
>>>>
>>>> Good day, Swift package manager
>>>> development folks.
>>>>
>>>>
>>>>
>>>> (There are at least two separate issues
>>>> being inquired about, but with the same
>>>> introductory context.)
>>>>
>>>>
>>>>
>>>> "Cloudy" deployment options derived
>>>> from or akin to CloudFoundry are
>>>> agonizingly locked-down environments.
>>>> Essentially Swift and all of its
>>>> dependencies and one's project's
>>>> dependencies must be stuffed into an
>>>> arbitrary directory (henceforth referred to
>>>> as "the hole," but usually /app/ ) and
>>>> build processes performed without any
>>>> root-user privileges. One consequence is
>>>> that one cannot use the OS'
>>>> package-management system to install
>>>> dependencies, but one must obtain them and
>>>> wrestle them into "the hole," instead. The
>>>> strategy seems rather silly.
>>>>
>>>>
>>>>
>>>> While developing a so-called
>>>> "buildpack" for Swift 3 projects to be
>>>> deployed via CloudFoundryish options and
>>>> utilizing the 'swift build' command, I have
>>>> come across a few issues.
>>>>
>>>>
>>>>
>>>> One issue is that 'swift build' wants
>>>> to do something with the
>>>> /usr/lib/swift/linux/x86_64/glibc.modulemap
>>>> file, but that file contains a hard-coded
>>>> path to a ///usr/include/complex.h
>>>> header-file. As is usually the case, this
>>>> hard-coded path will only work in a narrow
>>>> set of environments, which excludes "the
>>>> hole" that CloudFoundry provides. I have
>>>> attempted to use '-Xcc -I/the/hole' and
>>>> '-Xswiftc -I/the/hole' command-line
>>>> arguments, but I do not observe these paths
>>>> (nor sub-paths) being tried for the
>>>> complex.h header-file during complication.
>>>> I used 'strace' to trace the compilation
>>>> process, including all subprocesses.
>>>>
>>>>
>>>>
>>>> Is there some other mechanism to
>>>> instruct the Swift 3 package manager that
>>>> its [unfortunately] hard-coded paths are
>>>> relative to some particular path? If not,
>>>> would it be sensible to introduce some
>>>> logic to specify such a prefix path?
>>>>
>>>>
>>>>
>>>> Thank you for your time and attention.
>>>>
>>>> --
>>>>
>>>>
>>>> Shao Miller
>>>>
>>>> Synthetel Corporation
>>>>
>>>> T: +1.9053927729 <tel:%2B1.9053927729>
>>>> <tel:+1.9053927729 <tel:%2B1.9053927729>>
>>>>
>>>> E:swift-build-dev at synthetel.com
>>>>
>>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at synthetel.com
>>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at synthetel.com');>');>
>>>>
>>>> W:https://www.synthetel.com
>>>> <https://www.synthetel.com/>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>>
>>>> swift-build-dev mailing list
>>>>
>>>> swift-build-dev at swift.org
>>>>
>>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at swift.org
>>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at swift.org');>');>
>>>>
>>>> https://lists.swift.org/mailman/listinfo/swift-build-dev
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> swift-build-dev mailing list
>>>> swift-build-dev at swift.org
>>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at swift.org');>
>>>> https://lists.swift.org/mailman/listinfo/swift-build-dev
>>>>
>>>>
>>>
>>> _______________________________________________
>>> swift-build-dev mailing list
>>> swift-build-dev at swift.org
>>> <javascript:_e(%7B%7D,'cvml','swift-build-dev at swift.org');>
>>> https://lists.swift.org/mailman/listinfo/swift-build-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-build-dev/attachments/20160609/aa3e2d61/attachment.html>
More information about the swift-build-dev
mailing list