[swift-users] Communicating with dynamically loaded swift library
Geordie Jay
geojay at gmail.com
Sat Oct 7 09:33:20 CDT 2017
Ján Kosa <lopenka at gmail.com> schrieb am Sa. 7. Okt. 2017 um 15:27:
> I tried to use @_exported and it helped somewhat. While I still have same
> warnings, size of the PluginInterface library went down by 6mb (to 120kb)
> so it looks like Protobuf is no longer statically linked to it. However,
> size of PluginConsumer executable went up by same 6mb, it looks like it is
> linked there twice now.
>
To be clear: take protobuf out of the PluginConsumer dependencies.
Actually, I’m not sure which is which, but protobuf should only be listed
as a dependency of one package, where it is imported as @_exported. After
that, your other modules depend on the package that imports / exports
protobuf.
I also noticed interesting thing. If I build executable using `swift build`
> the size is around 17mb, when I generate xcode project and build it using
> that, size is around 200kb, but I get same warnings using both approaches
>
> On 7 October 2017 at 15:44, Geordie Jay <geojay at gmail.com> wrote:
>
>>
>> Ján Kosa <lopenka at gmail.com> schrieb am Sa. 7. Okt. 2017 um 13:34:
>>
>>> I tried swift package clean, but it didn't help
>>>
>>> "Try to ensure the plugin provider module (libA) is (only) being
>>> compiled into its standalone shared library file."
>>> How do I go about this? It is 3rd party module, it doesn't define any
>>> products (https://github.com/apple/swift-protobuf.git). Is there
>>> something I can do in my Package files to make sure it is loaded
>>> dynamically?
>>>
>>
>> When you compile a package depending on protobuf, all the relevant
>> symbols end up in your package’s library file. So here’s something you
>> might try:
>>
>> import protobuf into your own “PluginProvider” module (package), which
>> has a shared library product like this: ‘@_exported import Protobuf’ in
>> some compiled swift file. Then from the other dependent modules “import
>> PluginProvider” - the protobuf symbols should be available, and all from
>> one (nonconflicting) source.
>>
>> Geordie
>>
>>
>>
>>> On 6 October 2017 at 22:52, Geordie Jay <geojay at gmail.com> wrote:
>>>
>>>> I think SwiftPM is (incorrectly) compiling A.XYZ into each of the
>>>> modules that depend on it, as well as into your intended libA.so file.
>>>>
>>>> Try to ensure the plugin provider module (libA) is (only) being
>>>> compiled into its standalone shared library file. Try cleaning the swiftpm
>>>> build for one (swift package clean) and ensure the Package.swift files are
>>>> correctly set up to output the shared library.
>>>>
>>>> Sorry I can’t be more specific, I’ve had these same kinds of issues
>>>> before but I’m not 100% what they were.
>>>>
>>>> Geordie
>>>>
>>>>
>>>> Ján Kosa via swift-users <swift-users at swift.org> schrieb am Fr. 6.
>>>> Okt. 2017 um 14:41:
>>>>
>>>>> It worked! Took me a while to iron out details, but it is working now.
>>>>> Huge thanks sir, I will name my firstborn after you.
>>>>> Thanks for the @_cdecl("initializePlugin") tip as well, I didn't know
>>>>> about it and it will be very useful.
>>>>>
>>>>> I am having slightly related problem now (it was there before, but I
>>>>> ignored it for the time being), not sure if I should start a new thread?
>>>>>
>>>>> The PluginInterface module has one external dependency on module A,
>>>>> PluginConsumer has the dependency on module B which has dependency on same
>>>>> module A that the PluginInterface uses. When I load the plugin library, I
>>>>> get bunch of errors like:
>>>>>
>>>>> Class A.XYZ is implemented in both libPluginInterface.dylib and
>>>>> libMyPlugin.dylib
>>>>>
>>>>> I know why it is there, but I don't know how to get rid of it. I can't
>>>>> just remove dependency from PluginConsumer and use the one from
>>>>> PluginInterface (if that would even work?) because PluginConsumer does not
>>>>> depend on it directly, but it is going through module B first
>>>>>
>>>>> Cheers,
>>>>> Lope
>>>>>
>>>>> On 4 October 2017 at 22:17, Daniel Dunbar <daniel_dunbar at apple.com>
>>>>> wrote:
>>>>>
>>>>>> The way that I have done this in the past is pass a protocol as an
>>>>>> unsafe pointer to an exposed entry point:
>>>>>> ```swift
>>>>>> let entryPoint = dlsym(handle, “initializePlugin”)
>>>>>> guard entryPoint != nil else {
>>>>>> fatalError("missing plugin entry point:
>>>>>> \(pluginPath)")
>>>>>> }
>>>>>> typealias PluginInitializationFunc = @convention(c)
>>>>>> (UnsafeRawPointer) -> ()
>>>>>> let f = unsafeBitCast(entryPoint, to:
>>>>>> PluginInitializationFunc.self)
>>>>>> f(Unmanaged.passUnretained(self).toOpaque())
>>>>>> ```
>>>>>>
>>>>>> and then in the plugin convert back to the appropriate type:
>>>>>>
>>>>>> ```
>>>>>> @_cdecl("initializePlugin")
>>>>>> public func initializePlugin(_ ptr: UnsafeRawPointer) {
>>>>>> let manager =
>>>>>> Unmanaged<PluginManager>.fromOpaque(ptr).takeUnretainedValue()
>>>>>> ```
>>>>>>
>>>>>> HTH,
>>>>>> - Daniel
>>>>>>
>>>>>> On Oct 4, 2017, at 11:02 AM, Ján Kosa via swift-users <
>>>>>> swift-users at swift.org> wrote:
>>>>>>
>>>>>> Hello folks,
>>>>>>
>>>>>> I have been toying with dynamic libraries, trying to implement plugin
>>>>>> functionality. I was able to get to the point where I can call simple
>>>>>> function in loaded library, but I am having troubles starting more
>>>>>> sophisticated communication channel.
>>>>>>
>>>>>> There are 3 projects
>>>>>> - PluginConsumer is an app that loads plugin libraries
>>>>>> - MyPlugin is a plugin implementation, output is dynamic library that
>>>>>> PluginConsumer loads
>>>>>> - PluginInterface is common interface that both MyPlugin and
>>>>>> PluginConsumer use, so that they know how to communicate
>>>>>>
>>>>>> My first idea was to have PluginInterface be a simple SPM project
>>>>>> with single file where the bare-bones PluginInterface class would be:
>>>>>>
>>>>>>
>>>>>> open class PluginInterface {
>>>>>>
>>>>>> open func sayHi()
>>>>>>
>>>>>> }
>>>>>>
>>>>>>
>>>>>> Package.swift file:
>>>>>>
>>>>>>
>>>>>> // swift-tools-version:4.0
>>>>>>
>>>>>> import PackageDescription
>>>>>>
>>>>>> let package = Package(
>>>>>>
>>>>>> name: "PluginInterface",
>>>>>>
>>>>>> products: [ .library(name: "PluginInterface", type: .dynamic,
>>>>>> targets: ["PluginInterface"]) ],
>>>>>>
>>>>>> targets: [ .target(name: "PluginInterface") ]
>>>>>>
>>>>>> )
>>>>>>
>>>>>>
>>>>>>
>>>>>> UserPlugin is also very simple project containing only one file:
>>>>>>
>>>>>>
>>>>>> public func getPlugin() -> AnyObject {
>>>>>>
>>>>>> return MyPlugin()
>>>>>>
>>>>>> }
>>>>>>
>>>>>>
>>>>>> class MyPlugin: PluginInterface {
>>>>>>
>>>>>> override func sayHi() {
>>>>>>
>>>>>> print("Hi from my plugin")
>>>>>>
>>>>>> }
>>>>>>
>>>>>> }
>>>>>>
>>>>>> Package.swift:
>>>>>>
>>>>>>
>>>>>> // swift-tools-version:4.0
>>>>>>
>>>>>> import PackageDescription
>>>>>>
>>>>>> let package = Package(
>>>>>>
>>>>>> name: "MyPlugin",
>>>>>>
>>>>>> products: [ .library(name: "MyPlugin", type: .dynamic, targets: [
>>>>>> "MyPlugin"]) ],
>>>>>>
>>>>>> dependencies: [ .package(url: "url_to_PluginInterface", from:
>>>>>> "0.0.0"), ],
>>>>>>
>>>>>> targets: [
>>>>>>
>>>>>> .target(name: "PluginInterface", dependencies: [
>>>>>> "PluginInterface"]),
>>>>>>
>>>>>> .target(name: "MyPlugin", dependencies: ["PluginInterface"]),
>>>>>>
>>>>>> ]
>>>>>>
>>>>>> )
>>>>>>
>>>>>>
>>>>>> The PluginConsumer is bit more complicated, but here is relevant part
>>>>>> (lib loading and function calling):
>>>>>>
>>>>>>
>>>>>> typealias InitFunction = @convention(c) () -> AnyObject
>>>>>>
>>>>>>
>>>>>> let openRes = dlopen(pathToLib, RTLD_NOW|RTLD_LOCAL)
>>>>>>
>>>>>> if openRes != nil {
>>>>>>
>>>>>> defer {
>>>>>>
>>>>>> dlclose(openRes)
>>>>>>
>>>>>> }
>>>>>>
>>>>>> let symbolName = "mangled_symbol_name"
>>>>>>
>>>>>> let sym = dlsym(openRes, symbolName)
>>>>>>
>>>>>>
>>>>>> if sym != nil {
>>>>>>
>>>>>> let f: InitFunction = unsafeBitCast(sym, to: InitFunction.
>>>>>> self)
>>>>>>
>>>>>> let plugin = f() as? PluginInterface
>>>>>>
>>>>>> }
>>>>>>
>>>>>> }
>>>>>>
>>>>>> Package.swift file:
>>>>>>
>>>>>> // swift-tools-version:4.0
>>>>>>
>>>>>> import PackageDescription
>>>>>>
>>>>>> let package = Package(
>>>>>>
>>>>>> name: "PluginConsumer",
>>>>>>
>>>>>> dependencies: [ .package(url: "path_to_plugin_interface", from:
>>>>>> "0.0.0") ],
>>>>>>
>>>>>> targets: [ .target(name: "PluginConsumer", dependencies: [
>>>>>> "PluginConsumer"]) ]
>>>>>>
>>>>>> )
>>>>>>
>>>>>>
>>>>>> This all compiles nicely, MyPlugin project creates dylib file that
>>>>>> executable created by PluginConsumer can load, but the problem is with
>>>>>> following line:
>>>>>>
>>>>>> let plugin = f() as? PluginInterface
>>>>>>
>>>>>> Type of the plugin is MyPlugin, but from the consumer's view, it
>>>>>> doesn't inherit from PluginInterface so I can't call sayHi() method. I
>>>>>> assume this is because there is no relation between PluginInterface class
>>>>>> that compiler uses for MyPlugin project one that it uses for PluginConsumer
>>>>>> project. After library is loaded, they are two completely different classes
>>>>>> that happen to share same name. Is my assumption correct and how do I go
>>>>>> about fixing it?
>>>>>>
>>>>>> I had an idea I could make PluginInterface emit dynamic library that
>>>>>> would be dynamically linked by both MyPlugin and PluginConsumer, thus
>>>>>> making them share same PluginInterface class, but I can't figure out how to
>>>>>> do that (or if it's right way of doing this).
>>>>>>
>>>>>>
>>>>>> Any help appreciated :)
>>>>>>
>>>>>> Lope
>>>>>> _______________________________________________
>>>>>> swift-users mailing list
>>>>>> swift-users at swift.org
>>>>>> https://lists.swift.org/mailman/listinfo/swift-users
>>>>>>
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> swift-users mailing list
>>>>> swift-users at swift.org
>>>>> https://lists.swift.org/mailman/listinfo/swift-users
>>>>>
>>>>
>>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-users/attachments/20171007/7b260c01/attachment.html>
More information about the swift-users
mailing list