[swift-corelibs-dev] libdispatch prep for integration to the rest of swift-corelibs

[Tony Parker]

> On Jan 22, 2016, at 2:59 PM, Pierre Habouzit <phabouzit at apple.com> wrote:
> 
> FWIW today dispatch has that code when the objective-C runtime is used os OS X (in src/object.m).
> 
> 	Block_callbacks_RR callbacks = {
> 		sizeof(Block_callbacks_RR),
> 		(void (*)(const void *))&objc_retain,
> 		(void (*)(const void *))&objc_release,
> 		(void (*)(const void *))&_os_objc_destructInstance
> 	};
> 	_Block_use_RR2(&callbacks);
> 
> we may have to set it up with a similar trick for swift and that would solve the retain/release concern IMO.
> 
> 
> but that’s not the only concern, I sent a mail the other day, dispatch needs to:
> - get to the function pointer inside the block (so know at which offset it is)
> - if we want dispatch_block_create()d blocks to work, get some kind of support for it, to implement this function the right way:
> 
> DISPATCH_ALWAYS_INLINE
> static inline dispatch_block_private_data_t
> _dispatch_block_get_data(const dispatch_block_t db)
> {
> 	if (!_dispatch_block_has_private_data(db)) {
> 		return NULL;
> 	}
> 	// Keep in sync with _dispatch_block_create implementation
> 	uint8_t *x = (uint8_t *)db;
> 	// x points to base of struct Block_layout
> 	x += sizeof(struct Block_layout);
> 	// x points to base of captured dispatch_block_private_data_s object
> 	dispatch_block_private_data_t dbpd = (dispatch_block_private_data_t)x;
> 	if (dbpd->dbpd_magic != DISPATCH_BLOCK_PRIVATE_DATA_MAGIC) {
> 		DISPATCH_CLIENT_CRASH(dbpd->dbpd_magic,
> 				"Corruption of dispatch block object");
> 	}
> 	return dbpd;
> }
> 
> Though presumably, since dispatch_create_block() are created by the C++ runtime, these are C-like and probably don’t need anything. so the the only two issues should be retain/release of swift issued blocks, and the offset to the invoke function. If that offset could be the same in both worlds then that would be perfect.
> 
> -Pierre

If the swift compiler emits closures that look like Swift objects (and are retained/released with Swift ref counting functions), then the layout will probably be different.

In any case, I think the correct location for this stuff is not a library created by Foundation and imported by dispatch but instead exported functions from the Swift runtime (C code) that both libraries use as the truth for “how Swift blocks act when used in C code”.

- Tony

> 
>> On Jan 22, 2016, at 2:47 PM, Tony Parker via swift-corelibs-dev <swift-corelibs-dev at swift.org <mailto:swift-corelibs-dev at swift.org>> wrote:
>> 
>> Hi Dave,
>> 
>> What kind of layout does the Swift compiler emit for a Swift closure? If I call into a dispatch function that has a block argument, and that function calls Block_copy (from the blocks library), is everything retained correctly? The closure runtime.c calls out into a function emitted by the compiler to do that work. I would expect C code compiled by clang to follow the existing conventions but I’m not sure about Swift code compiled by swiftc.
>> 
>> - Tony
>> 
>>> On Jan 22, 2016, at 2:22 PM, David P Grove via swift-corelibs-dev <swift-corelibs-dev at swift.org <mailto:swift-corelibs-dev at swift.org>> wrote:
>>> 
>>> Hi,
>>> 
>>> Following up on the blocks discussion. I looked at the various Block header files and it looked there was a good chance things would just work. So, I did a quick & dirty hack of copying the .c and .h from foundation/closure into libdispatch/src/closure, building them, and linking with them instead of with libBlocksRuntime. All of the libdispatch tests that were working before, still work (not surprising), so it seems promising.
>>> 
>>> I speculate we would prefer to not have multiple copies of the blocks runtime used by an application. Therefore we need to get libdispatch and libFoundation to use the same one. The natural path would be to just build a blocks runtime library from the sources in foundation/closure and have foundation and dispatch link against that library and compile against its headers. I'd further guess that it would be preferred to leave the files in foundation/closure (not create some new project) and tweak the foundation build process so that it is possible to invoke it to build just the blocks runtime as a separate step. 
>>> 
>>> Does this sound plausible? If so, I can start working on it. Or is some other approach better?
>>> 
>>> thanks,
>>> 
>>> --dave
>>> 
>>> 
>>> <graycol.gif>Pierre Habouzit ---01/14/2016 10:09:28 PM---well dispatch also uses a bit of the blocks runtime because it uncorks the blocks and gets to the ca
>>> 
>>> From: Pierre Habouzit <phabouzit at apple.com <mailto:phabouzit at apple.com>>
>>> To: Philippe Hausler <phausler at apple.com <mailto:phausler at apple.com>>
>>> Cc: David P Grove/Watson/IBM at IBMUS, Swift Core Libs <swift-corelibs-dev at swift.org <mailto:swift-corelibs-dev at swift.org>>
>>> Date: 01/14/2016 10:09 PM
>>> Subject: Re: [swift-corelibs-dev] libdispatch prep for integration to the rest of swift-corelibs
>>> Sent by: phabouzit at apple.com <mailto:phabouzit at apple.com>
>>> 
>>> 
>>> 
>>> well dispatch also uses a bit of the blocks runtime because it uncorks the blocks and gets to the captured function pointer.
>>> 
>>> This is used pervasively, through this macro (internal.h):
>>> 
>>> #define _dispatch_Block_invoke(bb) \
>>> ((dispatch_function_t)((struct Block_layout *)bb)->invoke)
>>> 
>>> And we need to know the size of the Block_layout for a simple block (abused in _dispatch_block_get_data(), see inline_internal.h).
>>> 
>>> technically speaking dispatch only cares about the offsets/sizes and not about the rest of the layout.
>>> 
>>> -Pierre 
>>> On Jan 14, 2016, at 3:00 PM, Philippe Hausler via swift-corelibs-dev <swift-corelibs-dev at swift.org <mailto:swift-corelibs-dev at swift.org>> wrote:
>>> 
>>> As a note: our closure implementation in Foundation does NOT adhere to this (since it would mean that we would need to alter the c compiler to do so and that was not something that I have gotten to look at beyond a cursory glance.
>>> 
>>> So the change would be effectively something like this:
>>> 
>>> struct Block_layout {
>>> void *isa;
>>> #if DEPLOYMENT_RUNTIME_SWIFT
>>> uint32_t refCount;
>>> uint32_t weakRefCount;
>>> #else // not certain this branch can happen or not. we may need both
>>> volatile int32_t flags; // contains ref count
>>> int32_t reserved;
>>> #endif
>>> void (*invoke)(void *, ...);
>>> struct Block_descriptor_1 *descriptor;
>>> // imported variables
>>> };
>>> 
>>> and making the retain and release methods for blocks just call swift_retain and swift_release. But since blocks may be emitted by the compiler we would need to make certain that the compiler had a flag to emit those with the new layout and emit correct values for the refCount fields to give it immortality for things like global static blocks etc.
>>> 
>>> Then we would need to find some way to have the values for:
>>> 
>>> void * _NSConcreteStackBlock[32] = { 0 };
>>> void * _NSConcreteMallocBlock[32] = { 0 };
>>> void * _NSConcreteAutoBlock[32] = { 0 };
>>> void * _NSConcreteFinalizingBlock[32] = { 0 };
>>> void * _NSConcreteGlobalBlock[32] = { 0 };
>>> void * _NSConcreteWeakBlockVariable[32] = { 0 };
>>> 
>>> these guys emitted as objects as well so that the isa’s would be treated correctly.
>>> 
>>> I am not certain on how much of this lofty goal is actually needed; perhaps we should loop in one of the compiler team in on this to query what really needs to be done to properly support the @convention(block) syntax on linux. Perhaps I am overthinking this.
>>> On Jan 14, 2016, at 2:49 PM, David P Grove <groved at us.ibm.com <mailto:groved at us.ibm.com>> wrote:
>>> phausler at apple.com <mailto:phausler at apple.com> wrote on 01/13/2016 06:04:23 PM:
>>> > 
>>> > Tony brought up an important point about the prep for integration 
>>> > this morning: the blocks runtime from libblocksruntime-dev will be 
>>> > incompatible with the layout of blocks referenced from swift since 
>>> > the object size there is 2 uint32’s bigger to handle the RR (we 
>>> > don’t have objc). Without modifying clang and the runtime we won’t 
>>> > have a way to properly handoff blocks back and forth w/o objc.
>>> 
>>> Hi,
>>> 
>>> I'm guessing RR expands to Retain Release?
>>> 
>>> I've started exploring the source in swift-corelibs-foundation/closure.  Any chance there is a design doc laying out the object model used by the Swift implementation?  I'm sure I could figure it out from the source code, but it would be nice to be able to cheat and start with a an overview.
>>> 
>>> thanks,
>>> 
>>> --dave
>>> 
>>> > 
>>> > I am currently taking a look at this to see what we can do to add an
>>> > option to the clang code-gen to properly emit this structural 
>>> > difference. This isn’t a big issue for Foundation to CF since we 
>>> > don’t have many block APIs but dispatch is mostly blocks and that 
>>> > might pose an issue.
>>> > 
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> swift-corelibs-dev mailing list
>>> swift-corelibs-dev at swift.org <mailto:swift-corelibs-dev at swift.org>
>>> https://lists.swift.org/mailman/listinfo/swift-corelibs-dev <https://lists.swift.org/mailman/listinfo/swift-corelibs-dev>
>>> 
>>> 
>>> _______________________________________________
>>> swift-corelibs-dev mailing list
>>> swift-corelibs-dev at swift.org <mailto:swift-corelibs-dev at swift.org>
>>> https://lists.swift.org/mailman/listinfo/swift-corelibs-dev <https://lists.swift.org/mailman/listinfo/swift-corelibs-dev>
>> 
>> _______________________________________________
>> swift-corelibs-dev mailing list
>> swift-corelibs-dev at swift.org <mailto:swift-corelibs-dev at swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-corelibs-dev
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-corelibs-dev/attachments/20160122/1fabc5dd/attachment.html>