[swift-evolution] Pitch: Cross-module inlining and specialization

[Chris Lattner]

On Oct 5, 2017, at 1:30 PM, Joe Groff <jgroff at apple.com> wrote:
>> On Oct 4, 2017, at 9:24 PM, Chris Lattner <clattner at nondot.org> wrote:
>> On Oct 4, 2017, at 9:44 AM, Joe Groff <jgroff at apple.com> wrote:
>>>> I disagree.  The semantics being proposed perfectly overlap with the transitional plan for overlays (which matters for the next few years), but they are the wrong default for anything other than overlays and the wrong thing for long term API evolution over the next 20 years.
>>> 
>>> I disagree with this. 'inline' functions in C and C++ have to be backed by a symbol in the binary in order to guarantee function pointer identity, but we don't have that constraint. Without that constraint, there's almost no way that having a fallback definition in the binary is better:
>>> 
>>> - It becomes an ABI compatibility liability that has to be preserved forever. 
>> 
>> This seems like a marginal win at all.  Saying that you want to publish a symbol as public API but not have it be ABI is a bit odd.  What is the usecase (other than the Swift 3/4/5 transition period)?
> 
> I think it's a bigger win than you give it credit. If the function only exists in client code, then the library can much more aggressively deprecate and remove or replace the API, since it only has to worry about source compatibility and not deployed binary compatibility. If introducing `@inlinable` later further requires new clients to emit-into-client from that point on, so that the in-dylib entry point only exists for backward compatibility, then you get a free "linked-on-or-after" boundary where you can fix quirks or shed compatibility behavior in the inlinable version while preserving it in the binary.

I recognize the points you’re trying to make, but I’m not aware of sufficient experience with the model that makes me have any faith that it is the right default.  Consider:

1) Objective-C is widely known and proven for maintaining long-lived stable ABIs.  It worries (almost) equally about source and binary compatibility, because one without the other is odd.

2) C++ is widely known for both terrible source and binary compatibility, as well as for extensive code bloat.  It uses the model you’re describing.

Are you familiar with any system that uses the model you’re describing that provides good source/binary stability across api evolutions, and that does not inflict massive code bloat?


>>> - It increases binary size for a function that's rarely used, and which is often much larger as an outlined generic function than the simple operation that can be inlined into client code. Inlining makes the most sense when the inlined operation is smaller than a function call, so in many cases the net dylib + executable size would increase.
>> 
>> I can see this argument, but you’re basically saying that a sufficiently smart programmer can optimize code size based on (near) perfect knowledge of the symbol and all clients.  I don’t think this is realistic for a number of reasons.  In general, an API vendor has no way to know:
>> 
>> 1) how many clients it will have, potentially in multiple modules that get linked into a single app.
>> 2) on which types a generic function will be used with.
>> 3) what the code size tradeoffs ARE, e.g. if you have a large function that doesn’t use the archetype much, there is low bloat.
>> 
>> Furthermore, we have evidence from the C++ community that people are very eager to mark lots of things inlinable regardless of the cost of doing so.  Swift may end up being different, but programmers still have no general way to reason about code size given their declaration and without perfect knowledge of the clients.
>> 
>> The code of the approach I’m advocating is one *single* implementation gets generated in the module that defines the decl.  This can lead the N instantiations of exactly the same unspecialized code (consider the currying and other cases) in N different modules that end up in an app.  This seems like the right tradeoff.
> 
> If we're talking about inlinable functions, then we're already talking about functions generally on the small end of the scale,

What?  Programmers can and will mark tons of things inlinable.  This is well known from the C++ community.  I see nothing in your design that would lead to the conclusion that “only small functions” would get marked inlinable.

Even if we were dealing with disciplined programmers, it is a pretty meaningless and bad heuristic to make them decide about inlinability (in the sense you describe) since the profitability of inlining a function body depends on a ton of things.  This includes target processor architecture, but even more so the detailed behavior of the function body, and also the contextual information at specific calls sites (e.g. constant parameters).  Programmers will not be able to judge this sort of thing, even detailed cost models looking at specific examples are not perfect.

> so duplication is less of an issue. Furthermore, the duplication hazard only exists at dylib boundaries, since among static libraries we can still instantiate all the different instantiations of the function as ODR and let the linker fold them. 

This whole proposal is about those boundaries.  If not for those boundaries, this proposal wouldn’t exist.

>>> - It increases the uncertainty of the behavior client code sees. If an inlinable function must always be emitted in the client, then client code *always* gets the current definition. If an inlinable function calls into the dylib when the compiler chooses not to inline it, then you may get the current definition, or you may get an older definition from any published version of the dylib. Ideally these all behave the same if the function is inlinable, but quirks are going to be inevitable.
>> 
>> You’re saying that “if an API author incorrectly changes the behavior of their inlinable function” that your approach papers over the bug a little bit better.  I don’t see this as something that is important to design around.  Not least of which because it will produce other inconsistencies: what if a binary module A is built against the old version of that inlinable function and you app builds against a newer version?  Then you have the two inconsistent versions in your app again.
>> 
>> More generally though, an API vendor who does this has broken the fragile/inlinable contract, and they therefore invoked undefined behavior - c'est la vie.
> 
> The contract doesn't need to be nearly as strict with emitted-into-client code, though, since you can make fixes or behavior changes that only take hold when you recompile with a newer version of the library, linked-on-or-after style.

Which means that someone upgrading to a new version of your code - which has promised to be ABI stable - suddenly has an abi breakage.  I understand the value of linked-on-or-after checks, but these are extremely delicate and need to be carefully reasoned about.  They should not be the default based on applying an (appealing sounding) attribute to a function.

-Chris