[swift-evolution] [REVIEW] SE-0193 - Cross-module inlining and specialization

Thu Dec 21 14:42:52 CST 2017

Thanks for the answers, Slava. More inline below.

> On Dec 21, 2017, at 12:30 AM, Slava Pestov <spestov at apple.com> wrote:
> 
> Hi Paul,
> 
> Thanks for reviewing the proposal!
> 
>> On Dec 20, 2017, at 9:21 PM, Paul Cantrell <cantrell at pobox.com <mailto:cantrell at pobox.com>> wrote:
>> 
>> A concern: how would a library author reason about, and check for bugs in, the combinatorial explosion of old and new implementations that could exist simultaneously with this feature in use?
> 
> I don’t have a simple answer to this unfortunately, other than the author being really careful, perhaps keeping around build artifacts that were compiled against older versions of the library and testing those.
> 
>> That last paragraph gives a relatively trivial example, but the implications are daunting! If I understand correctly, anything in a library that uses any @inlinable or @abiPublic code must be prepared to deal with every possible combination of every past published implementation of that code. And that “every possible combination” is not per function, but per…call site?
>> 
>> Suppose we have this:
>> 
>>     // Module A
>> 
>>     @inlineable func bar() { ... }
>> 
>>     // Module B
>> 
>>     @inlineable func foo() {
>>         if whatever {
>>             bar(0)  // compiler decides to inline this...
>>         } else {
>>             bar(1)  // ...but not this, for whatever reason
>>         }
>>     }
>> 
>>     // Module C
>> 
>>     func baz() {
>>         foo()
>>     }
>> 
>> …and suppose B was compiled against A v1.0 but C was compiled against A v2.0. Then, if I’m following, it’s possible for bar(0) to use the 1.0 implementation but bar(1) to use the 2.0 impl. Do I have that right? It seems to be what the hash value example is getting at.
> 
> That is correct. Another example is if module A publishes an inlinable function, and module B and C depend on A, but B and C were compiled with different versions of A. Then a fourth module D that depends on B and C might see two different published versions of this function.

I am … horrified and intrigued! I suppose C++ headers have always had exactly the same problems, but never having been the maintainer of a C++ library, I never had to worry about it.

More follow-up “huh” questions:

1. Presumably the portions of A inlined into B and C remain sensitive to the version-specific memory layout of A? Or will ABI stability mean that the compiler can magically rearrange memory offsets in already-compiled code when the layout changes? (Apologies if this is a too-obvious question; this part of Swift is all a mystery to me.)

2. Is there some class of statically identifiable breaking changes that the compiler does (or should) detect to flag incompatible inlined code? e.g. some version of A inlined into B references A.foo, then A.foo is deleted in a later version of A, so mixing older B with newer A in a project gives a compile- or link-time error?

3. Does this need some sort of poison pill feature for other sorts of breaking changes that are not statically detectable? e.g. invariants of a data structure in A change in release 2.0, so the author of A says “it is an error to include A ≥2.0 in any project that inlined any of my code from a version <2.0.” Is this what you were getting at with the mention of @inlinable(2.0) in the proposal? Sounded like that part was about something else, but I didn’t really grasp it tbh.

> 
>> Or is this not as dangerous as I’m imagining it to be?
> 
> It *is* pretty dangerous, which is why I hope this feature is used judiciously by third-party binary frameworks. With source frameworks that are built together with an app and always recompiled, this is less of a concern.

Yes, frameworks+app built simultaneously are clearly the more common case. Though Carthage seems to be champing at the bit to create this problem, since it added a feature to download prebuilt binaries long before ABI stability! I can easily imagining this feature spreading via word of mouth as a “secret go faster switch,” and causing no end of problems in the wild.

Per this and my questions above, a proposal:

It might be safer — and better match the understanding of the typical user — to have @inlinable assume by default that an inlined version of any given method is only valid only for the specific version of the module it was inlined from. The compiler would by default flag any version mixing as an error, and require an explicit statement of compatibility intent for each piece of inlinable code to opt in to the danger zone of mixed versions.

Then inlinable code could opt in by specifying some sort of past and future compatibility contract, e.g. “inline-compatible with version 2.x of this module,” perhaps using syntax along the lines of the @available stuff Chris proposed elsewhere in this thread. IOW, my #3 just above would be necessary for the compiler to allow any version mixing whatsoever.

This would have the advantage of making the default behavior of inlinable much more foolproof, and also of nipping in the bud problems of “oh god there are still modules out there that inlined our version 1.0, we can never alter this class.”

Does all that make sense? I’m more than a little out of my depth here.

> Also we are using this feature extensively in the standard library, so as the standard library evolves we will learn and develop best practices, hopefully without too many hiccups :)

I am very curious to see how that develops!

Cheers, P

> 
> Slava
> 
>> 
>> Cheers, P
>> 
>> 
>>> On Dec 20, 2017, at 6:19 PM, Ted Kremenek via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>> 
>>> The review of "SE-0193 - Cross-module inlining and specialization" begins now and runs through January 5, 2018.
>>> 
>>> The proposal is available here:
>>> 
>>> https://github.com/apple/swift-evolution/blob/master/proposals/0193-cross-module-inlining-and-specialization.md <https://github.com/apple/swift-evolution/blob/master/proposals/0193-cross-module-inlining-and-specialization.md>
>>> Reviews are an important part of the Swift evolution process. All review feedback should be sent to the swift-evolution mailing list at:
>>> 
>>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
>>> or, if you would like to keep your feedback private, directly to the review manager. 
>>> 
>>> When replying, please try to keep the proposal link at the top of the message:
>>> 
>>> Proposal link: https://github.com/apple/swift-evolution/blob/master/proposals/0193-cross-module-inlining-and-specialization.md <https://github.com/apple/swift-evolution/blob/master/proposals/0193-cross-module-inlining-and-specialization.md>
>>> ...
>>> Reply text
>>> ...
>>> Other replies
>>> What goes into a review of a proposal?
>>> 
>>> The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. 
>>> 
>>> When reviewing a proposal, here are some questions to consider:
>>> 
>>> What is your evaluation of the proposal?
>>> 
>>> Is the problem being addressed significant enough to warrant a change to Swift?
>>> 
>>> Does this proposal fit well with the feel and direction of Swift?
>>> 
>>> If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
>>> 
>>> How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
>>> 
>>> Thanks,
>>> Ted Kremenek
>>> Review Manager
>>> 
>>> _______________________________________________
>>> swift-evolution mailing list
>>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20171221/f82a5720/attachment.html>