[swift-evolution] [RFC] "Library Evolution Support in Swift ('Resilience')"

[Drew Crawford]

I appreciate your responses.

> P.S. I hope I'm not coming off as antagonistic in these emails, particularly with the repeated responses. I think you've brought up a very important issue and it's critical that the ramifications of the model, whatever we decide on, are well-understood and can be meaningfully acted upon.

I hope I am equally not coming off as antagonistic.  I think this is a really important problem (security!  lawsuits!) and so we have to get it right.  I have a confrontational style at times, but I try to focus on ideas and not people.  And this proposal is, on balance, an amazing idea, we just have to iron out some details, and the necessary consequence of that is we have to focus on the more negative aspects.

>  I still think this is something better done on the client side.

Maybe we're not communicating?  I understand this proposal to say "if a library author says @inlineable in a dynamic library, the function may be inlined without any more warning on the client developer's part."  So I understand the current proposal to require consent only on the library side, and that will be sufficient for inlining to occur.

Let me give a developed example of the problem I forsee.  Let's say that there is a very popular third-party library for a very popular problem (JSON parsing, networking, app updating, pick your poison), and this library has been linked into hundreds or thousands of applications.

Through a circumstance of honest mistake, incompetence, or malice, the library has a security vulnerability.  Through a separate or coincident circumstance of honest mistake, incompetence, or malice, the function with the security vulnerability was declared inlineable.  As a matter of economics, none of the software developers depending on the library audited the library before using it, and as a matter of tooling, there was no practical way for developers like me, who forsaw this situation, to get a basic listing of what functions in the library were inlineable before deciding to avoid it.

This is a bad situation, although the part disregarding the inlineable aspect happens often enough that it is barely news these days (e.g. OpenSSL, Sparkle, AFNetworking, goto fail, to name recent examples).  Meanwhile application/client developers may not even be aware of security issues in their dependencies, or they may have gone out of business, they may consider the product to be at its end of life, or may take a long time to patch their software, etc.

There is little we can do about this in the iOS situation, but on OSX and particularly Linux, motivated end users can and do go around application/client developers, update their dynamic libraries, and resolve just such a situation as this.

I am worried about not so much the named examples of "good" inlining proffered in this thread (e.g. NSRect, UIKit, GCD, Swift standard library, etc.) but the "everything else" part of the ecosystem.  I assume Apple can afford to pay somebody to watch their inlineable functions, and I assume someone interested enough in performance to opt into "inlining" into their application understands the value of promptly patching their software.  (These may be bad assumptions for certain sensitive cases, but in the typical consumer-software case they are probably not a disaster.)  But if we consider a motivated attacker interested in developing "capabilities" against a large and vague unwashed mass of software, inlining-by-default significantly improves their opportunity.  And if we assume incompetence or honest mistake, inlining by default significantly raises the impact of a situation that is too common in our industry right now.

I believe that if inlining was "consented" on both sides (with some handwaving for a default policy for UIKit, standard library, and friends) the existing proposal that "clients may use the new implementation, or they may use the implementation from the time they were compiled, or they may use both inconsistently" might be fine, since anyone who ends up in that situation took steps to get there, and presumably they understood the risks of what they were doing.  IMO this is not so dangerous it should not be allowed, but it is so dangerous that someone should think carefully before they do it.  This feels like "-Ounchecked."  It is a power feature.

If we don't implement some dual consent like this (on library and client), then I think the "may use both inconsistently" language is too sharp an edge for a default behavior.  We need to implement some other safeguard against undefined behavior, because undefined behavior is a bad default for Swift.  Ideally the application could re-link itself (e.g. perhaps it contains a non-inlined payload that it dusts off and uses in this case), but in the interests of MVP I think a fail-to-start with a diagnostic would be a place to start.  Unfortunately fail-to-start might discourage upgrading an inlined library with a security bug (say you link with OpenSSL for example) but I prefer knowing I am insecure to not knowing.

I think either of those directions would separately address this issue.  I don't have a strong opinion about which path we should go.

> - Mark a dependency as version-locked; if the library changes in any way, refuse to launch.
> - Mark a dependency as non-inlineable.


Control is better than no control, but if the default is still "allow arbitrary libraries to inline with arbitrary applications" we are still in the case where an end user has hundreds of vulnerable applications with no mitigation, and that is very scary to me.

> Can you come up with something scarier that makes people less likely to use them on a whim? (This is by analogy with things like "UnsafePointer"; "@unsafe_inlineable" isn't quite right but gets at what I mean.)


The problem with "inlineable" is that it describes the benefit (high-performance!) but leaves the cost as an exercise to the reader.  Perhaps we should pick a word that describes the cost and leaves the benefit as an exercise to the reader.

I think the "cost" of "inlineable" is really that one cannot make changes to the functions later (as that will be undefined).  So we could say e.g. "@fixed", "@permanent", "@settled", "@unalterable", "@promised" "@immutable", "@irreversible", or similar.  I think that is better than "unsafe", since "unsafe" is a general danger, but inlining is not always dangerous, it is dangerous in the context of changes.  Something like "@immutable struct NSRect" sounds practically benign, while "@settled func verifyDigitalSignature" sounds terrifying, so this intuition maps well onto the actual level of danger.

Meanwhile I think most people who know enough to use this feature responsibly would be able to work out that making something "immutable" etc. would unlock more compiler optimizations, so I think such a word is adequate to describe the benefit.



> On Feb 12, 2016, at 5:06 PM, Jordan Rose <jordan_rose at apple.com> wrote:
> 
> I just re-read your initial objection, which had a slightly different idea: allow inlining, but if the library has changed when I go to run it, refuse to launch. I wanted to respond to that explicitly: this is no different from treating the library as part of your own distribution (what the document calls a "resilience domain"), and then not shipping it with your app. That means I see two options:
> 
> - Mark a dependency as version-locked; if the library changes in any way, refuse to launch.
> - Mark a dependency as non-inlineable.
> 
> I think these are both client-side controls, perhaps things you put in your Manifest.swift file. I'm not sure that we actually need or want both of them, but does this formulation seem reasonable to you?
> 
> Jordan
> 
> P.S. I hope I'm not coming off as antagonistic in these emails, particularly with the repeated responses. I think you've brought up a very important issue and it's critical that the ramifications of the model, whatever we decide on, are well-understood and can be meaningfully acted upon.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20160212/b222373b/attachment.html>