[swift-evolution] 100% bikeshed topic: DictionaryLiteral

Ted Kremenek kremenek at apple.com
Tue Jan 16 00:31:58 CST 2018


For some OS vendors it may be feasible to have multiple Swift runtimes installed on the system to be used by different apps.  That, however, is not an option if you actually want the libraries in the OS — in the case of macOS and iOS the OS frameworks — to actually use Swift themselves.  At the end of the day the frameworks in the framework stack have to be using the same Swift runtime in order to have Swift APIs, pass Swift objects around that are managed by the runtime, etc.

The Swift runtime was designed to actually allow the coexistence of two Swift runtimes: one for apps built before ABI stability that include a copy of the Swift runtime for their own use, and one for ABI stability that can be used both by the OS frameworks as well as apps built with the same version of Swift (or later) that had ABI stability.  In the case of runtime coexistence (which is to not break older Swift apps that aren’t rebuilt with a newer compiler) essentially the two runtimes are designed to essentially ignore each other.

> On Jan 15, 2018, at 11:48 AM, Wallacy via swift-evolution <swift-evolution at swift.org> wrote:
> 
> This. I asked that already and they not understand me very well.
> 
> I see no reason for each app to have its own version of "deprecated lib".
> 
> Also we can keep the libswiftCore_4.1.dylib, libswiftCore_5.0.dylib, libswiftCore_5.1.dylib etc on the SO too. The default target must be libswiftCore.dylib, but if need to use a specific feature on libswiftCore_4.1.dylib why not.
> 
> Em seg, 15 de jan de 2018 às 16:54, Jacob Williams via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> escreveu:
> Pardon my lack of knowledge in this area, but is there not also a 3rd option available?
> 
> (C) Split libswiftCore.dylib into two dylibs, both at the OS level. The *Deprecated.dylib would only be included when the application binary was compiled using a special -using-deprecated flag that signifies this app needs at least one of the deprecated APIs? Then the code signing verification you mention wouldn’t be a part of the apps that use deprecated APIs.
> 
> If this is not possible because all OS-level .dylibs MUST be included for whatever reason then just disregard me, as I’m not an expert on this area of Swift.
> 
> 
>> On Jan 12, 2018, at 5:43 PM, Ted Kremenek via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>> 
>> Hi Chris,
>> 
>> Instead of responding to each of your point bit-by-bit, I’ll try a different tactic to explain my reasoning — which may be wrong — by explaining how I see things top down with the tradeoffs they incur.  I’m going to say a bunch of things I know *you* know, but others are on this thread and I’ll state things for end-to-end clarity.
>> 
>> It seems to me that we are talking about two possible scenarios: (1) the status quo of keeping everything in libswiftCore.dylib or (2) splitting libswiftCore.dylib into two dylibs, one which includes some of the “deprecated” APIs.  I’ll enumerate what I think are the tradeoffs/benefits we will see with both scenarios, and see where the mismatch in our “talking past each other” is happening.
>> 
>> In both cases (A) and (B), with ABI stability the Standard Library has the option to ship in the OS.  Thus applications using Swift on (say) iOS would no longer need to include libswiftCore.dylib in the app when running on an OS that shipped with the Standard Library.
>> 
>> With that in mind, here are the tradeoffs as I see between scenarios (A) and (B):
>> 
>> 
>> (A) Status quo: Keep shipping everything in libswiftCore.dylib
>> 
>> - Applications running on an OS with the Standard Library would no longer have *any* of the currently libswift*.dylib’s embedded inside the application bundle because they would just use the one in the OS.
>> 
>> - One benefit of using libswift*.dylibs in the OS as opposed to those embedded inside the app bundle is that there is a non-neglible startup time improvement, as the code signing verification that happens when an app launches would be faster as it would not need to verify each of these dylibs that were previously embedded in the app.  We should consider this the new baseline for app startup time for an app running on an OS with an ABI stable Standard Library.
>> 
>> - We continue to support the deprecated APIs for some time, possibly indefinitely, even when better alternatives come around.
>> 
>> (B) Split libswiftCore.dylib into two dylibs, one that gets embedded in the app bundle
>> 
>> In the second scenario, we split out the deprecated APIs into a separate dylib, say libswiftCoreDeprecated.dylib.  That solution would have the following characteristics:
>> 
>> - Like scenario (A), app bundles would not need to embed libswiftCore.dylib when running on an OS that had an ABI-stable Standard Library.
>> 
>> - Compared to scenario (A), the OS shipping the Standard Library would have a slightly smaller libswiftCore.dylib that didn’t carry the bits for the deprecated APIs.  This would be a benefit in the code size for the OS, but not for the app itself.
>> 
>> - Any app using a deprecated API we put into libswiftCoreDeprecated.dylib (e.g., Mirrors) would need to embed libswiftCoreDeprecated.dylib inside their app bundle.  Compared to the new baseline I mentioned in (A), such apps would have a launch time regression once they started using any API in the libSwiftCoreDeprecated.dylib because code signing would need to verify the dylib, just like it does today with the libswiftCore.dylib that is embedded in every application.  They would also be slightly larger because the app bundle has the burden of hosting libswiftCoreDeprecated.dylib, instead of compared to scenario (A) where the implementations of the deprecated APIs are hosted by the libswiftCore.dylib that ships in the OS.
>> 
>> - Because of binary compatibility concerns, after Swift 5 we would *never* be able to “take out” any further APIs from libswiftCore.dylib and put them in libswiftCoreDeprecated.dylib.  This factorization can only happen once.
>> 
>> - There is some slight additional complexity added to the compiler and build system to support this spit of the Standard Library into multiple dylibs.  It’s not a huge concern but it does exist and it is not free both in terms of implementing and maintaining.
>> 
>> - We continue to support the deprecated APIs for some time, possibly indefinitely, even when better alternatives come around.  We *may* be able to completely sunset these APIs by having a future version of the Swift compiler simply refuse to build projects that use the deprecated (now obsoleted) APIs.  At that point, these apps need to move to using the newer API alternatives, or not upgrade to using a new Swift compiler.
>> 
>> 
>> With these points in mind, both scenarios are (by construction) very similar.  Scenario (B) potentially provides a bit more flexibility in outright sunsetting deprecated APIs down the road — but only the APIs we deem today should be on that path. In the future when we decide to deprecate APIs in the Standard Library we will need to follow the same kind of deprecation-and-obsoletion path as other binary frameworks in macOS/iOS, which may mean keeping API implementations around for a very long time just for the purpose of binary compatibility.
>> 
>> If you agree with me up to this point, to me this boils down to a tradeoff of some slightly increased opportunity to sunset some APIs in the future for a slight reduction in size of the Standard Library today (as it ships in the OS).  Most apps that don’t use these APIs won’t care either way, because even if we go with scenario B if they don’t use one of the deprecated APIs they will essentially be in scenario A (and thus get the benefit of using a Standard Library from the OS).  In scenario B, those apps that use the deprecated APIs will be slightly punished with slightly increased app bundle size and startup time.  Further, these APIs really can’t be deprecated right now (in terms of marking them deprecated in the module) until we have actual alternatives, so for the users that really want to use Mirrors today (which I agree may be small) that is the main tool they have to obtain that functionality.  Such users will pay a cost — albeit probably pretty minor — in scenario B.
>> 
>> I have not done the measurements of the impact in code size to the Standard Library of removing Mirrors, but I hypothesize it is relatively modest (I will look at verifying this hypothesis).
>> 
>> Do these observations align with what you are proposing, or am I missing a critical detail?
>> 
>> Ted
>> 
>>> On Jan 11, 2018, at 11:56 AM, Chris Lattner <clattner at nondot.org <mailto:clattner at nondot.org>> wrote:
>>> 
>>>> On Jan 11, 2018, at 12:01 AM, Ted Kremenek <kremenek at apple.com <mailto:kremenek at apple.com>> wrote:
>>>>> The nice thing about this is that only people who use these things would have to pay the cost, and you can directly message this by deprecating all the stuff in it.  Think about it as an overlay for the Swift standard library :-)
>>>> 
>>>> Hi Chris,
>>>> 
>>>> Even if we split the APIs into two sets, with an “ABI unstable” subset, we are still going to “carry round these ’suboptimal APIs’” because clients are using them today and those APIs — without an active plan to remove them — *are* a part of the Standard Library.  We’ve past the point in Swift where we can just rip them out (you said so yourself in a follow up email).
>>> 
>>> Who is “we” and how do you define carry?
>>> 
>>> I think you’re saying that they will remain part of the swift project (in the “SwiftDeprecated” module) for effectively ever.  If so, I agree, and that’s the point.  The point is that the code can continue to exist and service the few people who need it (and will become even fewer over the years) without causing *harm* to the Swift ecosystem over the near or long term.  That’s the feature.
>>> 
>>>> It feels to me that the main advantage of the overlay is a solution to remove out pieces of the Standard Library that will be used in practice less, and thus clients don’t “pay the cost” when they don’t use them.
>>> 
>>> Yes, exactly.
>>> 
>>>> But I think we should be honest about what that “cost” actually is, as these APIs aren’t going away right now.  These suboptimal APIs will stay around at least until better ones are created, and even then some existing clients will want to rely on the existing suboptimal APIs anyway even if others want to take advantage of newer, better APIs and idioms.
>>> 
>>> I’m not sure what you mean.  I’m talking about two things specifically:
>>> 
>>> 1) APIs that are already deprecated and have replacements.  The deprecated version goes in this library, avoid carrying them around forever.
>>> 
>>> 2) APIs that are used by extremely narrow audiences, which were added without much consideration and that have not been redesigned since Swift 1.  Mirror’s are an example of this.  I’m arguing that these should be moved to this SwiftDeprecated module - but not actually deprecated until a replacement exists.
>>> 
>>> I don’t see how this is bad for anyone, and I think this provides a significant process improvement compared to your current approach, which forces you to lock down APIs now without careful consideration just because they are not “important enough”.
>>> 
>>>> The main cost I see here that we save by having an “overlay for the Standard library" is code size.  
>>> 
>>> I don’t understand this point.
>>> 
>>>> On macOS/iOS it is true that applications embed a copy of the Standard Library within them and thus pay a real cost in the Standard Library’s actually payload size in their own app.  However, the only reason that is necessary today is because we don’t have ABI stability.  With ABI stability, other options for shipping the Standard Library — such as shipping it in the OS — become possible, and thus the burden of shouldering that “cost” of code size shifts around.  Then the tradeoffs of having an overlay for the Standard Library are different.  In a world with ABI stability, is it better to have a Standard Library "overlay" that only some clients use, and embed within their application bundle, than just eat that cost in the Standard Library to be shared by potentially multiple clients installed on a system?  I think it really comes down to the numbers.  In that case, are we really just talking about Mirrors?  I haven’t run the numbers here, but my intuition tells me we are talking about a relatively small code size impact overall.
>>> 
>>> This comment doesn’t make a lot of sense to me, perhaps we’re talking across each other.  The point of the design that I’m advocating for is that all apps (that deploy to iOS-next or later) get a immediate improvement from abi stability: the vast majority of the standard library is not put into their app bundle.  Further, the vastly most common apps - those that are not using mirrors or deprecated APIs, also do not include the SwiftDeprecated module. 
>>> 
>>> Only the tiny minority of apps that do use these APIs would have a copy of SwiftDeprecated in their app dylib, but they have a path to fix that, and they would still see a huge improvement from the bulk of the standard library being out of their app bundle.
>>> 
>>>> 
>>>> There’s also other potential performance implications from doing this split.  
>>> 
>>> This doesn’t make sense to me.  We’re not talking about taking comonly used apis here.  Mirrors are not performance sensitive at all, and neither are deprecated wrappers of renamed APIs.
>>> 
>>>> Being able to have the Standard Library be in the OS has potential major implications on the startup time of Swift applications.
>>> 
>>> It is great for app developers to have an incentive to stop using deprecated APIs! :-)
>>> 
>>>> Can you elaborate a bit more on the specific “cost” factors you are using when proposing to have a Standard Library overlay?
>>> 
>>> I think it is possible that you’re reading too much into the “overlay" word here, so ignore that.  The design I’m proposing is really simple:
>>> 
>>> 1. The vast majority of the standard library goes into the OS.  This includes all the string, array, and other stuff people use.
>>> 2. The deprecated wrappers and obscure APIs go into a new module, and that module doesn’t go into the OS.  Only people that use it pay to cost.
>>> 
>>> This isn’t particularly complicated, if give you new ways to manage the process of stdlib ABI stability, and I don’t see the downside.
>>> 
>>> -Chris
>>> 
>>> 
>>> 
>> 
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
> 
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
> https://lists.swift.org/mailman/listinfo/swift-evolution <https://lists.swift.org/mailman/listinfo/swift-evolution>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20180115/b2781569/attachment.html>


More information about the swift-evolution mailing list