[swift-evolution] 100% bikeshed topic: DictionaryLiteral

Tue Jan 16 15:09:08 CST 2018

Is interesting that the topic is going in this direction. The question
of "how an API “gets finalized” it's been something that has been a
little worrying for a while. The fact that a change in Swift goes from
Proposal to Oficial, part of the lang *for ever*, has bitten us in the
past already. My understanding is that it was one of the reasons to
introduce the requirement for a proposal to have an implementation
before hand.
But couldn't we have a better solution or this? For example what's the
plan with the async story? AFAIk in Rust they merge stuff into the
compiler and just mark it as not ready, forcing you to use a "nightly"
build (it's important to mention that they have a tool that helps them
pick the right lang. version easily) and enable specific feature
flags. If I understand correctly this is allowing them to test their
coroutines story without comiting to it before they're convince is the
right approach by putting in the hands of the community, not just by
being discussed by a small part of it in a mailing list.
I'm sure this can't be easy but I thought it was worth to raise it
seeing that the discussion lately is around finalizing APIs.
Cheers,

On Tue, Jan 16, 2018 at 6:39 AM, Ted Kremenek via swift-evolution
<swift-evolution at swift.org> wrote:
> Hi Nevin,
>
> I think this is an interesting perspective.  For me it begets the question
> of how an API “gets finalized”.  Currently anything that goes through the
> Swift Evolution proposal process that gets ratified is considered an
> official change to the language and Standard Library.  That feels like
> “finalized” to me, unless we want to introduce some new affordances to the
> process that allow us to introduce new APIs that are clearly going through a
> trial.  I think that’s a very interested idea (which is one Xiaodi
> proposed), but that’s not something we have today.
>
> On this thread we are talking about APIs that have been in the Standard
> Library for a while, and largely predate the Swift Evolution process.  What
> makes them less finalized versus other APIs with similar heritage?  These
> APIs are in use, albeit Chris points out that they are (most likely)
> uncommonly used.  A core deliverable of ABI stability, however, is that the
> majority of the Standard Library is in a binary stable form.  The only APIs
> I think we can consider candidates for a “deprecated” dylib that gets
> embedded in the app are “leaf” APIs that are uncommonly used by clients and
> not relied upon by other parts of the Standard Library that are commonly
> used by clients.  Practically speaking, however, I’m not convinced that set
> is so large to add the complexity/engineering cost/time of creating this
> separate dylib right now.  Those APIs can’t get ripped out right now — after
> all they’ve been in there for a while and we’re not deprecating them because
> there aren’t alternatives that exist yet anyway.  Thus these APIs need to
> continue to exist, and the benefits to me to create this separate dylib for
> those old “warty” APIs that we may — one day — want to replace seems
> marginal.
>
> I do think the idea of having a way to add *new* ABI-unstable APIs is very
> interesting, but I think the root of this discussion comes from talking
> about APIs that have been around for a while but feel like potential warts
> as we approach ABI stability.  I hate to say it, but there will be warts
> regardless.  There’s a lot to do for ABI stability — it’s making good
> progress — but not every last change we may have wanted to do will get all
> the attention we may have originally hoped it would.  Getting ABI stability
> done for Swift 5 is about prioritization and best effort, which includes
> accepting that not everything is going to be perfect.
>
> Ted
>
> On Jan 15, 2018, at 1:04 PM, Nevin Brackett-Rozinsky
> <nevin.brackettrozinsky at gmail.com> wrote:
>
> Alternatively, from a different perspective, rather than adding a new
> ABI-unstable library that gets bundled with apps, another way to look at it
> is that we already *have* an ABI-unstable library that gets bundled with
> apps. So we can instead think about introducing a new library with a stable
> ABI, which can be distributed with the OS.
>
> That way, as each piece of the existing standard library gets finalized, we
> can move it into the new ABI-stable library. Some pieces (eg. Mirror) will
> never make the jump and can be phased out eventually. And when future
> proposals make additions to the standard library, they can begin life in the
> original, existing, non-ABI-stable library.
>
> The net effect is the same as what we have been discussing, but a shift in
> viewpoint makes it clear that the ABI-stable library is the new thing we are
> adding, and the existing standard library can continue to serve a valuable
> purpose going forward.
>
> Nevin
>
>
>
> On Sun, Jan 14, 2018 at 9:04 PM, Ted Kremenek via swift-evolution
> <swift-evolution at swift.org> wrote:
>>
>>
>>
>> > On Jan 12, 2018, at 11:23 PM, Chris Lattner <clattner at nondot.org> wrote:
>> >
>> > On Jan 12, 2018, at 4:43 PM, Ted Kremenek <kremenek at apple.com> wrote:
>> >> Hi Chris,
>> >>
>> >> Instead of responding to each of your point bit-by-bit, I’ll try a
>> >> different tactic to explain my reasoning — which may be wrong — by
>> >> explaining how I see things top down with the tradeoffs they incur.  I’m
>> >> going to say a bunch of things I know *you* know, but others are on this
>> >> thread and I’ll state things for end-to-end clarity.
>> >
>> > Sounds good.
>> >
>> >> It seems to me that we are talking about two possible scenarios: (1)
>> >> the status quo of keeping everything in libswiftCore.dylib or (2) splitting
>> >> libswiftCore.dylib into two dylibs, one which includes some of the
>> >> “deprecated” APIs.  I’ll enumerate what I think are the tradeoffs/benefits
>> >> we will see with both scenarios, and see where the mismatch in our “talking
>> >> past each other” is happening.
>> >
>> > Right.
>> >
>> >> In both cases (A) and (B), with ABI stability the Standard Library has
>> >> the option to ship in the OS.  Thus applications using Swift on (say) iOS
>> >> would no longer need to include libswiftCore.dylib in the app when running
>> >> on an OS that shipped with the Standard Library.
>> >
>> > Right.  Please keep in mind that both approaches also eliminate all the
>> > overlay dylibs for those apps, and both approaches put the vast majority of
>> > the stdlib code into the OS as well.  The ‘deprecated’ dylib is probably
>> > going to be comparatively small.
>> >
>> >> With that in mind, here are the tradeoffs as I see between scenarios
>> >> (A) and (B):
>> >>
>> >> (A) Status quo: Keep shipping everything in libswiftCore.dylib
>> >>
>> >> - Applications running on an OS with the Standard Library would no
>> >> longer have *any* of the currently libswift*.dylib’s embedded inside the
>> >> application bundle because they would just use the one in the OS.
>> >
>> > Right.
>> >
>> >> - One benefit of using libswift*.dylibs in the OS as opposed to those
>> >> embedded inside the app bundle is that there is a non-neglible startup time
>> >> improvement, as the code signing verification that happens when an app
>> >> launches would be faster as it would not need to verify each of these dylibs
>> >> that were previously embedded in the app.  We should consider this the new
>> >> baseline for app startup time for an app running on an OS with an ABI stable
>> >> Standard Library.
>> >
>> > This happens with both models.  Refer back to the dyld optimization WWDC
>> > talk by Nick and Louis 2-3 years ago.  The big problem with Swift for
>> > startup time is that it adds half a dozen (or more) dylibs to your app
>> > bundle.  In the talk they make it clear that adding a single dylib is not a
>> > big deal, it is adding a bunch of dylibs that is the problem, particularly
>> > if they have interdependencies between them.
>>
>> Right — it is the large volume of dylibs that is the problem.
>>
>> >
>> > Neither approach presents this performance problem.
>>
>> Agreed — in practice neither scenario is much different.
>>
>> >
>> > Further, if it were important to solve this startup time problem, it is
>> > straight-forward to solve it for apps that do want to deploy backward (which
>> > will be almost all of them in NMOS).  You’d do this by merging all the
>> > dylibs into a single one in the app bundle instead of leaving them to be
>> > independently resolved at load time.
>> >
>> >> - We continue to support the deprecated APIs for some time, possibly
>> >> indefinitely, even when better alternatives come around.
>> >
>> > Agreed.
>> >
>> >> (B) Split libswiftCore.dylib into two dylibs, one that gets embedded in
>> >> the app bundle
>> >>
>> >> In the second scenario, we split out the deprecated APIs into a
>> >> separate dylib, say libswiftCoreDeprecated.dylib.  That solution would have
>> >> the following characteristics:
>> >>
>> >> - Like scenario (A), app bundles would not need to embed
>> >> libswiftCore.dylib when running on an OS that had an ABI-stable Standard
>> >> Library.
>> >>
>> >> - Compared to scenario (A), the OS shipping the Standard Library would
>> >> have a slightly smaller libswiftCore.dylib that didn’t carry the bits for
>> >> the deprecated APIs.  This would be a benefit in the code size for the OS,
>> >> but not for the app itself.
>> >
>> > The code size is small so it doesn’t matter much either way,
>>
>> Agreed.
>>
>> > but this actually is an advantage for the average app.  Part of the
>> > point of this is that most apps don’t use this stuff, so they wouldn’t
>> > include the dylib at all.  There is no cost for them in either launch time
>> > or size.
>> >
>> > To pick on Mirrors, the one example someone came up with is an API used
>> > by the swift on server community.  I haven’t heard of anyone using them in
>> > an iOS app (but of course I’m sure there is someone somewhere doing it :-)
>> >
>> >> - Any app using a deprecated API we put into
>> >> libswiftCoreDeprecated.dylib (e.g., Mirrors) would need to embed
>> >> libswiftCoreDeprecated.dylib inside their app bundle.  Compared to the new
>> >> baseline I mentioned in (A), such apps would have a launch time regression
>> >> once they started using any API in the libSwiftCoreDeprecated.dylib because
>> >> code signing would need to verify the dylib, just like it does today with
>> >> the libswiftCore.dylib that is embedded in every application.  They would
>> >> also be slightly larger because the app bundle has the burden of hosting
>> >> libswiftCoreDeprecated.dylib, instead of compared to scenario (A) where the
>> >> implementations of the deprecated APIs are hosted by the libswiftCore.dylib
>> >> that ships in the OS.
>> >
>> > The launch time comes from dyld, not code signing, but yes they would
>> > pay a very small cost.
>>
>> Yes, you’re right it is dyld.  For some reason I was thinking about code
>> signing.
>>
>> >  Again, I’d strongly recommend brushing up on nick + louis' WWDC talk
>> > that discusses this.
>> >
>> > More to the point though, it is *good* that there is some (small)
>> > pressure for people to stop using APIs that are deprecated in Swift 5 and
>> > earlier.  This is a one time opportunity. :-)
>>
>> Perhaps.  It feels (to me) like a rather minor opportunity considering we
>> need to still keep these APIs around and maintain them for some (perhaps
>> long) period of time.
>>
>> >
>> >> - Because of binary compatibility concerns, after Swift 5 we would
>> >> *never* be able to “take out” any further APIs from libswiftCore.dylib and
>> >> put them in libswiftCoreDeprecated.dylib.  This factorization can only
>> >> happen once.
>> >
>> > Correct, though Xiaodi’s point about gradually easing new APIs in with a
>> > similar approach is really really interesting and could be profound.
>>
>> Absolutely — it’s an interesting idea, but also orthogonal to this one.
>> I’d prefer to separate that into a different discussion as that one involves
>> a different set of concerns and parties interested in that discussion.
>>
>> >
>> >> - There is some slight additional complexity added to the compiler and
>> >> build system to support this spit of the Standard Library into multiple
>> >> dylibs.  It’s not a huge concern but it does exist and it is not free both
>> >> in terms of implementing and maintaining.
>> >
>> > Sure, but it also greatly eliminates the pressure to achieve perfection
>> > for a lot of API that simply isn’t important enough to fret that much about.
>> > This pays for itself in schedule time.
>>
>> To be honest I don’t see how it helps *that* much, and we may end up just
>> disagreeing here on the optics.  Since we have to keep these APIs around in
>> some way, regardless of whether or not we are happy with them, I don’t see
>> how it helps eliminate pressure.  I’m not that greatly concerned about
>> burning a small bit of space into the libswiftCore.dylib for some APIs we
>> hope to effectively sunset in the (unforeseeable) future.  OTOH, there are
>> plenty of things with ABI stability that I am concerned get done, which is
>> why efforts are focused on those.
>>
>> >
>> >> - We continue to support the deprecated APIs for some time, possibly
>> >> indefinitely, even when better alternatives come around.  We *may* be able
>> >> to completely sunset these APIs by having a future version of the Swift
>> >> compiler simply refuse to build projects that use the deprecated (now
>> >> obsoleted) APIs.  At that point, these apps need to move to using the newer
>> >> API alternatives, or not upgrade to using a new Swift compiler.
>> >
>> > Right: source compatibility isn’t a forever thing, even for existing
>> > Objective-C APIs vended by apple.  Giving people (e.g.) 3 years to migrate
>> > and remove them would probably be fine.  The key thing is that you never
>> > break *already shipped apps*, you only make more work for people who are
>> > actively maintaining/updating their app.
>>
>> Agreed.
>>
>> >
>> >> With these points in mind, both scenarios are (by construction) very
>> >> similar.
>> >
>> > I disagree, because I think you’re misunderstanding the performance
>> > impact of adding exactly one dylib to an app bundle.  How many dylibs does
>> > Swift 4 typically add to an app?  It’s probably up to 8 or more.
>>
>> I agree that the launch time cost of a single dynamic library is small.
>> My concern then is on the impact on people time of doing this factorization,
>> and adding the needed logic to the build system and compiler to support it.
>> It’s not insurmountable, but feels like a “nice to have” compared to other
>> tasks which are clearly quite urgent for ABI stability.  If we run out of
>> urgent things to do, I can see considering doing this.
>>
>> >
>> > 1 dylib isn’t a problem, check with Nick and Louis.
>> >
>> >> I have not done the measurements of the impact in code size to the
>> >> Standard Library of removing Mirrors, but I hypothesize it is relatively
>> >> modest (I will look at verifying this hypothesis).
>> >
>> > I agree.  The major motivation from my side isn’t the code size
>> > reduction, it is that we’ll have a better and brighter future with more
>> > clarity and less weird cruft left over - like the “DictionaryLiteral” type
>> > that sparked this whole thread.  Have you looked at it?  It’s crazy. :-)
>>
>> I think a bright future is still possible in either scenario.  From what I
>> can tell, if we don’t do anything, in the worst case there is some baggage
>> left in libswiftCore.dylib for the purposes of binary compatibility with
>> older apps.  We still are left with the opportunity to deprecate those APIs
>> in the future even if we can’t rip those symbols out now.
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution
>
>
>
>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>

-- 
Alejandro Martinez
http://alejandromp.com