[swift-evolution] Question regarding SE-0167 Swift Encoders

[Gwendal Roué]

> Le 31 mai 2017 à 19:15, Itai Ferber <iferber at apple.com> a écrit :
> 
> Hi Gwendal,
> 
> I hear your frustration. Some comments inline.

And I apologize for having written a rant :-)

>> On May 31, 2017, at 5:36 AM, Gwendal Roué <gwendal.roue at gmail.com <mailto:gwendal.roue at gmail.com>> wrote:
>> 
>> Itai,
>> 
>> (This email is not technical)
>> 
>> I'm not claiming that SE-0166 should be able to address all archival formats. I've been talking about GRDB to show at least one format that SE-0166 doesn't cover well. And should SE-0166 be fixed to support SQL (in the GRDB fashion), this does not mean that other developers won't eventually fight with SE-0166 until they understand it does not fit their bill.
> I’ll respond to the technical portion of this thread in the other email, but let me at least provide some background here. When working on this feature, we thought for a very long time about what we were looking to support with this feature, and how (feel free to take a look at the Alternatives Considered section of the proposal, though of course, there were more attempts and approaches before that).
> The majority of this thought was put into figuring out what the proper abstractions were for applying this new API — how can we abstract over different archival and serialization formats in a way that makes this useful?
> 
> In truth, if you try to abstract over all archival and serialization formats, the abstraction that you get is... the empty set. :) There are simply so many different things at odds with one another across different formats (JSON supports null values, plist does not; numbers are arbitrary precision in JSON, but not in plist or MessagePack or others; plist and MessagePack and others support binary data blobs, but JSON does not; etc.) that if you try to abstract over them all, you end up with nothing useful — an empty protocol that covers nothing.
> 
> So the key here is to try to strike a pragmatic balance between supporting some of the most common archival and serialization formats in a way that makes them useful, even if we have to handle special cases in some of them (e.g. null values in plist, binary data in JSON, etc.). It’s true that we cannot support them all, but in fact, we’re not looking to, because it would weaken the API.
> 
> I will respond to the comments specific to GRDB in the other thread, but this is a bit of background. Yes, there will always developers who will not be able to fit a serialization format into this API because it is fundamentally different in a way that cannot fit with the rest of the formats we’re looking to support. There’s nothing to be done about that. But, you mention this yourself.

I totally support this point of view. And JSON/Plist/NSCoding were the expected fundamentals.

For me the consequence of these intrinsic limitations / admitted caveats / necessary humbleness, should have blocked the inclusion of the this archival mechanism in the Standard Lib. Or at least its *fast* inclusion in the stdlib.

The reason for this is that unlike many others parts of the standard lib that are under constant scrutiny from the community, chances are low that many Coder/Decoder are/will be written.

To be short: don't you think it's too early to declare the archival library *stable*?

>> But there's something very special with SE-0166:
>> 
>> It's in the standard library, with all the backward-compatibility constraints that come with such a position.
>> 
>> IT'S BLESSED WITH CODE GENERATION.
>> 
>> I don't know if you, Michael LeHew, Tony Parker, and the core team, realize the importance of this insanely great privilege granted to this proposal.
> Believe me, I do, because we considered a lot of different approaches before settling on this. We wanted to avoid code generation for this reason — it has a privileged place within the compiler, it generates code which the user may not be able to introspect, etc.
> At the end of the day, though, we decided on this option because it provided the best user experience as part of the language in the vast majority of cases. There’s a lot to be said for that, and you mention this yourself, too.

So you want to improve the user experience... But below you attempt to lower the impact of code generation:

>> The lack of introspection and macros in Swift makes SE-0166 immensely attractive for a whole category of libraries.
>> 
>> When SE-0166 is lacking, should those libs ignore it, and lose CODE GENERATION, which means looking like it's still Swift 3?
>> 
>> Should those libs claim SE-0166 conformance, and raise runtime errors for invalid inputs (where "invalid" does not mean "invalid data", or "invalid code", but "impossible to fit in SE-0166" <=> "invalid library")?
> That being said, let’s separate the capabilities of the Codable API itself from the code generated by the compiler for it. While the code generation is a huge convenience for the majority of simple cases, it does just that — generate code for the simple cases. We cannot arbitrarily generate code to match arbitrary applications. Much more is possible with custom encode/decode implementations and custom CodingKeys than you might imagine, rather than just sticking to the default, compiler-generated implementation. (Data migration, format-specific encoded representations, multiple sets of CodingKeys, etc.)
> 
> If a library finds use for the Codable APIs only for the code generation, then I think that’s likely misapplication of the API.

That's what you hope. I predict that it will be abused in all kinds of awful ways, much worse than the GRDB attempt.

One year ago, all the rage was about dynamic runtime (http://inessential.com/2016/05/15/the_case_for_dynamic-swift_optimism <http://inessential.com/2016/05/15/the_case_for_dynamic-swift_optimism>), because memories were full of Objective-C. Next came countless blog posts about Swift Mirror hackery. Developers are *craving* for code generation.

There is a huge existing ecosystem for archival mechanisms that already avoid boilerplate code. Swift is a modern language, and does not err on the side of Java and C++, unless I'm mistaken. Scripting languages (Python, Ruby) have been generating code for years. Rust has Diesel (http://diesel.rs <http://diesel.rs/>). Some GRDB users use Sourcery (https://github.com/krzysztofzablocki/Sourcery <https://github.com/krzysztofzablocki/Sourcery>).

So I wouldn't talk about misapplication of the API, but more about a mismatch between the API and the developer needs.

Some libs will fit into SE-0166, and will be blessed with code generation. Some other libs won't fit, and won't be blessed with code generation. This will have a huge impact on their perceived quality.

> Attempting to use the Codable API to fit a square peg into a round hole will be frustrating because, well, it was designed for a singular purpose.

Yes, I understand. And your point of view is better than mine, because SE-0166 provides an actual *solution*, *today*, for *very frequent* use case. I can't help thinking that it's a little too soon for SE-0166, but I know very well that nobody will want to lose it, now that it is there.

The only reasonable way to reconcile our views, assuming I can be of any influence, is that SE-0166 should exist, but *not in the standard lib*. It's a matter of avoiding crippling stdlib with ad-hoc code that will not age very well and are nearly impossible to deprecate. It may be time to think about extending the number of core libraries.

Gwendal

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170531/2258541b/attachment.html>