[swift-evolution] [Proposal] Foundation Swift Archival & Serialization

Mon Mar 20 03:30:54 CDT 2017

> On 17 Mar 2017, at 17:07, Tony Parker <anthony.parker at apple.com> wrote:
> 
> Hi Karl,
> 
>> On Mar 17, 2017, at 12:20 AM, Karl Wagner via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>> 
>> 
>>> On 16 Mar 2017, at 21:48, Slava Pestov via swift-evolution <swift-evolution at swift.org <mailto:swift-evolution at swift.org>> wrote:
>>> 
>>> Hi Itai,
>>> 
>>> I’m wondering what the motivation is for keeping this as part of Foundation and not the standard library. It seems like you’re landing an implementation of this in the Foundation overlay on master, and another copy of all the code will have to go into swift-corelibs-foundation. This seems suboptimal. Or are there future plans to unify the Foundation overlay with corelibs-foundation somehow?
>>> 
>>> Also the implementation uses some Foundation-isms (NSMutableArray, NSNumber) and it would be nice to stick with idiomatic Swift as much as possible instead.
>>> 
>>> Finally you should take a look at the integer protocol work (https://github.com/apple/swift-evolution/blob/master/proposals/0104-improved-integers.md <https://github.com/apple/swift-evolution/blob/master/proposals/0104-improved-integers.md>) to replace the repetitive code surrounding primitive types, however I don’t know if this has landed in master yet.
>>> 
>>> Slava
>> 
>> I agree that the protocols should be part of the standard library rather than Foundation. As far as I can make out, the only part of this proposal that actually requires Foundation is the use of the “Data” type (which itself is a strange and often frustrating omission from the standard library). The actual concrete encoders can live in Foundation.
>> 
>> Generally my opinion is that the proposed feature is nice. Everybody hates NSCoder and having to write those required initialisers on your UIViews and whatnot. At its core, it’s not really very different from any other Swift archiving library which exists today, except that it’s backed with layer upon layer of compiler-generated magic to make it less verbose. The things I don’t like:
>> 
>> 1) While making things less verbose is commendable, automatically generating the encoding functions could be an anti-feature. “Codable” is for properties with persistable values only, which is a level of semantics which goes above the type-system. We don’t generate Equatable conformance for structs whose elements are all Equatable; it’s a big leap to go from “this data type is persistable” to “the value of this variable should be persisted” - for one thing, the value may not have meaning to others (e.g. a file-handle as an Int32) or it may contain sensitive user-data in a String. The encoding function isn’t just boilerplate - you *should* think about it; otherwise who knows what kind of data you’re leaking?
>> 
>> => One possible solution would be to have “Codable" as a regular protocol, and refine it with “AutoCodable" which contains the magic. Just so there is a little extra step where you think “do I want this to be generated?”.
> 
> The number one complaint we have about NSCoding (and this complaint predates Swift by a long shot) is that too much boilerplate is required for really simple data structures. Resolving this issue is one of our primary goals for this API.

Sure, but my point is that even for simple data structures, it’s worth thinking about what data you’re making available for persistence. That’s a level of semantic understanding which goes beyond the type-system: the compiler can’t tell if your String or Int contains per-instance or sensitive data. Also, this kind of synthesised behaviour doesn’t happen for any other protocol or library - not even for Equatable, which users probably ask for much more frequently than serialisation (just guessing).

> 
> There are a lot of benefits to keeping one protocol in place instead of making another one for “auto” codable: API which accepts Codable things does not need to have two entry points; there is one concept of Codable instead of two; you can “buy in” to more complex behavior by implementing parts of Codable (e.g., just the keys if you simply want to change the names of the JSON keys).

They wouldn’t need any new entry-points. AutoCodable would refine Codable; it wouldn’t add any requirements, just be an explicit declaration for the compiler to synthesise conformance because some data structure is expected to contain persistable values. Also, if I’m a programmer, and I see: “struct Foo: Codable”, I don’t know if the implementation of Codable exists in an extension somewhere (possibly in another file) or not.

> 
>> 
>> 2) More generally, though, I don’t support the idea of the Foundation module introducing extensions to the Swift language that no other developer of any other module has the chance to do, with an aside that some reflection API which doesn’t exist yet might possibly make it less magic in the future. My jaw hit the floor when I saw this was part of the proposal, and that it wasn’t for the standard library. Could you imagine, if somebody proposed their own magic protocols for AlamoFire or Kitura or any other Swift library? It would be a non-starter. It *should* be a non-starter.
>> 
>> => Do we have a policy about module-specific compiler magic such as this?
>> => Can we move any of the magic (e.g. CodableKeys generation) to the standard library?
>> 
>> I develop a lot for platforms, or for scenarios, where Foundation is not supported nor desirable. Considering that people are taught to prefer library code to rolling their own, and that humans are generally quite receptive to shortcuts at the expense of correctness, if this machinery exists at all we can expect it to spread. It would totally kill libraries such as SwiftJSON or whatever else you currently use. The fact that such a fundamental and widespread feature would now live in a monolithic module I can’t access would significantly spoil the language for me.
>> 
> 
> We’ve been very accepting of patches to port swift-corelibs-foundation to other architectures and platforms, including s390x (https://github.com/apple/swift-corelibs-foundation/pull/386 <https://github.com/apple/swift-corelibs-foundation/pull/386>), Android (https://github.com/apple/swift-corelibs-foundation/pull/622 <https://github.com/apple/swift-corelibs-foundation/pull/622>), and Cygwin (https://github.com/apple/swift-corelibs-foundation/pull/381 <https://github.com/apple/swift-corelibs-foundation/pull/381>). Foundation is extremely portable, so it should be available anywhere that Swift itself is possible to compile.
> 
> Foundation and the Swift standard library also have a more intimate relationship than most other libraries. This is especially relevant on Darwin, where we have special cases for bridging, implementations of various API on String, and more. It’s not a stretch to take advantage of that for this API as well.
> 

Sure, but there are some microcontroller environments I use regularly which I would love Swift to support one day. I’d like to be able to take advantage of focussed cross-platform libraries and get serialisation without requiring the whole Foundation module (which is rather heavyweight).

> Now, if we had all the language features we needed to avoid any compiler magic then we absolutely would have implemented this with those instead. However, those things aren’t here yet and for some of them (e.g. property behaviors or generalized existentials), there is no plan in place to add them in the near future. Working with JSON is such a common task for so many of our developers that we felt it was important to tackle the problem now with the tools we have at our disposal. 
> 

I’m still a bit uneasy about letting Foundation have access to special language features, though (beyond what is required for Obj-C compatibility - even then, where possible, using standard Swift idioms like initialisers rather than “as” casting). What is being described here goes beyond anything I’ve seen in the generics manifesto, so it’d be nice to have some idea about where this magic begins and ends and how we might open it up to 3rd-party libraries.

I think there could be an interesting path for that. The idea that, when all members of Foo conform to X, we can synthesise Foo: X by visiting each member and combining the values is something which crops up relatively frequently. For instance - it’s not all that different from the recent proposed extensions to Hashable, and it could go together with synthesising Equatable for enums when their payloads members are all Equatable.

I mean, really - users would appreciate synthesised conformances to Hashable and Equatable way more than Codable, but we haven’t done that yet. I think it’s because each level of the type-hierarchy represents an abstraction level we don’t want to assume semantics about. Why is the same not true of serialisation?

Perhaps we could generalise this feature - if not now then in the future. An “AutoX” pattern opens the door for AutoHashable and AutoEquatable as well as AutoCodable.

- Karl

> Of course, we will not be afraid to improve the API in the future as new features are added to the language.
> 
> - Tony
> 
>> - Karl
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org <mailto:swift-evolution at swift.org>
>> https://lists.swift.org/mailman/listinfo/swift-evolution
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170320/edd03661/attachment.html>