<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta http-equiv="Content-Type" content="text/xhtml; charset=utf-8">
</head>
<body>
<div style="font-family:sans-serif"><div style="white-space:normal">
<p dir="auto">Hi Colin,</p>
<p dir="auto">Thanks for your comments! Are you talking about <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">Codable</code> synthesis, or encoding in general?</p>
<p dir="auto">On 21 Mar 2017, at 8:44, Colin Barrett wrote:</p>
<p dir="auto"></p></div>
<div style="white-space:normal"><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px"><p dir="auto">Hi Itai,<br>
<br>
Glad to see these proposal! I'm curious, have you or the other Swift folks<br>
thought about how *users* of these new Codable protocols will interact with<br>
resilience domains?<br>
<br>
What I mean is that what appear to be private or internal identifiers, and<br>
thus changeable at will, may actually be fragile in that changing them will<br>
break the ability to decode archives encoded by previous versions.<br>
<br>
Making this safer could mean:<br>
- Encoding only public properties</p>
</blockquote></div>
<div style="white-space:normal">
<p dir="auto">Unfortunately, property accessibility in code does not always map 1-to-1 with accessibility for archival (nor do I think they should be tied to one another).<br>
There are certainly cases where you’d want to include private information in an archive, but that is not useful to expose to external clients, e.g., a struct/class version:</p>
<pre style="background-color:#F7F7F7; border-radius:5px 5px 5px 5px; margin-left:15px; margin-right:15px; max-width:90vw; overflow-x:auto; padding:5px; color:black" bgcolor="#F7F7F7"><code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0" bgcolor="#F7F7F7"><span style="color: #008800; font-weight: bold">public</span> <span style="color: #008800; font-weight: bold">struct</span> <span style="color: #BB0066; font-weight: bold">MyFoo</span> {
<span style="color: #888888">// Should be encoded.</span>
<span style="color: #008800; font-weight: bold">public</span> <span style="color: #008800; font-weight: bold">var</span> <span style="color: #996633">title</span>: <span style="color: #007020">String</span>
<span style="color: #008800; font-weight: bold">public</span> <span style="color: #008800; font-weight: bold">var</span> <span style="color: #996633">identifier</span>: <span style="color: #007020">Int</span>
<span style="color: #888888">// This should be encoded too — in case the struct changes in the</span>
<span style="color: #888888">// future, want to be able to refer to the payload version.</span>
<span style="color: #008800; font-weight: bold">private</span> <span style="color: #008800; font-weight: bold">let</span> <span style="color: #996633">version</span> = <span style="color: #6600EE; font-weight: bold">1.0</span>
}
</code></pre>
<p dir="auto">Of course, there can also be public properties that you don’t find useful to encode. At the moment, I’m not sure there’s a much better answer than "the author of the code will have to think about the representation of their data"; even if there were an easier way to annotate "I definitely want this to be archived"/"I definitely don’t want this to be archived", the annotation would still need to be manual.</p>
<p dir="auto">(The above applies primarily in the case of <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">Codable</code> synthesis; when implementing <code style="background-color:#F7F7F7; border-radius:3px; margin:0; padding:0 0.4em" bgcolor="#F7F7F7">Codable</code> manually I don’t think the compiler should ever prevent you from doing what you need.)</p>
<p dir="auto"></p></div>
<div style="white-space:normal"><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px"><p dir="auto">- Adding some form of indirection (a la ObjC non-fragile ivars?)</p>
</blockquote></div>
<div style="white-space:normal">
<p dir="auto">What do you mean by this?</p>
<p dir="auto"></p></div>
<div style="white-space:normal"><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px"><p dir="auto">- Compiler warning (or disallowing) changes to properties in certain<br>
situations.</p>
</blockquote></div>
<div style="white-space:normal">
<p dir="auto">We’ve thought about this with regards to identifying classes uniquely across renaming, moving modules, etc.; this is a resilience problem in general.<br>
In order for the compiler to know about changes to your code it’d need to keep state across compilations. While possible, this feels pretty fragile (and potentially not very portable).</p>
<ul>
<li>Compiler warns about changing a property? Blow away the cache directory!</li>
<li>Cloning the code to a new machine for the first time? Hmm, all the warnings went away…</li>
</ul>
<p dir="auto">This would be nice to have, but yes:</p>
<p dir="auto"></p></div>
<div style="white-space:normal"><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px"><p dir="auto">I imagine the specifics would need to follow the rest of the plans for<br>
resilience.</p>
</blockquote></div>
<div style="white-space:normal">
<p dir="auto">specifics on this would likely be in line with the rest of resilience plans for Swift in general.</p>
<p dir="auto"></p></div>
<div style="white-space:normal"><blockquote style="border-left:2px solid #777; color:#777; margin:0 0 5px; padding-left:5px"><p dir="auto">It's likely that this could be addressed by a future proposal, as for the<br>
time being developers can simply "not hold it wrong" ;)<br>
<br>
Thanks,<br>
-Colin<br>
<br>
On Wed, Mar 15, 2017 at 6:52 PM Itai Ferber via swift-evolution <<br>
swift-evolution@swift.org> wrote:<br>
</p>
<blockquote style="border-left:2px solid #777; color:#999; margin:0 0 5px; padding-left:5px; border-left-color:#999"><p dir="auto">Hi everyone,<br>
<br>
The following introduces a new Swift-focused archival and serialization<br>
API as part of the Foundation framework. We’re interested in improving the<br>
experience and safety of performing archival and serialization, and are<br>
happy to receive community feedback on this work.<br>
Because of the length of this proposal, the *Appendix* and *Alternatives<br>
Considered* sections have been omitted here, but are available in the full<br>
proposal <<a href="https://github.com/apple/swift-evolution/pull/639" style="color:#999">https://github.com/apple/swift-evolution/pull/639</a>> on the<br>
swift-evolution repo. The full proposal also includes an *Unabridged API* for<br>
further consideration.<br>
<br>
Without further ado, inlined below.<br>
<br>
— Itai<br>
<br>
Swift Archival & Serialization<br>
<br>
- Proposal: SE-NNNN <<a href="https://github.com/apple/swift-evolution/pull/639" style="color:#999">https://github.com/apple/swift-evolution/pull/639</a>><br>
- Author(s): Itai Ferber <<a href="https://github.com/itaiferber" style="color:#999">https://github.com/itaiferber</a>>, Michael LeHew<br>
<<a href="https://github.com/mlehew" style="color:#999">https://github.com/mlehew</a>>, Tony Parker <<a href="https://github.com/parkera" style="color:#999">https://github.com/parkera</a>><br>
- Review Manager: TBD<br>
- Status: *Awaiting review*<br>
- Associated PRs:<br>
- #8124 <<a href="https://github.com/apple/swift/pull/8124" style="color:#999">https://github.com/apple/swift/pull/8124</a>><br>
- #8125 <<a href="https://github.com/apple/swift/pull/8125" style="color:#999">https://github.com/apple/swift/pull/8125</a>><br>
<br>
Introduction<br>
<br>
Foundation's current archival and serialization APIs (NSCoding,<br>
NSJSONSerialization, NSPropertyListSerialization, etc.), while fitting<br>
for the dynamism of Objective-C, do not always map optimally into Swift.<br>
This document lays out the design of an updated API that improves the<br>
developer experience of performing archival and serialization in Swift.<br>
<br>
Specifically:<br>
<br>
- It aims to provide a solution for the archival of Swift struct and<br>
enum types<br>
- It aims to provide a more type-safe solution for serializing to<br>
external formats, such as JSON and plist<br>
<br>
Motivation<br>
<br>
The primary motivation for this proposal is the inclusion of native Swift<br>
enum and struct types in archival and serialization. Currently,<br>
developers targeting Swift cannot participate in NSCoding without being<br>
willing to abandon enum and structtypes — NSCoding is an @objc protocol,<br>
conformance to which excludes non-class types. This is can be limiting in<br>
Swift because small enums and structs can be an idiomatic approach to<br>
model representation; developers who wish to perform archival have to<br>
either forgo the Swift niceties that constructs like enumsprovide, or<br>
provide an additional compatibility layer between their "real" types and<br>
their archivable types.<br>
<br>
Secondarily, we would like to refine Foundation's existing serialization<br>
APIs (NSJSONSerialization and NSPropertyListSerialization) to better<br>
match Swift's strong type safety. From experience, we find that the<br>
conversion from the unstructured, untyped data of these formats into<br>
strongly-typed data structures is a good fit for archival mechanisms,<br>
rather than taking the less safe approach that 3rd-party JSON conversion<br>
approaches have taken (described further in an appendix below).<br>
<br>
We would like to offer a solution to these problems without sacrificing<br>
ease of use or type safety.<br>
Agenda<br>
<br>
This proposal is the first stage of three that introduce different facets<br>
of a whole Swift archival and serialization API:<br>
<br>
1. This proposal describes the basis for this API, focusing on the<br>
protocols that users adopt and interface with<br>
2. The next stage will propose specific API for new encoders<br>
3. The final stage will discuss how this new API will interop with<br>
NSCoding as it is today<br>
<br>
SE-NNNN provides stages 2 and 3.<br>
Proposed solution<br>
<br>
We will be introducing the following new types:<br>
<br>
- protocol Codable: Adopted by types to opt into archival. Conformance<br>
may be automatically derived in cases where all properties are also<br>
Codable.<br>
- protocol CodingKey: Adopted by types used as keys for keyed<br>
containers, replacing String keys with semantic types. Conformance may<br>
be automatically derived in most cases.<br>
- protocol Encoder: Adopted by types which can take Codable values and<br>
encode them into a native format.<br>
- class KeyedEncodingContainer<Key : CodingKey>: Subclasses of this<br>
type provide a concrete way to store encoded values by CodingKey.<br>
Types adopting Encoder should provide subclasses of<br>
KeyedEncodingContainer to vend.<br>
- protocol SingleValueEncodingContainer: Adopted by types which<br>
provide a concrete way to store a single encoded value. Types adopting<br>
Encoder should provide types conforming to<br>
SingleValueEncodingContainer to vend (but in many cases will be<br>
able to conform to it themselves).<br>
- protocol Decoder: Adopted by types which can take payloads in a<br>
native format and decode Codable values out of them.<br>
- class KeyedDecodingContainer<Key : CodingKey>: Subclasses of this<br>
type provide a concrete way to retrieve encoded values from storage by<br>
CodingKey. Types adopting Decoder should provide subclasses of<br>
KeyedDecodingContainer to vend.<br>
- protocol SingleValueDecodingContainer: Adopted by types which<br>
provide a concrete way to retrieve a single encoded value from storage.<br>
Types adopting Decoder should provide types conforming to<br>
SingleValueDecodingContainer to vend (but in many cases will be<br>
able to conform to it themselves).<br>
<br>
For end users of this API, adoption will primarily involve the Codable<br>
and CodingKey protocols. In order to participate in this new archival<br>
system, developers must add Codable conformance to their types:<br>
<br>
// If all properties are Codable, implementation is automatically derived:public struct Location : Codable {<br>
public let latitude: Double<br>
public let longitude: Double}<br>
public enum Animal : Int, Codable {<br>
case chicken = 1<br>
case dog<br>
case turkey<br>
case cow}<br>
public struct Farm : Codable {<br>
public let name: String<br>
public let location: Location<br>
public let animals: [Animal]}<br>
<br>
With developer participation, we will offer encoders and decoders<br>
(described in SE-NNNN, not here) that take advantage of this conformance to<br>
offer type-safe serialization of user models:<br>
<br>
let farm = Farm(name: "Old MacDonald's Farm",<br>
location: Location(latitude: 51.621648, longitude: 0.269273),<br>
animals: [.chicken, .dog, .cow, .turkey, .dog, .chicken, .cow, .turkey, .dog])let payload: Data = try JSONEncoder().encode(farm)<br>
do {<br>
let farm = try JSONDecoder().decode(Farm.self, from: payload)<br>
<br>
// Extracted as user types:<br>
let coordinates = "\(farm.location.latitude, farm.location.longitude)"} catch {<br>
// Encountered error during deserialization}<br>
<br>
This gives developers access to their data in a type-safe manner and a<br>
recognizable interface.<br>
Detailed design<br>
<br>
To support user types, we expose the Codable protocol:<br>
<br>
/// Conformance to `Codable` indicates that a type can marshal itself into and out of an external representation.public protocol Codable {<br>
/// Initializes `self` by decoding from `decoder`.<br>
///<br>
/// - parameter decoder: The decoder to read data from.<br>
/// - throws: An error if reading from the decoder fails, or if read data is corrupted or otherwise invalid.<br>
init(from decoder: Decoder) throws<br>
<br>
/// Encodes `self` into the given encoder.<br>
///<br>
/// If `self` fails to encode anything, `encoder` will encode an empty `.default` container in its place.<br>
///<br>
/// - parameter encoder: The encoder to write data to.<br>
/// - throws: An error if any values are invalid for `encoder`'s format.<br>
func encode(to encoder: Encoder) throws}<br>
<br>
By adopting Codable, user types opt in to this archival system.<br>
<br>
Structured types (i.e. types which encode as a collection of properties)<br>
encode and decode their properties in a keyed manner. Keys may be String-convertible<br>
or Int-convertible (or both), and user types which have properties should<br>
declare semantic key enums which map keys to their properties. Keys must<br>
conform to the CodingKey protocol:<br>
<br>
/// Conformance to `CodingKey` indicates that a type can be used as a key for encoding and decoding.public protocol CodingKey {<br>
/// The string to use in a named collection (e.g. a string-keyed dictionary).<br>
var stringValue: String? { get }<br>
<br>
/// Initializes `self` from a string.<br>
///<br>
/// - parameter stringValue: The string value of the desired key.<br>
/// - returns: An instance of `Self` from the given string, or `nil` if the given string does not correspond to any instance of `Self`.<br>
init?(stringValue: String)<br>
<br>
/// The int to use in an indexed collection (e.g. an int-keyed dictionary).<br>
var intValue: Int? { get }<br>
<br>
/// Initializes `self` from an integer.<br>
///<br>
/// - parameter intValue: The integer value of the desired key.<br>
/// - returns: An instance of `Self` from the given integer, or `nil` if the given integer does not correspond to any instance of `Self`.<br>
init?(intValue: Int)}<br>
<br>
For most types, String-convertible keys are a reasonable default; for<br>
performance, however, Int-convertible keys are preferred, and Encoders may<br>
choose to make use of Ints over Strings. Framework types should provide<br>
keys which have both for flexibility and performance across different types<br>
of Encoders. It is generally an error to provide a key which has neither<br>
a stringValue nor an intValue.<br>
<br>
By default, CodingKey conformance can be derived for enums which have<br>
either String or Int backing:<br>
<br>
enum Keys1 : CodingKey {<br>
case a // (stringValue: "a", intValue: nil)<br>
case b // (stringValue: "b", intValue: nil)}<br>
enum Keys2 : String, CodingKey {<br>
case c = "foo" // (stringValue: "foo", intValue: nil)<br>
case d // (stringValue: "d", intValue: nil)}<br>
enum Keys3 : Int, CodingKey {<br>
case e = 4 // (stringValue: "e", intValue: 4)<br>
case f // (stringValue: "f", intValue: 5)<br>
case g = 9 // (stringValue: "g", intValue: 9)}<br>
<br>
Coding keys which are not enums, have associated values, or have other<br>
raw representations must implement these methods manually.<br>
<br>
In addition to automatic CodingKey conformance derivation for enums,<br>
Codableconformance can be automatically derived for certain types as well:<br>
<br>
1. Types whose properties are all either Codable or primitive get an<br>
automatically derived String-backed CodingKeys enum mapping properties<br>
to case names<br>
2. Types falling into (1) and types which provide a CodingKeys enum (directly<br>
or via a typealias) whose case names map to properties which are all<br>
Codableget automatic derivation of init(from:) and encode(to:) using<br>
those properties and keys. Types may choose to provide a custom<br>
init(from:) or encode(to:) (or both); whichever they do not provide<br>
will be automatically derived<br>
3. Types which fall into neither (1) nor (2) will have to provide a<br>
custom key type and provide their own init(from:) and encode(to:)<br>
<br>
Many types will either allow for automatic derivation of all codability<br>
(1), or provide a custom key subset and take advantage of automatic method<br>
derivation (2).<br>
Encoding and Decoding<br>
<br>
Types which are encodable encode their data into a container provided by<br>
their Encoder:<br>
<br>
/// An `Encoder` is a type which can encode values into a native format for external representation.public protocol Encoder {<br>
/// Populates `self` with an encoding container (of `.default` type) and returns it, keyed by the given key type.<br>
///<br>
/// - parameter type: The key type to use for the container.<br>
/// - returns: A new keyed encoding container.<br>
/// - precondition: May not be called after a previous `self.container(keyedBy:)` call of a different `EncodingContainerType`.<br>
/// - precondition: May not be called after a value has been encoded through a prior `self.singleValueContainer()` call.<br>
func container<Key : CodingKey>(keyedBy type: Key.Type) -> KeyedEncodingContainer<Key><br>
<br>
/// Returns an encoding container appropriate for holding a single primitive value.<br>
///<br>
/// - returns: A new empty single value container.<br>
/// - precondition: May not be called after a prior `self.container(keyedBy:)` call.<br>
/// - precondition: May not be called after a value has been encoded through a previous `self.singleValueContainer()` call.<br>
func singleValueContainer() -> SingleValueEncodingContainer<br>
<br>
/// The path of coding keys taken to get to this point in encoding.<br>
var codingKeyContext: [CodingKey] { get }}<br>
// Continuing examples from before; below is automatically generated by the compiler if no customization is needed.public struct Location : Codable {<br>
private enum CodingKeys : CodingKey {<br>
case latitutude<br>
case longitude<br>
}<br>
<br>
public func encode(to encoder: Encoder) throws {<br>
// Generic keyed encoder gives type-safe key access: cannot encode with keys of the wrong type.<br>
let container = encoder.container(keyedBy: CodingKeys.self)<br>
<br>
// The encoder is generic on the key -- free key autocompletion here.<br>
try container.encode(latitude, forKey: .latitude)<br>
try container.encode(longitude, forKey: .longitude)<br>
}}<br>
public struct Farm : Codable {<br>
private enum CodingKeys : CodingKey {<br>
case name<br>
case location<br>
case animals<br>
}<br>
<br>
public func encode(to encoder: Encoder) throws {<br>
let container = encoder.container(keyedBy: CodingKeys.self)<br>
try container.encode(name<br>
<br>
</p>
</blockquote></blockquote></div>
<div style="white-space:normal">
</div>
</div>
</body>
</html>