[swift-evolution] Question regarding SE-0167 Swift Encoders

[Gwendal Roué]

Hello Itai,

Thanks for helping sorting things out.

I have since my initial question a better understanding of Codable, and I hope I can better express the trouble. I bump against the fact that SE-0166 and SE-0167 assume that there is a single kind of coding keys.

This is the case for JSON and Plist:
	
	{
		"a": "foo"
		"b": { ... }
	}

But imagine a different serialization format where we don't use the same kind of keys for values and objects. Values are stored on red keys, and objects on blue keys:
	
	{
		"a" (red): "foo"
		"b" (blue): { ... }
	}

This serialization format accepts keys with the same name, as long as they don't have the same color:
	
	{
		"a" (red): "foo"
		"a" (blue): { ... }
	}

This format is used by SQL rows in GRDB. A SQL row is both a set of columns with associated values (the "red" keys), plus a set of scopes with associated "view" on the row (the blue keys):

    let row = ...
    row["id"]                              // 1
    let scopedRpw = row.scoped(on: "foo")! // <Row "id":2, "foo": "bar">
    scopedRow["id"]                        // 2

If you wonder: "but why???": columns and scopes are what can make rows a suitable base for hierarchical decoding, just like JSON and PList. When a flat SQL row fetched from a joined query is seen as a hierarchical structure, several simple `init(row:)` initializers can get the rows they expect, and we load a complex graph of objects. I have high hopes (https://github.com/groue/GRDB.swift/issues/176#issuecomment-285938568).

I care about Codable because of the code generation is has been blessed with. I expect GRDB users to rush on Codable since they won't have any longer to write the decoding boilerplate.


> I have to confess that I’m not familiar with this concept, but let’s take a look:
> if let valueType = T.self as? DatabaseValueConvertible.Type {
>     // if column is missing, trigger the "missing key" error or return nil.
> } else if let complexType = T.self as? RowConvertible.Type {
>     // if row scope is missing, trigger the "missing key" error or return nil.
> } else {
>     // don't know what to do
>     fatalError("unsupported")
> }
> Is it appropriate for a type which is neither DatabaseValueConvertible nor RowConvertible to be decoded with your decoder? If not, then this warrants a preconditionFailure or an error of some sort, right? In this case, that would be valid.
> 
Yes it is, there's no point preventing this.

We can forget the GRDB DatabaseValueConvertible and RowConvertible protocals in this discussion - they have their purpose, but are irrelevant here, and I was wrong letting them in the discussion. Will you look at some updated code?

In practice, let's consider the `KeyedDecodingContainerProtocol.decode(_:forKey:)` method. The decoding container is asked for a type T. It does not know yet if T is a single-value or a keyed type. No problem: it delays the decision until the Decoder is asked for a container:
    
    struct RowKeyedDecodingContainer<Key: CodingKey>: KeyedDecodingContainerProtocol {
        func decode<T>(_ type: T.Type, forKey key: Key) throws -> T where T : Decodable {
            // Push the key, and wait until the decoder is asked for a container
            // so that we know if T is keyed, or single-value:
            return try T(from: RowDecoder(row: row, codingPath: codingPath + [key]))
        }
    }

    struct RowDecoder: Decoder {
        func container<Key>(keyedBy type: Key.Type) throws -> KeyedDecodingContainer<Key> {
            if let key = codingPath.last {
                // Asked for a keyed type: look for a row scope
                if let scopedRow = row.scoped(on: key!.stringValue) {
                    let container = RowKeyedDecodingContainer<Key>(row: scopedRow, codingPath: codingPath)
                    return KeyedDecodingContainer(container)
                } else {
                    throw DecodingError.keyNotFound...
                }
            } else {
                // Asked for a keyed type at the top level
                let container = RowKeyedDecodingContainer<Key>(row: row, codingPath: codingPath)
                return KeyedDecodingContainer(container)
            }
        }
        
        func singleValueContainer() throws -> SingleValueDecodingContainer {
            // Asked for a single-value type: look for a column
            return RowColumnDecodingContainer(row: row, column: codingPath.last!!.stringValue)
        }
    }

(Sorry for the bangs, I still have to understand how I should deal with nil coding keys)

This works pretty well so far.

But now let's consider the `KeyedDecodingContainerProtocol.decodeIfPresent(_:forKey:)` method. Now we have a problem. This method must return nil if the key is missing. But which key? We don't know if the decoded type is keyed, or single-value. We can't postpone the decision, as above. So we have to double guess:

My current implementation is the following:

    func decodeIfPresent<T>(_ type: T.Type, forKey key: Key) throws -> T? where T : Decodable {
        if let dbValue: DatabaseValue = row[key.stringValue] {
            // We don't know if T(from: Decoder) will request a single value
            // container, or a keyed container.
            //
            // Since the column is present, let's assume that T will ask for a
            // single value container (a column). This is our only opportunity
            // to turn NULL into nil. If T eventually asks for a keyed container
            // (a row scope), then the user will face a weird error.
            if dbValue.isNull {
                return nil
            } else {
                return try T(from: RowDecoder(row: row, codingPath: codingPath + [key]))
            }
        } else if row.scoped(on: key.stringValue) != nil {
            // We don't know if T(from: Decoder) will request a single value
            // container, or a keyed container.
            //
            // Since the row scope is present, let's assume that T will ask for
            // a keyed container (a row scope). If T eventually asks for a
            // single value container (a column), then the user will face a
            // weird error.
            return try T(from: RowDecoder(row: row, codingPath: codingPath + [key]))
        } else {
            // Both column and row scope are missing: we are sure that the value
            // is missing.
            return nil
        }
    }

But it's less than ideal, as expressed by the inline comments.

We could discuss potential solutions, but I first hope that I was able to clearly express the topic.

Gwendal

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170531/fe681a10/attachment.html>