[swift-evolution] Feedback on SE-0166 and SE-0167

[Gwendal Roué]

Hello,

I want to provide real-life feedback for the Swift Archival & Serialization (SE-0166) and Swift Encoders (SE-0167) proposals that currently ship in Swift 4 snapshots.

The context: I'm the author of GRDB.swift [1], a SQLite library that, among other goals, aims at easing the conversion between database rows and custom models (structs and class hierarchies):

	// Sample code
	let arthur = Player(name: "Arthur", score: 100)
	try arthur.insert(db)
	print(arthur.id)
	
	let topPlayers = try Player
		.order(Column("score").desc)
		.limit(10)
		.fetchAll(db) // [Player]

Due to the lack of any introspection in Swift, GRDB currently wants you to perform explicit conversion:

	struct Player {
		var id: Int64?
		let name: String
		let score: Int
	}
	
	extension Player : RowConvertible {
		init(row: Row) {
			id = row.value(named: "id")
			name = row.value(named: "name")
			score = row.value(named: "score")
		}
	}
	
	extension Player : TableMapping, MutablePersistable {
		static let databaseTableName = "player"
		var persistentDictionary: [String: DatabaseValueConvertible?] {
			return ["id": id, "name": name, "score: score]
		}
	}

That's enough, but that's still too much.

SE-0166 and SE-0167 sound like the promise that some boilerplate code could be automatically generated.

Along with JSONDecoder and PListDecoder, let's introduce DatabaseRowDecoder! The current state of the work is at https://github.com/groue/GRDB.swift/tree/Swift4


At first, it's very satisfying. Decodable keeps some of it promises:

	struct Player : RowConvertible, Decodable {
		static let databaseTableName = "player"
		var id: Int64?
		let name: String
		let score: Int
	}
	
	// Yeah, no more extra code necessary for this to work!
	let topPlayers = try Player
		.order(Column("score").desc)
		.limit(10)
		.fetchAll(db)

But there are some issues.


### Issue 1: SE-0166/0167 merge the concepts of keyed objects and values

This is a problem. Let's take this example:

	enum Color: Int, Codable {
		case blue, green, red
	}
	
	struct Flower : RowConvertible, Decodable {
		let name: String
		let color: Color
	}
	
The way to decode a color comes from KeyedDecodingContainerProtocol:

	protocol KeyedDecodingContainerProtocol {
		func decode<T>(_ type: T.Type, forKey key: Key) throws -> T where T : Decodable
		func decodeIfPresent<T>(_ type: T.Type, forKey key: Key) throws -> T? where T : Decodable
	}

But the ability to decode a Color from a database row comes from the DatabaseValueConvertible, which I can't invoke since I can't test if type T conforms to this protocol:

	struct RowKeyedDecodingContainer<Key: CodingKey>: KeyedDecodingContainerProtocol {
		let row: Row
		
		// Not OK: no support for values
		func decode<T>(_ type: T.Type, forKey key: Key) throws -> T where T : Decodable {
			if <T conforms to DatabaseValueConvertible>  {
				let databaseValue: DatabaseValue = row.value(named: key.stringValue)
				return T.fromDatabaseValue(databaseValue) 
			} else { ... }
		}
	}

So the current state of the Codable library disallow GRDB from supporting value properties which are not the trivial Int, Int32, etc. Of course, GRDB itself makes it possible, with explicit user code. But we're talking about removing boilerplate and relying on the code generation that Codable is blessed with, here. We're talking about sharing the immense privilege that Codable is blessed with.

However, if I can't decode **values**, I can still decode **complex keyed objects** (in this case the row behaves like a hierarchical container - a concept already present in GRDB and allows it to consume complex rows like results of joins):

	struct Book : RowConvertible, Decodable { ... }
	struct Author : RowConvertible, Decodable { ... }
	struct Pair : RowConvertible, Decodable {
		let book: Book
		let author: Author
	}
	
	struct RowKeyedDecodingContainer<Key: CodingKey>: KeyedDecodingContainerProtocol {
		let row: Row
	
		// OK, support for other decodable objects
		func decode<T>(_ type: T.Type, forKey key: Key) throws -> T where T : Decodable {
			if let scopedRow = row.scoped(on: key.stringValue) {
				return try T(from: RowDecoder(row: scopedRow, codingPath: codingPath + [key]))
			} else {
				throw DecodingError.keyNotFound(key, DecodingError.Context(codingPath: codingPath, debugDescription: "missing scope"))
			}
		}
	}

Yet this use case is much less frequent.

Is it possible to workaround this problem? Did I miss something?


### Issue 2: Encodable can not be used to derive other persistence strategies.

The use case here is to derive other types of persistence from Encodable (and take profit from the compiler-generated code).

For example, I want to write:

	extension MutablePersistable where Self: Encodable {
		// Required by MutablePersistable
		var persistentDictionary: [String: DatabaseValueConvertible?] {
			return ...
		}
	}

If it were possible, we could get the full picture, with all boilerplate removed:
	
	// Wouldn't it be great?
	struct Player : RowConvertible, MutablePersistable, Codable {
		static let databaseTableName = "player"
		var id: Int64?
		let name: String
		let score: Int
	}

	let arthur = Player(name: "Arthur", score: 100)
	try arthur.insert(db)
	print(arthur.id)

	let topPlayers = try Player
		.order(Column("score").desc)
		.limit(10)
		.fetchAll(db) // [Player]

Unfortunately, it's impossible: the Encodable protocol doesn't allow iteration on the coding keys. I can't generate anything useful.

Again, is it possible to workaround this problem? Did I miss something?


Thanks for your attention,
Gwendal Roué

[1] https://github.com/groue/GRDB.swift