<html><body><p><font size="2">Hi Brent,</font><br><font size="2">The answers are in line. </font><br><font size="2">gelareh</font><ul><ul><p><b>4.1 - TLS service protocol</b><br><font size="2"><br>The TLS service protocol describes the methods that the transport layer calls to handle transport-level events for the TLS service object.</font></ul></ul><br>I have a bunch of questions about the design you're presenting, and I think many of them ultimately stem from not understanding some of the high-level aspects of the proposal. For instance:<br><br>> * What types conform to this protocol? From the diagram, it looks like there's a type for each "engine"—a SecureTransportService, an OpenSSLService, etc.—and each instance represents a particular configuration of that engine. So you create a WhateverTLSService, configure it, and then hand it off (through several layers) to the transport management layer, which calls methods on it to handle various events. The transport management layer then uses that one TLSService to handle many connections. Is that correct?<br><br><br>[g]<br>More or less. As long as the implementation conforms to the protocol, it can use whatever underlying security library it wants. This way we can have a plug and play architecture which allows the user to pick the security library implementation of its choice (eg. LibreSSL, OpenSSL, etc)<br><br><br>> * What is the lifecycle of a connection? Does the TLSService create them itself, does the transport management layer create them and hand them off, or does the transport management layer retain control over them from beginning to end?<br><br><br>[g] The connection life cycle is handled by levels higher than TLS. For all intents and purposes, the transport management layers owns a TLS service delegate which calls the appropriate TLS-related functionality at the right time (eg. at socket creation time, at connection time, etc).<br><br><br>> * You discuss the higher layers creating a TLSService object and caching it in a property, then ultimately handing it down to the transport management layer, which then attaches it to socket objects. But presumably you can have many socket objects, possibly simultaneously. Are they all served by a single TLSService instance, or by many? If they share a TLSService, how does the TLSService know which socket is talking to it at a given moment? If they have separate ones, how does the transport management layer acquire a new one when it needs it?<br><br>[g]<br>Each socket instance would have its own TLS service delegate -- this is important because each socket might have its own specific TLS channel properties.<br><br><br>> You mention that this proposal is very small in scope, and it's fine to describe some of these details in general ways. For instance, you don't need to describe the interface to a Swift socket or the transport layer in detail. But currently, the description of these surrounding systems is *so* vague that I'm struggling to assess this design.<br><br>Perhaps some of these details have been described in other documents or meetings; if so, they really need to be presented in this document, too.<br><br>[g] fair enough. The problem is that right now in the servers working group, we have not defined *any* of our interfaces. This is the first proposal and we are trying to only pin the things which this service requires. <br>In any case, if you think other things should be included, please let us know.<br><br>
<ul><ul><b><font size="2">- onClientCreate</font></b></ul></ul>> Why do these methods all have "on" prefixes? I'm not totally sure I understand the intended usage here, but I see two possibilities:<br><br>> * These are imperative commands. `onAccept` says that the TLS engine should accept a connection, `onSend` means it should send some data, etc. In that case, these should not have any prefix—they should just be `accept`, `send`, etc.<br><br>> * These are essentially delegate methods notifying the TLS engine of an event. `onAccept` says that the system has accepted a connection and the TLS engine should do what it needs to do with it, `onSend` means the system is about to send data and it needs the TLS engine to modify it, etc. If so, Swift APIs more often use words like `should`, `will`, or `did` than `on`, particularly since they're more precise about the timing of the delegate method compared to the actual event.<br><br>> In either case, I don't think "on" is the best naming for these. It needlessly bucks platform conventions.<br><br>[g]<br>The latter description is the intended. Your points are fair and perhaps we can modify the delegate method names to, perhaps:<br><br>onClientCreate --> didClientCreate<br>onServerCreate --> didServerCreate<br>onDestroy --> willDestroy<br>onAccept --> didAccept(on connection: TransportManagementDelegate)<br>onConnect --> didConnect(on connection: TransportManagementDelegate)<br>onSend --> willSend(with data: UnsafeRawPointer, dataSize: Int))<br>onReceive --> willReceive(with data: UnsafeRawPointer, dataSize: Int))<br>
<ul><ul><font size="2">This will be called when a client I/O connection is created and appropriate TLS connection needs to be configured, including context creation, the handshake and connection verification.</font><br><font size="2"><br>This is a client only method.</font><br><font size="2"><br>///<br>/// Setup the contexts and process the TLSService configurations (certificates, etc)<br>///</font><br><font size="2"><br>func onClientCreate() throws</font><p><b><font size="2">- onServerCreate</font></b><br><font size="2"><br>This will be called when a server I/O connection is created and appropriate TLS connection needs to be setup, including context creation, the handshake and connection verification.</font><br><font size="2"><br>This is a server only method.</font><br><font size="2"><br>///<br>/// Setup the contexts and process the TLSService configurations (certificates, etc)<br>///</font><br><font size="2"><br>func onServerCreate() throws</font></ul></ul>> What are these methods supposed to do, exactly?<br><br>> * Do they put `self` into either a client state or a server state? If so, what happens if you call both, or neither, or call one twice? Would it be better to do this as part of initialization, or to have them make a client TLS object or server TLS object, or to require whatever code hands the TLSService to the TransportManager to pre-configure it as either client or server?<br><br><br>[g] <br>They setup the TLS contexts and process the certificates. They do put self in either a client or server state via associated parameter sets in OpenSSL and SecureTransport.<br>(I am pretty sure) if you call them multiple times, they simply over write the previous setting. But we can test this during implementation.<br><br>These methods are essentially initialization that gets called when the TransportManager decides it has enough information to set it as client or server.<br><br><br>> * Do they create a new instance that's either a client or a server? If so, how do they return it?<br>> * Do they configure something recently created as either client or server? If so, how do they access whatever they need to configure?<br><br>> Basically, what state are these supposed to operate upon?<br><br>[g]<br>Most of the functionality is the same for client or server except for setting the server/client flag/function in the underlying security library. The instance is stored in the delegate.<br><br>
<ul><ul><b><font size="2">- onDestroy</font></b><br><font size="2"><br>This will be called when an I/O instance connection is closed and any remaining TLS context needs to be destroyed.</font><br><font size="2"><br>This is both a client and server method.</font><br><font size="2"><br>///<br>/// Destroy any remaining contexts <br>///<br>func onDestroy()</font></ul></ul>> Is this called at the end of each connection, or is it called once when the transport management layer is totally finished with the TLSService, or are these the same thing?<br><br>> If per-connection, how does it know which connection?<br><br>> If during destruction, should we just class-constrain and use `deinit` for this purpose?<br><br>[g] this is per-connection and the TLS service would keep a context for the connection.<br>So the transport manager has a delegate instance pointing to this TLS service instance which itself has a context.<br>
<ul><ul><b><font size="2">- onAccept</font></b><br><font size="2"><br>This will be called once an I/O instance connection has been accepted, to setup the TLS connection, do the handshake and connection verification.</font><br><font size="2"><br>This is both a client and server method.</font><br><font size="2"><br>///<br>/// Processing on acceptance from a listening connection<br>/// <br>///<br>/// - Parameter IORef: The connected I/O instance<br>///<br>func onAccept(IORef: TransportManagementDelegate) throws</font></ul></ul>> I take it the parameter is some sort of abstraction wrapping e.g. a socket. So why is it called a `TransportManagementDelegate`? Shouldn't its name include words like `Connection` or `Socket` or `IOHandle` or something?<br><br><br>[g] we can simply call it ConnectionDelegate. I was trying to be consistent in my terminology.<br><br>> Do we want the parameter to be labeled `IORef`? That's not very idiomatic; it doesn't read well or follow the Swift naming guidelines.<br><br>[g] great feedback. Thanks!<br><br><br>> You say this is for both clients and servers. When does a TLS client have a listening connection that it `accept`s connections on?<br><br>Is it called at different times or in different ways than `onServerCreate`?<br><br>[g] Bad copy/paste! for both onConnect and onAccept. They are obviously server and client specific.
<ul><ul><b><font size="2">- onConnect</font></b><br><font size="2"><br>This will be called once a socket connection has been made, to setup the TLS connection, do the handshake and connection verification.</font><br><font size="2"><br>This is both a client and server method.</font><br><font size="2"><br>///<br>/// Processing on connection to a listening connection<br>///<br>/// - Parameter connectionRef: The connected I/O instance<br>///<br>func onConnect(IORef: TransportManagementDelegate) throws</font></ul></ul>The same as above, with appropriate substitutions.
<ul><ul><b><font size="2">- onSend</font></b><br><font size="2"><br>This will be called when data is to be written to an I/O instance. The input data buffer is written to the TLS connection associated with that I/O instance.</font><br><font size="2"><br>This is both a client and server method.</font><br><font size="2"><br>///<br>/// Low level writer<br>///<br>/// - Parameters:<br>/// - buffer: Buffer pointer<br>/// - bufSize: Size of the buffer<br>///<br>/// - Returns the number of bytes written. Zero indicates TLS shutdown, less than zero indicates error.<br>///<br>func onSend(buffer: UnsafeRawPointer, bufSize: Int) throws -> Int</font></ul></ul>> Is there a reason you use an UnsafeRawPointer and a buffer size, instead of using an UnsafeRawBufferPointer which would encapsulate both?
<p>[g] it was pointed our rightly by others that it's not good to have either of them :-) Right now, we are looking at have Data and more efficient versions.
<p><br><br>> Why is shutdown indicated with zero, rather than the return value being Optional and being nil? Why are errors signaled with negative values instead of being thrown? (Or are you saying that negative returns are invalid? That's different from saying "indicates error".)<br><br>[g] This is actually implementation details which shouldnt have been included in this pitch in the first place.<br><br><br>> If a TLSService return less than `bufSize`, will the enclosing later try to.re-send the remaining data in subsequent calls?<br><br>This sounds like the TLS engine owns the network connection (at least by this point) and is responsible for writing to it. Does that mean `accept` and `connect` take ownership of the connection and hold on to it? If you have several different simultaneous connections, how do you know which connection this should write to? Or does a given TLSService only own one connection at a time? If so, does the transport management layer create a new TLSService instance for each connection? How? If each TLSService is bound to one connection, shouldn't it be created already knowing the connection it's going to use?<br><br>[g] I believe I have talked about the relationship between connections and TLS service. <br>BlueSSLService implements a very similar protocol and you can get more information on it here:<br> <a href="https://github.com/IBM-Swift/BlueSSLService/blob/master/Sources/SSLService.swift">https://github.com/IBM-Swift/BlueSSLService/blob/master/Sources/SSLService.swift</a><br><br>
<ul><ul><b><font size="2">- onReceive</font></b><br><font size="2"><br>This will be called when data is to be read from an I/O instance. Encrypted data is read from TLS connection associated with that I/O instance and decrypted and written to the buffer passed in.</font><br><font size="2"><br>This is both a client and server method.</font><br><font size="2"><br>///<br>/// Low level reader<br>///<br>/// - Parameters:<br>/// - buffer: Buffer pointer<br>/// - bufSize: Size of the buffer<br>///<br>/// - Returns the number of bytes read. Zero indicates TLS shutdown, less than zero indicates error.<br>///<br>func onReceive(buffer: UnsafeMutableRawPointer, bufSize: Int) throws -> Int</font></ul></ul>> If I understand correctly, `buffer` is an uninitialized memory region that the type should fill with data. Is that correct?<br><br>Otherwise, the same as above, with appropriate substitutions.
<ul><ul><b><font size="4">5 - Non-goals</font></b><br><font size="2"><br>This proposal:</font><br><font size="2"><br>1- DOES NOT describe the TLS service configuration, which includes information on certificate types, formats and chains, cipher suites, etc. We expect this to be specified in a future proposal.</font><br><font size="2"><br>2- DOES NOT describe the TLS service trust policies, which define trust and validation policies of the incoming connection. We expect this to be specified in a future proposal.</font><br><font size="2"><br>3- DOES NOT describe the interface between the TLS service and the transport layer and any dependencies. We expect this to be specified in a future proposal.</font><br></ul></ul>> I feel like #3 in particular really hurts this proposal. It's impossible to evaluate this without at least a general idea of how the TLS service and the transport layer communicate. It's okay to handwave the details—for instance, you could say "Type X represents a network connection, and has methods to read, write, and close it", without describing those methods in detail—but without at least an overview of how this will be used, it's very difficult to evaluate.<br><br>I think this is probably a good design that just isn't being explained very clearly. I hope you can clarify some of these points.<br><br><font size="2">-- </font><br><font size="2">Brent Royal-Gordon</font><br><font size="2">Architechies</font><br><br><br><br><img width="16" height="16" src="cid:1__=8FBB0A64DFC3871B8f9e8a93df938690918c8FB@" border="0" alt="Inactive hide details for Brent Royal-Gordon ---04/01/2017 01:15:32 AM---> On Mar 20, 2017, at 1:04 PM, Gelareh Taban via swift"><font size="2" color="#424282">Brent Royal-Gordon ---04/01/2017 01:15:32 AM---> On Mar 20, 2017, at 1:04 PM, Gelareh Taban via swift-server-dev <swift-server-dev@swift.org> wrote</font><br><br><font size="2" color="#5F5F5F">From: </font><font size="2">Brent Royal-Gordon <brent@architechies.com></font><br><font size="2" color="#5F5F5F">To: </font><font size="2">Gelareh Taban/Austin/IBM@IBMUS</font><br><font size="2" color="#5F5F5F">Cc: </font><font size="2">swift-server-dev@swift.org, Bill Abt/Cambridge/IBM@IBMUS</font><br><font size="2" color="#5F5F5F">Date: </font><font size="2">04/01/2017 01:15 AM</font><br><font size="2" color="#5F5F5F">Subject: </font><font size="2">Re: [swift-server-dev] Draft proposal for TLS Service APIs (please review)</font><br><hr width="100%" size="2" align="left" noshade style="color:#8091A5; "><br><br>
<ul><ul>On Mar 20, 2017, at 1:04 PM, Gelareh Taban via swift-server-dev <<a href="mailto:swift-server-dev@swift.org"><u><font color="#0000FF">swift-server-dev@swift.org</font></u></a>> wrote:
<p><b>4.1 - TLS service protocol</b><br><font size="2"><br>The TLS service protocol describes the methods that the transport layer calls to handle transport-level events for the TLS service object.</font></ul></ul><br>I have a bunch of questions about the design you're presenting, and I think many of them ultimately stem from not understanding some of the high-level aspects of the proposal. For instance:<br><br>* What types conform to this protocol? From the diagram, it looks like there's a type for each "engine"—a SecureTransportService, an OpenSSLService, etc.—and each instance represents a particular configuration of that engine. So you create a WhateverTLSService, configure it, and then hand it off (through several layers) to the transport management layer, which calls methods on it to handle various events. The transport management layer then uses that one TLSService to handle many connections. Is that correct?<br><br>* What is the lifecycle of a connection? Does the TLSService create them itself, does the transport management layer create them and hand them off, or does the transport management layer retain control over them from beginning to end?<br><br>* You discuss the higher layers creating a TLSService object and caching it in a property, then ultimately handing it down to the transport management layer, which then attaches it to socket objects. But presumably you can have many socket objects, possibly simultaneously. Are they all served by a single TLSService instance, or by many? If they share a TLSService, how does the TLSService know which socket is talking to it at a given moment? If they have separate ones, how does the transport management layer acquire a new one when it needs it?<br><br>You mention that this proposal is very small in scope, and it's fine to describe some of these details in general ways. For instance, you don't need to describe the interface to a Swift socket or the transport layer in detail. But currently, the description of these surrounding systems is *so* vague that I'm struggling to assess this design.<br><br>Perhaps some of these details have been described in other documents or meetings; if so, they really need to be presented in this document, too.
<ul><ul><b><font size="2">- onClientCreate</font></b></ul></ul>Why do these methods all have "on" prefixes? I'm not totally sure I understand the intended usage here, but I see two possibilities:<br><br>* These are imperative commands. `onAccept` says that the TLS engine should accept a connection, `onSend` means it should send some data, etc. In that case, these should not have any prefix—they should just be `accept`, `send`, etc.<br><br>* These are essentially delegate methods notifying the TLS engine of an event. `onAccept` says that the system has accepted a connection and the TLS engine should do what it needs to do with it, `onSend` means the system is about to send data and it needs the TLS engine to modify it, etc. If so, Swift APIs more often use words like `should`, `will`, or `did` than `on`, particularly since they're more precise about the timing of the delegate method compared to the actual event.<br><br>In either case, I don't think "on" is the best naming for these. It needlessly bucks platform conventions.
<ul><ul><font size="2">This will be called when a client I/O connection is created and appropriate TLS connection needs to be configured, including context creation, the handshake and connection verification.</font><br><font size="2"><br>This is a client only method.</font><br><font size="2"><br>///<br>/// Setup the contexts and process the TLSService configurations (certificates, etc)<br>///</font><br><font size="2"><br>func onClientCreate() throws</font><p><b><font size="2">- onServerCreate</font></b><br><font size="2"><br>This will be called when a server I/O connection is created and appropriate TLS connection needs to be setup, including context creation, the handshake and connection verification.</font><br><font size="2"><br>This is a server only method.</font><br><font size="2"><br>///<br>/// Setup the contexts and process the TLSService configurations (certificates, etc)<br>///</font><br><font size="2"><br>func onServerCreate() throws</font></ul></ul>What are these methods supposed to do, exactly?<br><br>* Do they put `self` into either a client state or a server state? If so, what happens if you call both, or neither, or call one twice? Would it be better to do this as part of initialization, or to have them make a client TLS object or server TLS object, or to require whatever code hands the TLSService to the TransportManager to pre-configure it as either client or server?<br><br>* Do they create a new instance that's either a client or a server? If so, how do they return it?<br><br>* Do they configure something recently created as either client or server? If so, how do they access whatever they need to configure?<br><br>Basically, what state are these supposed to operate upon?
<ul><ul><b><font size="2">- onDestroy</font></b><br><font size="2"><br>This will be called when an I/O instance connection is closed and any remaining TLS context needs to be destroyed.</font><br><font size="2"><br>This is both a client and server method.</font><br><font size="2"><br>///<br>/// Destroy any remaining contexts <br>///<br>func onDestroy()</font></ul></ul>Is this called at the end of each connection, or is it called once when the transport management layer is totally finished with the TLSService, or are these the same thing?<br><br>If per-connection, how does it know which connection?<br><br>If during destruction, should we just class-constrain and use `deinit` for this purpose?
<ul><ul><b><font size="2">- onAccept</font></b><br><font size="2"><br>This will be called once an I/O instance connection has been accepted, to setup the TLS connection, do the handshake and connection verification.</font><br><font size="2"><br>This is both a client and server method.</font><br><font size="2"><br>///<br>/// Processing on acceptance from a listening connection<br>/// <br>///<br>/// - Parameter IORef: The connected I/O instance<br>///<br>func onAccept(IORef: TransportManagementDelegate) throws</font></ul></ul>I take it the parameter is some sort of abstraction wrapping e.g. a socket. So why is it called a `TransportManagementDelegate`? Shouldn't its name include words like `Connection` or `Socket` or `IOHandle` or something?<br><br>Do we want the parameter to be labeled `IORef`? That's not very idiomatic; it doesn't read well or follow the Swift naming guidelines.<br><br>You say this is for both clients and servers. When does a TLS client have a listening connection that it `accept`s connections on?<br><br>Is it called at different times or in different ways than `onServerCreate`?
<ul><ul><b><font size="2">- onConnect</font></b><br><font size="2"><br>This will be called once a socket connection has been made, to setup the TLS connection, do the handshake and connection verification.</font><br><font size="2"><br>This is both a client and server method.</font><br><font size="2"><br>///<br>/// Processing on connection to a listening connection<br>///<br>/// - Parameter connectionRef: The connected I/O instance<br>///<br>func onConnect(IORef: TransportManagementDelegate) throws</font></ul></ul>The same as above, with appropriate substitutions.
<ul><ul><b><font size="2">- onSend</font></b><br><font size="2"><br>This will be called when data is to be written to an I/O instance. The input data buffer is written to the TLS connection associated with that I/O instance.</font><br><font size="2"><br>This is both a client and server method.</font><br><font size="2"><br>///<br>/// Low level writer<br>///<br>/// - Parameters:<br>/// - buffer: Buffer pointer<br>/// - bufSize: Size of the buffer<br>///<br>/// - Returns the number of bytes written. Zero indicates TLS shutdown, less than zero indicates error.<br>///<br>func onSend(buffer: UnsafeRawPointer, bufSize: Int) throws -> Int</font></ul></ul>Is there a reason you use an UnsafeRawPointer and a buffer size, instead of using an UnsafeRawBufferPointer which would encapsulate both?<br><br>Why is shutdown indicated with zero, rather than the return value being Optional and being nil? Why are errors signaled with negative values instead of being thrown? (Or are you saying that negative returns are invalid? That's different from saying "indicates error".)<br><br>If a TLSService return less than `bufSize`, will the enclosing later try to.re-send the remaining data in subsequent calls?<br><br>This sounds like the TLS engine owns the network connection (at least by this point) and is responsible for writing to it. Does that mean `accept` and `connect` take ownership of the connection and hold on to it? If you have several different simultaneous connections, how do you know which connection this should write to? Or does a given TLSService only own one connection at a time? If so, does the transport management layer create a new TLSService instance for each connection? How? If each TLSService is bound to one connection, shouldn't it be created already knowing the connection it's going to use?
<ul><ul><b><font size="2">- onReceive</font></b><br><font size="2"><br>This will be called when data is to be read from an I/O instance. Encrypted data is read from TLS connection associated with that I/O instance and decrypted and written to the buffer passed in.</font><br><font size="2"><br>This is both a client and server method.</font><br><font size="2"><br>///<br>/// Low level reader<br>///<br>/// - Parameters:<br>/// - buffer: Buffer pointer<br>/// - bufSize: Size of the buffer<br>///<br>/// - Returns the number of bytes read. Zero indicates TLS shutdown, less than zero indicates error.<br>///<br>func onReceive(buffer: UnsafeMutableRawPointer, bufSize: Int) throws -> Int</font></ul></ul>If I understand correctly, `buffer` is an uninitialized memory region that the type should fill with data. Is that correct?<br><br>Otherwise, the same as above, with appropriate substitutions.
<ul><ul><b><font size="4">5 - Non-goals</font></b><br><font size="2"><br>This proposal:</font><br><font size="2"><br>1- DOES NOT describe the TLS service configuration, which includes information on certificate types, formats and chains, cipher suites, etc. We expect this to be specified in a future proposal.</font><br><font size="2"><br>2- DOES NOT describe the TLS service trust policies, which define trust and validation policies of the incoming connection. We expect this to be specified in a future proposal.</font><br><font size="2"><br>3- DOES NOT describe the interface between the TLS service and the transport layer and any dependencies. We expect this to be specified in a future proposal.</font><br></ul></ul>I feel like #3 in particular really hurts this proposal. It's impossible to evaluate this without at least a general idea of how the TLS service and the transport layer communicate. It's okay to handwave the details—for instance, you could say "Type X represents a network connection, and has methods to read, write, and close it", without describing those methods in detail—but without at least an overview of how this will be used, it's very difficult to evaluate.<br><br>I think this is probably a good design that just isn't being explained very clearly. I hope you can clarify some of these points.<br><br><font size="2">-- </font><br><font size="2">Brent Royal-Gordon</font><br><font size="2">Architechies</font><br><br><br><BR>
</body></html>