[swift-server-dev] Draft proposal for TLS Service APIs (please review)
Gelareh Taban
gtaban at us.ibm.com
Mon Apr 3 14:40:29 CDT 2017
Hi Brent,
The answers are in line.
gelareh
4.1 - TLS service protocol
The TLS service protocol describes the methods that the transport
layer calls to handle transport-level events for the TLS service
object.
I have a bunch of questions about the design you're presenting, and I think
many of them ultimately stem from not understanding some of the high-level
aspects of the proposal. For instance:
> * What types conform to this protocol? From the diagram, it looks like
there's a type for each "engine"—a SecureTransportService, an
OpenSSLService, etc.—and each instance represents a particular
configuration of that engine. So you create a WhateverTLSService, configure
it, and then hand it off (through several layers) to the transport
management layer, which calls methods on it to handle various events. The
transport management layer then uses that one TLSService to handle many
connections. Is that correct?
[g]
More or less. As long as the implementation conforms to the protocol, it
can use whatever underlying security library it wants. This way we can have
a plug and play architecture which allows the user to pick the security
library implementation of its choice (eg. LibreSSL, OpenSSL, etc)
> * What is the lifecycle of a connection? Does the TLSService create them
itself, does the transport management layer create them and hand them off,
or does the transport management layer retain control over them from
beginning to end?
[g] The connection life cycle is handled by levels higher than TLS. For all
intents and purposes, the transport management layers owns a TLS service
delegate which calls the appropriate TLS-related functionality at the right
time (eg. at socket creation time, at connection time, etc).
> * You discuss the higher layers creating a TLSService object and caching
it in a property, then ultimately handing it down to the transport
management layer, which then attaches it to socket objects. But presumably
you can have many socket objects, possibly simultaneously. Are they all
served by a single TLSService instance, or by many? If they share a
TLSService, how does the TLSService know which socket is talking to it at a
given moment? If they have separate ones, how does the transport management
layer acquire a new one when it needs it?
[g]
Each socket instance would have its own TLS service delegate -- this is
important because each socket might have its own specific TLS channel
properties.
> You mention that this proposal is very small in scope, and it's fine to
describe some of these details in general ways. For instance, you don't
need to describe the interface to a Swift socket or the transport layer in
detail. But currently, the description of these surrounding systems is *so*
vague that I'm struggling to assess this design.
Perhaps some of these details have been described in other documents or
meetings; if so, they really need to be presented in this document, too.
[g] fair enough. The problem is that right now in the servers working
group, we have not defined *any* of our interfaces. This is the first
proposal and we are trying to only pin the things which this service
requires.
In any case, if you think other things should be included, please let us
know.
- onClientCreate
> Why do these methods all have "on" prefixes? I'm not totally sure I
understand the intended usage here, but I see two possibilities:
> * These are imperative commands. `onAccept` says that the TLS engine
should accept a connection, `onSend` means it should send some data, etc.
In that case, these should not have any prefix—they should just be
`accept`, `send`, etc.
> * These are essentially delegate methods notifying the TLS engine of an
event. `onAccept` says that the system has accepted a connection and the
TLS engine should do what it needs to do with it, `onSend` means the system
is about to send data and it needs the TLS engine to modify it, etc. If so,
Swift APIs more often use words like `should`, `will`, or `did` than `on`,
particularly since they're more precise about the timing of the delegate
method compared to the actual event.
> In either case, I don't think "on" is the best naming for these. It
needlessly bucks platform conventions.
[g]
The latter description is the intended. Your points are fair and perhaps we
can modify the delegate method names to, perhaps:
onClientCreate --> didClientCreate
onServerCreate --> didServerCreate
onDestroy --> willDestroy
onAccept --> didAccept(on connection: TransportManagementDelegate)
onConnect --> didConnect(on connection: TransportManagementDelegate)
onSend --> willSend(with data: UnsafeRawPointer, dataSize: Int))
onReceive --> willReceive(with data: UnsafeRawPointer, dataSize: Int))
This will be called when a client I/O connection is created and
appropriate TLS connection needs to be configured, including context
creation, the handshake and connection verification.
This is a client only method.
///
/// Setup the contexts and process the TLSService configurations
(certificates, etc)
///
func onClientCreate() throws
- onServerCreate
This will be called when a server I/O connection is created and
appropriate TLS connection needs to be setup, including context
creation, the handshake and connection verification.
This is a server only method.
///
/// Setup the contexts and process the TLSService configurations
(certificates, etc)
///
func onServerCreate() throws
> What are these methods supposed to do, exactly?
> * Do they put `self` into either a client state or a server state? If so,
what happens if you call both, or neither, or call one twice? Would it be
better to do this as part of initialization, or to have them make a client
TLS object or server TLS object, or to require whatever code hands the
TLSService to the TransportManager to pre-configure it as either client or
server?
[g]
They setup the TLS contexts and process the certificates. They do put self
in either a client or server state via associated parameter sets in OpenSSL
and SecureTransport.
(I am pretty sure) if you call them multiple times, they simply over write
the previous setting. But we can test this during implementation.
These methods are essentially initialization that gets called when the
TransportManager decides it has enough information to set it as client or
server.
> * Do they create a new instance that's either a client or a server? If
so, how do they return it?
> * Do they configure something recently created as either client or
server? If so, how do they access whatever they need to configure?
> Basically, what state are these supposed to operate upon?
[g]
Most of the functionality is the same for client or server except for
setting the server/client flag/function in the underlying security library.
The instance is stored in the delegate.
- onDestroy
This will be called when an I/O instance connection is closed and any
remaining TLS context needs to be destroyed.
This is both a client and server method.
///
/// Destroy any remaining contexts
///
func onDestroy()
> Is this called at the end of each connection, or is it called once when
the transport management layer is totally finished with the TLSService, or
are these the same thing?
> If per-connection, how does it know which connection?
> If during destruction, should we just class-constrain and use `deinit`
for this purpose?
[g] this is per-connection and the TLS service would keep a context for the
connection.
So the transport manager has a delegate instance pointing to this TLS
service instance which itself has a context.
- onAccept
This will be called once an I/O instance connection has been
accepted, to setup the TLS connection, do the handshake and
connection verification.
This is both a client and server method.
///
/// Processing on acceptance from a listening connection
///
///
/// - Parameter IORef: The connected I/O instance
///
func onAccept(IORef: TransportManagementDelegate) throws
> I take it the parameter is some sort of abstraction wrapping e.g. a
socket. So why is it called a `TransportManagementDelegate`? Shouldn't its
name include words like `Connection` or `Socket` or `IOHandle` or
something?
[g] we can simply call it ConnectionDelegate. I was trying to be consistent
in my terminology.
> Do we want the parameter to be labeled `IORef`? That's not very
idiomatic; it doesn't read well or follow the Swift naming guidelines.
[g] great feedback. Thanks!
> You say this is for both clients and servers. When does a TLS client have
a listening connection that it `accept`s connections on?
Is it called at different times or in different ways than `onServerCreate`?
[g] Bad copy/paste! for both onConnect and onAccept. They are obviously
server and client specific.
- onConnect
This will be called once a socket connection has been made, to setup
the TLS connection, do the handshake and connection verification.
This is both a client and server method.
///
/// Processing on connection to a listening connection
///
/// - Parameter connectionRef: The connected I/O instance
///
func onConnect(IORef: TransportManagementDelegate) throws
The same as above, with appropriate substitutions.
- onSend
This will be called when data is to be written to an I/O instance.
The input data buffer is written to the TLS connection associated
with that I/O instance.
This is both a client and server method.
///
/// Low level writer
///
/// - Parameters:
/// - buffer: Buffer pointer
/// - bufSize: Size of the buffer
///
/// - Returns the number of bytes written. Zero indicates TLS
shutdown, less than zero indicates error.
///
func onSend(buffer: UnsafeRawPointer, bufSize: Int) throws -> Int
> Is there a reason you use an UnsafeRawPointer and a buffer size, instead
of using an UnsafeRawBufferPointer which would encapsulate both?
[g] it was pointed our rightly by others that it's not good to have either
of them :-) Right now, we are looking at have Data and more efficient
versions.
> Why is shutdown indicated with zero, rather than the return value being
Optional and being nil? Why are errors signaled with negative values
instead of being thrown? (Or are you saying that negative returns are
invalid? That's different from saying "indicates error".)
[g] This is actually implementation details which shouldnt have been
included in this pitch in the first place.
> If a TLSService return less than `bufSize`, will the enclosing later try
to.re-send the remaining data in subsequent calls?
This sounds like the TLS engine owns the network connection (at least by
this point) and is responsible for writing to it. Does that mean `accept`
and `connect` take ownership of the connection and hold on to it? If you
have several different simultaneous connections, how do you know which
connection this should write to? Or does a given TLSService only own one
connection at a time? If so, does the transport management layer create a
new TLSService instance for each connection? How? If each TLSService is
bound to one connection, shouldn't it be created already knowing the
connection it's going to use?
[g] I believe I have talked about the relationship between connections and
TLS service.
BlueSSLService implements a very similar protocol and you can get more
information on it here:
https://github.com/IBM-Swift/BlueSSLService/blob/master/Sources/SSLService.swift
- onReceive
This will be called when data is to be read from an I/O instance.
Encrypted data is read from TLS connection associated with that I/O
instance and decrypted and written to the buffer passed in.
This is both a client and server method.
///
/// Low level reader
///
/// - Parameters:
/// - buffer: Buffer pointer
/// - bufSize: Size of the buffer
///
/// - Returns the number of bytes read. Zero indicates TLS shutdown,
less than zero indicates error.
///
func onReceive(buffer: UnsafeMutableRawPointer, bufSize: Int) throws
-> Int
> If I understand correctly, `buffer` is an uninitialized memory region
that the type should fill with data. Is that correct?
Otherwise, the same as above, with appropriate substitutions.
5 - Non-goals
This proposal:
1- DOES NOT describe the TLS service configuration, which includes
information on certificate types, formats and chains, cipher suites,
etc. We expect this to be specified in a future proposal.
2- DOES NOT describe the TLS service trust policies, which define
trust and validation policies of the incoming connection. We expect
this to be specified in a future proposal.
3- DOES NOT describe the interface between the TLS service and the
transport layer and any dependencies. We expect this to be specified
in a future proposal.
> I feel like #3 in particular really hurts this proposal. It's impossible
to evaluate this without at least a general idea of how the TLS service and
the transport layer communicate. It's okay to handwave the details—for
instance, you could say "Type X represents a network connection, and has
methods to read, write, and close it", without describing those methods in
detail—but without at least an overview of how this will be used, it's very
difficult to evaluate.
I think this is probably a good design that just isn't being explained very
clearly. I hope you can clarify some of these points.
--
Brent Royal-Gordon
Architechies
From: Brent Royal-Gordon <brent at architechies.com>
To: Gelareh Taban/Austin/IBM at IBMUS
Cc: swift-server-dev at swift.org, Bill Abt/Cambridge/IBM at IBMUS
Date: 04/01/2017 01:15 AM
Subject: Re: [swift-server-dev] Draft proposal for TLS Service APIs
(please review)
On Mar 20, 2017, at 1:04 PM, Gelareh Taban via swift-server-dev <
swift-server-dev at swift.org> wrote:
4.1 - TLS service protocol
The TLS service protocol describes the methods that the transport
layer calls to handle transport-level events for the TLS service
object.
I have a bunch of questions about the design you're presenting, and I think
many of them ultimately stem from not understanding some of the high-level
aspects of the proposal. For instance:
* What types conform to this protocol? From the diagram, it looks like
there's a type for each "engine"—a SecureTransportService, an
OpenSSLService, etc.—and each instance represents a particular
configuration of that engine. So you create a WhateverTLSService, configure
it, and then hand it off (through several layers) to the transport
management layer, which calls methods on it to handle various events. The
transport management layer then uses that one TLSService to handle many
connections. Is that correct?
* What is the lifecycle of a connection? Does the TLSService create them
itself, does the transport management layer create them and hand them off,
or does the transport management layer retain control over them from
beginning to end?
* You discuss the higher layers creating a TLSService object and caching it
in a property, then ultimately handing it down to the transport management
layer, which then attaches it to socket objects. But presumably you can
have many socket objects, possibly simultaneously. Are they all served by a
single TLSService instance, or by many? If they share a TLSService, how
does the TLSService know which socket is talking to it at a given moment?
If they have separate ones, how does the transport management layer acquire
a new one when it needs it?
You mention that this proposal is very small in scope, and it's fine to
describe some of these details in general ways. For instance, you don't
need to describe the interface to a Swift socket or the transport layer in
detail. But currently, the description of these surrounding systems is *so*
vague that I'm struggling to assess this design.
Perhaps some of these details have been described in other documents or
meetings; if so, they really need to be presented in this document, too.
- onClientCreate
Why do these methods all have "on" prefixes? I'm not totally sure I
understand the intended usage here, but I see two possibilities:
* These are imperative commands. `onAccept` says that the TLS engine should
accept a connection, `onSend` means it should send some data, etc. In that
case, these should not have any prefix—they should just be `accept`,
`send`, etc.
* These are essentially delegate methods notifying the TLS engine of an
event. `onAccept` says that the system has accepted a connection and the
TLS engine should do what it needs to do with it, `onSend` means the system
is about to send data and it needs the TLS engine to modify it, etc. If so,
Swift APIs more often use words like `should`, `will`, or `did` than `on`,
particularly since they're more precise about the timing of the delegate
method compared to the actual event.
In either case, I don't think "on" is the best naming for these. It
needlessly bucks platform conventions.
This will be called when a client I/O connection is created and
appropriate TLS connection needs to be configured, including context
creation, the handshake and connection verification.
This is a client only method.
///
/// Setup the contexts and process the TLSService configurations
(certificates, etc)
///
func onClientCreate() throws
- onServerCreate
This will be called when a server I/O connection is created and
appropriate TLS connection needs to be setup, including context
creation, the handshake and connection verification.
This is a server only method.
///
/// Setup the contexts and process the TLSService configurations
(certificates, etc)
///
func onServerCreate() throws
What are these methods supposed to do, exactly?
* Do they put `self` into either a client state or a server state? If so,
what happens if you call both, or neither, or call one twice? Would it be
better to do this as part of initialization, or to have them make a client
TLS object or server TLS object, or to require whatever code hands the
TLSService to the TransportManager to pre-configure it as either client or
server?
* Do they create a new instance that's either a client or a server? If so,
how do they return it?
* Do they configure something recently created as either client or server?
If so, how do they access whatever they need to configure?
Basically, what state are these supposed to operate upon?
- onDestroy
This will be called when an I/O instance connection is closed and any
remaining TLS context needs to be destroyed.
This is both a client and server method.
///
/// Destroy any remaining contexts
///
func onDestroy()
Is this called at the end of each connection, or is it called once when the
transport management layer is totally finished with the TLSService, or are
these the same thing?
If per-connection, how does it know which connection?
If during destruction, should we just class-constrain and use `deinit` for
this purpose?
- onAccept
This will be called once an I/O instance connection has been
accepted, to setup the TLS connection, do the handshake and
connection verification.
This is both a client and server method.
///
/// Processing on acceptance from a listening connection
///
///
/// - Parameter IORef: The connected I/O instance
///
func onAccept(IORef: TransportManagementDelegate) throws
I take it the parameter is some sort of abstraction wrapping e.g. a socket.
So why is it called a `TransportManagementDelegate`? Shouldn't its name
include words like `Connection` or `Socket` or `IOHandle` or something?
Do we want the parameter to be labeled `IORef`? That's not very idiomatic;
it doesn't read well or follow the Swift naming guidelines.
You say this is for both clients and servers. When does a TLS client have a
listening connection that it `accept`s connections on?
Is it called at different times or in different ways than `onServerCreate`?
- onConnect
This will be called once a socket connection has been made, to setup
the TLS connection, do the handshake and connection verification.
This is both a client and server method.
///
/// Processing on connection to a listening connection
///
/// - Parameter connectionRef: The connected I/O instance
///
func onConnect(IORef: TransportManagementDelegate) throws
The same as above, with appropriate substitutions.
- onSend
This will be called when data is to be written to an I/O instance.
The input data buffer is written to the TLS connection associated
with that I/O instance.
This is both a client and server method.
///
/// Low level writer
///
/// - Parameters:
/// - buffer: Buffer pointer
/// - bufSize: Size of the buffer
///
/// - Returns the number of bytes written. Zero indicates TLS
shutdown, less than zero indicates error.
///
func onSend(buffer: UnsafeRawPointer, bufSize: Int) throws -> Int
Is there a reason you use an UnsafeRawPointer and a buffer size, instead of
using an UnsafeRawBufferPointer which would encapsulate both?
Why is shutdown indicated with zero, rather than the return value being
Optional and being nil? Why are errors signaled with negative values
instead of being thrown? (Or are you saying that negative returns are
invalid? That's different from saying "indicates error".)
If a TLSService return less than `bufSize`, will the enclosing later try
to.re-send the remaining data in subsequent calls?
This sounds like the TLS engine owns the network connection (at least by
this point) and is responsible for writing to it. Does that mean `accept`
and `connect` take ownership of the connection and hold on to it? If you
have several different simultaneous connections, how do you know which
connection this should write to? Or does a given TLSService only own one
connection at a time? If so, does the transport management layer create a
new TLSService instance for each connection? How? If each TLSService is
bound to one connection, shouldn't it be created already knowing the
connection it's going to use?
- onReceive
This will be called when data is to be read from an I/O instance.
Encrypted data is read from TLS connection associated with that I/O
instance and decrypted and written to the buffer passed in.
This is both a client and server method.
///
/// Low level reader
///
/// - Parameters:
/// - buffer: Buffer pointer
/// - bufSize: Size of the buffer
///
/// - Returns the number of bytes read. Zero indicates TLS shutdown,
less than zero indicates error.
///
func onReceive(buffer: UnsafeMutableRawPointer, bufSize: Int) throws
-> Int
If I understand correctly, `buffer` is an uninitialized memory region that
the type should fill with data. Is that correct?
Otherwise, the same as above, with appropriate substitutions.
5 - Non-goals
This proposal:
1- DOES NOT describe the TLS service configuration, which includes
information on certificate types, formats and chains, cipher suites,
etc. We expect this to be specified in a future proposal.
2- DOES NOT describe the TLS service trust policies, which define
trust and validation policies of the incoming connection. We expect
this to be specified in a future proposal.
3- DOES NOT describe the interface between the TLS service and the
transport layer and any dependencies. We expect this to be specified
in a future proposal.
I feel like #3 in particular really hurts this proposal. It's impossible to
evaluate this without at least a general idea of how the TLS service and
the transport layer communicate. It's okay to handwave the details—for
instance, you could say "Type X represents a network connection, and has
methods to read, write, and close it", without describing those methods in
detail—but without at least an overview of how this will be used, it's very
difficult to evaluate.
I think this is probably a good design that just isn't being explained very
clearly. I hope you can clarify some of these points.
--
Brent Royal-Gordon
Architechies
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-server-dev/attachments/20170403/e9721ed0/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <https://lists.swift.org/pipermail/swift-server-dev/attachments/20170403/e9721ed0/attachment.gif>
More information about the swift-server-dev
mailing list