[swift-server-dev] HTTP (server) APIs sketch of what we use internally

[Johannes Weiß]

Hi swift-server-dev,

First of all, sorry for the delay!

On the last HTTP meeting we were talking about how to represent HTTP/1.1 and how to handle trailers and request/response body streaming. I was describing what we use internally. There was some interest to see the actual APIs, please find them below.

The sketch doesn't cover the full spectrum of what we want to achieve. The main thing that's missing is that there is currently no data type for a HTTP response. We just build the response on the fly using the HTTPResponseWriter. IMHO that can easily be added by basically making the response headers, response code, etc a new type HTTPResponse. HTTPResponseWriter would then take a value of that.

The main design choices we made are
 - value type for HTTP request
 - having the HTTP body (&trailers) not part of the request type as we need to support request body streaming
 - writing the response is asynchronous, the result (success/failure) of the operation reported in the callback
 - headers not being a dictionary but its own data structure, basically [(String, String)] but when queried with a String, you'll get all the values as a list of strings. (headers["set-cookie"] returns ["foo", "bar"] for "Set-Cookie: foo\r\nSET-COOKIE: bar\r\n")

Not sure if that's interesting but our stuff is implemented on top of DispatchIO. We support both request body as well as response body streaming without sacrificing a thread per connection/request.

For HTTP, this is our lowest level API, we built a higher-level web framework on top of that and so far we're happy with what we got.

If you're interested, there's some very simple demo code (HTTP echo server) below the APIs.

Please let us know what you think.

--- SNIP ---
/* a web app is a function that gets a HTTPRequest and a HTTPResponseWriter and returns a function which processes the HTTP request body in chunks as they arrive */
public typealias WebApp = (HTTPRequest, HTTPResponseWriter) -> HTTPBodyProcessing

public struct HTTPRequest {
   public let method : HTTPMethod
   public let target : String /* e.g. "/foo/bar?buz=qux" */
   public let httpVersion : HTTPVersion
   public let headers : HTTPHeaders
}

public protocol HTTPResponseWriter: class {
   func writeResponse(status: HTTPResponseStatus, transferEncoding: HTTPTransferEncoding)

   func writeHeader(key: String, value: String)

   func writeTrailer(key: String, value: String)

   func writeBody(data: DispatchData) /* convenience */
   func writeBody(data: Data) /* convenience */
   func writeBody(data: DispatchData, completion: @escaping (Result<POSIXError, ()>) -> Void)
   func writeBody(data: Data, completion: @escaping (Result<POSIXError, ()>) -> Void)

   func done() /* convenience */
   func done(completion: @escaping (Result<POSIXError, ()>) -> Void))
   func abort()
}

public typealias HTTPBodyHandler = (HTTPBodyChunk) -> Void

public enum HTTPBodyProcessing {
    case discardBody /* if you're not interested in the body, equivalent to `.processBody { _ in }` */
    case processBody(handler: HTTPBodyHandler)
}

public enum HTTPBodyChunk {
   case chunk(data: DispatchData) /* a new bit of the HTTP request body has arrived */
   case failed(error: HTTPParserError) /* error while streaming the HTTP request body, eg. connection closed */
   case trailer(key: String, value: String) /* trailer has arrived (this we actually haven't implemented yet) */
   case end /* body and trailers finished */
}

public struct HTTPHeaders : Sequence {
   private let storage: [String:[String]]     /* lower cased keys */
   private let original: [(String, String)]   /* original casing */
   public var description: String { return original.description }

   public subscript(key: String) -> [String]
   public func makeIterator() -> IndexingIterator<Array<(String, String)>>
}

/* from here on just for completeness really */

public typealias HTTPVersion = (Int, Int)

public enum HTTPTransferEncoding {
   case identity(contentLength: UInt)
   case chunked
}

public enum HTTPResponseStatus: Equatable {
   /* use custom if you want to use a non-standard response code or
      have it available in a (UInt, String) pair from a higher-level web framework. */
   case custom(code: UInt, reasonPhrase: String)

   /* all the codes from http://www.iana.org/assignments/http-status-codes */
   case `continue`
   case switchingProtocols
   case processing
   case ok
   case created
   case accepted
   case nonAuthoritativeInformation
   case noContent
   case resetContent
   case partialContent
   case multiStatus
   case alreadyReported
   case imUsed
   case multipleChoices
   case movedPermanently
   case found
   case seeOther
   case notModified
   case useProxy
   case temporaryRedirect
   case permanentRedirect
   case badRequest
   case unauthorized
   case paymentRequired
   case forbidden
   case notFound
   case methodNotAllowed
   case notAcceptable
   case proxyAuthenticationRequired
   case requestTimeout
   case conflict
   case gone
   case lengthRequired
   case preconditionFailed
   case payloadTooLarge
   case uriTooLong
   case unsupportedMediaType
   case rangeNotSatisfiable
   case expectationFailed
   case misdirectedRequest
   case unprocessableEntity
   case locked
   case failedDependency
   case upgradeRequired
   case preconditionRequired
   case tooManyRequests
   case requestHeaderFieldsTooLarge
   case unavailableForLegalReasons
   case internalServerError
   case notImplemented
   case badGateway
   case serviceUnavailable
   case gatewayTimeout
   case httpVersionNotSupported
   case variantAlsoNegotiates
   case insufficientStorage
   case loopDetected
   case notExtended
   case networkAuthenticationRequired
}

public enum HTTPMethod {
   /* everything that http_parser.[ch] supports */
   case DELETE
   case GET
   case HEAD
   case POST
   case PUT
   case CONNECT
   case OPTIONS
   case TRACE
   case COPY
   case LOCK
   case MKCOL
   case MOVE
   case PROPFIND
   case PROPPATCH
   case SEARCH
   case UNLOCK
   case BIND
   case REBIND
   case UNBIND
   case ACL
   case REPORT
   case MKACTIVITY
   case CHECKOUT
   case MERGE
   case MSEARCH
   case NOTIFY
   case SUBSCRIBE
   case UNSUBSCRIBE
   case PATCH
   case PURGE
   case MKCALENDAR
   case LINK
   case UNLINK
}
--- SNAP ---

Here's the demo code for a simple echo server

--- SNIP ---
serve { (req, res) in
    if req.target == "/echo" {
        guard req.httpVersion == (1, 1) else {
            /* HTTP/1.0 doesn't support chunked encoding */
            res.writeResponse(status: .httpVersionNotSupported, transferEncoding: .identity(contentLength: 0))
            res.done()
            return .discardBody
        }
        res.writeResponse(status: .ok, transferEncoding: .chunked)
        return .processBody { chunk in
            switch chunk {
                case .chunk(let data):
                    res.writeBody(data: data)
                case .end:
                    res.done()
                default:
                    res.abort()
            }
        }
    } else { ... }
}
--- SNAP ---

-- 
  Johannes