[swift-users] file io in Swift 2.2 for Linux (would like to be pointed in the right direction)

Jens Alfke jens at mooseyard.com
Fri May 20 10:47:57 CDT 2016

> On May 20, 2016, at 1:16 AM, Quinn The Eskimo! via swift-users <swift-users at swift.org> wrote:
> One of the nice things about the file system is that it has reasonable performance and error characteristics such that you /can/ deal with it synchronously.  Revel in that fact!

That’s only really true at small scales, for local filesystems, under light I/O loads. This happens to be true most of the time* for typical Mac/iOS app needs, but it’s not universal.

I believe most people in the Swift community today come from a Mac/iOS background, but as Linux support matures, and with IBM’s backing, that’s going to be less and less true in the future. Me, I’m one of the oddball “mobile guys” at a database company [Couchbase] whose customers [eBay, PayPal, Citi, etc.] very commonly have multi-terabyte data sets.

> Consider the NSXMLDocument example again.  This high-level one-shot API is layered on top of a low-level one-shot API.

The trouble with this is that those one-shot APIs tend to suck when working with data coming from the network, because you have to buffer the entire data set in memory, and you can’t start parsing it until the entire response is finished. If it does finish — there are protocols like XMPP/Jabber that use indefinitely-long streams of structured data like XML and can’t be parsed at all with one-shot APIs. I’ve recently had to replace NSJSONSerialization with a 3rd-party streaming JSON parser library, because the latency of reading large JSON documents over HTTP was getting in the way of performance and memory requirements.

Sorry if this is off-topic! What’s relevant is that I’d hate to see Apple’s historical bias toward one-shot I/O get in the way of Swift’s Foundation framework having a kick-ass stream API.


* But not always. I can attest that during OS startup, when dozens of processes are contending for the disk, reading a 10MB file can take an achingly long time by normal standards. This caused lots of “fun” during performance tuning of the PubSub framework. Also, remember the days when devs at Apple still had NFS-based home directories?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-users/attachments/20160520/2d8422a4/attachment.html>

More information about the swift-users mailing list