[swift-evolution] Pitch: Progress Tracking in Swift

[Tony Parker]

Hi Charles,

I see Rob already pointed this out, but I’ll respond as well since I own NSProgress and I also was involved in the WWDC session he references.

NSProgress actually received a significant upgrade in iOS 9 and OS X 10.11 to address most of your concerns. I’ve never really liked the ‘current’ progress idea either, so I’ve changed Foundation to make using explicit progress objects possible (and encouraged). There are still cases where the idea of having a current progress is useful. For example, when there is no opportunity to grab a progress object, because the method you are calling is a proxy, or a global function, etc.).

> On Jan 17, 2016, at 6:14 PM, Charles Srstka via swift-evolution <swift-evolution at swift.org> wrote:
> 
> Introduction:
> 
> This is a proposal for a native progress-tracking mechanism in Swift.
> 
> Motivation:
> 
> As most of us know, Foundation includes NSProgress, a class that makes it fairly simple to implement progress tracking in complex applications. What makes NSProgress nice is the way that it builds a tree of progress objects, so that each method or function only needs to take into account the work that needs to be done in that specific context, and the parent NSProgress object can interpret that progress in the context of its own work to be done. NSProgress also provides a support for common issues like checking whether the user has cancelled an operation. However, there are a few issues with NSProgress:
> 
> 1. Because NSProgress relies on a thread-local global variable to store the current progress object, it is impossible to know, outside of documentation, whether any particular method supports NSProgress or not, and if the documentation is inadequate, one must resort to trial-and-error to determine whether NSProgress is supported (this has been noted before: http://oleb.net/blog/2014/03/nsprogress/).


You can now create progress trees explicitly. There are two parts to this:

    @available(OSX 10.11, *)
    public /*not inherited*/ init(totalUnitCount unitCount: Int64, parent: NSProgress, pendingUnitCount portionOfParentTotalUnitCount: Int64)

and a new protocol:

public protocol NSProgressReporting : NSObjectProtocol {
    @available(OSX 10.9, *)
    public var progress: NSProgress { get }
}

So you can mark classes that publish progress with the protocol, and clients can create their trees using the new initializer.

> 
> 2. NSProgress relies on a paired register-deregister pattern in order to obtain and release "current" status. One must balance every becomeCurrentWithPendingUnitCount() call with a resignCurrent() call, or Bad Things™ will happen. However, this pattern does not work well with Swift's error handling mechanism:
> 
> progress.becomeCurrentWithPendingUnitCount(1)
> try self.doSomeThingThatMightThrow()
> progress.resignCurrent() // If an error occurs, this will never be called!

Yes, agreed. I would encourage the discrete progress if possible, but I’ve had lots of requests for a block-based version of becomeCurrent. This is another good reason to add one.

> 
> 3. It is very easy to accidentally add a child NSProgress object if one does not realize that a particular method supports NSProgress. A couple ways this can happen:
> 
> func doSomething() {
>    let progress = NSProgress(totalUnitCount: 10)
> 
>    progress.becomeCurrentWithPendingUnitCount(10)
>    doSomethingThatSupportsNSProgress()
>    doSomethingThatWeDontExpectSupportsNSProgressButDoes() // whoops, we just picked up a child accidentally
>    progress.resignCurrent()
> }
> 
> func doSomething() {
>    let progress = NSProgress(totalUnitCount: 10)
> 
>    progress.becomeCurrentWithPendingUnitCount(10)
>    doSomethingThatSupportsNSProgress()
>    progress.resignCurrent()
> 
>    doSomethingThatWeDontExpectSupportsNSProgressButDoes() // whoops, this one just picked up our parent's NSProgress and made it our sibling
> }
> 
> This is particularly problematic when you consider the obvious workaround for the problem described in #2:
> 
> func doSomething() {
>    let progress = NSProgress(totalUnitCount: 10)
> 
>    progress.becomeCurrentWithPendingUnitCount(10)
>    defer { progress.resignCurrent() }
> 
>    doSomethingThatSupportsNSProgress()
>    doSomethingThatWeDontExpectSupportsNSProgressButDoes() // whoops
> }
> 
> I haven't figured out exactly how to reproduce this reliably yet, but I have noticed that NSProgress objects can sometimes end up with a fractionCompleted larger than 1.0 when this occurs.

I changed the algorithm for how we add child progress objects, so now we only will add the first one. This wasn’t strictly backwards compatible, but it was far too easy to add more progress objects than you intended.

> 
> 4. Because NSProgress uses a thread-local global variable for the current progress, one must be vigilant about jumping through specific hoops when spinning off a task in a new thread, or else the child task's NSProgress object will not be properly connected to its parent.
> 

Yup, again another problem solved by using the discrete progress objects.

> 5. NSProgress posts KVO notifications on the main thread.

This actually isn’t true. NSProgress posts KVO notifications on the thread in which the change was made.

> In addition to complicating the interface and causing undefined behavior if one mistakenly binds it to a UI element without inserting something in between to forward the KVO notifications to the main thread, this also hurts performance in the worker thread, since KVO notifications carry a non-trivial performance penalty, involving not only message sends but also packaging things into an NSDictionary for the "change" parameter. Notifications for multiple properties are fired every time completedUnitCount is changed, which then can cause similar KVO notifications to occur up the family tree, depending on how many KVO observers are installed. This can show up as a non-trivial performance cost in Instruments if the NSProgress object is updated frequently enough.
> 

There is a cost associated with updating progress, it’s true. We can always do more to optimize it (and we continue to do so), but there comes a point at which the updater of the progress object has to add some intelligence of its own. This is why the documentation and release notes encourage callers to avoid doing things like updating the progress for each byte in a file (because that is far too many updates).

- Tony

> Proposed Solution:
> 
> I propose a system that follows the example of Swift's excellent do/try/catch error-handling system. A "reports" keyword can be added to a function or method's declaration, similar to "throws":
> 
> func myFunc() reports -> () {
>    ...
> }
> 
> By default, the "reports" keyword will introduce a "let" constant named "progress" to the function's scope, much like "catch" implicitly adds "error". The name of this constant could be customized, if desired:
> 
> func myFunc() reports(myNiftyProgressObject) {
>    ...
> }
> 
> The created progress object is automatically added as a child to the progress object which is implicitly passed into the function by the calling function or method.
> 
> The interface to the progress object would be similar to NSProgress, with a "prepare" method taking the total unit count, as well as a "cancelled" property and a "cancel()" function helping keep track of whether or not an operation is cancelled. However, the "becomeCurrent" and "resignCurrent" methods are replaced with a "report" keyword which precedes a function or method call, similar to "try", although the "report" keyword includes a parameter taking the number of units corresponding to this operation:
> 
> func myFunc() throws, reports {
>    progress.prepare(10)
> 
>    report(5) foo()
> 
>    if progress.cancelled { throw NSCocoaError.UserCancelledError }
> 
>    report(5) try bar() // yes, "report" can be combined with "try"
> }
> 
> Spinning off an operation onto another thread or queue is no problem:
> 
> func myFunc() reports {
>    progress.prepare(10)
> 
>    dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0)) {
>        report(10) foo() // the progress object is passed as a parameter to the function, so no need to specially make it current on this thread first
>    }
> }
> 
> Unlike "try", "report" is not required. If we don't care about the progress of a method or function, we can call it without "report" and its progress will be ignored. In this example, the "someRelativelyTrivialOperationThatReports"'s progress object will *not* be added as a child to "progress":
> 
> func myFunc() reports {
>    progress.prepare(10)
> 
>    someRelativelyTrivialOperationThatReports()
>    report(10) someNonTrivialOperationThatReports()
> }
> 
> If a function calls only one reporting function, the "prepare" method can be omitted and the parameter can be left off of "report", and the parent progress object will simply be passed through:
> 
> func myFunc() reports {
>    report doAllTheActualWork()
> }
> 
> To pass in a progress object other than the default progress object (for example, when creating the object at the very top of the tree), simply pass the progress object in as an optional second parameter to "reports":
> 
> func startEverything() {
>    let progress = Progress()
> 
>    progress.prepare(10)
> 
>    report(10, progress) someFuncThatReports()
> }
> 
> In the function that does the actual work, of course the progress object has a "completed" property that tracks the completed units:
> 
> func doTheActualWork() reports {
>    let someArrayOfThings = ...
> 
>    progress.prepare(someArrayOfThings.count)
> 
>    for eachThing in someArrayOfThings {
>        // do the work
> 
>        progress.completed += 1
>    }
> }
> 
> Observing the progress object uses a closure-based system, containing a "willUpdateHandler", a "didUpdateHandler", and a "cancelHandler":
> 
> progress.didUpdateHandler = { (completed: Int, total: Int) -> () in
>    print("\(completed) units complete out of \(total)")
> }
> 
> progress.cancelHandler = {
>    print("the user cancelled the operation")
> }
> 
> The separation between will- and did- handlers is to facilitate interoperability with NSProgressIndicator and other Objective-C objects; one could call willChangeValueForKey() and didChangeValueForKey() in the handlers to let the KVO system know about the changes if needed. If desired, a small wrapper could be added to Foundation to translate the progress object's notifications into KVO notifications.
> 
> The progress object also includes a property specifying the dispatch queue on which the handlers will be run. By default this is the main queue, but this can be customized as needed:
> 
> progress.dispatchQueue = myDispatchQueue
> 
> Of course, any changes to the "completed" property in a child progress object would be bubbled up the family tree, causing any handlers on the parent progress objects to fire, and any changes to the "cancelled" property in a parent progress object would be propagated to its children, similar to how NSProgress currently works.
> 
> If there is interest in this, I could flesh out the interface and implementation to the progress object a bit.
> 
> What do you all think?
> 
> Charles
> 
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20160119/48b2580a/attachment.html>