[swift-evolution] [Concurrency] Fixing race conditions in async/await example

[Xiaodi Wu]

On Mon, Aug 28, 2017 at 16:10 Adam Kemp via swift-evolution <
swift-evolution at swift.org> wrote:

> I know what the proposal said. I’m making a case that there is value in
> doing it differently.
>
> The composability of futures is valuable. Mixing and matching async/await
> with futures is also valuable. The queue-returning behavior that you can
> get from futures is also valuable, and building async/await on top of
> futures means async/await can get that for free.
>

Why couldn't you mix and match async/await and futures and get the
queue-return behavior of futures if futures are built on top of async/await
instead off the other way around?

Maybe you don’t value those things, which is fine. But I do, and maybe
> other people do too. That’s why we’re having a discussion about it.
>
> It can also be valuable having a minimal implementation, but we have to
> acknowledge that it comes with a downside as well. The problem with doing a
> minimal implementation is that you can be stuck with the consequences for a
> long time. I want to make sure that we’re not stuck with the consequences
> of a minimal implementation that doesn’t adequately address the problems
> that async/await should be addressing. I’d hate for Swift to get an
> async/await that is so weak that it has to be augmented by tedious
> boilerplate code before it’s useful.
>
>
> On Aug 28, 2017, at 1:54 PM, Wallacy <wallacyf at gmail.com> wrote:
>
> We don't need to this now!
>
> Again: (Using proposal words)
>
> "It is important to understand that this is proposing compiler support
> that is completely concurrency runtime-agnostic. This proposal does not
> include a new runtime model (like "actors") - it works just as well with
> GCD as with pthreads or another API. Furthermore, unlike designs in other
> languages, it is independent of specific coordination mechanisms, such as
> futures or channels, allowing these to be built as library feature"
>
> and
>
> "This proposal does not formally propose a Future type, or any other
> coordination abstractions. There are many rational designs for futures, and
> a lot of experience working with them. On the other hand, there are also
> completely different coordination primitives that can be used with this
> coroutine design, and incorporating them into this proposal only makes it
> larger."
>
> and
>
> We focus on task-based concurrency abstractions commonly encountered in
> client and server applications, particularly those that are highly event
> driven (e.g. responding to UI events or requests from clients). This does
> not attempt to be a comprehensive survey of all possible options, nor does
> it attempt to solve all possible problems in the space of concurrency.
> Instead, it outlines a single coherent design thread that can be built over
> the span of years to incrementally drive Swift to further greatness.
>
> and
>
> This proposal has been kept intentionally minimal, but there are many
> possible ways to expand this in the future.
>
> ....
>
> The point is: No Future type is indeed proposed yet!
>
> The proposal try to include de "minimum" required to implement a basic
> async/await to solve the problem created by the GCD! (Pyramid of doom)
>
> The question is: How do you do the same using dispatch_async ?
> dispatch_async also does not return nothing to do what you are intentend do
> do!
>
> Algo, by Swift 5 manifesto, there's no compromise to make a "complete"
> concurrency model by this time!
>
> My intention is only make parity to dispatch_async, but also make the
> ground free to make more complex implementation like Futures in another
> round on top of this one.
>
> This 'async T' can be a real type in the future? Maybe will... But doesn't
> matter now! Now we only need to is some kind of type which need to be
> unwrapped using await before use. Maybe this intermediary/virtual type can
> be a real thing and gain some abilities at some point! Maybe a full Future
> type, why not?
>
> Em seg, 28 de ago de 2017 às 17:33, Adam Kemp <adam.kemp at apple.com>
> escreveu:
>
>> How would these anonymous types get composed? If I wanted to implement a
>> function that takes a collection of futures and wait on it, how would I do
>> that? That is, how would I implement the equivalent of C#’s Task.WhenAll
>> and Task.WhenAny methods?
>>
>> More generally, how do you pass one of these typeless futures to some
>> other function so that we can do the waiting there?
>>
>>
>> On Aug 28, 2017, at 1:23 PM, Wallacy <wallacyf at gmail.com> wrote:
>>
>> And that's why I (and others) are suggesting:
>>
>> func processImageData1a() async -> Image {
>>   let dataResource  = async loadWebResource("dataprofile.txt") // No
>> future type here... Just another way to call dispatch_async under the hood.
>>   let imageResource = async loadWebResource("imagedata.dat")
>>
>>   // ... other stuff can go here to cover load latency...
>>
>>   let imageTmp    = await decodeImage(dataResource, imageResource) //
>> Compiles force await call here...
>>   let imageResult = await dewarpAndCleanupImage(imageTmp)
>>   return imageResult
>> }
>>
>> And now we gain all advantages of async/await again without to handle
>> with one more type.
>>
>> Em seg, 28 de ago de 2017 às 17:07, Adam Kemp via swift-evolution <
>> swift-evolution at swift.org> escreveu:
>>
>>> I think the biggest tradeoff is clearer when you look at the examples
>>> from the proposal where futures are built on top of async/await:
>>>
>>> func processImageData1a() async -> Image {
>>>   let dataResource  = Future { await loadWebResource("dataprofile.txt") }
>>>   let imageResource = Future { await loadWebResource("imagedata.dat") }
>>>
>>>   // ... other stuff can go here to cover load latency...
>>>
>>>   let imageTmp    = await decodeImage(dataResource.get(),
>>> imageResource.get())
>>>   let imageResult = await dewarpAndCleanupImage(imageTmp)
>>>   return imageResult
>>> }
>>>
>>>
>>> With this approach you have to wrap each call site to create a future.
>>> Compare to this:
>>>
>>> func processImageData1a() -> Future<Image> {
>>>   let dataResourceFuture  = loadWebResource("dataprofile.txt”);
>>>   let imageResourceFuture = loadWebResource("imagedata.dat”);
>>>
>>>   // ... other stuff can go here to cover load latency...
>>>
>>>   let imageTmp    = await decodeImage(await dataResourceFuture, await
>>> imageResourceFuture)
>>>   let imageResult = await dewarpAndCleanupImage(imageTmp)
>>>   return imageResult
>>> }
>>>
>>>
>>> Here, not only are the explicit wrappers gone, but this function itself
>>> can be used with either await or as a future. You get both options with one
>>> implementation.
>>>
>>> As I’ve mentioned before, C#’s implementation is not tied to any one
>>> particular futures implementation. The Task type is commonly used, but
>>> async/await does not directly depend on Task. Instead it works with any
>>> return type that meets certain requirements (detailed here:
>>> https://blogs.msdn.microsoft.com/pfxteam/2011/01/13/await-anything/).
>>> Swift could do this using a protocol, which can be retroactively applied
>>> using an extension.
>>>
>>> Obviously for this to be useful we would need some kind of existing
>>> future implementation, but at least we wouldn’t be tied to any particular
>>> one. That would mean library maintainers who have already been using their
>>> own futures implementations could quickly adopt async/await in their code
>>> without having to rewrite their futures library or throw wrappers around
>>> every usage of async/await. They could just adopt a protocol (using an
>>> extension, even) and get async/await support for free.
>>>
>>> The downside is that this feature would be specific to the async/await
>>> use case rather than a generic coroutine implementation (i.e., there would
>>> have to be a separate compiler transform for yield return). It’s not clear
>>> to me why it should be a goal to have just one generic coroutine feature.
>>> The real-world usages of async/await and yield return are different enough
>>> that I’m not convinced we could have a single compiler feature that meets
>>> the needs of both cleanly.
>>>
>>> On Aug 27, 2017, at 7:35 PM, Florent Vilmart <florent at flovilmart.com>
>>> wrote:
>>>
>>> Adam, you’re completely right, languages as c# and JS have been through
>>> the path before, (callback, Promises , async/await) I believe Chris’s goal
>>> it to avoid building a promise implementation and go straight to a
>>> coroutines model, which is more deeply integrated with the compiler. I
>>> don’t see a particular trade off, pursuing that route, and the main benefit
>>> is that coroutines can power any asynchronous metaphor (Signals, Streams,
>>> Futures, Promises etc...) which is not true of Futures so i would tend to
>>> think that for the long run, and to maximize usability, async/await/yield
>>> would probably be the way to go.
>>>
>>> On Aug 27, 2017, 22:22 -0400, Adam Kemp <adam.kemp at apple.com>, wrote:
>>>
>>> As has been explained, futures can be built on top of async/await (or
>>> the other way around). You can have the best of both worlds. We are not
>>> losing anything by having this feature. It would be a huge improvement to
>>> have this as an option.
>>>
>>> However, using futures correctly requires more nested closures than you
>>> have shown in your examples to avoid blocking any threads. That's why
>>> you're not seeing the advantage to async/await. You're comparing examples
>>> that have very different behaviors.
>>>
>>> That said, I have also expressed my opinion that it is better to build
>>> async/await on top of futures rather than the other way around. I believe
>>> it is more powerful and cleaner to make async/await work with any arbitrary
>>> future type (via a protocol). The alternative (building futures on top of
>>> async/await) requires more code when the two are mixed. I very much prefer
>>> how it's done in C#, where you can freely mix the two models without having
>>> to resort to ad-hoc wrappers, and you can use async/await with any futures
>>> implementation you might already be using.
>>>
>>> I really think we should be having more discussion about the tradeoffs
>>> between those two approaches, and I'm concerned that some of the opinions
>>> about how C# does it are not based on a clear and accurate understanding of
>>> how it actually works in that language.
>>>
>>> --
>>> Adam Kemp
>>>
>>> On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt at gmail.com>
>>> wrote:
>>>
>>> The async/await is very similar to the proposed Future (as I posed
>>> earlier) with regard to completion-handler code, they both re-write the
>>> imported completion-handler function using a closure, the relevant sentence
>>> from the Async Proposal is:
>>>
>>> "Under the hood, the compiler rewrites this code using nested closures
>>> ..."
>>>
>>>
>>> Unlike the proposed future code the async code is not naturally
>>> parallel, in the running example the following lines from the async code
>>> are run in series, i.e. await blocks:
>>>
>>>   let dataResource  = await loadWebResource("dataprofile.txt")
>>>   let imageResource = await loadWebResource("imagedata.dat")
>>>
>>> The equivalent lines using the proposed Future:
>>>
>>>   let dataResource  = loadWebResource("dataprofile.txt")
>>>   let imageResource = loadWebResource("imagedata.dat")
>>>
>>> Run in parallel and therefore are potentially faster assuming that
>>> resources, like cores and IO, are available.
>>>
>>> Therefore you would be better using a Future than an async, so why
>>> provide an async unless you can make a convincing argument that it allows
>>> you to write a better future?
>>>
>>>   -- Howard.
>>>
>>> On 28 August 2017 at 09:59, Adam Kemp <adam.kemp at apple.com> wrote:
>>>
>>>> This example still has nested closures (to create a Future), and still
>>>> relies on a synchronous get method that will block a thread. Async/await
>>>> does not require blocking any threads.
>>>>
>>>> I’m definitely a fan of futures, but this example isn’t even a good
>>>> example of using futures. If you’re using a synchronous get method then
>>>> you’re not using futures properly. They’re supposed to make it easy to
>>>> avoid writing blocking code. This example just does the blocking call on
>>>> some other thread.
>>>>
>>>> Doing it properly would show the benefits of async/await because it
>>>> would require more nesting and more complex error handling. By simplifying
>>>> the code you’ve made a comparison between proper asynchronous code (with
>>>> async/await) and improper asynchronous code (your example).
>>>>
>>>> That tendency to want to just block a thread to make it easier is
>>>> exactly why async/await is so valuable. You get simple code while still
>>>> doing it correctly.
>>>>
>>>> --
>>>> Adam Kemp
>>>>
>>>> On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution <
>>>> swift-evolution at swift.org> wrote:
>>>>
>>>> The running example used in the white paper coded using a Future is:
>>>>
>>>> func processImageData1() -> Future<Image> {
>>>>     return AsynchronousFuture { _ -> Image in
>>>>         let dataResource  = loadWebResource("dataprofile.txt") //
>>>> dataResource and imageResource run in parallel.
>>>>         let imageResource = loadWebResource("imagedata.dat")
>>>>         let imageTmp      = decodeImage(dataResource.get ??
>>>> Resource(path: "Default data resource or prompt user"), imageResource.get
>>>> ?? Resource(path: "Default image resource or prompt user"))
>>>>         let imageResult   =  dewarpAndCleanupImage(imageTmp.get ??
>>>> Image(dataPath: "Default image or prompt user", imagePath: "Default image
>>>> or prompt user"))
>>>>         return imageResult.get ?? Image(dataPath: "Default image or
>>>> prompt user", imagePath: "Default image or prompt user")
>>>>     }
>>>> }
>>>>
>>>> This also avoids the pyramid of doom; the pyramid is avoided by
>>>> converting continuation-handlers into either a sync or future, i.e. it is
>>>> the importer that eliminates the nesting by translating the code
>>>> automatically.
>>>>
>>>> This example using Future also demonstrates three advantages of Future:
>>>> they are naturally parallel (dataResource and imageResource lines run in
>>>> parallel), they timeout automatically (get returns nil if the Future has
>>>> taken too long), and if there is a failure (for any reason including
>>>> timeout) it provides a method of either detecting the failure or providing
>>>> a default (get returns nil on failure).
>>>>
>>>> There are a three of other advantages a Future has that this example
>>>> doesn’t show: control over which thread the Future runs on, Futures can be
>>>> cancelled, and debugging information is available.
>>>>
>>>> You could imagine `async` as a syntax sugar for Future, e.g. the above
>>>> Future example could be:
>>>>
>>>> func processImageData1() async -> Image {
>>>>     let dataResource  = loadWebResource("dataprofile.txt") //
>>>> dataResource and imageResource run in parallel.
>>>>     let imageResource = loadWebResource("imagedata.dat")
>>>>     let imageTmp      = decodeImage(dataResource.get ?? Resource(path:
>>>> "Default data resource or prompt user"), imageResource.get ??
>>>> Resource(path: "Default image resource or prompt user"))
>>>>     let imageResult   =  dewarpAndCleanupImage(imageTmp.get ??
>>>> Image(dataPath: "Default image or prompt user", imagePath: "Default image
>>>> or prompt user"))
>>>>     return imageResult.get ?? Image(dataPath: "Default image or prompt
>>>> user", imagePath: "Default image or prompt user")
>>>> }
>>>>
>>>> Since an async is sugar for Future the async runs as soon as it is
>>>> created (as soon as the underlying Future is created) and get returns an
>>>> optional (also cancel and status would be still be present). Then if you
>>>> want control over threads and timeout they could be arguments to async:
>>>>
>>>> func processImageData1() async(queue: DispatchQueue.main, timeout:
>>>> .seconds(5)) -> Image { ... }
>>>>
>>>> On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <
>>>> florent at flovilmart.com> wrote:
>>>>
>>>>> Howard, with async / await, the code is flat and you don’t have to
>>>>> unowned/weak self to prevent hideous cycles in the callbacks.
>>>>> Futures can’t do that
>>>>>
>>>>> On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution <
>>>>> swift-evolution at swift.org>, wrote:
>>>>>
>>>>> With both he now built in promises in Node8 as well as libraries like
>>>>> Bluebird there was ample time to evaluate them and convert/auto convert at
>>>>> times libraries that loved callback pyramids of doom when the flow grows
>>>>> complex into promise based chains. Converting to Promises seems magical for
>>>>> the simple case, but can quickly descend in hard to follow flows and hard
>>>>> to debug errors when you move to non trivial multi path scenarios. JS is
>>>>> now solving it with their implementation of async/await, but the point is
>>>>> that without the full picture any single solution would break horribly in
>>>>> real life scenarios.
>>>>>
>>>>> Sent from my iPhone
>>>>>
>>>>> On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution <
>>>>> swift-evolution at swift.org> wrote:
>>>>>
>>>>> My argument goes like this:
>>>>>
>>>>>   1. You don't need async/await to write a powerful future type; you
>>>>> can use the underlying threads just as well, i.e. future with async/await
>>>>> is no better than future without.
>>>>>
>>>>>   2. Since future is more powerful, thread control, cancel, and
>>>>> timeout, people should be encouraged to use this; instead because
>>>>> async/await are language features they will be presumed, incorrectly, to be
>>>>> the best way, consequently people will get into trouble with deadlocks
>>>>> because they don't have control.
>>>>>
>>>>>   3. async/await will require some engineering work and will at best
>>>>> make a mild syntax improvement and at worst lead to deadlocks, therefore
>>>>> they just don't carry their weight in terms of useful additions to Swift.
>>>>>
>>>>> Therefore, save some engineering effort and just provide a future
>>>>> library.
>>>>>
>>>>> To turn the question round another way, in two forms:
>>>>>
>>>>>   1. What can async/wait do that a future can't?
>>>>>
>>>>>   2. How will future be improved if async/await is added?
>>>>>
>>>>>
>>>>>   -- Howard.
>>>>>
>>>>> On 26 August 2017 at 02:23, Joe Groff <jgroff at apple.com> wrote:
>>>>>
>>>>>>
>>>>>> On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>  In particular a future that is cancellable is more powerful that
>>>>>> the proposed async/await.
>>>>>>
>>>>>>
>>>>>> It's not more powerful; the features are to some degree disjoint. You
>>>>>> can build a Future abstraction and then use async/await to sugar code that
>>>>>> threads computation through futures. Getting back to Jakob's example,
>>>>>> someone (maybe the Clang importer, maybe Apple's framework developers in an
>>>>>> overlay) will still need to build infrastructure on top of IBActions and
>>>>>> other currently ad-hoc signalling mechanisms to integrate them into a more
>>>>>> expressive coordination framework.
>>>>>>
>>>>>> -Joe
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> swift-evolution mailing list
>>>>> swift-evolution at swift.org
>>>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>>>>
>>>>> --
>>>> -- Howard.
>>>>
>>>> _______________________________________________
>>>> swift-evolution mailing list
>>>> swift-evolution at swift.org
>>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>>>
>>>>
>>>
>>> _______________________________________________
>>> swift-evolution mailing list
>>> swift-evolution at swift.org
>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>>
>>
>>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170829/d1a3d4db/attachment-0001.html>