[swift-evolution] [Concurrency] Fixing race conditions in async/await example
Howard Lovatt
howard.lovatt at gmail.com
Tue Aug 29 03:22:51 CDT 2017
@David,
Using the `Future` library based on GCD that I have previously posted your
example would be:
let image = preprocessImage(downloadImage()) // These first two lines
run in parallellet text = translate(downloadText())render(image:
image.get ?? defaultImage, text: text.get ?? defaultText)
The main difference, and I would argue an improvement, is that the `Future`
version handles errors.
So what advantage does async/await have over a `Future` library we can
write today?
-- Howard.
On 29 August 2017 at 15:28, David Hart via swift-evolution <
swift-evolution at swift.org> wrote:
>
> On 29 Aug 2017, at 02:22, Xiaodi Wu via swift-evolution <
> swift-evolution at swift.org> wrote:
>
> On Mon, Aug 28, 2017 at 16:10 Adam Kemp via swift-evolution <
> swift-evolution at swift.org> wrote:
>
>> I know what the proposal said. I’m making a case that there is value in
>> doing it differently.
>>
>> The composability of futures is valuable. Mixing and matching async/await
>> with futures is also valuable. The queue-returning behavior that you can
>> get from futures is also valuable, and building async/await on top of
>> futures means async/await can get that for free.
>>
>
> Why couldn't you mix and match async/await and futures and get the
> queue-return behavior of futures if futures are built on top of async/await
> instead off the other way around?
>
>
> We could, but the syntax is much worse. Contrast:
>
> *async/await built on top of Futures*
>
> let image = preprocessImage(downloadImage())let text = translate(downloadText())
> await render(image: image, text: text)
>
>
> *Futures built on top of async/await*
>
> let image = Future(downloadImage).then({ preprocessImage($0) })let text = Future(downloadText).then({ translate($0) })
> await render(image: image.get(), text: text.get())
>
>
> Maybe you don’t value those things, which is fine. But I do, and maybe
>> other people do too. That’s why we’re having a discussion about it.
>>
>> It can also be valuable having a minimal implementation, but we have to
>> acknowledge that it comes with a downside as well. The problem with doing a
>> minimal implementation is that you can be stuck with the consequences for a
>> long time. I want to make sure that we’re not stuck with the consequences
>> of a minimal implementation that doesn’t adequately address the problems
>> that async/await should be addressing. I’d hate for Swift to get an
>> async/await that is so weak that it has to be augmented by tedious
>> boilerplate code before it’s useful.
>>
>>
>> On Aug 28, 2017, at 1:54 PM, Wallacy <wallacyf at gmail.com> wrote:
>>
>> We don't need to this now!
>>
>> Again: (Using proposal words)
>>
>> "It is important to understand that this is proposing compiler support
>> that is completely concurrency runtime-agnostic. This proposal does not
>> include a new runtime model (like "actors") - it works just as well with
>> GCD as with pthreads or another API. Furthermore, unlike designs in other
>> languages, it is independent of specific coordination mechanisms, such as
>> futures or channels, allowing these to be built as library feature"
>>
>> and
>>
>> "This proposal does not formally propose a Future type, or any other
>> coordination abstractions. There are many rational designs for futures, and
>> a lot of experience working with them. On the other hand, there are also
>> completely different coordination primitives that can be used with this
>> coroutine design, and incorporating them into this proposal only makes it
>> larger."
>>
>> and
>>
>> We focus on task-based concurrency abstractions commonly encountered in
>> client and server applications, particularly those that are highly event
>> driven (e.g. responding to UI events or requests from clients). This does
>> not attempt to be a comprehensive survey of all possible options, nor does
>> it attempt to solve all possible problems in the space of concurrency.
>> Instead, it outlines a single coherent design thread that can be built over
>> the span of years to incrementally drive Swift to further greatness.
>>
>> and
>>
>> This proposal has been kept intentionally minimal, but there are many
>> possible ways to expand this in the future.
>>
>> ....
>>
>> The point is: No Future type is indeed proposed yet!
>>
>> The proposal try to include de "minimum" required to implement a basic
>> async/await to solve the problem created by the GCD! (Pyramid of doom)
>>
>> The question is: How do you do the same using dispatch_async ?
>> dispatch_async also does not return nothing to do what you are intentend do
>> do!
>>
>> Algo, by Swift 5 manifesto, there's no compromise to make a "complete"
>> concurrency model by this time!
>>
>> My intention is only make parity to dispatch_async, but also make the
>> ground free to make more complex implementation like Futures in another
>> round on top of this one.
>>
>> This 'async T' can be a real type in the future? Maybe will... But
>> doesn't matter now! Now we only need to is some kind of type which need to
>> be unwrapped using await before use. Maybe this intermediary/virtual type
>> can be a real thing and gain some abilities at some point! Maybe a full
>> Future type, why not?
>>
>> Em seg, 28 de ago de 2017 às 17:33, Adam Kemp <adam.kemp at apple.com>
>> escreveu:
>>
>>> How would these anonymous types get composed? If I wanted to implement a
>>> function that takes a collection of futures and wait on it, how would I do
>>> that? That is, how would I implement the equivalent of C#’s Task.WhenAll
>>> and Task.WhenAny methods?
>>>
>>> More generally, how do you pass one of these typeless futures to some
>>> other function so that we can do the waiting there?
>>>
>>>
>>> On Aug 28, 2017, at 1:23 PM, Wallacy <wallacyf at gmail.com> wrote:
>>>
>>> And that's why I (and others) are suggesting:
>>>
>>> func processImageData1a() async -> Image {
>>> let dataResource = async loadWebResource("dataprofile.txt") // No
>>> future type here... Just another way to call dispatch_async under the hood.
>>> let imageResource = async loadWebResource("imagedata.dat")
>>>
>>> // ... other stuff can go here to cover load latency...
>>>
>>> let imageTmp = await decodeImage(dataResource, imageResource) //
>>> Compiles force await call here...
>>> let imageResult = await dewarpAndCleanupImage(imageTmp)
>>> return imageResult
>>> }
>>>
>>> And now we gain all advantages of async/await again without to handle
>>> with one more type.
>>>
>>> Em seg, 28 de ago de 2017 às 17:07, Adam Kemp via swift-evolution <
>>> swift-evolution at swift.org> escreveu:
>>>
>>>> I think the biggest tradeoff is clearer when you look at the examples
>>>> from the proposal where futures are built on top of async/await:
>>>>
>>>> func processImageData1a() async -> Image {
>>>> let dataResource = Future { await loadWebResource("dataprofile.txt")
>>>> }
>>>> let imageResource = Future { await loadWebResource("imagedata.dat") }
>>>>
>>>> // ... other stuff can go here to cover load latency...
>>>>
>>>> let imageTmp = await decodeImage(dataResource.get(),
>>>> imageResource.get())
>>>> let imageResult = await dewarpAndCleanupImage(imageTmp)
>>>> return imageResult
>>>> }
>>>>
>>>>
>>>> With this approach you have to wrap each call site to create a future.
>>>> Compare to this:
>>>>
>>>> func processImageData1a() -> Future<Image> {
>>>> let dataResourceFuture = loadWebResource("dataprofile.txt”);
>>>> let imageResourceFuture = loadWebResource("imagedata.dat”);
>>>>
>>>> // ... other stuff can go here to cover load latency...
>>>>
>>>> let imageTmp = await decodeImage(await dataResourceFuture, await
>>>> imageResourceFuture)
>>>> let imageResult = await dewarpAndCleanupImage(imageTmp)
>>>> return imageResult
>>>> }
>>>>
>>>>
>>>> Here, not only are the explicit wrappers gone, but this function itself
>>>> can be used with either await or as a future. You get both options with one
>>>> implementation.
>>>>
>>>> As I’ve mentioned before, C#’s implementation is not tied to any one
>>>> particular futures implementation. The Task type is commonly used, but
>>>> async/await does not directly depend on Task. Instead it works with any
>>>> return type that meets certain requirements (detailed here:
>>>> https://blogs.msdn.microsoft.com/pfxteam/2011/01/13/await-anything/).
>>>> Swift could do this using a protocol, which can be retroactively applied
>>>> using an extension.
>>>>
>>>> Obviously for this to be useful we would need some kind of existing
>>>> future implementation, but at least we wouldn’t be tied to any particular
>>>> one. That would mean library maintainers who have already been using their
>>>> own futures implementations could quickly adopt async/await in their code
>>>> without having to rewrite their futures library or throw wrappers around
>>>> every usage of async/await. They could just adopt a protocol (using an
>>>> extension, even) and get async/await support for free.
>>>>
>>>> The downside is that this feature would be specific to the async/await
>>>> use case rather than a generic coroutine implementation (i.e., there would
>>>> have to be a separate compiler transform for yield return). It’s not clear
>>>> to me why it should be a goal to have just one generic coroutine feature.
>>>> The real-world usages of async/await and yield return are different enough
>>>> that I’m not convinced we could have a single compiler feature that meets
>>>> the needs of both cleanly.
>>>>
>>>> On Aug 27, 2017, at 7:35 PM, Florent Vilmart <florent at flovilmart.com>
>>>> wrote:
>>>>
>>>> Adam, you’re completely right, languages as c# and JS have been through
>>>> the path before, (callback, Promises , async/await) I believe Chris’s goal
>>>> it to avoid building a promise implementation and go straight to a
>>>> coroutines model, which is more deeply integrated with the compiler. I
>>>> don’t see a particular trade off, pursuing that route, and the main benefit
>>>> is that coroutines can power any asynchronous metaphor (Signals, Streams,
>>>> Futures, Promises etc...) which is not true of Futures so i would tend to
>>>> think that for the long run, and to maximize usability, async/await/yield
>>>> would probably be the way to go.
>>>>
>>>> On Aug 27, 2017, 22:22 -0400, Adam Kemp <adam.kemp at apple.com>, wrote:
>>>>
>>>> As has been explained, futures can be built on top of async/await (or
>>>> the other way around). You can have the best of both worlds. We are not
>>>> losing anything by having this feature. It would be a huge improvement to
>>>> have this as an option.
>>>>
>>>> However, using futures correctly requires more nested closures than you
>>>> have shown in your examples to avoid blocking any threads. That's why
>>>> you're not seeing the advantage to async/await. You're comparing examples
>>>> that have very different behaviors.
>>>>
>>>> That said, I have also expressed my opinion that it is better to build
>>>> async/await on top of futures rather than the other way around. I believe
>>>> it is more powerful and cleaner to make async/await work with any arbitrary
>>>> future type (via a protocol). The alternative (building futures on top of
>>>> async/await) requires more code when the two are mixed. I very much prefer
>>>> how it's done in C#, where you can freely mix the two models without having
>>>> to resort to ad-hoc wrappers, and you can use async/await with any futures
>>>> implementation you might already be using.
>>>>
>>>> I really think we should be having more discussion about the tradeoffs
>>>> between those two approaches, and I'm concerned that some of the opinions
>>>> about how C# does it are not based on a clear and accurate understanding of
>>>> how it actually works in that language.
>>>>
>>>> --
>>>> Adam Kemp
>>>>
>>>> On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt at gmail.com>
>>>> wrote:
>>>>
>>>> The async/await is very similar to the proposed Future (as I posed
>>>> earlier) with regard to completion-handler code, they both re-write the
>>>> imported completion-handler function using a closure, the relevant sentence
>>>> from the Async Proposal is:
>>>>
>>>> "Under the hood, the compiler rewrites this code using nested closures
>>>> ..."
>>>>
>>>>
>>>> Unlike the proposed future code the async code is not naturally
>>>> parallel, in the running example the following lines from the async code
>>>> are run in series, i.e. await blocks:
>>>>
>>>> let dataResource = await loadWebResource("dataprofile.txt")
>>>> let imageResource = await loadWebResource("imagedata.dat")
>>>>
>>>> The equivalent lines using the proposed Future:
>>>>
>>>> let dataResource = loadWebResource("dataprofile.txt")
>>>> let imageResource = loadWebResource("imagedata.dat")
>>>>
>>>> Run in parallel and therefore are potentially faster assuming that
>>>> resources, like cores and IO, are available.
>>>>
>>>> Therefore you would be better using a Future than an async, so why
>>>> provide an async unless you can make a convincing argument that it allows
>>>> you to write a better future?
>>>>
>>>> -- Howard.
>>>>
>>>> On 28 August 2017 at 09:59, Adam Kemp <adam.kemp at apple.com> wrote:
>>>>
>>>>> This example still has nested closures (to create a Future), and still
>>>>> relies on a synchronous get method that will block a thread. Async/await
>>>>> does not require blocking any threads.
>>>>>
>>>>> I’m definitely a fan of futures, but this example isn’t even a good
>>>>> example of using futures. If you’re using a synchronous get method then
>>>>> you’re not using futures properly. They’re supposed to make it easy to
>>>>> avoid writing blocking code. This example just does the blocking call on
>>>>> some other thread.
>>>>>
>>>>> Doing it properly would show the benefits of async/await because it
>>>>> would require more nesting and more complex error handling. By simplifying
>>>>> the code you’ve made a comparison between proper asynchronous code (with
>>>>> async/await) and improper asynchronous code (your example).
>>>>>
>>>>> That tendency to want to just block a thread to make it easier is
>>>>> exactly why async/await is so valuable. You get simple code while still
>>>>> doing it correctly.
>>>>>
>>>>> --
>>>>> Adam Kemp
>>>>>
>>>>> On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution <
>>>>> swift-evolution at swift.org> wrote:
>>>>>
>>>>> The running example used in the white paper coded using a Future is:
>>>>>
>>>>> func processImageData1() -> Future<Image> {
>>>>> return AsynchronousFuture { _ -> Image in
>>>>> let dataResource = loadWebResource("dataprofile.txt") //
>>>>> dataResource and imageResource run in parallel.
>>>>> let imageResource = loadWebResource("imagedata.dat")
>>>>> let imageTmp = decodeImage(dataResource.get ??
>>>>> Resource(path: "Default data resource or prompt user"), imageResource.get
>>>>> ?? Resource(path: "Default image resource or prompt user"))
>>>>> let imageResult = dewarpAndCleanupImage(imageTmp.get ??
>>>>> Image(dataPath: "Default image or prompt user", imagePath: "Default image
>>>>> or prompt user"))
>>>>> return imageResult.get ?? Image(dataPath: "Default image or
>>>>> prompt user", imagePath: "Default image or prompt user")
>>>>> }
>>>>> }
>>>>>
>>>>> This also avoids the pyramid of doom; the pyramid is avoided by
>>>>> converting continuation-handlers into either a sync or future, i.e. it is
>>>>> the importer that eliminates the nesting by translating the code
>>>>> automatically.
>>>>>
>>>>> This example using Future also demonstrates three advantages of
>>>>> Future: they are naturally parallel (dataResource and imageResource lines
>>>>> run in parallel), they timeout automatically (get returns nil if the Future
>>>>> has taken too long), and if there is a failure (for any reason including
>>>>> timeout) it provides a method of either detecting the failure or providing
>>>>> a default (get returns nil on failure).
>>>>>
>>>>> There are a three of other advantages a Future has that this example
>>>>> doesn’t show: control over which thread the Future runs on, Futures can be
>>>>> cancelled, and debugging information is available.
>>>>>
>>>>> You could imagine `async` as a syntax sugar for Future, e.g. the above
>>>>> Future example could be:
>>>>>
>>>>> func processImageData1() async -> Image {
>>>>> let dataResource = loadWebResource("dataprofile.txt") //
>>>>> dataResource and imageResource run in parallel.
>>>>> let imageResource = loadWebResource("imagedata.dat")
>>>>> let imageTmp = decodeImage(dataResource.get ?? Resource(path:
>>>>> "Default data resource or prompt user"), imageResource.get ??
>>>>> Resource(path: "Default image resource or prompt user"))
>>>>> let imageResult = dewarpAndCleanupImage(imageTmp.get ??
>>>>> Image(dataPath: "Default image or prompt user", imagePath: "Default image
>>>>> or prompt user"))
>>>>> return imageResult.get ?? Image(dataPath: "Default image or prompt
>>>>> user", imagePath: "Default image or prompt user")
>>>>> }
>>>>>
>>>>> Since an async is sugar for Future the async runs as soon as it is
>>>>> created (as soon as the underlying Future is created) and get returns an
>>>>> optional (also cancel and status would be still be present). Then if you
>>>>> want control over threads and timeout they could be arguments to async:
>>>>>
>>>>> func processImageData1() async(queue: DispatchQueue.main, timeout:
>>>>> .seconds(5)) -> Image { ... }
>>>>>
>>>>> On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <
>>>>> florent at flovilmart.com> wrote:
>>>>>
>>>>>> Howard, with async / await, the code is flat and you don’t have to
>>>>>> unowned/weak self to prevent hideous cycles in the callbacks.
>>>>>> Futures can’t do that
>>>>>>
>>>>>> On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution <
>>>>>> swift-evolution at swift.org>, wrote:
>>>>>>
>>>>>> With both he now built in promises in Node8 as well as libraries like
>>>>>> Bluebird there was ample time to evaluate them and convert/auto convert at
>>>>>> times libraries that loved callback pyramids of doom when the flow grows
>>>>>> complex into promise based chains. Converting to Promises seems magical for
>>>>>> the simple case, but can quickly descend in hard to follow flows and hard
>>>>>> to debug errors when you move to non trivial multi path scenarios. JS is
>>>>>> now solving it with their implementation of async/await, but the point is
>>>>>> that without the full picture any single solution would break horribly in
>>>>>> real life scenarios.
>>>>>>
>>>>>> Sent from my iPhone
>>>>>>
>>>>>> On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution <
>>>>>> swift-evolution at swift.org> wrote:
>>>>>>
>>>>>> My argument goes like this:
>>>>>>
>>>>>> 1. You don't need async/await to write a powerful future type; you
>>>>>> can use the underlying threads just as well, i.e. future with async/await
>>>>>> is no better than future without.
>>>>>>
>>>>>> 2. Since future is more powerful, thread control, cancel, and
>>>>>> timeout, people should be encouraged to use this; instead because
>>>>>> async/await are language features they will be presumed, incorrectly, to be
>>>>>> the best way, consequently people will get into trouble with deadlocks
>>>>>> because they don't have control.
>>>>>>
>>>>>> 3. async/await will require some engineering work and will at best
>>>>>> make a mild syntax improvement and at worst lead to deadlocks, therefore
>>>>>> they just don't carry their weight in terms of useful additions to Swift.
>>>>>>
>>>>>> Therefore, save some engineering effort and just provide a future
>>>>>> library.
>>>>>>
>>>>>> To turn the question round another way, in two forms:
>>>>>>
>>>>>> 1. What can async/wait do that a future can't?
>>>>>>
>>>>>> 2. How will future be improved if async/await is added?
>>>>>>
>>>>>>
>>>>>> -- Howard.
>>>>>>
>>>>>> On 26 August 2017 at 02:23, Joe Groff <jgroff at apple.com> wrote:
>>>>>>
>>>>>>>
>>>>>>> On Aug 25, 2017, at 12:34 AM, Howard Lovatt <howard.lovatt at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> In particular a future that is cancellable is more powerful that
>>>>>>> the proposed async/await.
>>>>>>>
>>>>>>>
>>>>>>> It's not more powerful; the features are to some degree disjoint.
>>>>>>> You can build a Future abstraction and then use async/await to sugar code
>>>>>>> that threads computation through futures. Getting back to Jakob's example,
>>>>>>> someone (maybe the Clang importer, maybe Apple's framework developers in an
>>>>>>> overlay) will still need to build infrastructure on top of IBActions and
>>>>>>> other currently ad-hoc signalling mechanisms to integrate them into a more
>>>>>>> expressive coordination framework.
>>>>>>>
>>>>>>> -Joe
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> swift-evolution mailing list
>>>>>> swift-evolution at swift.org
>>>>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>>>>>
>>>>>> --
>>>>> -- Howard.
>>>>>
>>>>> _______________________________________________
>>>>> swift-evolution mailing list
>>>>> swift-evolution at swift.org
>>>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> swift-evolution mailing list
>>>> swift-evolution at swift.org
>>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>>>
>>>
>>>
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
>
>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170829/6ddd0a74/attachment.html>
More information about the swift-evolution
mailing list