[swift-evolution] [Concurrency] Fixing race conditions in async/await example

Wallacy wallacyf at gmail.com
Tue Aug 29 11:02:17 CDT 2017


In this example i think we lose clarity, just looking for the code we cant
know if this two line will run on parallel or not!
Also, image.get blocks the thread, in this case we need the await anyway!
And `async` can throws too... So the error handler can be pretty similar.

let image = async preprocessImage(downloadImage()) // These first two
lines run in parallel and I can "see" the async keyword.let text =
async translate(downloadText())await render(image: image ??
defaultImage, text: text ?? defaultText) // No blocking!


Like i said before! Today's, the proposal only lack two things over the
`Future`....
Parallel computing: Can be implemented by a third party library or a
personal one, but i don't think this is a good approach to the first
version.
Coordination: This we can wait! And why? Because coordination, can be made
in different ways, maybe is more suitable to a standard library
class/function, not a language level resource.

Also, coordination cant be applied to all variants of the runtimes in the
same way! async/await as language level  works just as well with GCD as
with pthreads or another API. And coordination is a compromise that we can
make after that one.

Em ter, 29 de ago de 2017 às 05:23, Howard Lovatt via swift-evolution <
swift-evolution at swift.org> escreveu:

> @David,
>
> Using the `Future` library based on GCD that I have previously posted your
> example would be:
>
> let image = preprocessImage(downloadImage()) // These first two lines run in parallellet text = translate(downloadText())render(image: image.get ?? defaultImage, text: text.get ?? defaultText)
>
>
> The main difference, and I would argue an improvement, is that the
> `Future` version handles errors.
>
> So what advantage does async/await have over a `Future` library we can
> write today?
>
>
>   -- Howard.
>
> On 29 August 2017 at 15:28, David Hart via swift-evolution <
> swift-evolution at swift.org> wrote:
>
>>
>> On 29 Aug 2017, at 02:22, Xiaodi Wu via swift-evolution <
>> swift-evolution at swift.org> wrote:
>>
>> On Mon, Aug 28, 2017 at 16:10 Adam Kemp via swift-evolution <
>> swift-evolution at swift.org> wrote:
>>
>>> I know what the proposal said. I’m making a case that there is value in
>>> doing it differently.
>>>
>>> The composability of futures is valuable. Mixing and matching
>>> async/await with futures is also valuable. The queue-returning behavior
>>> that you can get from futures is also valuable, and building async/await on
>>> top of futures means async/await can get that for free.
>>>
>>
>> Why couldn't you mix and match async/await and futures and get the
>> queue-return behavior of futures if futures are built on top of async/await
>> instead off the other way around?
>>
>>
>> We could, but the syntax is much worse. Contrast:
>>
>> *async/await built on top of Futures*
>>
>> let image = preprocessImage(downloadImage())let text = translate(downloadText())
>> await render(image: image, text: text)
>>
>>
>> *Futures built on top of async/await*
>>
>> let image = Future(downloadImage).then({ preprocessImage($0) })let text = Future(downloadText).then({ translate($0) })
>> await render(image: image.get(), text: text.get())
>>
>>
>> Maybe you don’t value those things, which is fine. But I do, and maybe
>>> other people do too. That’s why we’re having a discussion about it.
>>>
>>> It can also be valuable having a minimal implementation, but we have to
>>> acknowledge that it comes with a downside as well. The problem with doing a
>>> minimal implementation is that you can be stuck with the consequences for a
>>> long time. I want to make sure that we’re not stuck with the consequences
>>> of a minimal implementation that doesn’t adequately address the problems
>>> that async/await should be addressing. I’d hate for Swift to get an
>>> async/await that is so weak that it has to be augmented by tedious
>>> boilerplate code before it’s useful.
>>>
>>>
>>> On Aug 28, 2017, at 1:54 PM, Wallacy <wallacyf at gmail.com> wrote:
>>>
>>> We don't need to this now!
>>>
>>> Again: (Using proposal words)
>>>
>>> "It is important to understand that this is proposing compiler support
>>> that is completely concurrency runtime-agnostic. This proposal does not
>>> include a new runtime model (like "actors") - it works just as well with
>>> GCD as with pthreads or another API. Furthermore, unlike designs in other
>>> languages, it is independent of specific coordination mechanisms, such as
>>> futures or channels, allowing these to be built as library feature"
>>>
>>> and
>>>
>>> "This proposal does not formally propose a Future type, or any other
>>> coordination abstractions. There are many rational designs for futures, and
>>> a lot of experience working with them. On the other hand, there are also
>>> completely different coordination primitives that can be used with this
>>> coroutine design, and incorporating them into this proposal only makes it
>>> larger."
>>>
>>> and
>>>
>>> We focus on task-based concurrency abstractions commonly encountered in
>>> client and server applications, particularly those that are highly event
>>> driven (e.g. responding to UI events or requests from clients). This does
>>> not attempt to be a comprehensive survey of all possible options, nor does
>>> it attempt to solve all possible problems in the space of concurrency.
>>> Instead, it outlines a single coherent design thread that can be built over
>>> the span of years to incrementally drive Swift to further greatness.
>>>
>>> and
>>>
>>> This proposal has been kept intentionally minimal, but there are many
>>> possible ways to expand this in the future.
>>>
>>> ....
>>>
>>> The point is: No Future type is indeed proposed yet!
>>>
>>> The proposal try to include de "minimum" required to implement a basic
>>> async/await to solve the problem created by the GCD! (Pyramid of doom)
>>>
>>> The question is: How do you do the same using dispatch_async ?
>>> dispatch_async also does not return nothing to do what you are intentend do
>>> do!
>>>
>>> Algo, by Swift 5 manifesto, there's no compromise to make a "complete"
>>> concurrency model by this time!
>>>
>>> My intention is only make parity to dispatch_async, but also make the
>>> ground free to make more complex implementation like Futures in another
>>> round on top of this one.
>>>
>>> This 'async T' can be a real type in the future? Maybe will... But
>>> doesn't matter now! Now we only need to is some kind of type which need to
>>> be unwrapped using await before use. Maybe this intermediary/virtual type
>>> can be a real thing and gain some abilities at some point! Maybe a full
>>> Future type, why not?
>>>
>>> Em seg, 28 de ago de 2017 às 17:33, Adam Kemp <adam.kemp at apple.com>
>>> escreveu:
>>>
>>>> How would these anonymous types get composed? If I wanted to implement
>>>> a function that takes a collection of futures and wait on it, how would I
>>>> do that? That is, how would I implement the equivalent of C#’s Task.WhenAll
>>>> and Task.WhenAny methods?
>>>>
>>>> More generally, how do you pass one of these typeless futures to some
>>>> other function so that we can do the waiting there?
>>>>
>>>>
>>>> On Aug 28, 2017, at 1:23 PM, Wallacy <wallacyf at gmail.com> wrote:
>>>>
>>>> And that's why I (and others) are suggesting:
>>>>
>>>> func processImageData1a() async -> Image {
>>>>   let dataResource  = async loadWebResource("dataprofile.txt") // No
>>>> future type here... Just another way to call dispatch_async under the hood.
>>>>   let imageResource = async loadWebResource("imagedata.dat")
>>>>
>>>>   // ... other stuff can go here to cover load latency...
>>>>
>>>>   let imageTmp    = await decodeImage(dataResource, imageResource) //
>>>> Compiles force await call here...
>>>>   let imageResult = await dewarpAndCleanupImage(imageTmp)
>>>>   return imageResult
>>>> }
>>>>
>>>> And now we gain all advantages of async/await again without to handle
>>>> with one more type.
>>>>
>>>> Em seg, 28 de ago de 2017 às 17:07, Adam Kemp via swift-evolution <
>>>> swift-evolution at swift.org> escreveu:
>>>>
>>>>> I think the biggest tradeoff is clearer when you look at the examples
>>>>> from the proposal where futures are built on top of async/await:
>>>>>
>>>>> func processImageData1a() async -> Image {
>>>>>   let dataResource  = Future { await
>>>>> loadWebResource("dataprofile.txt") }
>>>>>   let imageResource = Future { await loadWebResource("imagedata.dat") }
>>>>>
>>>>>   // ... other stuff can go here to cover load latency...
>>>>>
>>>>>   let imageTmp    = await decodeImage(dataResource.get(),
>>>>> imageResource.get())
>>>>>   let imageResult = await dewarpAndCleanupImage(imageTmp)
>>>>>   return imageResult
>>>>> }
>>>>>
>>>>>
>>>>> With this approach you have to wrap each call site to create a future.
>>>>> Compare to this:
>>>>>
>>>>> func processImageData1a() -> Future<Image> {
>>>>>   let dataResourceFuture  = loadWebResource("dataprofile.txt”);
>>>>>   let imageResourceFuture = loadWebResource("imagedata.dat”);
>>>>>
>>>>>   // ... other stuff can go here to cover load latency...
>>>>>
>>>>>   let imageTmp    = await decodeImage(await dataResourceFuture, await
>>>>> imageResourceFuture)
>>>>>   let imageResult = await dewarpAndCleanupImage(imageTmp)
>>>>>   return imageResult
>>>>> }
>>>>>
>>>>>
>>>>> Here, not only are the explicit wrappers gone, but this function
>>>>> itself can be used with either await or as a future. You get both options
>>>>> with one implementation.
>>>>>
>>>>> As I’ve mentioned before, C#’s implementation is not tied to any one
>>>>> particular futures implementation. The Task type is commonly used, but
>>>>> async/await does not directly depend on Task. Instead it works with any
>>>>> return type that meets certain requirements (detailed here:
>>>>> https://blogs.msdn.microsoft.com/pfxteam/2011/01/13/await-anything/).
>>>>> Swift could do this using a protocol, which can be retroactively applied
>>>>> using an extension.
>>>>>
>>>>> Obviously for this to be useful we would need some kind of existing
>>>>> future implementation, but at least we wouldn’t be tied to any particular
>>>>> one. That would mean library maintainers who have already been using their
>>>>> own futures implementations could quickly adopt async/await in their code
>>>>> without having to rewrite their futures library or throw wrappers around
>>>>> every usage of async/await. They could just adopt a protocol (using an
>>>>> extension, even) and get async/await support for free.
>>>>>
>>>>> The downside is that this feature would be specific to the async/await
>>>>> use case rather than a generic coroutine implementation (i.e., there would
>>>>> have to be a separate compiler transform for yield return). It’s not clear
>>>>> to me why it should be a goal to have just one generic coroutine feature.
>>>>> The real-world usages of async/await and yield return are different enough
>>>>> that I’m not convinced we could have a single compiler feature that meets
>>>>> the needs of both cleanly.
>>>>>
>>>>> On Aug 27, 2017, at 7:35 PM, Florent Vilmart <florent at flovilmart.com>
>>>>> wrote:
>>>>>
>>>>> Adam, you’re completely right, languages as c# and JS have been
>>>>> through the path before, (callback, Promises , async/await) I believe
>>>>> Chris’s goal it to avoid building a promise implementation and go straight
>>>>> to a coroutines model, which is more deeply integrated with the compiler. I
>>>>> don’t see a particular trade off, pursuing that route, and the main benefit
>>>>> is that coroutines can power any asynchronous metaphor (Signals, Streams,
>>>>> Futures, Promises etc...) which is not true of Futures so i would tend to
>>>>> think that for the long run, and to maximize usability, async/await/yield
>>>>> would probably be the way to go.
>>>>>
>>>>> On Aug 27, 2017, 22:22 -0400, Adam Kemp <adam.kemp at apple.com>, wrote:
>>>>>
>>>>> As has been explained, futures can be built on top of async/await (or
>>>>> the other way around). You can have the best of both worlds. We are not
>>>>> losing anything by having this feature. It would be a huge improvement to
>>>>> have this as an option.
>>>>>
>>>>> However, using futures correctly requires more nested closures than
>>>>> you have shown in your examples to avoid blocking any threads. That's why
>>>>> you're not seeing the advantage to async/await. You're comparing examples
>>>>> that have very different behaviors.
>>>>>
>>>>> That said, I have also expressed my opinion that it is better to build
>>>>> async/await on top of futures rather than the other way around. I believe
>>>>> it is more powerful and cleaner to make async/await work with any arbitrary
>>>>> future type (via a protocol). The alternative (building futures on top of
>>>>> async/await) requires more code when the two are mixed. I very much prefer
>>>>> how it's done in C#, where you can freely mix the two models without having
>>>>> to resort to ad-hoc wrappers, and you can use async/await with any futures
>>>>> implementation you might already be using.
>>>>>
>>>>> I really think we should be having more discussion about the tradeoffs
>>>>> between those two approaches, and I'm concerned that some of the opinions
>>>>> about how C# does it are not based on a clear and accurate understanding of
>>>>> how it actually works in that language.
>>>>>
>>>>> --
>>>>> Adam Kemp
>>>>>
>>>>> On Aug 27, 2017, at 6:02 PM, Howard Lovatt <howard.lovatt at gmail.com>
>>>>> wrote:
>>>>>
>>>>> The async/await is very similar to the proposed Future (as I posed
>>>>> earlier) with regard to completion-handler code, they both re-write the
>>>>> imported completion-handler function using a closure, the relevant sentence
>>>>> from the Async Proposal is:
>>>>>
>>>>> "Under the hood, the compiler rewrites this code using nested closures
>>>>> ..."
>>>>>
>>>>>
>>>>> Unlike the proposed future code the async code is not naturally
>>>>> parallel, in the running example the following lines from the async code
>>>>> are run in series, i.e. await blocks:
>>>>>
>>>>>   let dataResource  = await loadWebResource("dataprofile.txt")
>>>>>   let imageResource = await loadWebResource("imagedata.dat")
>>>>>
>>>>> The equivalent lines using the proposed Future:
>>>>>
>>>>>   let dataResource  = loadWebResource("dataprofile.txt")
>>>>>   let imageResource = loadWebResource("imagedata.dat")
>>>>>
>>>>> Run in parallel and therefore are potentially faster assuming that
>>>>> resources, like cores and IO, are available.
>>>>>
>>>>> Therefore you would be better using a Future than an async, so why
>>>>> provide an async unless you can make a convincing argument that it allows
>>>>> you to write a better future?
>>>>>
>>>>>   -- Howard.
>>>>>
>>>>> On 28 August 2017 at 09:59, Adam Kemp <adam.kemp at apple.com> wrote:
>>>>>
>>>>>> This example still has nested closures (to create a Future), and
>>>>>> still relies on a synchronous get method that will block a thread.
>>>>>> Async/await does not require blocking any threads.
>>>>>>
>>>>>> I’m definitely a fan of futures, but this example isn’t even a good
>>>>>> example of using futures. If you’re using a synchronous get method then
>>>>>> you’re not using futures properly. They’re supposed to make it easy to
>>>>>> avoid writing blocking code. This example just does the blocking call on
>>>>>> some other thread.
>>>>>>
>>>>>> Doing it properly would show the benefits of async/await because it
>>>>>> would require more nesting and more complex error handling. By simplifying
>>>>>> the code you’ve made a comparison between proper asynchronous code (with
>>>>>> async/await) and improper asynchronous code (your example).
>>>>>>
>>>>>> That tendency to want to just block a thread to make it easier is
>>>>>> exactly why async/await is so valuable. You get simple code while still
>>>>>> doing it correctly.
>>>>>>
>>>>>> --
>>>>>> Adam Kemp
>>>>>>
>>>>>> On Aug 27, 2017, at 4:00 PM, Howard Lovatt via swift-evolution <
>>>>>> swift-evolution at swift.org> wrote:
>>>>>>
>>>>>> The running example used in the white paper coded using a Future is:
>>>>>>
>>>>>> func processImageData1() -> Future<Image> {
>>>>>>     return AsynchronousFuture { _ -> Image in
>>>>>>         let dataResource  = loadWebResource("dataprofile.txt") //
>>>>>> dataResource and imageResource run in parallel.
>>>>>>         let imageResource = loadWebResource("imagedata.dat")
>>>>>>         let imageTmp      = decodeImage(dataResource.get ??
>>>>>> Resource(path: "Default data resource or prompt user"), imageResource.get
>>>>>> ?? Resource(path: "Default image resource or prompt user"))
>>>>>>         let imageResult   =  dewarpAndCleanupImage(imageTmp.get ??
>>>>>> Image(dataPath: "Default image or prompt user", imagePath: "Default image
>>>>>> or prompt user"))
>>>>>>         return imageResult.get ?? Image(dataPath: "Default image or
>>>>>> prompt user", imagePath: "Default image or prompt user")
>>>>>>     }
>>>>>> }
>>>>>>
>>>>>> This also avoids the pyramid of doom; the pyramid is avoided by
>>>>>> converting continuation-handlers into either a sync or future, i.e. it is
>>>>>> the importer that eliminates the nesting by translating the code
>>>>>> automatically.
>>>>>>
>>>>>> This example using Future also demonstrates three advantages of
>>>>>> Future: they are naturally parallel (dataResource and imageResource lines
>>>>>> run in parallel), they timeout automatically (get returns nil if the Future
>>>>>> has taken too long), and if there is a failure (for any reason including
>>>>>> timeout) it provides a method of either detecting the failure or providing
>>>>>> a default (get returns nil on failure).
>>>>>>
>>>>>> There are a three of other advantages a Future has that this example
>>>>>> doesn’t show: control over which thread the Future runs on, Futures can be
>>>>>> cancelled, and debugging information is available.
>>>>>>
>>>>>> You could imagine `async` as a syntax sugar for Future, e.g. the
>>>>>> above Future example could be:
>>>>>>
>>>>>> func processImageData1() async -> Image {
>>>>>>     let dataResource  = loadWebResource("dataprofile.txt") //
>>>>>> dataResource and imageResource run in parallel.
>>>>>>     let imageResource = loadWebResource("imagedata.dat")
>>>>>>     let imageTmp      = decodeImage(dataResource.get ??
>>>>>> Resource(path: "Default data resource or prompt user"), imageResource.get
>>>>>> ?? Resource(path: "Default image resource or prompt user"))
>>>>>>     let imageResult   =  dewarpAndCleanupImage(imageTmp.get ??
>>>>>> Image(dataPath: "Default image or prompt user", imagePath: "Default image
>>>>>> or prompt user"))
>>>>>>     return imageResult.get ?? Image(dataPath: "Default image or
>>>>>> prompt user", imagePath: "Default image or prompt user")
>>>>>> }
>>>>>>
>>>>>> Since an async is sugar for Future the async runs as soon as it is
>>>>>> created (as soon as the underlying Future is created) and get returns an
>>>>>> optional (also cancel and status would be still be present). Then if you
>>>>>> want control over threads and timeout they could be arguments to async:
>>>>>>
>>>>>> func processImageData1() async(queue: DispatchQueue.main, timeout:
>>>>>> .seconds(5)) -> Image { ... }
>>>>>>
>>>>>> On Sat, 26 Aug 2017 at 11:00 pm, Florent Vilmart <
>>>>>> florent at flovilmart.com> wrote:
>>>>>>
>>>>>>> Howard, with async / await, the code is flat and you don’t have to
>>>>>>> unowned/weak self to prevent hideous cycles in the callbacks.
>>>>>>> Futures can’t do that
>>>>>>>
>>>>>>> On Aug 26, 2017, 04:37 -0400, Goffredo Marocchi via swift-evolution <
>>>>>>> swift-evolution at swift.org>, wrote:
>>>>>>>
>>>>>>> With both he now built in promises in Node8 as well as libraries
>>>>>>> like Bluebird there was ample time to evaluate them and convert/auto
>>>>>>> convert at times libraries that loved callback pyramids of doom when the
>>>>>>> flow grows complex into promise based chains. Converting to Promises seems
>>>>>>> magical for the simple case, but can quickly descend in hard to follow
>>>>>>> flows and hard to debug errors when you move to non trivial multi path
>>>>>>> scenarios. JS is now solving it with their implementation of async/await,
>>>>>>> but the point is that without the full picture any single solution would
>>>>>>> break horribly in real life scenarios.
>>>>>>>
>>>>>>> Sent from my iPhone
>>>>>>>
>>>>>>> On 26 Aug 2017, at 06:27, Howard Lovatt via swift-evolution <
>>>>>>> swift-evolution at swift.org> wrote:
>>>>>>>
>>>>>>> My argument goes like this:
>>>>>>>
>>>>>>>   1. You don't need async/await to write a powerful future type; you
>>>>>>> can use the underlying threads just as well, i.e. future with async/await
>>>>>>> is no better than future without.
>>>>>>>
>>>>>>>   2. Since future is more powerful, thread control, cancel, and
>>>>>>> timeout, people should be encouraged to use this; instead because
>>>>>>> async/await are language features they will be presumed, incorrectly, to be
>>>>>>> the best way, consequently people will get into trouble with deadlocks
>>>>>>> because they don't have control.
>>>>>>>
>>>>>>>   3. async/await will require some engineering work and will at best
>>>>>>> make a mild syntax improvement and at worst lead to deadlocks, therefore
>>>>>>> they just don't carry their weight in terms of useful additions to Swift.
>>>>>>>
>>>>>>> Therefore, save some engineering effort and just provide a future
>>>>>>> library.
>>>>>>>
>>>>>>> To turn the question round another way, in two forms:
>>>>>>>
>>>>>>>   1. What can async/wait do that a future can't?
>>>>>>>
>>>>>>>   2. How will future be improved if async/await is added?
>>>>>>>
>>>>>>>
>>>>>>>   -- Howard.
>>>>>>>
>>>>>>> On 26 August 2017 at 02:23, Joe Groff <jgroff at apple.com> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> On Aug 25, 2017, at 12:34 AM, Howard Lovatt <
>>>>>>>> howard.lovatt at gmail.com> wrote:
>>>>>>>>
>>>>>>>>  In particular a future that is cancellable is more powerful that
>>>>>>>> the proposed async/await.
>>>>>>>>
>>>>>>>>
>>>>>>>> It's not more powerful; the features are to some degree disjoint.
>>>>>>>> You can build a Future abstraction and then use async/await to sugar code
>>>>>>>> that threads computation through futures. Getting back to Jakob's example,
>>>>>>>> someone (maybe the Clang importer, maybe Apple's framework developers in an
>>>>>>>> overlay) will still need to build infrastructure on top of IBActions and
>>>>>>>> other currently ad-hoc signalling mechanisms to integrate them into a more
>>>>>>>> expressive coordination framework.
>>>>>>>>
>>>>>>>> -Joe
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> swift-evolution mailing list
>>>>>>> swift-evolution at swift.org
>>>>>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>>>>>>
>>>>>>> --
>>>>>> -- Howard.
>>>>>>
>>>>>> _______________________________________________
>>>>>> swift-evolution mailing list
>>>>>> swift-evolution at swift.org
>>>>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> swift-evolution mailing list
>>>>> swift-evolution at swift.org
>>>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>>>>
>>>>
>>>>
>>> _______________________________________________
>>> swift-evolution mailing list
>>> swift-evolution at swift.org
>>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>>
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>
>>
>>
>> _______________________________________________
>> swift-evolution mailing list
>> swift-evolution at swift.org
>> https://lists.swift.org/mailman/listinfo/swift-evolution
>>
>>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution at swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-evolution/attachments/20170829/2e8c1200/attachment-0001.html>


More information about the swift-evolution mailing list