[swift-dev] Profiling ARC

[David P Grove]

Sorry, I shouldn't have assumed that everyone know what the Computer
Language Benchmark Game was.

There's a set of 10 toy benchmarks written in 30 different languages that
are available here: http://benchmarksgame.alioth.debian.org/   The webpage
also lets you see the results of regular performance runs and see how the
various programs stack up against each other.

As usual with small benchmarks, there are lots of ways in which the
programs aren't realistic.

After dispatch became available on Linux with Swift 3, I had a side project
of going through the Swift implementations and adding concurrency to a few
Swift programs that didn't have it already.

regex-dna is the main outlier for Swift (which is why I had profiled it).
There's enough ARC overhead that using dispatch actually made it slower...
	The sequential version for of regex-dna for Swift is:
http://benchmarksgame.alioth.debian.org/u64q/program.php?test=regexdna&lang=swift&id=2
	My slower concurrent version is:
http://benchmarksgame.alioth.debian.org/u64q/program.php?test=regexdna&lang=swift&id=3

I suspect that the main fix for improving the performance of this program
is actually doing something in CoreFoundation, but I got sidetracked and
didn't finish looking into it.

--dave



From:	Michael Gottesman <mgottesman at apple.com>
To:	David P Grove/Watson/IBM at IBMUS
Cc:	Jiho Choi <jray319 at gmail.com>, swift-dev <swift-dev at swift.org>
Date:	02/17/2017 05:32 PM
Subject:	Re: [swift-dev] Profiling ARC
Sent by:	mgottesman at apple.com




      On Feb 17, 2017, at 11:30 AM, David P Grove via swift-dev <
      swift-dev at swift.org> wrote:



      swift-dev-bounces at swift.org wrote on 02/16/2017 09:48:28 PM:
      >
      > I was curious about the overhead of ARC and started profiling some
      > benchmarks found in the Computer Language Benchmark Game (http://
      > benchmarksgame.alioth.debian.org/u64q/measurements.php?lang=swift).

      > So far, it seems that ARC sequence optimization is surprisingly
      good
      > and most benchmarks don't have to perform ARC operations as often
      as
      > I expected.  I have some questions regarding this finding.
      >
      > I compiled all benchmarks with "-O -wmo" flags and counted the
      > number of calls to ARC runtime (e.g., swift_rt_swift_retain) using
      Pin.
      >
      > 1. Reference counting is considered to have high overhead due to
      > frequent counting operations which also have to be atomic.  At
      least
      > for the benchmarks I tested, it is not the case and there is almost

      > no overhead.  Is it expected behavior?  Or is it because the
      > benchmarks are too simple (they are all single-file programs)?  How

      > do you estimate the overhead of ARC would be?
      >

      hmm,  I wonder if your method of profiling is really finding all the
      ARC operations.  The Swift version of regex-dna is about 25x slower
      than the Java version (on Linux).  I looked at some prof profiles
      about a month ago and at the time roughly 80% of all execution
      samples were attributed to swift_retain/swift_release operations
      coming from CoreFoundation's regex implementation.


Question. Where is this regex-dna benchmark, is it in the swift benchmark
suite?



      --dave

      (See attached file: regex-dna.svg)


      <regex-dna.svg>_______________________________________________
      swift-dev mailing list
      swift-dev at swift.org
      https://lists.swift.org/mailman/listinfo/swift-dev


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-dev/attachments/20170217/12d090ca/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: not available
URL: <https://lists.swift.org/pipermail/swift-dev/attachments/20170217/12d090ca/attachment.gif>