[swift-users] Help! Slicing an array is very expensive

[Michael Gottesman]

Could you file a bug report? bugs.swift.org?

Michael

> On May 9, 2017, at 12:59 AM, Howard Lovatt via swift-users <swift-users at swift.org> wrote:
> 
> My mistake. If I create a new array I get the problem. EG:
> 
> import Foundation
> 
> func elapsed(s: DispatchTime, e: DispatchTime) -> Double {
>     return Double(e.uptimeNanoseconds - s.uptimeNanoseconds) / 1_000_000_000
> }
> 
> let s = 250_000_000
> var a = [UInt8]()
> a.reserveCapacity(s)
> 
> let sa = DispatchTime.now()
> for i in 1 ... s {
>     a.append(0)
> }
> let ea = DispatchTime.now()
> print("Append time: \(elapsed(s: sa, e: ea)) s")
> 
> let st = DispatchTime.now()
> a = Array(a[0 ..< (s >> 1)])
> let et = DispatchTime.now()
> print("Trim time: \(elapsed(s: st, e: et)) s")
> print("a count: \(a.count), capacity: \(a.capacity)")
> 
> 
> Prints:
> 
> Append time: 2.65726525 s
> Trim time: 36.954417356 s
> a count: 125000000, capacity: 125001696
> 
>   -- Howard.
> 
> On 9 May 2017 at 17:53, Howard Lovatt <howard.lovatt at gmail.com <mailto:howard.lovatt at gmail.com>> wrote:
> I find trimming relative to appending OK on my 6 year old MacBook Pro. EG:
> 
> import Foundation
> 
> func elapsed(s: DispatchTime, e: DispatchTime) -> Double {
>     return Double(e.uptimeNanoseconds - s.uptimeNanoseconds) / 1_000_000_000
> }
> 
> let s = 250_000_000
> var a = [UInt8]()
> a.reserveCapacity(s)
> 
> let sa = DispatchTime.now()
> for i in 1 ... s {
>     a.append(0)
> }
> let ea = DispatchTime.now()
> print("Append time: \(elapsed(s: sa, e: ea)) s")
> 
> let st = DispatchTime.now()
> let ha = a[0 ..< (s >> 1)]
> let et = DispatchTime.now()
> print("Trim time: \(elapsed(s: st, e: et)) s")
> 
> print("ha count: \(ha.count), capacity: \(ha.capacity)")
> 
> 
> Prints:
> 
> Append time: 2.612397389 s
> Trim time: 0.000444132 s
> ha count: 125000000, capacity: 125000000
> 
>   -- Howard.
> 
> On 9 May 2017 at 12:56, Kelvin Ma via swift-users <swift-users at swift.org <mailto:swift-users at swift.org>> wrote:
> Depending on what you’re trying to do with the data, you might be better off using an UnsafeBufferPointer and allocating and reallocating that, C-style.
> 
> On Mon, May 8, 2017 at 7:01 PM, Philippe Hausler via swift-users <swift-users at swift.org <mailto:swift-users at swift.org>> wrote:
> I wonder if Data might be a better tool for the job here since it is it’s own slice type and would avoid copying large amounts of data into temporary buffers.
> 
> > On May 8, 2017, at 16:47, Rick Mann via swift-users <swift-users at swift.org <mailto:swift-users at swift.org>> wrote:
> >
> > I have this C library that interacts with some hardware over the network that produces a ton of data. It tells me up front the maximum size the data might be so I can allocate a buffer for it, then does a bunch of network requests downloading that data into the buffer, then tells me when it's done and what the final, smaller size is.
> >
> > Thanks to previous discussions on the list, I settled on using a [UInt8] as the buffer, because it let me avoid various .withUnsafePointer{} calls (I need the unsafe buffer pointer to live outside the scope of the closures). Unfortunately, When I go to shrink the buffer to its final size with:
> >
> >    self.dataBuffer = Array(self.dataBuffer![0 ..< finalBufferSize])
> >
> > This ends up taking over 2 minutes to complete (on an iPad Pro). finalBufferSize is very large, 240 MB, but I think it's doing a very naive copy.
> >
> > I've since worked around this problem, but is there any way to improve on this?
> >
> > Thanks,
> >
> > --
> > Rick Mann
> > rmann at latencyzero.com <mailto:rmann at latencyzero.com>
> >
> >
> > _______________________________________________
> > swift-users mailing list
> > swift-users at swift.org <mailto:swift-users at swift.org>
> > https://lists.swift.org/mailman/listinfo/swift-users <https://lists.swift.org/mailman/listinfo/swift-users>
> 
> _______________________________________________
> swift-users mailing list
> swift-users at swift.org <mailto:swift-users at swift.org>
> https://lists.swift.org/mailman/listinfo/swift-users <https://lists.swift.org/mailman/listinfo/swift-users>
> 
> 
> _______________________________________________
> swift-users mailing list
> swift-users at swift.org <mailto:swift-users at swift.org>
> https://lists.swift.org/mailman/listinfo/swift-users <https://lists.swift.org/mailman/listinfo/swift-users>
> 
> 
> 
> _______________________________________________
> swift-users mailing list
> swift-users at swift.org
> https://lists.swift.org/mailman/listinfo/swift-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-users/attachments/20170509/8ecc0198/attachment.html>