[swift-users] raw buffer pointer load alignment

[Kelvin Ma]

On Thu, Nov 9, 2017 at 10:37 AM, Johannes Weiß <johannesweiss at apple.com>
wrote:

> Hi Kelvin,
>
> > On 9 Nov 2017, at 12:30 am, Kelvin Ma <kelvin13ma at gmail.com> wrote:
> >
> > For context, the problem I’m trying to solve is efficiently parsing JPEG
> chunks. This means reading each chunk of the JPEG from a file into a raw
> buffer pointer, and then parsing the chunk according to its expected
> layout. For example, a frame header chunk looks like this:
> >
> > 0                 1                 2                 3
> 4                  5              6
> > [ precision:UInt8 |          height:UInt16            |
>  width:UInt16            |   Nf:UInt8   | ... ]
> >
> > what I want to do is to be able to load the height and width into
> something I can pass into UInt16.init(bigEndian:) without failing because
> of alignment. I’ve thought of several options but none of them seem to be
> great.
> >
> > 1 - bind the entire buffer to UInt8.self, and then do buffer[1] <<
> UInt8.bitWidth | buffer[2]. probably most straightforward, but doesn’t
> generalize well at all to larger Int types.
> >
> > 2 - copy MemoryLayout<UInt16>.size bytes from offset 1 into the
> beginning of a new raw buffer, aligned to MemoryLayout<UInt16>.alignment,
> and do load(fromByteOffset: 0, as: UInt16.self) from that. Seems very
> inefficient because you have to allocate a new heap buffer copy everything
> over and then free it just to hold the bytes in the right alignment.
> >
> > 3 - use withUnsafeMutablePointer(to:_:) on a local variable of type
> UInt16, cast it to a raw pointer, and copy MemoryLayout<UInt16>.sizebytes
> into it. Like 2 it involves declaring a temporary variable which is
> annoying, and also, while the default initialization isn’t that big a
> problem, it’s introducing a meaningless value into the source code and can
> be problematic for non-integer types. Also, wasn’t Swift supposed to be
> designed so that Optional is the only thing which has a “default” value;
> Bool does not default to false and Int does not default to 0. Default
> constructors are evil.
>
> I agree. However, that meaningless value would just exist very temporarily
> in a function. I think you'd need a fancier type system to express 'this is
> an uninitialised value on the stack that can only be read after it has been
> written to'. Sure you could use a local Int16? but that'd come with some
> overhead.
>
> With endianness I still think you can use that function below and you'll
> get it super efficient. The compiler will (likely) inline that whole
> function anyway.
>
> What's the problem with the local temporary variable? You'd need that in C
> too. Maybe can you post the C code that you'd like to write? Then we can
> work from there and create some Swift code that does the same.
>
>
> enum Endianness {
>     case little
>     case big
> }
>
> func integerFromBuffer<T: FixedWidthInteger>(_ pointer:
> UnsafeRawBufferPointer, index: Int, endianness: Endianness = .big) -> T {
>     precondition(index >= 0)
>     precondition(index <= pointer.count - MemoryLayout<T>.size)
>
>     var value = T()
>     withUnsafeMutableBytes(of: &value) { valuePtr in
>         valuePtr.copyBytes(from: UnsafeRawBufferPointer(start:
> pointer.baseAddress!.advanced(by: index),
>                                                         count:
> MemoryLayout<T>.size))
>     }
>     switch endianness {
>         case .little:
>             return value.littleEndian /* does nothing on little endian,
> swaps on big */
>         case .big:
>             return value.bigEndian /* does nothing on big endian, swaps on
> little */
>     }
> }
>
> -- Johannes
>

This solution works for what I’m doing right now. However, a semi-separate
issue comes up when T is a more complex type (or a class type). There
zero-parameter initializers don’t make sense. This sounds like something
that should be supported by Builtin.


>
>
> >
> >
> >
> > On Wed, Nov 8, 2017 at 7:49 PM, Johannes Weiß <johannesweiss at apple.com>
> wrote:
> > Hi Kelvin,
> >
> > > On 8 Nov 2017, at 5:40 pm, Kelvin Ma <kelvin13ma at gmail.com> wrote:
> > >
> > > yikes there’s no less verbose way to do that? and if the type isn’t an
> integer there’s no way to avoid the default initialization? Can this be
> done with opaques or something?
> >
> > well, it's 5 lines for the generic case to rule all the integers. You
> could just put that in a function and never think about it again, right?
> >
> > func integerFromBuffer<T: FixedWidthInteger>(_ pointer:
> UnsafeRawBufferPointer, index: Int) -> T {
> >     precondition(index >= 0)
> >     precondition(index <= pointer.count - MemoryLayout<T>.size)
> >
> >     var value = T()
> >     withUnsafeMutableBytes(of: &value) { valuePtr in
> >         valuePtr.copyBytes(from: UnsafeRawBufferPointer(start:
> pointer.baseAddress!.advanced(by: index),
> >                                                         count:
> MemoryLayout<T>.size))
> >     }
> >     return value
> > }
> >
> > should work (untested). Also you might need to handle endianness.
> >
> > Regarding types that are not integers, what types are you thinking of?
> For normal Swift types the layout isn't guaranteed so you can't just read
> the bytes from somewhere. For C types where the layout is known you can
> just use the above code and either relax the constraint or specialise it to
> the very type you need. The only requirement of the type (besides that it's
> layout is defined) is that is has an empty constructor.
> >
> > This isn’t what I’m trying to do atm but does this mean it’s not
> possible to save a memory dump of a Swift struct to a file and then read it
> back in from the file to reconstitute it? Also the empty constructor
> requirement is problematic as explained before, especially when the type
> isn’t super simple like an Int.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.swift.org/pipermail/swift-users/attachments/20171109/a723161f/attachment.html>