How can you create an "aligned" array of bytes and read from it?

Question

I want to be able to use the UnsafeRawBufferPointer.load(fromByteOffset:as:) method to read a number of bytes from a [UInt8] array and their corresponding type as an UnsignedInteger, FixedWidthInteger at each read.

In both approaches that follow, a "Fatal error: load from misaligned raw pointer" exception is raised, since load expects the underlying data to be aligned in memory.

I have tried using a ContiguousArray

    var words: ContiguousArray<UInt8> = [0x01, 0x00, 0x03, 0x0a, 0x00, 0x01, 0x00, 0xec, 0xfa, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7c, 0x20, 0x00, 0x00, 0xe0, 0x88, 0x47, 0xa3, 0xd6, 0x6b, 0xd6, 0x01, 0x4c, 0xff, 0x08]
    
    var offset = 0
    let byte = words.withUnsafeBytes { $0.load(fromByteOffset: offset, as: UInt8.self) }
    offset += MemoryLayout<UInt8>.size
    let bytes = words.withUnsafeBytes { $0.load(fromByteOffset: offset, as: UInt16.self) }

    XCTAssertEqual(byte, UInt8(littleEndian: 0x01))
    XCTAssertEqual(bytes, UInt16(littleEndian: 0x0003))

Allocating and initialising a UnsafeMutablePointer

    var words: [UInt8] = [0x01, 0x00, 0x03, 0x0a, 0x00, 0x01, 0x00, 0xec, 0xfa, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7c, 0x20, 0x00, 0x00, 0xe0, 0x88, 0x47, 0xa3, 0xd6, 0x6b, 0xd6, 0x01, 0x4c, 0xff, 0x08]
    
    let uint8Pointer = UnsafeMutablePointer<UInt8>.allocate(capacity: words.count)
    uint8Pointer.initialize(from: &words, count: words.count)
    
    let rawPointer = UnsafeMutableRawPointer(uint8Pointer)
    
    var offset = 0
    let byte = UInt8(bigEndian: rawPointer.load(fromByteOffset: offset, as: UInt8.self))
    offset += MemoryLayout<UInt8>.size
    let bytes = UInt16(bigEndian: rawPointer.load(fromByteOffset: offset, as: UInt16.self))

    rawPointer.deallocate()

    uint8Pointer.deinitialize(count: words.count)
    uint8Pointer.deallocate()

    XCTAssertEqual(byte, UInt8(littleEndian: 0x01))
    XCTAssertEqual(bytes, UInt16(littleEndian: 0x0003))

Can you please point out where my misunderstanding lies in each and provide a working example?

Answer 1

You can now just use loadUnaligned(fromByteOffset:as:) :

    var words: [UInt8] = [0x01, 0x00, 0x03, 0x0a, 0x00, 0x01, 0x00, 0xec, 0xfa, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x7c, 0x20, 0x00, 0x00, 0xe0, 0x88, 0x47, 0xa3, 0xd6, 0x6b, 0xd6, 0x01, 0x4c, 0xff, 0x08]
    
    var offset = 0
    let byte = words[0] // doesn't need an unaligned access
    offset += 1
    let bytes = words.withUnsafeBytes {
        UInt16(bigEndian: $0.loadUnaligned(fromByteOffset: offset, as: UInt16.self))
    }

    precondition(byte == UInt8(littleEndian: 0x01))
    precondition(bytes == UInt16(littleEndian: 0x0003))

Answer 2

One way to guarantee an “aligned” array of bytes is by using the same UnsignedInteger, FixedWidthInteger type. Since the array is homogenous, alignment is guaranteed by the size of each type.

e.g. a [UInt8] array that holds byte types

        var array: [UInt8] = [0x01, 0x00, 0x03]

        var offset = 0
        let byte = array.withUnsafeBytes { $0.load(fromByteOffset: offset, as: UInt8.self) }

        offset += MemoryLayout<UInt8>.size
        let bytee = array.withUnsafeBytes { $0.load(fromByteOffset: offset, as: UInt8.self) }

        offset += MemoryLayout<UInt8>.size
        let byteee = array.withUnsafeBytes { $0.load(fromByteOffset: offset, as: UInt8.self) }

        XCTAssertEqual(byte, 0x01)
        XCTAssertEqual(bytee, 0x00)
        XCTAssertEqual(byteee, 0x03)

e.g. a [UInt8] array that holds 2 byte types

        var array: [UInt8] = [0x01, 0x00, 0x03, 0x0a, 0x00, 0x01]

        var offset = 0
        let bytes = UInt16(bigEndian: array.withUnsafeBytes { $0.load(fromByteOffset: offset, as: UInt16.self) })

        offset += MemoryLayout<UInt16>.size
        let bytess = UInt16(bigEndian: array.withUnsafeBytes { $0.load(fromByteOffset: offset, as: UInt16.self) })

        offset += MemoryLayout<UInt16>.size
        let bytesss = UInt16(bigEndian: array.withUnsafeBytes { $0.load(fromByteOffset: offset, as: UInt16.self) })

        XCTAssertEqual(bytes, 0x0100)
        XCTAssertEqual(bytess, 0x030a)
        XCTAssertEqual(bytesss, 0x0001)

The misunderstanding lies in that, it is not possible to have an "aligned" array of bytes when types stored in the array can't be aligned given their mismatched size.

The size of UInt8 being 1 byte and the size of UInt16 being 2 bytes in this case creating an unaligned array that is not supported by UnsafeRawBufferPointer.load(fromByteOffset:as:).

Answer 3

There are multiple problems:

1. UnsafeMutablePointer makes no alignment guarantees. One could use POSIX functions (like posix_memalign) instead, but that does not really help if ...

2. The values are at arbitrarily byte offsets in the byte array. As an example, if a: UInt16, b: UInt8, c: UInt32 are packed (in this order)

   aa aa bb cc cc cc cc
   

   then there is no way to align this buffer such that all integer values are properly aligned.

3. Unfortunately, there are no methods yet to load or initialize a value from unaligned data. (This has been suggested (1, 2) but not yet implemented). It does not matter if you use Array, ContiguousArray, or Data as the byte source.

The safest way is still to load the bytes and use bit shifting to combine them to the values:

let byteArray: Array<UInt8> = [0x01, 0x00, 0x03, 0x0a]
var offset = 0
let ui8 = byteArray[offset]
print(ui8) // 1
offset += MemoryLayout<UInt8>.size
let ui16 = UInt16(byteArray[offset]) << 8 + UInt16(byteArray[offset + 1])
print(ui16) // 3

An alternative is to copy the bytes (and do the byte-order conversion):

let byteArray: Array<UInt8> = [0x01, 0x00, 0x03, 0x0a]
var offset = 0
var ui8 = UInt8(0)
_ = withUnsafeMutableBytes(of: &ui8, { byteArray.copyBytes(to: $0, from: offset...) } )
print(ui8) // 1
offset += MemoryLayout<UInt8>.size
var ui16 = UInt16(0)
_ = withUnsafeMutableBytes(of: &ui16, { byteArray.copyBytes(to: $0, from: offset...) } )
print(UInt16(bigEndian: ui16)) // 3