Using zerocopy to convert types to and from bytes safely

Converting types to and from bytes

To convert types to and from bytes, we use the zerocopy library to do this safely. It primarily works through three core marker traits, which are implemented for primitive types and can be implemented for your own types with #[derive()]. They indicate what operations can be done on a type:

FromBytes indicates that a type may safely be converted from an arbitrary byte sequence. This means that every possible byte pattern is valid for that type - as such, this cannot be implemented for most enums.
AsBytes indicates that a type may safely be converted to a byte sequence. This means that the type contains no uninitialized memory, including padding bytes.
Unaligned indicates that a type's alignment requirement is 1.

Converting a type to bytes (preferred)

Converting a struct reference to byte slice is zero-cost and doesn't require a copy - so it's preferred to convert from type to bytes rather than the reverse.

// This is necessary to call methods provided by these traits.
use zerocopy::{AsBytes, FromBytes};

// This unlocks converting to and from bytes,
// and does a compile time check to ensure this is safe.
#[derive(AsBytes, FromBytes)]
#[derive(Clone, Copy)]  // Let `Foo` be copied cheaply: it is 2 registers.
#[repr(C)]  // This is necessary to guarantee a stable layout
struct Foo {
    x: u32,
    y: u16,
    z: u16,
}

// Create a `Foo` containing all zeroes, provided by `FromBytes`.
let mut foo: Foo = Foo::new_zeroed();
foo.x = 10;
foo.z = 42;

// Convert a `&Foo` to `&[u8]`.
let foo_bytes: &[u8] = foo.as_bytes();
assert_eq!(foo_bytes, &[10, 0, 0, 0, 0, 0, 42, 0]);

// Convert a `&mut Foo` to `&mut [u8]`.
let foo_bytes: &mut [u8] = foo.as_bytes_mut();
foo_bytes[4] = 27;
assert_eq!(foo.y, 27);

// This works for slices automatically: convert a `&[u32]` to `&[u8]`.
let chunk: [u32; 4] = [1,2,3,4];
assert_eq!(chunk.as_bytes(), [1, 0, 0, 0, 2, 0, 0, 0, 3, 0, 0, 0, 4, 0, 0, 0]);

// Example reading the board ID from storage.
let mut board_id = MaybeUninit::uninit();
let board_id = Storage::read(  // This ref points to the newly written data
    BOARD_INFO_PAGE,
    BOARD_ID_OFFSET,
    (&mut board_id).into()  // Converts to Out<InfoBoardId>
);

Converting a type from bytes

Converting a byte slice into a struct is not zero-cost, because size and alignment checks must be performed. You have three options:

Require your byte slice is properly aligned, meaning you can directly convert the reference without copying.
Copy out the data from the byte slice.
Dealign the type you're converting to.

Converting references directly

This does no copy, but requires the input byte slice be aligned correctly. The primary API to do this is zerocopy::LayoutVerified. LayoutVerified is a "witness type", meaning it wraps a byte slice and represents it having been checked for size and alignment for a certain type.

Its API is not incredibly ergonomic, but these examples should help:

use zerocopy::LayoutVerified;
let mut foo_bytes: [u8; 8] = [1, 1, 0, 0, 5, 0, 10, 0];

// This does a size/align check, and returns an `Option` based on whether the check succeeded.
match LayoutVerified::<&[u8], Foo>::new(&foo_bytes) {
    Some(foo) => {
        // The input byte array *was* properly aligned and sized for Foo.
        // We can use the Deref impl `LayoutVerified` provides...
        assert_eq!(foo.x, 0x101);
        // Or, we can convert directly to a `&Foo`.
        let foo: &Foo = foo.into_ref();
        assert_eq!(foo.y, 5);
    },
    None => {
        // We did nothing to ensure that `foo_bytes` was aligned to `u32`, so either branch
        // may be taken.
        println!("the input byte array was not properly aligned");
    }
}

// Only care about the byte slice being *at least* as long as the type?
// Use `LayoutVerified::new_from_prefix`:
match LayoutVerified::<&[u8], Foo>::new_from_prefix(&[1,2,3,4,5,6,7,8,9,10]) {
    Some((foo, rest)) => {
        assert_eq!(foo.x, 0x04030201);
        assert_eq!(rest, [9, 10]);
    },
    None => {
        println!("the input byte array was not properly aligned");
    }
}

// For mutability, use `LayoutVerified<&mut [u8], T>`
match LayoutVerified::<&mut [u8], Foo>::new(&mut foo_bytes) {
    Some(mut foo) => {
        // The input byte array *was* properly aligned and sized for Foo.
        // We can use the DerefMut impl `LayoutVerified` provides...
        foo.x = 0x202;
        // Or, we can convert directly to a `&mut Foo`.
        let foo: &mut Foo = foo.into_mut();
        assert_eq!(foo.y, 5);
        foo.z = 100;
    },
    None => {
        // We did nothing to ensure that `foo_bytes` was aligned to `u32`, so either branch
        // may be taken.
        println!("the input byte array was not properly aligned");
    }
}

// What if you need to convert bytes to a slice of objects?
// You can do this with LayoutVerified::new_slice.
let val = [0u8; 24];
match LayoutVerified::<&[u8], [Foo]>::new_slice(&val) {
    Some(foo) => {
        // The input array *was* properly aligned and sized for a slice of `Foo`.
        // 3 == 24 / sizeof(Foo)
        assert_eq!(foo.len(), 3);
    },
    None => {
        println!("the input byte array was not properly aligned");
    }
}

Copy the bytes out into aligned memory

If we don't know that the input byte slice is properly aligned, we can perform a copy.

// This still has to do a size check, so it returns an `Option`. If we have an array with a
// known size at compile time, the optimizer should remove the `None` branch that panics.
let foo = Foo::read_from(&foo_bytes[..]).unwrap();

// However, if the type has a known size at compile time, we can use `zerocopy::transmute!`.
// This converts a [u32; 2] to [u16; 4] by copying (which may be optimized out).
let val: [u32; 2] = zerocopy::transmute!([0u16, 5, 10, 20]);

// This copies a `[u32; 2]` to a `Foo` safely and with no cost, because they have the same size.
let foo: Foo = zerocopy::transmute!([0u32, 10]);

Use a type with no alignment requirement

By removing the need for your struct to be aligned, there are no copies. You can do this by using unaligned primitive types from zerocopy::byteorder, the zerocopy::Unalign struct, or by declaring your struct #[repr(packed)].

// zerocopy::byteorder comes with unaligned types for primitives - they also support explicit endianness.
use zerocopy::byteorder::{U32, BE, LE};
let val: [u8; 8] = [1, 2, 3, 4, 5, 6, 7, 8];
// This will never fail because `val` is small enough.
let val_u32le = LayoutVerified::<&[u8], [U32<LE>; 2]>::new(&val).unwrap().into_ref();
assert_eq!(val_u32le[0].get(), 0x04030201);

let val_u32be = LayoutVerified::<&[u8], [U32<BE>; 2]>::new(&val).unwrap().into_ref();
assert_eq!(val_u32be[0].get(), 0x01020304);

// For custom structs, zerocopy provides an `Unalign` struct that dealigns what it wraps, copying out
// into an aligned type when needed.
use zerocopy::Unalign;
let foo: &Unalign<Foo> = LayoutVerified::<&[u8], Unalign<Foo>>::new(&val).unwrap().into_ref();
assert_eq!(foo.get().z, 0x0807);