Skip to content

Instantly share code, notes, and snippets.

@ccdle12

ccdle12/Rust-Cheat-Sheet.md

Last active September 29, 2023 01:51
Show Gist options
  • Save ccdle12/48ec24f4e25b3f289b873a1d32b41980 to your computer and use it in GitHub Desktop.
Save ccdle12/48ec24f4e25b3f289b873a1d32b41980 to your computer and use it in GitHub Desktop.
Download ZIP
My Cheat Sheet for programming in Rust.
Raw
Rust-Cheat-Sheet.md

Rust Cheat Sheet

Covers different areas of programming and the best practices/components explained.

Frequently used crates

A crate that contains a trait StructOpt. Allows a structure Opt to be converted
from command line arguments (clap)

A command line tool crate.

Index

Arrays vs Slices vs Vectors

Arrays

  • Arrays are fixed length contiguous blocks of memory.
  • They must be declared as [TYPE, SIZE].
  • Arrays in rust are allocated on the stack and not pointing to the heap.
  • Arrays in rust are a sequence of values not pointers.
    • This is probably why arrays can be copied.
let ages: [u32, 5] = [1, 2, 3, 4, 5];

Arrays don't change ownership apparently?

fn change_array_ownership(x: [u32; 5]) {
    println!("changed ownership");
    println!("x: {:?}", x[0]);
}

fn main() {
    // an array.
    // Arrays are copied? - Ownership doesn't seem to change?
    let a: [u32; 5] = [1, 2, 3, 4, 5];
    change_array_ownership(a);
    println!("a: {:?}", a[0]);
}

The above demo, shows that we can accept an array and not lose ownership in the outer scope.

TODO:

  • can you return arrays? [x]

Slices

A slice is an array that isn't known at compile time.

Slices can be viewed as a borrow of an array.

let b: &[u32] = &[1, 2, 3, 4, 5, 6, 7];

// Since we are accepting a slice that has an unknown size at compile time,
// we could try to access an index that doesn't exist.
fn pass_slice(b: &[u32]) {
    println!("In pass_slice(): {:?}", b[10]);
}

Slices can also get a subset of a slice.

// Subset of a slice.
let c: &[u32] = &a[0..3];
println!("c: {:?}", c[1])

Slices can be returned via a function, but since slices are essentially pointers to arrays, we need to specify the lifetime of the slice.

// Return a subset of a slice.
fn subset_slice<'a>(b: &'a [u32]) -> &'a [u32] {
    &b[1..2]
}

// Subset of a slice.
let c: &[u32] = &a[0..3];
println!("c: {:?}", c[1]);
let subset: &[u32] = subset_slice(c);
println!("subset: {:?}", subset[0]);

Vectors

  • Allocates to the heap and owns the allocation.
let v = vec![1, 2, 3, 4];      // A Vec<i32> with length 4.
let v: Vec<i32> = Vec::new();  // An empty vector of i32s.

Async

Async programming traits and objects.

Arc

Atomic Refernce Counted

A unique pointer that allows ownership of a value T allocated on the heap.

Arc requires clone implementation (i think). When clone is called on an Arc instance, it creates another Arc that points to the same value on the heap as the source Arc and increases the reference count.

When the last Arc pointer is destroyed, the source Arc is dropped from memory.

By default Arc does not allow a mutable reference to the value.

In order to make Arc mutable it needs to be wrapped in a Mutex, RwLock or Atmoic.

store: Arc<Mutex<HashMap<String, String>>>,

The above has an Arc that wraps a mutex to a HashMap. This indicates that this Arc will be muted from different threads.

Futures

A crate that contains the trait Future.

Similar to promise in Javascript.

So to put it simply there are two constructs - Executor and Future.

The Executor polls the Future to see if it is ready.

Simple Example:

trait Future {
    type Output;
    fn poll(&mut self, waker: &Waker) -> Poll<Self::Output>
}

enum Poll<T> {
    Ready(T),
    Pending,
}

Advantages of Futures

  • Cancelling a future is really easy - we just stop polling it
    • In an event loop model, once its on the queue, its hard to cancel it

State Machines

  • Each future is a state machine on a single heap allocation

Mutex

Mutex mutual exclusion primitive that blocks threads from accessing a resource.

It uses a lock and try_lock - ensures the resource is only ever accessed when the mutex is locked.

Arc<Mutex<HashMap<String, String>>>
...
self.store.lock().unwrap().insert(key, value);

Waker

https://docs.rs/futures/0.3.1/futures/task/struct.Waker.html

A struct that is a handler for waking up a task and notifying an executor.

Has methods:

  • wake()
  • wake_by_ref()
  • will_wake()

waker_ref

https://docs.rs/futures/0.3.1/futures/task/trait.ArcWake.html

Part of the futures::task::ArcWake trait.

Converts a type that is wrapped in an Arc into a Waker.

There are two methods that can be called to wake up:

  • waker() -> converts Arc
  • waker_ref() -> converts &Arc

Implementaiton method:

  • wake_by_ref(arc_self: &Arc<Self>)

Pin

A trait that gurantees an object won't ever be moved.

Context

Used as the context of an async task

It wraps a waker

Has a method:

from_waker() converts a ref of a waker to a Context

BoxFuture

An type alias for Box<Future + Send>

Binary and Bytes

Byte Array

Creates a Byte Array of x number of bytes.

let mut buffer: [u8; 5] = [0, 5];

let mut buffer = [0_u8, 5];

Best Practice for using Byte Arrays

XOR

From the rust crypto library it seems like the best practice is to use GenericArrays to declare different types of arrays with generic lengths at compile time.

Also it xors two slices, but mutates the first slice.

// XOR bytes in place.
pub fn xor(a: &mut [u8], b: &[u8]) {
    assert_eq!(a.len(), b.len());
    for (a, b) in a.iter_mut().zip(b) {
        *a ^= *b;
    }
}

fn main() {
    let mut a: [u8; 5] = [0x01, 0x02, 0x03, 0x04, 0x05];
    let b: [u8; 5] = [0x06, 0x07, 0x08, 0x09, 0x0A];

    println!("BEFORE: {:?}", &a);
    xor(&mut a, &b);
    println!("AFTER: {:?}", &a);
}

Print bytes as string from byte array

let mut buffer = [0u8];

String::from_utf8_lossy(&buffer);

Ones and Twos Compliment

Ones compliment is the inversion of bits in a byte/word.

01101 -> 10010

Twos compliment is the inversion of bits in a byte/word + 1 to the least significat bit?

This is used to display a signed (negative/positive) number.

1. Invert bytes/word using ones compliment.
01000001 -> 10111110

2. Convert ones compliment to twos compliment.

  a. Invert the bits like ones compliment.
     4 = 100
  b. Add 0 to the front.
     4 = 0100
  c. Invert the bits.
     1011
  d. Add 1 to the inversion.
     1 + 1011 = 1100
  
  e. Calculating twos compliment.
     8421
     ----
     1100 = (-8 + 4 = -4) = -4

Twos compliment is calculated using the left most bit as the sign. If its set, calculate the 2s place as a negative.

Example:

-8 = 1100
 8 = 01000

Notation

Terms:

  • Least significat bit (rightmost)
  • bits - 1 bit
  • nibbles - 4 bits
  • bytes - 8 bits
  • halfwords - 16 bits
  • words - 32 bits
  • doublewords - 64 bits

Set bit:

A bit is set when the bit is 1.

A bit is not set (off) when it is 0.

Basic Formulas

Concatenate two bytes

Takes two bytes and concatenates the two bytes as u16.

  • Shift the left concatenation by 8 bytes
  • OR the shifted left with the right, this should keep the bytes the same on each side of the concatenation as a u16
  let left = 1;
  let right = 4;
  let result = ((left as u16) << 8) | right as u16;

Turn off right most bit

Turns off the right most bit of a byte/word, this doesn't mean it turns off the last bit, but the right most significant bit which is the most right bit that is set (1).

Performs a Bitwise AND of a & (a-1).

The idea is that a-1 will always be different from a being a power of 2 or 0.

The right most bit of a compared with a-1 will always be different.

a = 0
or
a = 1

Formula:

a & (a - 1)

Example:

 a:     00000101
 a - 1: 00000100
 after: 00000100

 d before bit: 01011000
        d - 1: 01010111
 after:        01010000

Turn on right most bit

Turns on the rigt most bit. If byte/word is none it sets it to all 1.

Performs a Bitwise OR of a | (a+1).

Formula:

a | (a + 1)

The right most bit it the right most not set bit.

        v--- The right most bit.
0000 10101

Example:

 b before bit: 00001001
        b + 1: 00001010
        after: 00001011

Turn off trailing ones

Turns off trailing ones in a word/byte.

Meaning ones become zeroes. 1 => 0.

Formula:

a & (a + 1)

Example:

          f before bit: 01011011
                 f + 1: 01011100
Turn off trailing ones: 01011000

Turn on trailing zeroes

Turns on trailing zeroes in a word/byte.

Meaning zeroes become ones. 0 => 1.

Formula:

a & (a + 1)

Example:

           d before bit: 01011000
                  d - 1: 01010111
Turn on trailing zeroes: 01011111

Print in binary

let a = 5;

println!("a in binary: {}", format("{:#b}", a))

Operations

Bitwise AND

Bitwise AND compares each bit between two numbers/bytes.

If two bits are 1 == 1 then 1 is returned in the output at the index position.

If two bits are NOT 1 == 1, then 0 is returned in the output at the index position.

Example:

a = (0000 0101)
b = (0000 1001)
---------------
c = (0000 0001)

Code:

/// Returns the result of a bitwise AND.
int bitwise_and(int a) {
    return ~a;
}

Bitwise OR

Bitwise OR compares each bit between two numbers/bytes.

If at least one of the bits is a 1 || 0 then 1 is returned in the output at the index.

If both bits are NOT 1, then 0 is returned in the output at the index postion.

Example:

a = (0000 0101)
b = (0000 1001)
---------------
c = (0000 1101)

Code:

/// Returns the result of a bitwise OR.
fn bitwise_or(a: u8, b: u8) -> u8 {
    a | b
}

Bitwise NOT

Bitwise NOT takes one number/byte and inverts all the bits.

If a bit is 1, NOT will invert it to 0.

Example:

a = (0000 0101)
---------------
c = (1111 1010)

/// Returns the result of a bitwise OR.
fn bitwise_not(a: u8) -> u8 {
    !a
}

Bitwise XOR

Bitwise XOR compares each bit between two numbers/bytes.

If the two compared bits are different 1 && 0 then 1 is returned in the output at the index.

If both bits are the SAME, then 0 is returned in the output at the index postion.

Example:

a = (0000 0101)
b = (0000 1001)
---------------
c = (0000 1100)

Code:

/// Returns the result of a bitwise OR.
fn bitwise_xor(a: u8, b: u8) -> u8 {
    a ^ b
}

Left Shift

Shifts the bits to the left in an integer/byte by x number of places.

Example:

b = (0000 1001) << 1
--------------------
b = (0001 0010)

Code:

fn left_shift(a: u8, amount: u8) -> u8 {
    let c = a;
    c << amount
}

Right Shift

Shifts the bits to the right in an integer/byte by x number of places.

Example:

b = (0000 1001) >> 1
--------------------
b = (0000 0100)

Code:

// Shifts the integer by amount.
fn right_shift(a: u8, amount: u8) -> u8 {
  let c = a;
  c >> amount;
}

Bin folder

This contains cli files.

Different cli files can be called using

cargo run --bin <name-of-file>

Using local library in Bin folder

When using a local library (files one level up) in a .rs in /bin.

We need to use the name of the cargo project found in Cargo.toml in the import

use kvs::KvsServer

In Cargo.toml:

  [package]
~ name = "kvs"
  version = "0.1.0"
  authors = ["ccdle12 <[email protected]>"]
  edition = "2018"

Cargo and Depedencies

The following explains how to use cargo and build dependencies.

Add a dependency

Adds a cargo create

$ cargo add "name of crate"

Building binaries

A binary can be built as debug or release.

Debug

cargo build

Will build the binaries at target/debug.

Release

cargo build --release

Will build the binaries at target/release

Creating a lib project

Create a Rust library project.

cargo init --lib

Search for a crate

$ cargo search "name of crate"

$ cargo search serde 
> serde = "1.0.97"                          # A generic serialization/deserialization framework
serde_json_experimental = "1.0.29-rc1"    # A JSON serialization file format
serde_json = "1.0.40"                     # A JSON serialization file format
typescript-definitions = "0.1.10"         # serde support for exporting Typescript definitions
erased-serde = "0.3.9"                    # Type-erased Serialize and Serializer traits
serde_rustler = "0.0.3"                   # Serde Serializer and Deserializer for Rustler NIFs
serde_any = "0.5.0"                       # Dynamic serialization and deserialization with the format chosen at runtime
serde_gelf = "0.1.6"                      # Gelf serialization using serde.
serde_yaml = "0.8.9"                      # YAML support for Serde
serde-feature-hack = "0.2.0"              # A hack to allow having a feature named serde while having serde as a dependency
... and 804 crates more (use --limit N to see more)

Importing a local project

A local/library project can be imported.

NOTE: - Need to actually try this

In Cargo.toml:

...
[dependencies]
my_library = { path = "~/code/my_library" }

Installing Rust and Cargo

To automate the installation

curl https://sh.rustup.rs -sSf | sh -s -- -y

Modules

The following explains how the lib.rs file works in terms of importing modules and declaring publicly exposed modules.

  1. Declare any pre-processors for the crate
  2. Create docstring for the library //!<something>
  3. Import the modules used in the project mod <something>
  4. Declare publicly facing modules pub use <something>
#![deny(missing_docs)]
//! A simple key/value store.

#[macro_use]
extern crate log;


mod client;
mod common;
mod engines;
mod error;
mod server;

pub use client::KvsClient;
pub use engines::{KvStore, KvsEngine, SledKvsEngine};
pub use error::{KvsError, Result};
pub use server::KvsServer;

Mod Import

  • mod <x> imports a module in the library allowing it to be used in other modules of the library.

Importing other modules in from the same library can be achieved using: (in the specific module file)

Because we declared the module crate in lib.rs we can use it in client.rs

use crate::common::{GetResponse, RemoveResponse, Request, SetResponse};
use crate::{KvsError, Result};
  • Also if there is a subfolder, the folder needs to have mod.rs file to allow to be importable in lib.rs.
  • It also declares the modules used at this folder level using mod kvs and declares publicly available structs/traits using pub use.

Example:

//! This module provides various key value storage engines.

use crate::Result;

/// Trait for a key value storage engine.
pub trait KvsEngine {
    /// Sets the value of a string key to a string.
    ///
    /// If the key already exists, the previous value will be overwritten.
    fn set(&mut self, key: String, value: String) -> Result<()>;

    /// Gets the string value of a given string key.
    ///
    /// Returns `None` if the given key does not exist.
    fn get(&mut self, key: String) -> Result<Option<String>>;

    /// Removes a given key.
    ///
    /// # Errors
    ///
    /// It returns `KvsError::KeyNotFound` if the given key is not found.
    fn remove(&mut self, key: String) -> Result<()>;
}

mod kvs;
mod sled;

pub use self::kvs::KvStore;
pub use self::sled::SledKvsEngine;

Exporting modules

Allows a module to be publicly exportable from the library.

pub use client::KvsClient;

Using own modules and crates

Imports public structs/enums etc... that were made available via the lib.rs file.

To access modules from within the project use crate::{...}

use crate::{CSError, Result};
...

Turn off dead code warnings

Turns off dead code warnings aka unused functinons. Particularly useful for libraries. This example sets it in lib.rs which is applied to all modules in the library.

#![allow(dead_code)]
//! A library for cryptography implementations.

Big Number

Currently using this big number library: https://github.com/rust-num/num-bigint

  • Requires a random number library: cargo add rand
  • Requires a the number trains: cargo add num_traits

Dependencies should look as follows:

[dependencies]
num-bigint = { version = "0.2", features = ["rand"] }
rand = "0.5"
num-traits = "0.2.8"

Imported in lib.rs as:

//! Demonstrates the use of a XOR cipher.

extern crate num_bigint as bigint;
extern crate num_traits;
extern crate rand;

mod xor_cipher;

Generates a random number according to this bit size. In the below example, its a signed number of 1000 bits.

let mut rng = rand::thread_rng();
let random_big_num = rng.gen_bigint(1000);

Parse Bytes

Parsing Bytes converting it to a BigUint. Needs a RADIX.

fn bytes_to_biguint(message: &[u8]) -> BigUint {
    BigUint::parse_bytes(message, RADIX)?
}

To String

Converts a biguint to string according to a radix.

fn big_uint_to_str(message: BigUint) -> String {
    message.to_str_radix(RADIX)
}

Clap CLI

Currently I'm using Clap and structopt to create a command line interface.

1. Update Cargo.toml to use the following dependencies

[dependencies]
clap = "2.33.0"
structopt = "0.2.18"

2. Create the following folder structure

The bin folder will be the compiled folder. This is where the Clap crate wil be used to create the cli.

├── bin
│   └── cs_cli.rs
├── cs_manager.rs
├── error.rs
└── lib.rs

3. Update lib.rs to declare all the modules

Lib contains all the modules/folders NOT in the the /bin folder.

extern crate failure;
#[macro_use]
extern crate failure_derive;

pub use cs_manager::CSManager;
pub use error::{CSError, Result};

mod cs_manager;
mod error;

4. Update clap cli file in /bin

Line 2: calls the modules from outside /bin using cs_cli. The reason being cs_cli is the name of the package found in Cargo.toml.

Line 1+3: StructOpt is the crate used to create a command line parser using a struct.

Line 4-16: Declaration of an enum Opt that applies structopt. The enum contains the commands of the cli and the parameters.

1. extern crate structopt;
2. use cs_cli::{CSManager, Result};
3. use structopt::StructOpt;

4. #[derive(Debug, StructOpt)]
   #[structopt(
      name = "cs_cli",
      about = "A cli tool to  calculate portfolio allocations"
   )]
   enum Opt {
      /// Caluclates a given USD value, split into allocations for each product.
      #[structopt(name = "calc")]
      Calc {
          #[structopt(help = "The amount in USD to calculate the allocation")]
          amount: f64,
      },
16. }

fn main() -> Result<()> {
    match Opt::from_args() {
        Opt::Calc { amount } => {
            let cs_manager = CSManager::from_default()?;
            let alloc_suggestion = cs_manager.calc_allocation(amount);

            match alloc_suggestion {
                Ok(s) => {
                    println!("{:?}", s);
                    std::process::exit(0);
                }
                Err(e) => {
                    println!("Error: {:?}", e);
                    std::process::exit(1);
                }
            }
        }
    }
}

Optional Parameters

Using Option<> lets the parameter be optional.

...
#[structopt(short = "a", help = "The server address as IP:PORT")]
address: Option<String>,
...

Documentation

The following explains how to write documenation.

Open Project Documentation

$ cargo doc --open

Open Project Documentation without Depedencies

$ cargo doc --no-deps --open

Example Code in Documenation

Functions should have code examples in the documentation strings.

  /// Sets a string value according to a key.
  /// If the key already exists the value will be overwritten.
  ///
  /// Example:
  ///
  /// ```rust
  /// # use kvs::KvStore;
  /// # use std::env;
  /// let key = "hello";
  /// let value = "world";
  ///
  /// let current_dir = env::current_dir().unwrap();
  ///
  /// let mut kv_store = KvStore::open(&current_dir).unwrap();
  /// kv_store.set(key.to_string(), value.to_string()).unwrap();
  /// ```

Enums

Enums are similar to structs/classes but are similar to C-style enums.

The below is an enum Command with variants - Set, Get, Remove with their own fields.

  /// Command is an enum with each possible command of the database. Each enum
  /// command will be serialized to a log file and used as the basis for populating/
  /// updating an in-memory key/value store.
  #[derive(Serialize, Deserialize, Debug)]
  pub enum Command {      
    Set { key: String, value: String },
    Get { key: String },
    Remove { key: String },
  }

Error Handling

Custom Error Implementation

This is NOT the current best practice, this is simply to display how to do it in the simplest way possible.

  1. The custom error must implement the error::Error trait (identifies as an error)
  2. The custom error must implement the std::fmt::Display trait for the error message to be displayed
use std::error;
use std::fmt;

#[derive(Debug)]
struct NegativeError;

// Implements the error trait.
impl error::Error for NegativeError {}

// Formats the display message NegativeError.
impl fmt::Display for NegativeError {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        write!(f, "Number is below 0")
    }

}

Current Best Practice

NOTE:

Requires:

Needs the failure dependencies.

failure = "0.1.5"
failure_derive = "0.1.5"

Failure crate needs to be imported in lib.rs and the macros for failure_derive.

extern crate failure;
#[macro_use]
extern crate failure_derive;

Current Best Practice Implementation

Current strategy is to create an error.rs file.

The error.rs file uses:

  • A custom enum for the project
  • Each variant is an error type that can be raised
  • Implements From for each non-custom error type
  • The file exports a Result<T, CustomErrorEnum>
    • This will be used throughout the projet when returning a Result<> by using ?
/// The custom error type for this project. Each different error type will be
/// added as an enum variant.
#[derive(Fail, Debug)]
pub enum KvStoreError {
    /// Used for errors that are miscellaneous and/or cannot be explained.
    #[fail(display = "An unknown error has occurred")]
    UnknownError,

    /// Error for a key not found in the key value store.
    #[fail(display = "Key not found")]
    KeyNotFoundError,

    /// Serde serialization and deserialization errors.
    #[fail(display = "{}", _0)]
    Serde(#[cause] serde_json::Error),

    /// Standard Input/Output errors.
    #[fail(display = "{}", _0)]
    IOError(#[cause] std::io::Error),
}

impl From<serde_json::Error> for KvStoreError {
    fn from(err: serde_json::Error) -> KvStoreError {
        KvStoreError::Serde(err)
    }
}

impl From<std::io::Error> for KvStoreError {
    fn from(err: std::io::Error) -> KvStoreError {
        KvStoreError::IOError(err)
    }
}

/// Shorthand alias for Result in this project, uses the concrete implementation
/// of KvStoreError.
pub type Result<T> = std::result::Result<T, KvStoreError>;

  • An example of using the error throughout the project.

  • We can use ? when a function returns a Result.

pub fn open(path: &Path) -> Result<KvStore> {
        let mut path_buf = PathBuf::from(path);
        create_dir_all(&path_buf)?;

If a function could return an error, return it wrapped in Result<type>.

If a function doesn't return anything then:

fn some_func() -> Result<()> {
    ...
    Ok(())
}

Returns a result wrapping (). If it reaches teh end of the function it needs to return an Ok(()).

Importing in bin folder:

use <name_of_project>::{Result};

Importing in library project scope:

use crate::{KvStoreError, Result};

File System

The following explains how to interact with the File Sytem in Rust.

BufWriter and BufReader

BufWriter and BufReader is a wrapper of reader and writer. It has slightly better performance when used for Reads/Writes that perform frequent calls.

BufRead:

use std::fs::File;
use std::io::prelude::*; // bulk import of many common io traits.
use std::io::BufReader;

fn main() {
    let f = File::open("foo.txt").unwrap();
    let mut reader = BufReader::new(f);
    let mut buffer = String::new();

    reader.read_line(&mut buffer).unwrap();
    println!("buffer: {:?}", buffer);
}

BufWrite:

use std::fs::File;
use std::io::prelude::*; // bulk import of many common io traits.
use std::io::BufWriter;

let mut writer = BufWriter::new(File::create("bar.txt").unwrap());

for _i in 0..10 {
  writer.write(b"world\n").unwrap();
}

BufReader - Reading from a file and serialzing to enum or struct

Example opens a file using a BufReader and iterating over each line in the file using ...lines().

Serde_json is used to serialize the read line from a str to an enum Command.

for line in BufReader::new(file_handler).lines() {
    let cmd: Command = serde_json::from_str(&line?)?;

    if let Command::Set { key, value } = &cmd {
        store.insert(key.to_string(), value.to_string());
    };

    if let Command::Remove { key } = &cmd {
        store.remove(&key.to_string());
    };
}

BufWriter Appending to a file

Uses the OpenOptions to open a file to be appended to and uses the BufWriter to write to the file.

use std::fs::File;
use std::fs::OpenOptions;
use std::io::prelude::*; // bulk import of many common io traits.
use std::io::{BufReader, BufWriter};

let file_handler = OpenOptions::new()
   .append(true)
   .open("foo.txt")
   .expect("failed to open file");

let mut writer = BufWriter::new(file_handler);

for _i in 0..10 {
  writer.write(b"foobar\n").unwrap();
}

Create folder path recursively if files and folders are missing

create_dir_all() is part of the std::fs module. It creates a folder path given a borrowed PathBuf. This will only create the files/folders if they don't exist.

use std::fs::create_dir_all

create_dir_all(&path_buf)?;

DirEntry

Returns an iterator with all the files and folders.

use std::fs::{self, File};
use std::io::prelude::*;
use std::io::SeekFrom;

fn main() {
    let mut f = File::open("foo.txt").unwrap();

    let result = f.seek(SeekFrom::Start(33)).unwrap();
    println!("result: {}", result);

    fs::read_dir(".")
        .unwrap()
        .for_each(|x| println!("{:?}", x.unwrap().path()));
}

Get the current directory

current_dir() part of the std::env module. Returns a PathBuf of the current directory.

https://doc.rust-lang.org/std/env/fn.current_dir.html

use std::env

let current_dir = env::current_dir()?;

Get all directories and files in a path as an iterator

Returns all files and folders in a directory path as an iterator.

use std::fs::read_dir;
use std::path::PathBuf;
...

let path = PathBuf::from("./");
let _directories = read_dir(&path)
     .unwrap()
      .for_each(|x| println!("dir: {:?}", x));
>
dir: Ok(DirEntry("./Cargo.toml"))
dir: Ok(DirEntry("./target"))
dir: Ok(DirEntry("./Cargo.lock"))
dir: Ok(DirEntry("./world"))
dir: Ok(DirEntry("./.gitignore"))
dir: Ok(DirEntry("./hello"))
dir: Ok(DirEntry("./.git"))
dir: Ok(DirEntry("./3"))
dir: Ok(DirEntry("./src"))

Line Writer

LineWriter is part of the std::io module.

LineWriter is a wrapper of BufWriter, the difference is LineWriter deoes not write the contents of it's buffer until a \n is written or detected.

use std::io::prelude::*;
use std::io::LineWriter;
use std::path::PathBuf;

fn main() {
    let path = PathBuf::from("hello.txt");

    let file_handler = OpenOptions::new()
        .write(true)
        .create(true)
        .open(&path)
        .expect("failed to create file using path");

    let mut file_writer = LineWriter::new(file_handler);

    file_writer.write_all(b"This is the first line.").unwrap();
    assert_eq!(fs::read_to_string("hello.txt").unwrap(), "");

    file_writer.write_all(b"\n").unwrap();
    assert_eq!(
        fs::read_to_string("hello.txt").unwrap(),
        "This is the first line.\n"
    );
}

PathBuf

PathBuf is part of the std::path module.

A PathBuf is essentially like a String. It allows the building of folder/directory paths represented as Strings and other functions to extend/manipulate a path.

https://doc.rust-lang.org/std/path/struct.PathBuf.html

let mut path_buf = PathBuf::from(&path);

path_buf.push("log");
path_buf.set_extension("txt");

OpenOptions

OpenOptions is part of the std::fs module.

It is a builder, allows opening a file with certain parameters.

https://doc.rust-lang.org/std/fs/struct.OpenOptions.html

Open a file if it exists, if not create it.
OpenOptions::new()
  .write(true)
  .create(true)
  .open(&path)
  .expect("failed to creat file using path");
 
Open a file and write/append to it.
let file_handler = OpenOptions::new()
      .append(true)
      .open(&path)
      .expect("failed to open folder");
      
Open a file as read only.
OpenOptions::new()
    .read(true)
    .open(&path)
    .expect("failed to open file as read only");

Seek

Moves a cursor by x amount of bytes.

use std::fs::File;
use std::io::prelude::*;
use std::io::SeekFrom;

fn main() {
    let mut f = File::open("foo.txt").unwrap();

    let result = f.seek(SeekFrom::Start(33)).unwrap();
    println!("result: {}", result);
}

HashMaps

An key/value pair datastructure

Get

use std::collections::HashMap;

let hm = HashMap::new();

hm.insert("hello", "world");
let value = hm.get("hello");

Insert

use std::collections::HashMap;

let hm = HashMap::new();

hm.insert("hello", "world");

Remove

use std::collections::HashMap;

let hm = HashMap::new();

hm.insert("hello", "world");
hm.remove("hello");

Hash Library

Cargo.toml

sha2 = "0.8.0"

Module

extern crate sha2;

Importing

use sha2::{Digest, Sha256}

Creatinga hasher

let hasher = Sha256::new();

Hasher Result

Returns the result of the hasher and resets it to be used again.

hasher.result_reset()

Iterators

Collect

Collects the results of an iterator to the type specified.

Enumerate

Returns the iterator object + the index.

Flat Map

An iterator that flattens a nested structure of iterators.

let words = ["my", "name", "is", "chris"];
// flat_map:
// Input: {Iterator} -> Logic: {Takes the iterator and flattens the whole collection} ->
// Output: {Items are flat}.
// In below example:
// {} = Iterator
// Words: { Characters: {} }
// Basically iterator in and interator.
let flattened_words: String = words.iter().flat_map(|s| s.chars()).collect();

> "mynameischris"

Filter

Filters a an iterator according to some logic, it returns an iterator.

let n_2 = vec![10, 12, 1235, 565, 347, 45, 6, 3, 25, 7, 8];
let n_2_iter = n_2.into_iter();

let n_3_iter = n_2_iter.filter(|x| x % 2 == 0);
n_3_iter.for_each(|x| println!("x: {}", x));

>
x: 10
x: 12
x: 6
x: 8

For Each

A basic iterator that acts as a loop over an iterable.

let _directories = read_dir(&path)
.unwrap()
.for_each(|x| println!("dir: {:?}", x));

>
dir: Ok(DirEntry("./Cargo.toml"))
dir: Ok(DirEntry("./target"))
dir: Ok(DirEntry("./Cargo.lock"))
dir: Ok(DirEntry("./world"))
dir: Ok(DirEntry("./.gitignore"))
dir: Ok(DirEntry("./hello"))
dir: Ok(DirEntry("./.git"))
dir: Ok(DirEntry("./3"))
dir: Ok(DirEntry("./src"))

Map

NOTE: map is a iterator implementation. It can Map Option -> Option but can also map Iter() -> Iter().

The below example is an iterator map that transforms an Option<T> -> Option<U>.

fn foo(x: Option<i32>) -> Option<i32> {
    x.map(|y| y + 2 as i32)
}

fn main() {
    let y = foo(Some(3));
    println!("y: {}", y.unwrap());
}

Pass in an option, the function takes it, performs something on the value and wraps it again in an option.

If we pass a None to the map iterator, it will just output a None.

fn foo(x: Option<i32>) -> Option<i32> {
    x.map(|y| y + 2 as i32)
}

fn main() {
    let y = foo(None);
    println!("y: {:?}", y);
}

Sum

Sums a given iterator.

Expands on the Map example by summing the iterator accessed in map which is an iterator of f64.

let portfolio: Vec<(String, f64)> = vec![("ASD", 100), ("WMA", 200)];
let total: f64 = portfolio.iter().map(|x| x.1).sum();

print("{}", total);
> 200

Itertools

An external crate the has added iterators.

Install via:

$ cargo add itertools

Import via:

use itertools::Itertools;

Unique

Ensures no duplicates in an iterator.

let row: Vec<String> = test_row_1
          .split(",")
          .map(|x| x.to_string())
          .filter(|x| x != "")
          .unique()
          .collect();

Lifetimes

Lifetimes are a way for the Rust compiler to know the lifetime of a borrowed object.

Borrowed objects should not:

  • Outlive its original

Lifetime variables

Because object is borrowed and we are passing into the Multiplier struct, we need to tell Rust that the lifetime of the borrow will live at least as long as Multiplier. Since we have given Multiplier the lifetime scope of <'a>.

struct Multiplier<'a> {
  object: &'a Object,
  mult: u32,
}

Logical Expressions

If let

A good way to match enums.

The below takes a an enum variables Command and matches it to Command::Set. If it matches we can use the enum field key.

if let Command::Set { key } = cmd {
  println!("{}", key);
}

Loops

Range Loops

for i in 0..10 {
  ...
}

While Loops

loop {...}

Loop with params

let i: usize = 0;

while i < 10 as usize {
    i += 1 as usize;
}

Logging

Enables basic logging in a project.

Importing:

[dependencies]
log = "0.4.6"
env_logger = "0.6.1"

Env Logger needs to be built first:

env_logger::builder().filter_level(LevelFilter::Info).init();

Macros

Macros is a form meta programming, allows us to write code that writes code.

Macros can be called from anywhere.

Captures

Different capture types for a macro.

* expr : expression
* ident : identifier
* block : a statement or block of code wrapped in {...}

Simple Macro

Simple macro that takes in a variable and prints a message Hey! <name>.

macro_rules! hey {
  ($x:expr) => {
    println!("Hey {}!", $x);
  }
}

Variadic Expression Macro

A macro that matches a variadiac expression and acts on each one.

  • , is the delimiter (the variadic argument is separated by a comma)
  • * repeats the pattern inside of $($x:expr)
  • $($x:expr),* repeats the expression for each item separated by a comma
  • $( println!("Hey {}!", $x); )* print each item repeatedly *
macro_rules! hey {
  ( $($x:expr),* ) => {
    $( println!("Hey {}!", $x); )*
  }
}

Creating Expressions with custom syntax

We can also use macros to create specific syntax that is not native rust style.

// Map! macro
macro_rules! map {
    // We create an expression that uses => to map k:v.
    // We could easily have used ",".
    ( $($key:expr => $value:expr),* ) => {{
        let mut hm = HashMap::new();
        $( hm.insert($key, $value); )*;
        hm
    }};
}

> let hash_map = map!(
        1 => "some val",
        2 => "other val"
    );

Match

match keyword can act as a switch statment, matching the result of an expression.

fn main() {
    let number = 13;
    // TODO ^ Try different values for `number`

    println!("Tell me about {}", number);
    match number {
        // Match a single value
        1 => println!("One!"),
        // Match several values
        2 | 3 | 5 | 7 | 11 => println!("This is a prime"),
        // Match an inclusive range
        13...19 => println!("A teen"),
        // Handle the rest of cases
        _ => println!("Ain't special"),
    }

    let boolean = true;
    // Match is an expression too
    let binary = match boolean {
        // The arms of a match must cover all the possible values
        false => 0,
        true => 1,
        // TODO ^ Try commenting out one of these arms
    };

    println!("{} -> {}", boolean, binary);
}

Listener

Memory Management

Tips on memory management.

Converting a reference to an owned variable

Usually structs will have a function to_<some-type>(). This will move the reference of that type to an owned variable.

path_buf.to_path_buf();
some_str.to_string();

Dereferencing

The following example is dereferencing an integer type. This is because in rust we cannot compare a &integer to an integer as in we cannot do &integer == integer.

We need to use the * to dereference as follows:

assert!(*get_some_int.get("integer_key").unwrap() == 100)

Option

Used to wrap some value or None (Null). It's used very frequently.

fn check_optional(optional: Option<Box<i32>>) {
    match optional {
        Some(ref p) => println!("has value {}", p),
        None => println!("has no value"),
    }
}

Option - Take()

Takes the value of Option, leaving None.

match self.tail.take() {
	Some(v) => ...
	None => ...
}

Option - ok_or()

Transforms a Option (Some, None) to Result (Ok, Err).

fn bar(input: Option<i32>) -> Result<i32, NegativeError> {
    foo(input).ok_or(NegativeError)

    // ORIGINAL
    // match foo(input) {
    //     Some(n) => Ok(n),
    //     None => Err(NegativeError)
    // }
}

Option - Filter

A method on an Option type.

Filter returns None if the value is None. It runs a predicate (function, logic). If true, returns the value wrapped in as Some(x). If it fails, returns None.

// Echos back the users i32 if it is not negative. Returns an error if its negative.
fn foo(input: Option<i32>) -> Option<i32> {
    input.filter(|x| *x >= 0)
    
    // match input {
    //     Some(x) => {
    //       if x < 0 {
    //           return None;
    //       }
    //       Some(x)
    //     },
    //     None => None
    // }
}

Serialization

The following explains serialization in Rust.

Serde

A crate that is used for serializing and deserializing Rust data structures.

Serde_json

A serde crate specific to serializing and deseriliazing Rust data structures to json.

Write a enum as json to a file.
File can be any struct that implements Writer.

let file = File::create("log.txt")?;
let remove_cmd = Command::Remove { key };

serde_json::to_writer(file, &remove_cmd)?;

Result

Result is an enum type part of the std module.

It is used when returning an expected value wrapped in an Ok(something) which is a success. Or an error, either thrown or explicit Err(e) => ....

Result:

enum Result<T, E> {
   Ok(T),
   Err(E),
}

Example:

pub fn get(&self, key: String) -> Result<Option<String>> {
    // Clone the value from the store.
    let value = self.store.get(&key).cloned();
    self.write_cmd(&Command::Get { key }, self.log_file_append_only()?)?;

    match value {
      Some(v) => Ok(Some(v)),
      None => Err(KvStoreError::KeyNotFoundError),
    }
}

Strings

Convert string to bytes

let msg = "hello, world!";
msg.as_bytes();

Append a character to string

let c = '\n';
let msg = String::from("hello, world!");
msg.push(c);

println!("{}", msg);
> "hello, world!\n"

Structs

Classes/structs in rust.

struct Something {
  member_variable: u32,
  another_variable: vector<u8>,
}

impl Something {
    // publicly facing
    pub fn new() {...}

    // struct method 'borrows' self, meaning it does not consume itself.
    pub fn foo(&self) {...}

    // private method.
    fn bar(&self) {...}
}

Constant pre-defined variables

constant predefined variables for rust structs need to be declared in the impl.

  impl HMAC {
+     /// Constants for the implementation of the HMAC.
+     const IPAD: [u8; 1] = [0x36];
+     const OPAD: [u8; 1] = [0x5c];

TCP Listener

Setup a TCP Listener. That allows connections via TCP.

This uses the TcpListener from std::net::TcpListener.

listerning.incoming() - returns an iterator over the connections received, also blocks.

let listener = TcpListener::bind("127.0.0.1:443").unwrap();

for stream in listener.incoming() {
    println!("Hello, World");
}

Writing to a buffer

  • use std::io::{Read, Write}

Imports the io package so that TcpStream can use Read and Write.

  • let mut buffer = [0u8] - Creates an array of bytes. Has 1 byte = 0.
  • stream.unwrap().read(&mut buffer) - Reads the received bytes from the stream to the buffer.
use std::io::{Read, Write};
use std::net::{TcpListener, TcpStream};

/// Server for the Key/Value store.
pub struct KvsServer {}

impl KvsServer {
    pub fn new() -> KvsServer {
        KvsServer {}
    }

    pub fn run(&self) {
        let listener = TcpListener::bind("127.0.0.1:443").unwrap();

        let mut buffer = [0u8];

        for stream in listener.incoming() {
            stream.unwrap().read(&mut buffer);
            println!("buffer: {:?}", String::from_utf8_lossy(&buffer));
        }
    }
}

TCP Streamer

Setups up a TCP Streamer. Allows a user to connect to TCP Server.

  • use std::io::prelude::*;

Allows TcpStream to use Read/Write.

  • stream.write(&[3]).unwrap()

Writes bytes to the stream.

use std::io::prelude::*;
use std::net::TcpStream;

pub struct KvsClient {}

impl KvsClient {
    pub fn new() -> KvsClient {
        KvsClient {}
    }

    pub fn connect(&self) {
        let mut stream = TcpStream::connect("127.0.0.1:443").unwrap();

        stream.write(&[3]).unwrap();
    }
}

Testing

The following explains how to run tests.

Testing Docs

Documents should and have example code that is testable.

To run document code tests:

$ cargo test --doc

Run individual Tests

$ cargo test "name of test"

Run a group of Tests

$ cargo test cli_
  • Will run all tests prefixed with cli_.

Enable print statements in tests

$ cargo run -- --nocapture

Fail/Panic a test

Sometimes we want to fail a test if it reaches a certain part of a program.

if <something>:
  panic!();

Test for a failure/panic

Tests that a function call WILL test for a panic/failure

#[test]
#[should_panic]
fn some_test() {...}

Traits

BorrowMut

https://doc.rust-lang.org/std/borrow/trait.BorrowMut.html

A really useful trait. It mutably borrows the owned value.

Some(old) => old.borrow_mut().next = Some(new.clone())

In the above example old is an Rc with a RefCell inside.

old.borrow_mut() returns the Refcell borrowed mutably.

Clone and Copy

Clone is trait that allows an object to be copied in memory.

The difference between Copy and Clone is that Clone allow sthe programmer to implement a complex Cone and not simply just copying bits to a new location.

For example if we were to make a Copy of a String (Using Clone) we would need to create a buffer in memory with the same size and capacity, and then copy the bits over into each index.

We can use derive if each field in a struct implements Clone

#[derive(Debug,Clone)]
struct Person {
    first_name: String,
    last_name: String,
}

Copy is a market trait, it has no methods to implement and simply just copies bits. Only possible with fields that implement Copy, we can't use a String field here.

#[derive(Debug,Clone,Copy)]
struct Point {
    x: f32,
    y: f32,
    z: f32
}

derive

A macro that allows trait implementation without the boiler plate implemenation. Almost all modules from the std library allows this and certain external libraries like serde.

Example:

Using dervie

#[derive(Debug)]
struct Pet {
    name: String,
}

If we didn't use derive, we would have to implement with boilerplate:

use std::fmt;

struct Pet {
    name: String,
}

impl fmt::Debug for Pet {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        match self {
            Pet { name } => {
                let mut debug_trait_builder = f.debug_struct("Pet");

                let _ = debug_trait_builder.field("name", name);

                debug_trait_builder.finish()
            }
        }
    }
}

dyn

Used to mark that a method is referencing a trait NOT a struct

// trait objects (new dyn syntax)
&Foo     => &dyn Foo
&mut Foo => &mut dyn Foo
Box<Foo> => Box<dyn Foo>

// structs (no change)
&Bar
&mut Bar
Box<Bar>

From

link

The From trait is used to convert one type to another type.

It's very useful for converting different error types to a custom error type. This is useful because all errors are one type but we still retain the error message of the preconversion type.

Example:

use std::fs;
use std::io;
use std::num;

enum CliError {
    IoError(io::Error),
    ParseError(num::ParseIntError),
}

impl From<io::Error> for CliError {
    fn from(error: io::Error) -> Self {
        CliError::IoError(error)
    }
}

impl From<num::ParseIntError> for CliError {
    fn from(error: num::ParseIntError) -> Self {
        CliError::ParseError(error)
    }
}

fn open_and_parse_file(file_name: &str) -> Result<i32, CliError> {
    let mut contents = fs::read_to_string(&file_name)?;
    let num: i32 = contents.trim().parse()?;
    Ok(num)
}

Send

A trait that allows a type to be transferred across threads.

pub trait KvsEngine: Clone + Send + 'static {}

Sized

Indicates a typed constant with a known size at compile time.

ToSocketAddrs

A trait that is from std::net::ToSocketAddrs.

It can convert a value into a Ipv4 or Ipv6 Address.

pub fn run<A: ToSocketAddrs>(&self, addr: A) -> Result<()> {
~         let listener = TcpListener::bind(addr)?;

Tuples

Access

Accessing a tuple:

let some_tuple: (String, u64) = ("Some_string".to_string(), 100);
println!("{}", some_tuple.1);

> 100

Declaration

Tuples can be declared by:

let some_tuple: (String, u64) = ("Some_string".to_string(), 100);

Writing Tests

Assertions

Simple assertion:

assert!(1 == 1)

Test Declaration

Tests should be declared as a module

#[cfg(test)]
mod initialization_tests {
    use super::*;

    #[test]
    fn allocate_products() {
    }
}

Test for thrown errors or panics

Vectors

Converting to an iterator

Although easily confused, vectors do not implement the iterator trait.

let v = vec![1, 2, 3];
let mut iter = v.into_iter();

Concatenate vectors

Method to concatenate vectors, especially useful in cryptography when we need to concatenate bytes.

let preimage: Vec<u8> = [
+             big_uint_to_str(outer_xor).as_bytes(),
+             hasher.result_reset().as_slice(),
+         ]
+         .concat();
```
@juanviamonte
Copy link

Thanks for this !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment