Skip to content

Instantly share code, notes, and snippets.

@esemeniuc
Last active June 23, 2024 22:57
Show Gist options
  • Save esemeniuc/ad889c617cc4297043fc344136be563f to your computer and use it in GitHub Desktop.
Save esemeniuc/ad889c617cc4297043fc344136be563f to your computer and use it in GitHub Desktop.
Rust zstd-rs compression with dictionary
// dict
let mut dict_file =
std::fs::File::open("dictionaryFile").unwrap();
let mut dict = Vec::new();
dict_file.read_to_end(&mut dict).unwrap();
// compress
let mut enc_buf = [0; 1_000_000];
let encoder_dict = zstd::dict::EncoderDictionary::copy(&dict, 0);
let mut compressor =
zstd::bulk::Compressor::with_prepared_dictionary(&encoder_dict).unwrap();
// decompress
let mut dec_buf = [0; 1_000_000];
let decoder_dict = zstd::dict::DecoderDictionary::copy(&dict);
let mut decompressor =
zstd::bulk::Decompressor::with_prepared_dictionary(&decoder_dict).unwrap();
for _i in 0..100 {
let enc_bytes_written = compressor
.compress_to_buffer(&input_bytes, &mut enc_buf)
.unwrap();
let dec_bytes_written = decompressor
.decompress_to_buffer(&enc_buf[..enc_bytes_written], &mut dec_buf)
.unwrap();
println!("enc: {enc_bytes_written}, dec: {dec_bytes_written}");
assert_eq!(&dec_buf[..dec_bytes_written], input_bytes.as_slice());
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment