Skip to content

Instantly share code, notes, and snippets.

@prasanthj
Created June 28, 2017 08:52
Show Gist options
  • Save prasanthj/7ecbfadaf61bc88046b62f869d7d3dc5 to your computer and use it in GitHub Desktop.
Save prasanthj/7ecbfadaf61bc88046b62f869d7d3dc5 to your computer and use it in GitHub Desktop.
configs to tune leveldb
These are tunings from https://github.com/basho/leveldb
Video: https://www.youtube.com/watch?v=vo88IdglU_8
https://github.com/google/leveldb/blob/master/include/leveldb/options.h#L83
- write buffer size - default 4MB (increase this for higher performance) (try 10x or 100x will get 72% performance)
https://github.com/google/leveldb/blob/master/db/skiplist.h#L98
- if increasing write buffer size, also increase skip list max height. Since skip list is probabilistic,
expected number of keys in write buffer has to specified ahead. (change it from 2^12 to 2^17, affects read and write performance)
- enable bloom filter (stored at top of sstables)
- look for write amplifications (same keys gets written to multiple levels during compaction to maintain sortedness at each level)
Bigger files will reduce write amplification? (google write amplication level 11, basho level 5.7 and new release level 3?)
- only one compaction thread and only one queue where compaction works are added.
This can lead to stalls caused by compactions on other DBs
- Thread Group of size 5 for optimal compaction performance. RW tiered locks across following (IMM - Immutable Memory Buffer or old write buffer)
IMM to Level 0 highest priority
Level 0 to Level 1 next highest priority
Level 1+ after that
- Write throttle (backpressure) instead of default write blocking
- check file handles config
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment