Created
June 28, 2017 08:52
-
-
Save prasanthj/7ecbfadaf61bc88046b62f869d7d3dc5 to your computer and use it in GitHub Desktop.
configs to tune leveldb
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
These are tunings from https://github.com/basho/leveldb | |
Video: https://www.youtube.com/watch?v=vo88IdglU_8 | |
https://github.com/google/leveldb/blob/master/include/leveldb/options.h#L83 | |
- write buffer size - default 4MB (increase this for higher performance) (try 10x or 100x will get 72% performance) | |
https://github.com/google/leveldb/blob/master/db/skiplist.h#L98 | |
- if increasing write buffer size, also increase skip list max height. Since skip list is probabilistic, | |
expected number of keys in write buffer has to specified ahead. (change it from 2^12 to 2^17, affects read and write performance) | |
- enable bloom filter (stored at top of sstables) | |
- look for write amplifications (same keys gets written to multiple levels during compaction to maintain sortedness at each level) | |
Bigger files will reduce write amplification? (google write amplication level 11, basho level 5.7 and new release level 3?) | |
- only one compaction thread and only one queue where compaction works are added. | |
This can lead to stalls caused by compactions on other DBs | |
- Thread Group of size 5 for optimal compaction performance. RW tiered locks across following (IMM - Immutable Memory Buffer or old write buffer) | |
IMM to Level 0 highest priority | |
Level 0 to Level 1 next highest priority | |
Level 1+ after that | |
- Write throttle (backpressure) instead of default write blocking | |
- check file handles config |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment