Those are the redb benchmarks with different key value stores and the two different versions of LMDB living on two different branches: mdb.master and mdb.master3. The benchmarks were done on my iMac M3 (2023).
| redb | lmdb (mdb.master) | lmdb (mdb.master3) | rocksdb | sled | sanakirja | |
|---|---|---|---|---|---|---|
| bulk load | 2.97s | 982.90ms | 987.29ms | 5.40s | 4.49s | 865.30ms |
| individual writes | 443.95ms | 7.30ms | 7.37ms | 2.49ms | 403.99ms | 52.14ms |
| batch writes | 4.07s | 2.09s | 2.10s | 222.74ms | 1.38s | 3.63s |
| len() | 2.96µs | **125. |
| // This script can literally replace the whole changes I made on this Rust program to alterate | |
| // my dataset to be able to compute scores. You can find the function by follozing this link: | |
| // <https://github.com/Kerollmops/musicbrainz-artist-scoring/blob/901ea2d35707477880f328211c76945a15b2410b/src/main.rs> | |
| // The file extension is .rhai.rs just to be able to highlight the code. | |
| // You can run this Rhai script on the Rhai Playground: | |
| // <https://rhai.rs/playground/stable> | |
| // Here is the context and document Meilisearch provides. | |
| // It's just the setup part of the script |
| // Meilisearch is setting the document for you, this way. | |
| // Note we use the rhai.rs file extension to benefit from | |
| // the Rust Syntax Highlighting only, it's Rhai nothing more. | |
| let doc = #{ title: "star wars" }; | |
| print(doc); | |
| // -- script starts here -- |
| [ | |
| { | |
| "bindings": { | |
| "cmd-shift-[": "pane::ActivatePrevItem", | |
| "cmd-shift-]": "pane::ActivateNextItem", | |
| "ctrl-pagedown": "pane::ActivatePrevItem", | |
| "ctrl-pageup": "pane::ActivateNextItem", | |
| "ctrl-shift-tab": "pane::ActivateNextItem", | |
| "ctrl-tab": "pane::ActivatePrevItem", | |
| "cmd-+": "zed::IncreaseBufferFontSize" |
| # Create an API KEY there: https://pushover.net/apps/build | |
| # Find your user TOKEN here: https://pushover.net | |
| # | |
| # Note that the `title` field is optional and don't forget to put double quotes | |
| # around your argument when calling this function. | |
| # | |
| # Here is a small example of pipping logs through pushover: | |
| # | |
| # nohup tail -f nohup.out | while read l; do pushover "$l"; done & | |
| # |
| /* | |
| * par_build_tree.cpp | |
| * | |
| * Move this in the examples folder of Annoy and compile it like the other c++ example. | |
| * | |
| * Created on: Dec 6, 2023 | |
| * Author: Clément Renault | |
| */ |
| {} |
| //! A small program that computes the stats of an LMDB Meilisearch index. | |
| //! | |
| //! ```cargo | |
| //! [dependencies] | |
| //! anyhow = "1.0.71" | |
| //! clap = { version = "4.3.5", features = ["derive"] } | |
| //! heed = "0.20.0-alpha.1" | |
| //! ``` | |
| use std::path::PathBuf; |
We extract the movie titles along with the crew names and much more. You can also look at what BurntSushi have done for renaming series titles. The original IMDb datasets (TSVs) can be found on there official documentation.
We use the imdb-sqlite python script to help us do that. The command line generates an SQLite database in the imdb.db file.
pip install imdb-sqlite