Skip to content

Instantly share code, notes, and snippets.

@stereobooster
Last active July 18, 2017 22:09
Show Gist options
  • Save stereobooster/64c7207c89c9525357e05d9886dfb428 to your computer and use it in GitHub Desktop.
Save stereobooster/64c7207c89c9525357e05d9886dfb428 to your computer and use it in GitHub Desktop.
https://stereobooster.github.io/reference-data-as-dependency
http://lampwww.epfl.ch/papers/idealhashtrees.pdf
Task
1. compact file format for faster read from disk
magpack? protobuf? pbf
2. traverse all file, leave only popular bits. Cache LRU
mini mime
3. be able to unload db from memory, if not required
4. use effiecient in memory representation instead of objects
zero copy
http://bsonspec.org/spec.html DateTime
6. prebuild index
pasword db
7. Read only, so no locks or transactions
+-------+
| file | - effective format representation. Format with optional schema
+-------+
|
+-------+
| index | - optional, depends on data type. For example for LIKE pattern% binary tree will do
+-------+
|
+-------+
| cache | - optional, cap number of simultenious objects
+-------+
|
+--------------+
| access layer | - effective in memory representation
+--------------+
## gems
- https://github.com/discourse/mini_mime
- https://github.com/stereobooster/micro_mime
- https://github.com/bdmac/strong_password/pull/14
- https://github.com/SixArm/sixarm_ruby_unaccent/issues/1
- https://github.com/nicotaing/yaml_record
- https://github.com/byroot/frozen_record
- https://github.com/zilkey/active_hash
- https://github.com/hexorx/countries
## npm
- https://github.com/zeke/all-the-cities/pull/3
## lisp
- https://picolisp.com/wiki/?home
- http://www.cliki.net/CommonLispPrevalence
## natural lang js
https://github.com/nhunzaker/speakeasy
https://github.com/NaturalNode/natural
https://github.com/nlp-compromise/compromise
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment