We've built an example (simple) rocksdb server available at: http://authbox-cdn.s3.amazonaws.com/mirror/sada-sliding
The server should run inside an ubuntu docker container with flags:
./sada-sliding --port 9015 --rocksdb_db_path /data \
--rocksdb_create_if_missing_one_off --version_timestamp_ms 9999999999999
This will start it up, and create the database on the first run. What we want is a python wrapper script to backup and restore the database on start if it isn't already present (the /data/CURRENT file will exist for any active database.)
The server speaks redis protocol, so redis-cli will aid in testing/debugging.
To estimate the number of seconds the server has been running in the last hour: SHCARD time 3600 That should also help with testing backup/restore is working as expected.
Each backup is made up of a number of immutable SST files, as well as a number of append-only LOG files, an append-only MANIFEST and a CURRENT file (which is simply a file pointer to the manifest)
== Backup script ==
To start a backup send the FREEZE command; which will have output as follows:
127.0.0.1:9907> FREEZE
1) "/000056.sst"
2) "/000053.sst"
3) "/000009.sst"
4) "/CURRENT"
5) "/MANIFEST-000046:514"
6) "/000051.log:63805250"
7) "/000054.log:64418528"
8) "/000055.log:29768989"
Each of the sst files should be uploaded (in parallel) to google cloud store if they are not already present. The log files contain a size... any files in gcloud should be appended with data until they reach the specified size.
The CURRENT/MANIFEST files, as well as the file listing (to aid in recovering a backup) should be uploaded seperately into a per-backup directory.
== Recover script ==
The recovery script should essentially do the inverse of the above. Grab the file listing of the most recent backup made and then download all the neccessary files to the /data directory.