de-sh

June 7th - 13th: As had been discussed in last week's conversations with my mentor @little-wallace, it was noted that I had to focus not on writing an interface with EBS but rather find out locations in the code of components/raftstore and figure out ways of removing redundant writes made to both KVDB and RaftDB, this was something I had not been clear upon in the first week, but I have started work on reading the details of raftstore::store, starting with the FSM module, where there are multiple details that are a bit unclear for me, this is especially due to the fact that I am extremely new to the source-code. I intended to figure out the exact mechanism of write to DB, by further researching through the source code.

de-sh

June 14th - 20th: I have been continuing to work on raftstore as guided by @little-wallace but he has been unable to work with me due to a hectic schedule that he is currently facing at his full time position, thus my GSoC designated mentor @yiwu-arbug has decided to take back charge of my project and will be guiding me in the venture from here on.

de-sh

The syntax of rust is readability driven and there is immense attention to detail given on simple interfaces to achieve functional programming approach of computation with map, filter and fold allowing for an iter item to be operated on, which sometimes increases the readability of the program in question.

Introducing Closures

A closure is the concept of isolating the values that a function is operating on, it's 'environment', popularized in functional programming. We are using an anonymous function in our program, which is passed literally as a value instead of being a call. To define an anonymous function, closure in rust we are going to use the | operator and place the arguments between it, before defining the function's behaviour right after.

let inc = |i| i+1;
println!("{}", inc(1));

Run in Playground

de-sh

June 21-27: Conversed with Yi Wu by video call, I have internal exams going on, but I have started work on tasks as pointed out by him on rust-rocksdb#514.

de-sh

June 28 - July 4: I am unable to keep a note of the activities as the weeks are progressing due to a hectic load from the experimentative nature of work alotted to me, but it seems like I am not doing bad and have been going in the correct direction. I have no experience of working with CPP code compilation at the scale of the rocksdb-cloud project, but it seems like the CMakeLists.txt I have written is somewhat working from the edits that @yiwu-arbug has committed to the repo. I will take notes and document the same here when the entire process is done with.

de-sh

In the cloud native world, databases are more than just stores of data, they act as data exchanges, transporters and in some form as data processors. In this scheme of things, a database is in effect composed of micro-services to make it truly cloud native. Log Structured Merge Trees are a datastructure standard that fits well for use in this realm of high volume, high velocity 'data-engines' as I like to call them.

One of the databases that many people refer to in this field is Google's BigTable, infamous in the space for having started a conversation. But we are not going to be talking about that and infact refer to a product born out of Facebook, rocksdb. MyRocks is a database that works at scale and is distributed, building on top of the standards set by MySQL, with a core written in CPP known as rocksdb. Rocksdb-Cloud is a set of tools that were added on top of this engine that exponentially increases the ability of this database as well as add the ability for it to work on a public cloud setting, makin

de-sh

July 5-11: Work on rust-rocksdb-cloud bindings progressed with experimentation in writing an interface ccloud wrapper that is inspired from crocksdb C-ABI of the [rust-rocskdb]. It doesn't seem to be a working option and I am not entirely sure if this is the way to go forward, I have requested Yi to reconsider the creation of an entirely separate interface that is not contained within the same crate as the rocksdb interface. Some trial interfaces that I have created includes a layer of code that effectively invokes CloudEnv::NewAwsEnv() but there are issues in compilation.

de-sh

July 12-18: Work on rust-rocksdb bindings for rocksdb-cloud progressed with addition of some more code changes for compilation to be successful, apparently the code wasn't compiling due to certain changes in rocksdb that weren't carried out equally among the forks tikv/rocksdb~6.4.tikv and rockset/rocksdb-cloud~v6.7.3, including the inclusion of remote compaction that we have planned to include in the future and aren't pursuing at the get go, these are PluggableCompactionParam and PluggableCompactionResult defined in pluggable_compaction.h, so we have decided to use rockset/rocksdb-cloud~v5.18.3 instead.

As the work continued, I was able to make the changes necessary for this, have included them in local an

de-sh

July 19-25: As Yi had decided to get me invested in re-writing the rocksdb-cloud repo as a plugin to tikv/rocksdb, there were some editions made to the codebase modernizing it from tag v5.18.3 to work seamlessly and compile for tikv/rocksdb at v6.4.tikv. The re-structured code removing all except the cloud components in tikv/rocksdb#182 required some reworking of tikv/rocksdb, by publicising the LogReporter struct, is being achieved in #181.

With respect to addition of an interface to these cloud features through the rust-rocksdb ffi code, I was able to restructure it in such a way that now we can create a AWS S3 based instance of rocksdb env, a [rudimentry test](https://github.com/tikv/rust-rocksdb/blob/45b28cd4c99b7d37380df441

de-sh

#[derive(Clone, Serialize, Deserialize, PartialEq, Debug, Configuration)]
#[serde(default)]
#[serde(rename_all = "kebab-case")]
pub struct DbConfig {
    #[config(skip)]
    pub info_log_level: LogLevel,
    #[serde(with = "rocks_config::recovery_mode_serde")]
    #[config(skip)]
    pub wal_recovery_mode: DBRecoveryMode,
    #[config(skip)]