Russell Brown russelldb

API

HTTP

Entry point for all object operations: https://github.com/basho/riak_kv/blob/1.4.2/src/riak_kv_wm_object.erl

delete_resource/2 takes RequestData(Request header, ex: vclock) and Context(Record containing: Bucket, Key, Client): https://github.com/basho/riak_kv/blob/1.4.2/src/riak_kv_wm_object.erl#L888

Put

API

Starting: https://github.com/basho/riak_kv/blob/1.4.2/src/riak_kv_wm_object.erl#L619
We create a new riak_object and populate the various fields with the headers, metadata supplied by the client.
Big suprise, we eventually call riak_client:put: https://github.com/basho/riak_kv/blob/1.4.2/src/riak_client.erl#L143
If/when the client returns any errors these are handled in handle_common_errors and it is nice to return human readable errors to client :)

Riak_client

Here's where I understand the state of the art to be:

In this INRIA tech report, Shapiro, Preguiça, Baquero and Zawirski (SPBZ) prove, amongst other things, that a sufficient condition for CRDTs to achieve eventual consistency on networks which may reorder and duplicate packets (which I'll call flaky networks, henceforth) is that
1. the underlying datatype forms a semilattice,
2. messages are full states,
3. incoming messages are combined with the node's current state using the least-upper-bound operation in the semilattice.
It's possible to relax condition 2 and still achieve eventual consistency over flaky networks by fragmenting the state into independent parts and transmitting updates to each part separately. For instance, in the G-Set CRDT (an add-only bitset) one can transmit only the index of the element to be added.
In [these slides from a talk at Dagstuhl](http://www.dagstuhl.de/mat/Files/13/13081/13081.BaqueroCarlos.Sl

riak-admin force-remove should not exist.

It's Friday evening, almost time to head out of the office for a nice long weekend. Your Riak cluster has been running along, everything fine. All of a sudden, the SSD in one of your Riak nodes decides to go up in a ball of flames. So you, being the good sysadmin that you are, immediately hop on the phone with your hardware vendor and order a new SSD. They tell you that you'll have it on Monday morning. Clearly you can't leave a broken node in your Riak environment, so you'll want to remove it from the cluster. You sit down at your terminal, hop on to a working Riak node and type

riak-admin force-remove [email protected]

NOOOOOOOOOOOOOOOOOOOOOOOOO!!!!

CvRDTs are (almost?) as general as they can be

What are you talking about, and why should I care?

Now that we live in the Big Data, Web 3.14159 era, lots of people want to build databases that are too big to fit on a single machine. But there's a problem in the form of the CAP theorem, which states that if your network ever partitions (a machine goes down, or part of the network loses its connection to the rest) then you can keep consistency (all machines return the same answer to

Step 1: From you logs I have decoded the bucket/key

binary_to_term(<<131,109,0,0,0,9,85,114,108,84,111,83,99,97,110>>).
<<"UrlToScan">>
binary_to_term(<<131,109,0,0,0,61,104,116,116,112,37,51,97,37,50,102,37,50,102,119,119,119,46,97,110,104,114,105,46,110,101,116,37,50,102,37,51,102,102,101,101,100,37,51,100,114,115,115,50,37,50,54,97,109,112,37,51,98,112,37,51,100,55,51,49,49,48>>).
<<"http%3a%2f%2fwww.anhri.net%2f%3ffeed%3drss2%26amp%3bp%3d73110">>

Step 2: Please run the below snippits from riak attach to identify the owning partitions of this bucket/key

on the guest SmartOS VM

pkg_add http://pkgsrc.joyent.com/sdc6/2012Q1/i386/All/smtools
# note this is going to shut down the VM
sm-prepare-image

	-module(riak_metrics).
	-compile(export_all).

	main([NodeName0, Cookie, Length, Command]) ->
	LocalName = '[email protected]',
	NodeName = list_to_atom(NodeName0),
	case net_kernel:start([LocalName]) of
	{ok, _} ->
	erlang:set_cookie(node(), list_to_atom(Cookie)),
	case net_kernel:hidden_connect_node(NodeName) of

	([email protected])1> {ok, C} = riak:local_client().
	{ok,{riak_client,'[email protected]',undefined}}
	([email protected])2> Object = riak_object:new(<<"testbucket">>, <<"testkey">>, <<"testvalue">>).
	{r_object,<<"testbucket">>,<<"testkey">>,
	[{r_content,{dict,0,16,16,8,80,48,
	{[],[],[],[],[],[],[],[],[],[],[],[],[],[],...},
	{{[],[],[],[],[],[],[],[],[],[],[],[],...}}},
	<<"testvalue">>}],
	[],
	{dict,1,16,16,8,80,48,