yokozuna AAE

This is a demo of Active Anti-Entropy (AAE) in action on Yokozuna.

Three modifications were made to the app.config

Set ring_creation_size to 8.
Comment out the lager console logger (KV AAE is still spammy)
Add {yokozuna, [{entropy_tick, 5000}]}

Put the entropy manager in manual mode

([email protected])1> yz_entropy_mgr:manual_mode().
 ok

Create an index and set the hook

([email protected])2> yz_index:create("people").
ok
([email protected])3> yz_kv:install_hook(<<"people">>).
ok

Write an object

([email protected])4> {ok, C} = riak:local_client().
{ok,{riak_client,'[email protected]',undefined}}
([email protected])5> B = <<"people">>.
<<"people">>
([email protected])6> K = <<"ryan">>.
<<"ryan">>
([email protected])7> Obj = riak_object:new(B, K, <<"My name is Ryan Zezeski and I'm from bawlmore hon.">>, "text/plain").
{r_object,<<"people">>,<<"ryan">>,..}
([email protected])8> C:put(Obj).

Verify it is indexed

$ curl  'http://localhost:8091/search/people?q=text:bawlmore&wt=json' 2> /dev/null | jsonpp | grep 'numFound'
      "numFound": 1,

Discover preflist for object

([email protected])9> BKey = {B, K}.
{<<"people">>,<<"ryan">>}
([email protected])10> F = fun(BKey) ->
([email protected])10>       Ring = yz_misc:get_ring(transformed),
([email protected])10>       DocIdx = riak_core_util:chash_key(BKey),
([email protected])10>       riak_core_ring:preflist(DocIdx, 3, Ring)
([email protected])10>     end.
#Fun<erl_eval.6.82930912>
([email protected])11> Preflist = F(BKey).
[{182687704666362864775460604089535377456991567872,
  '[email protected]'},
 {365375409332725729550921208179070754913983135744,
  '[email protected]'},
 {548063113999088594326381812268606132370974703616,
  '[email protected]'}]

Delete all doc replicas from solr

The preflist first needs to be converted to logical format as that is what yokozuna uses.

([email protected])12> LPL = yz_misc:convert_preflist(Preflist,logical).
[{2,'[email protected]'},
 {3,'[email protected]'},
 {4,'[email protected]'}]
([email protected])13> Ps = [P || {P,_} <- LPL].
[2,3,4]
([email protected])14> [yz_solr:delete("people", "ryan_" ++ integer_to_list(P)) || P <- Ps].
[ok,ok,ok]

Verify docs are deleted

$ curl  'http://localhost:8091/search/people?q=text:bawlmore&wt=json' 2> /dev/null | jsonpp | grep 'numFound'
      "numFound": 0,

Request hashtree exchange for one of the partition/preflists combinations

yz_entropy_mgr:manual_exchange(182687704666362864775460604089535377456991567872, {182687704666362864775460604089535377456991567872,3}).

Below is the output in the log

2012-10-04 15:47:03.708 [info] <0.521.0>@yz_index_hashtree:handle_call:134 Updating tree for partition 182687704666362864775460604089535377456991567872 preflist {182687704666362864775460604089535377456991567872,3}
2012-10-04 15:47:03.708 [info] <0.356.0>@index_hashtree:handle_call:167 Updating tree: (vnode)=182687704666362864775460604089535377456991567872 (preflist)={182687704666362864775460604089535377456991567872,3}
2012-10-04 15:47:03.708 [info] <0.886.0>@yz_exchange_fsm:update_trees:117 Moving to key exchange
2012-10-04 15:47:03.708 [info] <0.886.0>@yz_exchange_fsm:key_exchange:127 Starting key exchange for partition 182687704666362864775460604089535377456991567872 preflist {182687704666362864775460604089535377456991567872,3}
2012-10-04 15:47:03.709 [info] <0.892.0>@yz_exchange_fsm:read_repair_keydiff:154 Anti-entropy forced read repair and re-index: <<"people">>/<<"ryan">>
2012-10-04 15:47:03.727 [info] <0.886.0>@yz_exchange_fsm:key_exchange:147 Finished key exchange for partition 182687704666362864775460604089535377456991567872 preflist {182687704666362864775460604089535377456991567872,3}
2012-10-04 15:47:03.728 [info] <0.478.0>@yz_entropy_mgr:maybe_clear_exchange:203 Untracking exchange: 182687704666362864775460604089535377456991567872

Verify data was re-indexed

$ curl  'http://localhost:8091/search/people?q=text:bawlmore&wt=json' 2> /dev/null | jsonpp | grep 'numFound'
      "numFound": 1,

rzezeski/gist:3834987