Skip to content

Instantly share code, notes, and snippets.

@rzezeski
rzezeski / sample-lines.sh
Created February 16, 2011 05:28
Pull a random sample of lines from the files in a directory (http://www.progski.net/blog/2011/random_lines_in_bash.html)
#! /bin/bash
#
# Pull a random sample of size 1000 from a directory of files and
# write to standard out.
#
# Usage:
# ./sample-lines /path/to/dir > mysample.log
if [ $# -lt 1 ]
then
@rzezeski
rzezeski / gist:1602953
Created January 12, 2012 20:38
check luwak file ancestor
{ok, C} = riak:local_client().
{ok, F} = luwak_file:get(C, <<"filename">>).
PrevRoot = hd(luwak_file:get_property(F, ancestors)).
V = riak_object:get_value(F).
V2 = lists:keyreplace(root, 1, V, {root, PrevRoot}).
TempF = riak_object:apply_updates(riak_object:update_value(F, V2)).
Bytes = luwak_io:get_range(C, TempF, 0, Length). % where Length is number of bytes you want to read
@rzezeski
rzezeski / gist:2287383
Created April 2, 2012 21:34
fprof PB search/mapred

Get conn

{ok, Pid} = riakc_pb_socket:start_link("127.0.0.1", 8081). 

Set variables

Bucket = <<"spam">>.
Query = <<"Scott">>.
Phases = [ {reduce,{modfun, riak_kv_mapreduce, reduce_identity},[{reduce_phase_only_1, true}],true} ].
@rzezeski
rzezeski / gist:2485252
Created April 25, 2012 01:28
New `riak-admin transfers` output
@rzezeski
rzezeski / gist:3250870
Created August 3, 2012 19:53
detect bad merge index files
#!/usr/bin/env escript
%% -*- erlang -*-
-include_lib("kernel/include/file.hrl").
-compile(export_all).
-define(LOG(S), io:format(S)).
-define(LOG(S,A), io:format(S,A)).
main(Dirs) ->
CodePath = case os:getenv("RIAK_LIB") of
@rzezeski
rzezeski / gist:3662010
Created September 7, 2012 00:45
yokozuna - use logical partitions
Cut disk usage by half and makes query easier to debug if necessary.
$ curl -vv 'http://localhost:8091/search/fruit?q=text:apple&wt=json' | jsonpp | tee result.out
{
"responseHeader": {
"status": 0,
"QTime": 126,
"params": {
"shards": "127.0.0.1:7981/solr/fruit",
@rzezeski
rzezeski / gist:3834987
Created October 4, 2012 17:05
yokozuna AAE

This is a demo of Active Anti-Entropy (AAE) in action on Yokozuna.

Three modifications were made to the app.config

  1. Set ring_creation_size to 8.

  2. Comment out the lager console logger (KV AAE is still spammy)

  3. Add {yokozuna, [{entropy_tick, 5000}]}

@rzezeski
rzezeski / gist:3914565
Created October 18, 2012 20:40
start of yokozuna tagging
curl -XPUT -H 'content-type: text/plain' -H 'x-riak-meta-yz-tags: X-Riak-Meta-Name_s' -H 'x-riak-meta-name_s: zezeski' 'http://localhost:8098/riak/people/ryan' -d "You gonna drink that?"
curl 'http://localhost:8098/search/people?q=X-Riak-Meta-Name_s:zezeski&wt=json' | jsonpp
{
"responseHeader": {
"status": 0,
"QTime": 22,
"params": {
"shards": "127.0.0.1:8093/solr/people",
@rzezeski
rzezeski / gist:3941804
Created October 23, 2012 21:42
lmao erlang:decode_packet
17:39:48 <{mochiweb_acceptor,init,3}> {erlang,decode_packet,
[httph,
<<"x-riak-meta-user_s: foo\r\nx-riak-meta-description_t: bar\r\nContent-Length: 153426\r\nExpect: 100-continue\r\n\r\n">>,
[]]}
17:39:48 <{mochiweb_acceptor,init,3}> {erlang,decode_packet,3} -> {ok,
{http_header,
0,
"X-Riak-Meta-User_s",
undefined,
@rzezeski
rzezeski / gist:3996286
Created November 1, 2012 20:30
merge index forced compaction
SVNodes = riak_core_vnode_manager:all_vnodes(riak_search_vnode).
GetMIPid = fun(VNodePid) -> element(3,element(4,element(4,element(2,hd(element(2,lists:nth(3,lists:nth(5,element(4,sys:get_status(VNodePid)))))))))) end.
[merge_index:compact(GetMIPid(VNodePid)) || {_,_,VNodePid} <- SVNodes].