Skip to content

Instantly share code, notes, and snippets.

@elliottcordo
Last active August 29, 2015 13:57
Show Gist options
  • Save elliottcordo/9534516 to your computer and use it in GitHub Desktop.
Save elliottcordo/9534516 to your computer and use it in GitHub Desktop.
output transform
--item sim
REGISTER '/home/hduser/libs/pig-redis.jar';
raw = LOAD '/user/movie_lens_rec_item_similarity'
USING PigStorage('\t') as (item1:chararray, item2:chararray, rating:chararray);
exp_tuple = FOREACH raw GENERATE item1, TOTUPLE(item2, rating);
STORE exp_tuple INTO 'dummy' USING com.hackdiary.pig.RedisStorer('zset','192.168.56.1');
--item base
REGISTER '/home/hduser/libs/pig-redis.jar' ;
raw = LOAD '/user/movie_lens_rec_item_based'
USING PigStorage('\t') as (user:chararray, results:chararray);
exp_1 = FOREACH raw
GENERATE user, FLATTEN(TOKENIZE(results,',')) as results;
exp_2 = FOREACH exp_1
GENERATE CONCAT('U-',user) as user,
STRSPLIT(REPLACE(REPLACE(results,'\\]',''),'\\[',''),':') as results;
STORE exp_2 INTO 'dummy' USING com.hackdiary.pig.RedisStorer('zset','192.168.56.1');
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment