David Snyder davidsnyder

‘foo.pig.erb’:

fips = LOAD '<%= in_path %>' AS (fips_id:int,state_name:chararray);
DUMP fips;

Ruby irb interpreter:

require 'swineherd'
script = Swineherd::Script.new('foo.pig.erb',{"in_path" => "fips_to_state.tsv"}) 
script.run(:run_mode => :local,:combine_splits => false,:map_tasks => 10,:param => {:foo => "bar"}) 
--> ENV['PIG_OPTS'] = '-Dmapred.map.tasks=10 -Dpig.splitCombination=false'

Factual API Wrapper

The ruby-factual gem is clunky and out of date, so I tossed together a script to access the new v3 API.

The v3 Factual API encourages 2-legged OAuth for requests. Request a key here:

http://developer.factual.com/display/docs/Factual+Developer+APIs+Version+3

require 'factual'

OpenMenu API Wrapper

Check the documentation here http://openmenu.org/api/

require 'open_menu'

apikey = '1219-ab23-139b-de3-c23'
client = OpenMenu::Client.new(apikey)

menu_uuid = '12db-12b3-23bde-23bef'

	-- 1,2000,Dinosaur Planet,3.74
	-- 2,2000,Isle of Man TT 2004 Review,3.53
	-- 3,1990,Character,3.64
	-- 4,1990,Paula Abdul's Get Up & Dance,2.72
	movie_decades = LOAD '$DECADE_TITLES' USING PigStorage(',') AS (movie_id:chararray,decade:int,title:chararray,avg_rating:float);
	snipped_decades = FOREACH movie_decades GENERATE movie_id,decade;

	-- 9983,462930,3
	-- 9983,1149472,4
	-- 9983,238407,4

	=> #<Icss::RecordField:0x7f7189e34458 @type=#<Icss::PrimitiveType:0x7f7189fbca28 @name=:string, @mysql_name="VARCHAR", @ruby_klass=String, @pig_name="chararray">, @doc="Word type headword - see pp.4-5", @name="head_word", @is_reference=true>
	irb(main):007:0> f.length
	=> 3
	irb(main):008:0> f.class
	=> Icss::RecordField
	irb(main):009:0> f.keys
	=> [:name, :doc, :type]

	require 'yaml'
	require 'json'
	require 'icss'
	require 'icss/brevity'

	yaml_file = File.read("config.yaml")

	#load a YAML ICSS
	yaml_icss = Icss::Protocol.receive YAML.load(yaml_file)