Skip to content

Instantly share code, notes, and snippets.

@MarkRoddy
Created August 2, 2013 02:31
Show Gist options
  • Save MarkRoddy/6137097 to your computer and use it in GitHub Desktop.
Save MarkRoddy/6137097 to your computer and use it in GitHub Desktop.
input_data = LOAD '/tmp/foobar' USING PigStorage(',') AS (some_id:int, some_val:chararray);
smaller_data = FILTER input_data BY (some_id > 5) OR (some_val == 'foobar') OR (some_val == 'bizbaz');
STORE smaller_data INTO '/tmp/smallerdata';
2013-08-01 22:22:37,977 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimi\
zer - MR plan size before optimization: 1
2013-08-01 22:22:37,991 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimi\
zer - MR plan size after optimization: 1
#--------------------------------------------------
# Map Reduce Plan
#--------------------------------------------------
MapReduce node 1-51
Map Plan
Store(fakefile:org.apache.pig.builtin.PigStorage) - 1-50
|
|---Filter[bag] - 1-38
| |
| Or[boolean] - 1-49
| |
| |---Or[boolean] - 1-45
| | |
| | |---Greater Than[boolean] - 1-41
| | | |
| | | |---Project[int][0] - 1-39
| | | |
| | | |---Constant(5) - 1-40
| | |
| | |---Equal To[boolean] - 1-44
| | |
LogicalExpressionOperator | | |---Project[chararray][1] - 1-42
| | |
| | |---Constant(foobar) - 1-43
| |
| |---Equal To[boolean] - 1-48
| |
| |---Project[chararray][1] - 1-46
| |
| |---Constant(bizbaz) - 1-47
|
|---New For Each(false,false)[bag] - 1-37
| |
| Cast[int] - 1-34
| |
| |---Project[bytearray][0] - 1-33
| |
| Cast[chararray] - 1-36
| |
| |---Project[bytearray][1] - 1-35
|
|---Load(file:///tmp/foobar:PigStorage(',')) - 1-32--------
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment