Created
July 5, 2013 08:43
-
-
Save infectious/5933004 to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
## Question: I want to use this same ETL for a few different dimensions. How can I use a parameter (cols) to employ the 'columns' option in the load block (line 28/29 below)? | |
parameter :today do | |
8.day.ago.to_date | |
end | |
parameter :dimension do | |
"placement" | |
end | |
parameter :cols do | |
case dimension.downcase | |
when "ad" then :id, :name, :type, :width, :height | |
when "placement" then :id, :name, :start_date, :end_date, :budget | |
when "video" then [:id, :name, :duration] | |
else [:id, :name] | |
end | |
end | |
extract :ARC do | |
from "tubemogul/PlayTime_*_GEN-*_ST-#{today.ymd}_END-#{today.ymd}_DIM-#{filename}.tsv.gz" | |
separator "\t" | |
end | |
load :PDW do | |
into :"tubemogul__#{dimension.downcase}s" | |
insert_ignore | |
#columns cols # cannot get to work | |
#columns [:id, :name, :start_date, :end_date, :budget] #works | |
end |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment