Skip to content

Instantly share code, notes, and snippets.

@rms1000watt
Created August 30, 2019 18:50
Show Gist options
  • Save rms1000watt/6ae6383d6dc4a6003ad2eb1fbc0eb3d5 to your computer and use it in GitHub Desktop.
Save rms1000watt/6ae6383d6dc4a6003ad2eb1fbc0eb3d5 to your computer and use it in GitHub Desktop.
Create Table and Partitions in Athena via AWS CLI bash
aws athena start-query-execution \
--query-string "$(cat create-table.sql)" \
--query-execution-context Database=someathenadatabase \
--result-configuration OutputLocation=s3://some-s3-bucket-for-athena-results/aws-cli
# Do this multiple times for each partition.. probably jinja2 script it
aws athena start-query-execution \
--query-string "ALTER TABLE table_1 ADD PARTITION (year='2019',month='4',day='1') location 's3://some-bucket-where-structured-data-is/path/2019/4/1';" \
--query-execution-context Database=someathenadatabase \
--result-configuration OutputLocation=s3://some-s3-bucket-for-athena-results/aws-cli
CREATE EXTERNAL TABLE IF NOT EXISTS table_1 (
key_1 string,
key_2 string,
key_3 double
)
PARTITIONED BY (year string, month string, day string)
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.RegexSerDe'
WITH SERDEPROPERTIES (
'serialization.format' = '1',
'input.regex' = '([^ ]*)\t([^ ]*)\t([-.0-9]*)')
LOCATION 's3://some-bucket-where-structured-data-is/path/';
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment