Skip to content

Instantly share code, notes, and snippets.

@itochu0523
Last active January 7, 2016 06:31
Show Gist options
  • Save itochu0523/a5fff0b5c3b03055ade8 to your computer and use it in GitHub Desktop.
Save itochu0523/a5fff0b5c3b03055ade8 to your computer and use it in GitHub Desktop.
$ td connector:guess Sample.yml -o Sample_load.yml
$ td connector:issue load.yml --database td_sample_db --table td_sample_table \
--time-column created_at
$ td connector:preview Sample_load.yml
+-------+---------+----------+---------------------+
| id | company | customer | created_at |
+-------+---------+----------+---------------------+
| 11200 | AA Inc. | David | 2015-03-31 06:12:37 |
| 20313 | BB Imc. | Tom | 2015-04-01 01:00:07 |
| 32132 | CC Inc. | Fernando | 2015-04-01 10:33:41 |
| 40133 | DD Inc. | Cesar | 2015-04-02 05:12:32 |
| 93133 | EE Inc. | Jake | 2015-04-02 14:11:13 |
+-------+---------+----------+---------------------+
$ td connector:preview Sample_load.yml
in:
type: s3
access_key_id: XXXXXXXXXX
secret_access_key: YYYYYYYYYY
bucket: sample_bucket
# path the the *.csv or *.tsv file on your s3 bucket
path_prefix: path/to/sample_file
out:
mode: append
in:
type: s3
access_key_id: XXXXXXXXXX
secret_access_key: YYYYYYYYYY
bucket: sample_bucket
path_prefix: path/to/sample_file
parser:
charset: UTF-8
newline: CRLF
type: csv
delimiter: ','
quote: '"'
escape: ''
skip_header_lines: 1
columns:
- name: id
type: long
- name: company
type: string
- name: customer
type: string
- name: created_at
type: timestamp
format: '%Y-%m-%d %H:%M:%S'
out:
mode: append
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment