Last active
June 21, 2019 16:18
-
-
Save itochu0523/b30652f9183ef637e99a to your computer and use it in GitHub Desktop.
セグメントデータを可視化してみる ref: https://qiita.com/itochu0523/items/3702fead73686b2aecc2
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
in: | |
type: postgresql | |
host: 00.00.00.00 | |
user: tank_user | |
password: "XXXXXXXXXXXXXXXX" | |
database: datatank | |
query: | | |
SELECT os,device,flag,count | |
FROM device_master | |
out: | |
type: elasticsearch_2x | |
nodes: | |
- {host: localhost, port: 9300} | |
index: device | |
index_type: test | |
bulk_actions: 200000 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
$ curl --create-dirs -o ~/bin/embulk -L http://dl.embulk.org/embulk-latest.jar | |
$ chmod +x ~/bin/embulk | |
$ cat << 'EOF' >> ~/.bash_profile | |
export PATH=$PATH:$HOME/bin | |
EOF | |
$ source ~/.bash_profile |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
$ embulk gem install embulk-input-postgresql | |
$ embulk gem install embulk-output-elasticsearch |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
https://www.elastic.co/downloads/elasticsearch | |
こちらのサイトより最新版をダウンロード。 | |
最終的には2.1.1。 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
https://www.elastic.co/downloads/kibana | |
上記のサイトで最新版をダウンロード | |
あとはtarを展開して適当に配置 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
https://github.com/muga/embulk-output-elasticsearch/tree/support-multiple-versions_3gems | |
branchを切り替えてビルドし、 | |
$ git co support-multiple-versions_3gems | |
$ ./gradlew gem | |
$ embulk gem install embulk-output-elasticsearch_2x-0.1.8.gem | |
$ embulk gem list |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
$ psql -h XX.XX.XX.XX -U tank_user datatank | |
> select * from device_master |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
$ embulk run config/config.yml |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment