elasticsearch の触ってみたメモ

01elasticsearch-memo.md

index一覧取得

$ curl -XGET http://localhost:9200/_aliases

見づらいので、見やすくするには「jq」コマンド利用する

$ curl -XGET http://tag100:9200/_aliases | jq .
{
  "syslog.secure.rdb103-2013.11.18": {
    "aliases": {}
  },
  "syslog.cron.rdb102-2013.11.20": {
    "aliases": {}
  },
  "syslog.maillog.rdb102-2013.11.20": {
    "aliases": {}
  },
  ...snip...
  
}

delete-index.md

Indexの削除

$ curl -XDELETE http://localhost:9200/httpd.access_log-2013.11.20
{"ok":true,"acknowledged":true}

get-mapping.md

カラム構造(mapping)取得

$ curl -XGET http://localhost:9200/httpd.access_log-2013.11.20/_mapping

いつもの通り、見辛いのでjq 使って見やすくする

$ curl -XGET http://localhost:9200/httpd.access_log-2013.11.20/_mapping | jq .
{
  "httpd.access_log-2013.11.20": {
    "httpd": {
      "properties": {
        "vhost": {
          "type": "string"
        },
        "user": {
          "type": "string"
        },
        "uri": {
          "fields": {
            "uri_not_analyzed": {
              "include_in_all": false,
              "index_options": "docs",
              "omit_norms": true,
              "index": "not_analyzed",
              "type": "string"
            },
            "uri": {
              "type": "string"
            }
          },
          "type": "multi_field"
        },
        "timestamp": {
          "type": "string"
        },
        "ident": {
          "type": "string"
        },
        "host": {
          "type": "string"
        },
        "forwardedhost": {
          "type": "string"
        },
        "forwardedfor": {
          "type": "string"
        },
        "cookie": {
          "type": "string"
        },
        "agent": {
          "fields": {
            "agent_not_analyzed": {
              "include_in_all": false,
              "index_options": "docs",
              "omit_norms": true,
              "index": "not_analyzed",
              "type": "string"
            },
            "agent": {
              "type": "string"
            }
          },
          "type": "multi_field"
        },
        "_key": {
          "type": "string"
        },
        "@timestamp": {
          "format": "dateOptionalTime",
          "type": "date"
        },
        "method": {
          "type": "string"
        },
        "protocol": {
          "type": "string"
        },
        "query": {
          "type": "string"
        },
        "referer": {
          "fields": {
            "referer_not_analyzed": {
              "include_in_all": false,
              "index_options": "docs",
              "omit_norms": true,
              "index": "not_analyzed",
              "type": "string"
            },
            "referer": {
              "type": "string"
            }
          },
          "type": "multi_field"
        },
        "req": {
          "fields": {
            "req_not_analyzed": {
              "include_in_all": false,
              "index_options": "docs",
              "omit_norms": true,
              "index": "not_analyzed",
              "type": "string"
            },
            "req": {
              "type": "string"
            }
          },
          "type": "multi_field"
        },
        "response_time": {
          "type": "string"
        },
        "size": {
          "type": "string"
        },
        "status": {
          "type": "string"
        }
      }
    }
  }
}

mapping-template.md

カラム構造(mapping)設定

参考サイト

Kibana + ElasticSearch + fluentd でDBスロークエリログなどを集計し表示したい http://blog.tkmr.org/post/kibana-elasticsearch-fluentd-db

fluentd + Elasticsearch + kibana + siege でお手軽に web サイトのレスポンスタイムを可視化する試み http://inokara.hateblo.jp/entry/2013/11/20/072844

curl -X PUT http://localhost:9200/_template/slow_log_template -d '
{
  "template" : "mysql.slow_query*",
  "order" : 1,
  "mappings" : {
    "mysql-slow_query": {
      "properties": {
        "host": {
          "type" : "multi_field",
          "fields" : {
            "host" : {"type" : "string", "index" : "analyzed"},
            "host_not_analyzed" : {"type" : "string", "index" : "not_analyzed"}
          }
        },
        "sql": {
          "type" : "multi_field",
          "fields" : {
            "sql" : {"type" : "string", "index" : "analyzed"},
            "sql_not_analyzed" : {"type" : "string", "index" : "not_analyzed"}
          }
        }
      }
    }
  }
}
'