logstash-grok-pattern-vpc-flow-log

vpc-flowlog

# VPC Flow Log fields
# version account-id interface-id srcaddr dstaddr srcport dstport protocol packets bytes start end action log-status
# http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/flow-logs.html#flow-log-records
VPCFLOWLOG %{NUMBER:version:int} %{NUMBER:account-id} %{NOTSPACE:interface-id} %{NOTSPACE:srcaddr} %{NOTSPACE:dstaddr} %{NOTSPACE:srcport:int} %{NOTSPACE:dstport:int} %{NOTSPACE:protocol:int} %{NOTSPACE:packets:int} %{NOTSPACE:bytes:int} %{NUMBER:start:int} %{NUMBER:end:int} %{NOTSPACE:action} %{NOTSPACE:log-status}

Thank you!

Gr8 job m8

for the latest vpc flow logs format, i am using below pattern

%{NUMBER:version:int} %{NOTSPACE:vpc-id} %{NOTSPACE:region} %{NOTSPACE:subnet-id} %{NOTSPACE:instance-id} %{NOTSPACE:interface-id} %{NUMBER:account-id} %{NOTSPACE:type} %{NOTSPACE:srcaddr} %{NOTSPACE:dstaddr} %{NOTSPACE:srcport:int} %{NOTSPACE:dstport:int} %{NOTSPACE:pkt-srcaddr} %{NOTSPACE:pkt-dstaddr} %{NOTSPACE:protocol:int} %{NOTSPACE:bytes:int} %{NOTSPACE:packets:int} %{NUMBER:start:int} %{NUMBER:end:int} %{NOTSPACE:action} %{NOTSPACE:tcp-flags} %{NOTSPACE:log-status}

an update to this for version 5:

%{NUMBER:version:int} %{NUMBER:account-id} %{NOTSPACE:interface-id} %{NOTSPACE:srcaddr} %{NOTSPACE:dstaddr} %{NOTSPACE:srcport:int} %{NOTSPACE:dstport:int} %{NOTSPACE:protocol:int} %{NOTSPACE:packets:int} %{NOTSPACE:bytes:int} %{NUMBER:start:int} %{NUMBER:end:int} %{NOTSPACE:action} %{NOTSPACE:log-status} %{NOTSPACE:vpc-id} %{NOTSPACE:subnet-id} %{NOTSPACE:instance-id} %{NOTSPACE:tcp-flags} %{NOTSPACE:type} %{NOTSPACE:pkt-srcaddr} %{NOTSPACE:pkt-dstaddr} %{NOTSPACE:region} %{NOTSPACE:az-id} %{NOTSPACE:sublocation-type} %{NOTSPACE:sublocation-id} %{NOTSPACE:flow-direction} %{GREEDYDATA:other}