Skip to content

Instantly share code, notes, and snippets.

@matthewbdaly
Last active May 19, 2023 16:31
Show Gist options
  • Save matthewbdaly/20743250ce3aa3ce4077783b4f4facd2 to your computer and use it in GitHub Desktop.
Save matthewbdaly/20743250ce3aa3ce4077783b4f4facd2 to your computer and use it in GitHub Desktop.
Ingest pipeline for Apache logs with GeoIP
{
"description": "Ingest pipeline created by text structure finder",
"processors": [
{
"grok": {
"field": "message",
"patterns": [
"\\[%{HTTPDATE:timestamp}\\] .*?%{GREEDYDATA:domain_name}:%{NUMBER:port} .*?%{IP:ipaddress}.*? .*? .*? .*?%{QUOTEDSTRING:request} %{INT:status_code} %{INT:size} %{QUOTEDSTRING:referrer} %{QUOTEDSTRING:user_agent}"
],
"ecs_compatibility": "v1"
}
},
{
"date": {
"field": "timestamp",
"formats": [
"dd/MMM/yyyy:HH:mm:ss XX"
]
}
},
{
"convert": {
"field": "port",
"type": "long",
"ignore_missing": true
}
},
{
"convert": {
"field": "size",
"type": "long",
"ignore_missing": true
}
},
{
"convert": {
"field": "status_code",
"type": "long",
"ignore_missing": true
}
},
{
"remove": {
"field": "timestamp"
}
},
{
"geoip": {
"field": "ipaddress",
"target_field": "geolocation"
}
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment