Odoo Logstash Filter

filter

filter {
  if [type] == "odoo" {
    grok {
      match => { "message" => "%{ODOOLOG}" }
    }
    date {
      match => [ "timestamp", "yyyy-MM-dd HH:mm:ss,SSS" ]
    }
  }
}

input

input {
  file {
    type => "odoo"
    path => "/path/to/server.log"
    codec => multiline {
      pattern => "^%{TIMESTAMP_ISO8601} "
      negate => true
      what => previous
    }
  }
}

pattern

WERKZEUGREQUESTTIMESTAMP %{MONTHDAY}/%{MONTH}/%{YEAR} %{TIME}
WERKZEUGREQUEST %{IPORHOST:client} %{USER:ident} %{USER:auth} \[%{WERKZEUGREQUESTTIMESTAMP:request_timestamp}\] "%{WORD:request_type} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response} -
ODOOMISC %{GREEDYDATA}
ODOOLOG %{TIMESTAMP_ISO8601:timestamp} %{POSINT:pid} %{LOGLEVEL:level} (?:%{USERNAME:user}|\?) %{PROG:module}: (?:%{WERKZEUGREQUEST}|%{ODOOMISC:misc})

It could be interesting to consider the case of http error response code with stack trace separately. This append with only a specific module name : openerp.addons.website.models.ir_http followed by answer code and code name, and then stack trace.

HTTPCODE [1-5][0-9][0-9]
HTTPDESCRIPTION [a-zA-Z0-9\s_-]*
HTTPMODULE openerp.addons.website.models.ir_http: %{HTTPCODE:http_code} %{HTTPDESCRIPTION:http_description}:%{GREEDYDATA:stacktrace}
WERKZEUGREQUESTTIMESTAMP %{MONTHDAY}/%{MONTH}/%{YEAR} %{TIME}
WERKZEUGREQUEST %{IPORHOST:client} %{USER:ident} %{USER:auth} [%{WERKZEUGREQUESTTIMESTAMP:request_timestamp}] "%{WORD:request_type} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response} -
ODOOMISC %{GREEDYDATA}
OTHERMODULE %{PROG:module}: (?:%{WERKZEUGREQUEST}|%{ODOOMISC:misc})
ODOOLOG %{TIMESTAMP_ISO8601:timestamp} %{POSINT:pid} %{LOGLEVEL:level} (?:%{USERNAME:user}|?) (?:%{HTTPMODULE}|%{OTHERMODULE})

Not yet tested, just beginning to look after odoo log analytics

In combination with Filebeat :
filebeat.yml : (using regex cause filebeat doesn't know TIMESTAMP_ISO8601 grok pattern)

  - {
    document_type: odoo,
    paths: ['{{ odoo_config_logfile }}'],
    multiline: {
      pattern: "^(?>\\d\\d){1,2}-(?:0?[1-9]|1[0-2])-(?:(?:0[1-9])|(?:[12][0-9])|(?:3[01])|[1-9])[T ](?:2[0123]|[01]?[0-9]):?(?:[0-5][0-9])(?::?(?:(?:[0-5][0-9]|60)(?:[:.,][0-9]+)?))?(?:Z|[+-](?:2[0123]|[01]?[0-9])(?::?(?:[0-5][0-9])))?",
      negate: true,
      match: after
      }
    }

Logstash filter :

filter {
  if [type] == "odoo" {
    date {
      match => [ "timestamp", "yyyy-MM-dd HH:mm:ss,SSS" ]
    }
    grok {
      match => { "message" => "%{ODOOLOG}" }
    }

  }
}