Skip to content

Instantly share code, notes, and snippets.

@FilBot3
Last active December 7, 2016 16:24
Show Gist options
  • Select an option

  • Save FilBot3/ebc8592263cc6afa88dc800d01c4040d to your computer and use it in GitHub Desktop.

Select an option

Save FilBot3/ebc8592263cc6afa88dc800d01c4040d to your computer and use it in GitHub Desktop.
ELK LogStash config for JDBC log gathering
input {
jdbc {
jdbc_driver_library => "/var/sqljdbc_4.1/enu/sqljdbc41.jar"
jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
jdbc_connection_string => "jdbc:sqlserver://DATABSESERVER;user=USER;password=PASSWORD;"
jdbc_user => "USER"
# parameters => { "DATE" => "2016-10-19" }
schedule => "* * * * *"
#clean_run => true
statement => "SELECT ID,Date,Level,Application,Logger,Message,MachineName,UserName,CallSite,Thread,Exception,Stacktrace,Client FROM [service_Cloud].[dbo].[service_Log] WHERE ID > :sql_last_value and (Level <> 'Debug' AND Level <> 'Info' OR Logger LIKE '%Metric%')"
# The line below is used for initial run, and then commented out.
# Kept for reference purposes
#statement => "SELECT ID,Date,Level,Application,Logger,Message,MachineName,UserName,CallSite,Thread,Exception,Stacktrace,Client FROM [service_Cloud].[dbo].[service_Log] WHERE ID > 29379421 AND ID > :sql_last_value and (Level <> 'Debug' AND Level <> 'Info' OR Logger LIKE '%Metric%')"
use_column_value => true
tracking_column => ID
last_run_metadata_path => "/opt/logstash/service_last_run_metadata_Log.txt"
}
}
output
{
#stdout { codec => rubydebug }
elasticsearch {
hosts => ["elkcrt.copmany.com:9200"]
index => "chd-%{+YYYY.MM.dd}"
#duplicate values wont be present on ES server . Same ID as in DB
#document_id => "%{ID}"
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment