Fortune database Logstash Demonstration

001-logstash-input.conf

input {
  file {
    path => "/home/vinh/development/projects/geek-fortune-cookies/databases/programming-wisdom"
    sincedb_path => "/dev/null"
    start_position => "beginning"
    codec => multiline {
      pattern => "^\%"
      negate => true
      what => "previous"
      charset => "UTF-8"
    }
  }
}

002-logstash-filter.conf

filter {
  mutate {
    gsub => [
      "message", "\%", "",
      "message", "\n", ""
    ]
  }
  grok {
    match => { "message" => "%{GREEDYDATA:quote}\s+ --\s%{GREEDYDATA:person}" }
    remove_field => [ "message" ]
  }
  mutate {
    gsub => [
      "quote", "\s+", " "
    ]
  }
}

003-logstash-output.conf

output {
  stdout { codec => "rubydebug" }
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "quotes"
    document_type => "doc"
    template => "/home/vinh/demos/logstash-config/fortune-cookies/quotes_template.json"
    template_name => "quotes"
  }
}

index-template.json

{
  "index_patterns": [
    "quotes"
  ],
  "order": 1,
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 1
  },
  "mappings": {
    "doc": {
      "dynamic": "false",
      "properties": {
        "quote": {
          "type": "text"
        },
        "person": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 64
            }
          }
        }
      }
    }
  }
}

programming-wisdom

%
We build our computers the way we build our cities:
   over time, without a plan, on top of ruins.
    -- Ellen Ullman
%
A programmer was arrested for writing unreadable code.
    He refused to comment.
    -- Twitter, "Mr. Drinks On Me"
%
"Good programming is good writing."
    -- John Shore
%
"The most important single aspect of software development
    is to be clear about what you are trying to build."
    -- Bjarne Stroustrup
%
"A program is like a poem: you cannot write a poem without writing it."
    -- E.W Dijkstra
%
"The best error message is the one that never shows up."
    -- Thomas Fuchs
%
“Every programmer is an author.”
    -- Sercan Leylek
%
"An evolving system increases its complexity unless work is done to reduce it."
    -- Meir Lehman
%
"Theory is when you know something, but it doesn’t work.
 Practice is when something works, but you don’t know why.
 Programmers combine theory and practice: Nothing works and they don’t know why."
    -- Unknown
%
"One of the best programming skills you can have is knowing when to walk away for awhile."
    -- Oscar Godson
%
“Any fool can write code that a computer can understand.
 Good programmers write code that humans can understand.”
    -- Martin Fowler
%
Make sure that the old quote "they muddy the water so that it seems deep"
doesn't describe your engineering style/codebases.
    -- Sarah Drasner
%
“Before software can be reusable it first has to be usable.”
    -- Ralph Johnson