imweijh

AUDIT type=%{WORD:audit_type} msg=audit\(%{NUMBER:audit_epoch}:%{NUMBER:audit_counter}\): user pid=%{NUMBER:audit_pid} uid=%{NUMBER:audit_uid} auid=%{NUMBER:audit_audid} subj=%{WORD:audit_subject} msg=%{GREEDYDATA:audit_message} 
AUDITLOGIN type=%{WORD:audit_type} msg=audit\(%{NUMBER:audit_epoch}:%{NUMBER:audit_counter}\): login pid=%{NUMBER:audit_pid} uid=%{NUMBER:audit_uid} old auid=%{NUMBER:old_auid} new auid=%{NUMBER:new_auid} old ses=%{NUMBER:old_ses} new ses=%{NUMBER:new_ses}

imweijh

input {
  generator {
    count => 1
    lines => [
      "qid=ABCD1 first",
      "qid=XYZ2 first",
      "qid=ABCD1 second",
      "qid=XYZ2 second",
      "qid=ZZZZ first",
      "qid=ABCD1 third"

imweijh

yum install -y wget
wget http://rpms.adiscon.com/v8-stable/rsyslog.repo
mv rsyslog.repo /etc/yum.repos.d/rsyslog.repo
yum info rsyslog --skip-broken
yum install -y rsyslog
rsyslogd -version

imweijh

--[[
Provides custom authorization for nginx.
See the `nginx_authorize_by_lua.conf` for the Nginx config. This lua file is referenced in the config
See testWebserverAccess.sh for unit tests.

To Run nginx (make sure you have the lua, config and htpasswd file):
$ /usr/local/openresty/nginx/sbin/nginx -c /etc/nginx/conf/nginx_authorize_by_lua.conf

Logs are available at: /usr/local/openresty/nginx/logs/lua.log
To write to the log:

imweijh

{
  "metrics" : {
    "order" : 0,
    "template" : "metrics-*",
    "settings" : {
      "index" : {
        "refresh_interval" : "5s"
      }
    },
    "mappings" : {

imweijh

#!/bin/bash
echo "Building NGINX along with Echo module"

# install prerequisites
yum -y install gcc gcc-c++ make zlib-devel pcre-devel openssl-devel

# download the Echo module
curl -L -O 'https://github.com/openresty/echo-nginx-module/archive/v0.58.tar.gz'
tar -xzvf v0.58.tar.gz && rm v0.58.tar.gz
mv echo-nginx-module-0.58 /tmp/echo-nginx-module

imweijh

/var/www/example.com/logs/*.log {
        daily
        missingok
        rotate 52
        compress
        delaycompress
        notifempty
        create 0640 www-data adm
        sharedscripts
        prerotate

imweijh

On Tue Oct 27, 2015, history.state.gov began buckling under load, intermittently issuing 500 errors. Nginx's error log was sprinkled with the following errors:

2015/10/27 21:48:36 [crit] 2475#0: accept4() failed (24: Too many open files) 2015/10/27 21:48:36 [alert] 2475#0: *7163915 socket() failed (24: Too many open files) while connecting to upstream...

An article at http://www.cyberciti.biz/faq/linux-unix-nginx-too-many-open-files/ provided directions that mostly worked. Below are the steps we followed. The steps that diverged from the article's directions are marked with an *.

1. • Instead of using su to run ulimit on the nginx account, use ps aux | grep nginx to locate nginx's process IDs. Then query each process's file handle limits using cat /proc/pid/limits (where pid is the process id retrieved from ps). (Note: sudo may be necessary on your system for the cat command here, depending on your system.)
2. Added fs.file-max = 70000 to /etc/sysctl.conf
3. Added `nginx soft nofile 1

imweijh

Latency numbers every programmer should know

L1 cache reference ......................... 0.5 ns
Branch mispredict ............................ 5 ns
L2 cache reference ........................... 7 ns
Mutex lock/unlock ........................... 25 ns
Main memory reference ...................... 100 ns             
Compress 1K bytes with Zippy ............. 3,000 ns  =   3 µs
Send 2K bytes over 1 Gbps network ....... 20,000 ns  =  20 µs
SSD random read ........................ 150,000 ns  = 150 µs

Read 1 MB sequentially from memory ..... 250,000 ns = 250 µs

imweijh

wordlist created from original 41G stash via:

grep -rohP '(?<=:).*$' | uniq > breachcompilation.txt

Then, compressed with:

7z a breachcompilation.txt.7z breachcompilation.txt

Size: