StevenACoffman

Fluent-bit rocks

A short survey of log collection options and why you picked the wrong one. 😜

Who am I? Where am I from?

I'm Steve Coffman and I work at Ithaka. We do JStor (academic journals) and other stuff. How big is it?

Number	what it means
101,332,633	unique visitors in 2017

Neo23x0

# IMPORTANT!
# This gist has been transformed into a github repo 
# You can find the most recent version there:
# https://github.com/Neo23x0/auditd

#      ___             ___ __      __
#     /   | __  ______/ (_) /_____/ /
#    / /| |/ / / / __  / / __/ __  / 
#   / ___ / /_/ / /_/ / / /_/ /_/ /  
#  /_/  |_\__,_/\__,_/_/\__/\__,_/   

bsedat

FROM elixir:1.4.5 as asset-builder-mix-getter

ENV HOME=/opt/app

RUN mix do local.hex --force, local.rebar --force
# Cache elixir deps
COPY config/ $HOME/config/
COPY mix.exs mix.lock $HOME/
COPY apps/myproject_web/mix.exs $HOME/apps/myproject_web/
COPY apps/myproject_web/config/ $HOME/apps/myproject_web/config/

hvelarde

# get total requests by status code
awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -rn

# get top requesters by IP
awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head | awk -v OFS='\t' '{"host " $2 | getline ip; print $0, ip}'

# get top requesters by user agent
awk -F'"' '{print $6}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head

# get top requests by URL

wolfeidau

#!/usr/bin/env ruby

require 'bundler/setup'
require 'cloudformation-ruby-dsl/cfntemplate'
require 'cloudformation-ruby-dsl/spotprice'
require 'cloudformation-ruby-dsl/table'

template do
  value AWSTemplateFormatVersion: '2010-09-09'

whiteinge

#!/usr/bin/env python
# coding: utf-8
'''
Validate input file(s) against a Voluptuous schema

Pass a dash to read from stdin.
'''
from __future__ import print_function

import imp

brianleroux

sns.publish({
  TargetArn: device.arn,
  MessageStructure: 'json',
  Message: JSON.stringify({
    default: 'you will never see this muah!',
    APNS_SANDBOX: JSON.stringify({
      aps: {
        'alert': '',
        'content-available': 1
      },

junegunn

# GIT heart FZF
# -------------

is_in_git_repo() {
  git rev-parse HEAD > /dev/null 2>&1
}

fzf-down() {
  fzf --height 50% --min-height 20 --border --bind ctrl-/:toggle-preview "$@"
}

ericclemmons

Using <details> in GitHub

Suppose you're opening an issue and there's a lot noisey logs that may be useful.

Rather than wrecking readability, wrap it in a <details> tag!

<details>
 Summary Goes Here

jamesottaway

version: '2.0'
services:
  couchpotato:
    image: linuxserver/couchpotato
    ports:
      - 5050:5050
    volumes:
      - couchpotato:/config:rw
      - movies:/movies:rw
      - downloads:/downloads:rw