sahilsk

Kafka Cheat Sheet

Display Topic Information

$ kafka-topics.sh --describe --zookeeper localhost:2181 --topic beacon
Topic:beacon	PartitionCount:6	ReplicationFactor:1	Configs:
	Topic: beacon	Partition: 0	Leader: 1	Replicas: 1	Isr: 1
	Topic: beacon	Partition: 1	Leader: 1	Replicas: 1	Isr: 1

lukeplausin

# Count total EBS based storage in AWS
aws ec2 describe-volumes | jq "[.Volumes[].Size] | add"
# Count total EBS storage with a tag filter
aws ec2 describe-volumes --filters "Name=tag:Name,Values=CloudEndure Volume qjenc" | jq "[.Volumes[].Size] | add"
# Describe instances concisely
aws ec2 describe-instances | jq '[.Reservations | .[] | .Instances | .[] | {InstanceId: .InstanceId, State: .State, SubnetId: .SubnetId, VpcId: .VpcId, Name: (.Tags[]|select(.Key=="Name")|.Value)}]'
# Wait until $instance_id is running and then immediately stop it again
aws ec2 wait instance-running --instance-id $instance_id && aws ec2 stop-instances --instance-id $instance_id

# Get 10th instance in the account

so0k

Kubectl output options

Let's look at some basic kubectl output options.

Our intention is to list nodes (with their AWS InstanceId) and Pods (sorted by node).

We can start with:

kubectl get no

radabass

El Capitan OSX Vagrant port forwarding rules to use privileged ports 80 and 443

Due to the OSX limitations in ports below 1024, in order to use them without running as root the virtualbox headless you can do the following workaround, (remember the command ipfw is deprecated on El Capitan)

In the Vagrant file use ports over 1024, for instance change 80 and 443 to 8080 and 8043.

  # Apache
  config.vm.network "forwarded_port", guest: 80, host: 8080

bastman

Docker - How to cleanup (unused) resources

Once in a while, you may need to cleanup resources (containers, volumes, images, networks) ...

delete volumes

// see: https://github.com/chadoe/docker-cleanup-volumes

$ docker volume rm $(docker volume ls -qf dangling=true)

$ docker volume ls -qf dangling=true | xargs -r docker volume rm

a-h

filter {
  ## WebLogic Server Http Access Log
  if [type] == "weblogic-access" {
    grok {
      match => [ "message", "%{IP:client} - - \[(?<timestamp>%{MONTHDAY}[./-]%{MONTH}[./-]%{YEAR}:%{TIME}\s+%{ISO8601_TIMEZONE})] \"%{WORD:verb} %{URIPATHPARAM:uri}\s+HTTP.+?\" %{NUMBER:status} %{NUMBER:response_time}" ]
    }
    date {
      match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
    }
  }

mmaassen

input {
  ## WebLogic Server Log
  file {
    type => "weblogic"
    path => [ "/var/log/weblogic/domain/managedserver.log" ]
    codec => multiline {
      pattern => "^####"
      negate => true
      what => previous
    }

dreampuf

#!/bin/bash

# Origianl version : http://www.keepthingssimple.net/2012/10/creating-requests-per-time-graph-from-nginx-or-apache-access-log/ 
# This version is a port to Mac OSX
# dreampuf (http://huangx.in)

# useage :
#   graphit.sh logfile.log
#   open result-2013-06-04.png 

p1nox

Using Meld merging tool on Mac

There are two ways of installing meld on osx (May 2023), using brew and .dmg package (from @yousseb). Since I found https://yousseb.github.io/meld/, I've installed it with .dmg package, but having macOS Ventura Version 13.4 (22F66) in place, it's not even starting for me. So I tried brew installation, and the application is working as expected, including symlink to start it from the terminal.

Using brew formulae: https://formulae.brew.sh/cask/meld

brew install --cask meld

# set meld as your default git mergetool

dongyuwei

if (!Date.prototype.toISOString) {
    Date.prototype.toISOString = function () {
        function pad(n) { return n < 10 ? '0' + n : n; }
        function ms(n) { return n < 10 ? '00'+ n : n < 100 ? '0' + n : n }
        return this.getFullYear() + '-' +
            pad(this.getMonth() + 1) + '-' +
            pad(this.getDate()) + 'T' +
            pad(this.getHours()) + ':' +
            pad(this.getMinutes()) + ':' +
            pad(this.getSeconds()) + '.' +