Process Basics

All processes, with params + hierarchy

ps auxww -H

Show all ruby-related PIDs and processes

pgrep -fl ruby

What is a process doing?

strace -f -p $PID

What files does a process have open? (also nice to detect ruby version of a process)

lsof -p $PID

Flavors of kill

kill xxxx
kill xxxx yyyy zzzz
pkill <name of process>
pkill -f <word in process name>

Keep an eye on a process

watch 'ps aux | grep ruby'

MEMORY

How much mem is free?

free -m
cat /proc/meminfo

Learn how to read output

Are we swapping? First line is avg since boot

vmstat 1

List the top 10 memory hogs

ps aux --sort=-resident|head -11

Detect OOM and other bad things

for i in messages kern.log syslog; do egrep -i "s[ie]g|oo(m|ps)" /var/log/$i{,.0}; done

Disable OOM killer for a process

echo -17 > /proc/$PID/oom_adj

DISK/FILES

Check reads/writes per disk

iostat -xnk 5

Files (often logs) marked for deletion but not yet deleted

lsof | grep delete

Overview of all disks

df -h

Usage of this dir and all subdirs

du -hs

Find files over 100MB

find . -size +100M

Low hanging fruit for free space. Check /var/log too...

ls -al /tmp

Find files created within the last 7 days

find . -mtime -7

Find files older than 14 days

find . -mtime +14 -type f -name '*.gz'

Delete files older than 14 days

find *.gz -mtime +14 -type f -exec rm {} \;

Monitor a log file for an IP or anything else

tail -f file.log | grep 192.168.1.1

Generate a large file (count * bs = total bytes)

dd if=/dev/zero of=file.txt count=1024 bs=102

NETWORK

TCP sockets in use

lsof -nPi tcp

Get IP/Ethernet info

ip addr
ifconfig

host <=> IP resolution

host 192.168.1.1
host MyRouter

Curl, display headers (I), follow redirects (L)

curl -LI http://google.com

Traceroute with stats over time (top for traceroute) Requires install

mtr google.com

Traceroute using TCP to avoid ICMP blockage

tcptraceroute google.com

List any IP blocks/rules

iptables -L

Drop any network requests from IP

iptables -I INPUT -s 66.75.84.220 -j DROP

Show traffic by port

iftop

Show all ports listening with process PID

netstat -tlnp

D/L speed test (don't run in prod! :)

wget cachefly.cachefly.net/100mb.test -O /dev/null

TERMINAL & SCREEN

Start a screen session as the current user

screen -x

Join/re-attach to a screen session

screen -r

Record a terminal session

script filename.out 2> filename.timing

Playback a recorded terminal session

scriptreplay filename.timing filename.out

TIPS N TRICKS

Run Previous command as root

sudo !!

Change to last working dir

cd -

Run something forever

while true;do ruby ghetto.rb;done

DATABASES

tail all queries hitting mysql. Learn more

pt-query-digest --processlist h=localhost --print --no-report --user xxxx --password *****

Connect to production mysql locally on port 3307 via ssh Learn More

ssh -L 3307:localhost:3306 user@hostname -N

oddlyzen/devops_shell_commands.md