check max limit of fd

Background

Exception in thread "qtp859985937-5877" java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached

Howto

cat /proc/sys/fs/file-max
104331942

check the number of current fd

watch 'ls /proc/*/fd/|wc -l'
...
3160409
3160694
...

check processes that uses fd more than others:

for pid in /proc/[0-9]*; do p=$(basename $pid); printf "%4d FDs for PID %6d; command=%s\n" $(ls $pid/fd | wc -l) $p "$(ps -p $p -o comm=)"; done | sort -nr

Check file descriptor

 ls -l /proc/id/fd > /tmp/openfiles
 wc -l /tmp/openfiles
 cat /tmp/openfiles

Checking the sockets for the process

ss -p | grep 96426 > /tmp/ssprocess

count number of thread

ps -eLf | wc -l

top 100 process that has many threads

ps -eLf | awk '{print $2}' | sort | uniq -c | sort -k1 -n -r | head -100

kubosuke/java-oom-thread-limit.md

Background

Howto