sudo -u postgres -i
eg. aws-vault
$ aws-vault exec profile-name -- aws s3 ls
aws-vault: error: exec: Error execing process: cannot allocate memory
Check free -h
Run
ps -rss -er rss,pid,command --sort
# r only running processes
# s
# e all processes
# rss resident set size (non-swapped physical mem that a task has used)
# pid process ID
# command (only the executable name)
# some differences with GNU --sort, BSD etc
Or system has run out of file handles, or socket buffers, and has lots of memory left.
simplest, trace all system calls (sudo strace..)
strace df -h
output something like
open(....) = 3
open (the system call) (...) some file and mode 3 the system call return value
can strrace a running pid
strace -p some_pid
(then Ctrl C)
To get relative timestamps, -r Prefix with clock time, -t Prefix with clock time, -tt Prefix with clock time, -ttt
use -c flag to get cumulative times for each call
strace -c -p somepid
Can get instruction points, -i
To get command time, time spent in system calls, (start, end times) -T
Can filter on specific system calls -e trace=write -e trace=process -e trace=file -e trace=memory -e trace=network -e trace=signal
Redirect output -o some_file.txt
Futex. fast userspace mutex. kernel system call. used to implement basic locking.