Shell Cheat Sheet - my various thoughts and things-to-remember regarding shells

shell-cheat-sheet.md

Shell Cheat Sheet

Using jq to find shait in a JSON-file

Using the public API from Bring https://api.bring.com/pickuppoint/api/pickuppoint/no/all I needed to find all pickup point names, and the longest name. Used jq for this:

jq '.pickupPoint[] | .name' response.json >> pickup-points.out
awk '{ if (length($0) > length(longest)) longest = $0 } END { print longest }' pickup-points.out

Allow for TTY / Sudo from cygwin or whatever

Note: works on RHEL atleast..

Put this in the sudoers-file (normal sudoers, not the .d-specific one..)

sudo visudo
##
## Allow specific commands to be executed without the need of password
##
## @since ??.??.??
## @author <you>
##
<you> ALL = NOPASSWD: /home/<you>/script/whatever.sh
Defaults:<you> !requiretty

Run any sudo-based command from outside:

ssh -i my-awsome-priv-key <you>@<server> './script/whatever.sh'

Oh jeah..the whatever.sh-script contains sudo commands. If you'd like to run sudo-commands straight in the ssh command, you'd most likely have to also add the following stuff to your sudoers-file:

Defaults:<you> visiblepw

Change from CSH to Bash

When not wanting to have csh as your default shell, perform the following simple actions:

# 1. Execute the bash-shell upon login
echo -e "\nexec /bin/bash --login\n" >> .cshrc
# 2. Set default shell in your .profile-file:
echo -e "\nSHELL=/bin/bash\nexport SHELL\n" >> .profile
# 3. Relog :-)

Note: if your account exists within the /etc/passwd, you can also just use the following command:

chsh -s `which bash`

Enable X11 forwarding on RedHat

sudo yum install -y xorg-x11-server-Xorg xorg-x11-xauth xorg-x11-apps
sudo service sshd restart
# relog
xclock
# should see a nice little clock app in your fwd X11 clock

LINUX / UNIX

Finn ut om det er noen hengende connections

while true; do sudo netstat -ntpa | grep TIME_WAIT | wc -l; sleep 1; done

get slowest tests running in Junit

grep -h "<testcase" `find . -iname "TEST-*.xml"` | sed 's/<testcase time="\(.*\)" classname="\(.*\)" name="\(.*\)".*/\1\t\2.\3/' | sort -rn | head

Print tree

yum install tree
tree .
tree /path/to/folder
find ./ -type d -print | sed -e 's;[^/]*/;|____;g;s;____|; |;g'

Block in firewall

# Append rule
sudo iptables -A INPUT -s n.n.n.n -j DROP
# Remove rule
sudo iptables -D INPUT -s n.n.n.n -j DROP
sudo service iptables save

Safely remove a file

shred --remove --iterations=3 --force --zero <file>

Convert file-format (-f ENCODING the encoding of the input | -t ENCODING the encoding of the output)

iconv -f UTF-8 -t ISO-8859-1 in.txt > out.txt

TUNNEL TO A SPECIFIC SERVER ON A SPECIFIC PORT

ssh -L <port_local_server>:localhost:<port_on_remote_server> <user>@<remote_server_ip>

e.g:

ssh -L 7017:localhost:7017 [email protected]

Lytt til 80 p <client_localhost> via <remote_ip> port 96. Logg p server med @<remote_server_ip>

ssh -L 80:10.235.128.36:96 [email protected]

Forward til en port p lokalen, gjennom en annen port

ssh -L 1515:localhost:3389 Administrator@localhost

TUNNEL TO A SPECIFIC SERVER ON A SPECIFIC PORT + FORWARDING [requires route to exists]

ssh -L 9999:10.250.24.65:80 -R 9999:134.47.155.71:9999 [email protected]
ssh -L 7000:10.181.48.84:80 [email protected]

Get Remote Desktop to work from a server that is outside our own blocked network

ssh -L 3391:localhost:3389 [email protected]

WGET

wget -S -O/dev/null --header='Cookie: nms=148.121.175.18:8080' --header="Host: wintest.telenor.no" http://153.110.190.180:82/privat/minesider/nms

Wget multiple times

for i in {1..40}; do wget --header "Cookie:session=1234" --no-proxy -O - http://local.whatwhat.com:8080/something/pdf/REF/REF.pdf & done

Se alt av disk usage i en katalog

du -skh *

Gzip til katalog

cd /tmp/; gtar cvzf /tmp/tss.tar.gz 

Tar og Gzip

tar -c <folder> | gzip -[1-9] > <filename>.tar.gz	

Gzip .. noe

tar -pczf <name>.tar.gz <output_navn>

Finne ut hvor mange linjer av hver enkelt duppeditt :-)

wc -l apache.error.log.1308009600

Finne antall linjer basert p tekst:

cat apache.error.log.1317600000 | grep "nocookie*" | wc -l

Finne endra filer ila. et antall dager

find . -name "ddd" -mtime -<dager> -print

Finne filer uten  f noe "permission denied"

find . -name "zlib" -print 2> /dev/null

Finne fil og s kopiere

find . -iname "*-8.1*jar" -exec cp "{}" /home/t769765/tmp/libs/. \;

For  finne hvilken jar-fil en bestemt klasse ligger inne i

for q in `find . -name \*.jar`; do jar tf $q | /usr/xpg4/bin/grep -q org/hibernate/ejb/HibernatePersistence  && echo $q; done

For  finne hvilken jar-fil en bestemt klasse ligger i (UTEN path)

find . -name '*.jar' | while read f; do jar tf "$f" | grep "IesSamlServlet" && echo "[Located in $f]"; done

For  finne en string i en fil, bruk f¯lgende s¯kestreng

find . -type f -exec grep "projects" {} -print 2> /dev/null \;

Slette mapper

find . -name ".svn" -exec rm -rf {} \;

Simulere en "server"

# Source:
	[Unix] nc -l -p 7200 p dest
	[Linux] nc -l 7200 p dest
# Klient:
	telnet p ip : port p source

install p en Linux-server

yum [install|erase|update] <navn>

Sjekke om port er oppe med netcat

netcat -z -w3 <ip> <port>; echo $?; # 1 == not up, 0 == up!

Starte en enkel "webserver" som lytter på requests (POST, GET, whatever)

    while true ; do (echo -e "HTTP/1.1\n\n OK") | nc -l 8888 ; done

dos2unix recursive

sudo find . *.* -exec dos2unix {} \;

Finn alle grupper definert p unix:

cat /etc/group |cut -d: -f1 | grep <groupname>

Routing-tabell - "full view"

route -rnv

Finn alle mem-modules p Linux (Ubuntu, Kubuntu o.l)

sudo dmidecode --type 17 | less

Finn alle folders med write-access

find / -xdev -type d \( -perm -0002 -a ! -perm -1000 \) -print 2> /dev/null;

Solaris - Finne ut hvilken bit-mode kj¯res:

isainfo -kv

Legge til bruker:

sudo useradd -c "<comment>" --create-home --home-dir /home/<> --user-group <username>

Legge til bruker, grupper og pwd

sudo useradd -c "Whatevz" --create-home --home-dir /home/<> --user-group <username>
sudo usermod -a -G wheel <username>
echo <username> | sudo passwd --stdin <password>

Legge til og fjerne grupper for bruker

sudo usermod -a -G wheel t769765
sudo usermod -G "" t769765

Legge til bruker i EC2:

#add bruker (useradd)
#ssh-keygen -b 1024 -f testuser4 -t dsa
#mkdir .ssh
#chmod 700 .ssh
#cat testuser1.pub > .ssh/authorized_keys
#chmod 600 .ssh/authorized_keys
#chown testuser1:testuser1 .ssh
#chown testuser1:testuser1 .ssh/authorized_keys

"Not enough diskspace" - Ubuntu Linux

Det er for mange gamle pakker installert p linuxen din. Bare kj¯r f¯lgende, og ting blir slettaa..

dpkg -l 'linux-*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d' | xargs sudo apt-get -y purge

Full informasjon om SunOs (Solaris)

prtdiag -v

Full informasjon om Linux (RHEL)

sudo dmidecode
lscpu
lspci
free -m

Broadcast-meldinger:

wall << end MELDING end <enter>
echo "<msg>" | wall

Drepe prosesser p Linux/Solaris via s¯k

kill -9 `ps aux | grep <keyword> | grep -v grep | awk '{print $2}'`

PakkehÂndteringer

#SUN Solaris
	pkginfo
		pkginfo | grep apache
	pkgrm apacheENIGMATF 

Swap

#FREE
free
#SWAPMOUNT
swapon -s

Generere passord

apg

Sette opp NTP-server (Network Time Protocol)

# SERVER
	sudo apt-get install ntpd
	sudo vi /etc/ntp.conf
		server ntp.ubuntu.com iburst #send 8 packages instead of just the default one
		server 127.127.1.0
		fudge 127.127.1.0 stratum 10
	sudo service ntp restart
	ntpq -c lpeer
# CLIENT
	sudo vi /etc/ntp.conf
		server <IP> iburst
	sudo service ntp stop
	sudo service ntp start
	ntpq -c lpeer # verify that the Local-variant is replying and all is well.
	# check the date on both servers. if not synced, try to use the sudo ntpq -g to re-initiate the server

Rename all files within a folder

This example shows how to simply rename IntTests -> CtxTests.

# print'em
for i in `find . -name '*IntTest.java'`; do echo "$i"|awk -F'IntTest' '{print $1"CtxTest."$2}' ; done;
# move'em
for i in `find . -name '*IntTest.java'`; do mv $i `echo "$i"|awk -F'IntTest' '{print $1"CtxTest."$2}'` ; done;

Remove files older than (date)

find . -type f -newermt 2007-01-01 ! -newermt 2016-08-01 -exec rm {} \;

Ping a certain service n times to figure out the status

#!/bin/bash
echo "####STARTING SCRIPT TO CHECK PING####";
while true; do curl -s https://test-idp.telenor.no/system/ping --no-keepalive | grep "Hostname:" | awk -F": " '/1/ {print $2}' | awk -F"<" -v date="$(date +"[%d-%m-%Y %H:%M:%S] ")" '{print date $1}'; done;

Zip all found items

find . -regex ".*whatever-[0-9]+.log" -mtime +12 -exec tar cfz {}.tar.gz {} \;

WINDOWS

Tail en fil

Get-Content '<fil-sti>' -Wait

Finn alle porter som det lyttes pÂ:

netstat -an | find /i "listening"

Logge av remote brukere p en Windows Server

#a) Finn brukere
	qwinsta /server:tns-sko-22-021
#b) Steng ID
	logoff 2 /server:tns-sko-22-021

Registrere en service p Windows

sc.exe create winsecapps binpath= C:\_server\start_all.bat DisplayName= "WIN Security Applications" password= "telenor1814" type= own start= auto

Legge inn trusted sites..

[HKEY_CURRENT_USER\Software\Policies\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap\Domains\telenor.no*.fiks-staging] "http"=dword:00000001

[HKEY_CURRENT_USER\Software\Policies\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap\Domains\telenor.no*.fiks-prodtest] "http"=dword:00000001

[HKEY_CURRENT_USER\Software\Policies\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap\Domains\telenor.no*.fiks-systest] "http"=dword:00000001

[HKEY_CURRENT_USER\Software\Policies\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap\Domains\telenor.no*.fiks-test] "http"=dword:00000001

[HKEY_CURRENT_USER\Software\Policies\Microsoft\Windows\CurrentVersion\Internet Settings\ZoneMap\Domains\telenor.no*.fiks-dev] "http"=dword:00000001

RANDOM

JAVA i Path. Husk  ta touch + relogg! Legges i .bashrc, nederst

JAVA_HOME=/usr/jdk/jdk1.7.0_05
export JAVA_HOME
MAVEN_HOME=/progs/apache-maven-3.0.4
export MAVEN_HOME
export PATH=$PATH:$JAVA_HOME/bin:$MAVEN_HOME/bin

Antall threads for apacher:

for pid in `ps U wwwrun | grep httpd | grep -v grep | awk '{ print $1 }'`;
do echo Apache Worker Server $pid has `ps ms -p $pid | wc -l` threads;
done

Virtual Box

Install a Virtual Box image using ISO

# Choose settings > Storage > Add DVD/CD > Point to *.iso or *.raw or similar
# (Save)
# (Boot device)
# (Hit F12, choose CD/DVD as startup, follow instructions..)

Ubuntu @ Telenor

Logge inn p corp.telenor.no med Domainjoin-gui

domainjoin-gui (with details Computer-name: "t769765-stationarypc", Domain: "CORP.TELENOR.NO")

sudo domainjoin-cli join TELENOR\t769765

sudo domainjoin-cli join TELENOR t769765

sudo domainjoin-cli join corp.telenor.no t769765

sudo domainjoin-cli join schema.telenor.no t769765

sudo domainjoin-cli join schema.corp.telenor.no t769765

sudo domainjoin-cli join corp.telenor.no t769765

To verify that all this stuff works, try to ping one of the COS-Build servers @ IP/DNS: 134.47.116.55/tns-sko-371-011.corp.telenor.no

Vulnerabilities

env x='() { :;}; echo vulnerable' bash -c "echo this is a test"

Problems

Issues with the number of processes attached on Linux

In cases where a process (typically java) spawns more processes that is currently defined for a user on Linux systems, you need to raise the actual # of processes to allow spawning to be allowed. See below :-)

cat /proc/sys/fs/file-max
3264982 
# ulimit -u
128331
[root@ukgl-oim-vp02 limits.d]# cat /proc/sys/kernel/threads-max
256662
[root@ukgl-oim-vp02 limits.d]# ps -u oracle -L | wc -l
1025 
# cat 90-nproc.conf 
# Default limit for number of user's processes to prevent
# accidental fork bombs.
# See rhbz #432903 for reasoning.

*          soft    nproc     1024
root       soft    nproc     unlimited 
oracle     soft    nproc     10240

/etc/security/limits.d/90-nproc.conf  
https://access.redhat.com/solutions/30316 

Locate information about VM image

      cat /proc/meminfocat         /proc/cpuinfocat         /etc/security/limits*cat         /var/lib/pgsql/9.6/data/postgresql.conf           # CentOS/RHELcat         /etc/postgresql/9.6/main/postgresql.conf         # Debian cat /etc/fstabcat         /etc/redhat-release           # CentOS/RHELcat         /etc/debian_version         # Debianmountsystemctl         zram         statuslsblkpvdislayvgdisplaylvdisplaylsmod         /sbin/sysctl -afdisk -l /dev/

        blockdev --get-ra /dev/           # For each of your mounted         diskssmartctl -a /dev/<whatever your first disk is, i.e. sdarpm -qa | geel -iE ‘postgres|kernel|psql|e2fs|ext|xfs|btrfs’         # CentOS/RHEL         dpkg --list | grep -iE ‘postgres|linux-image|psql|e2fs|ext|xfs|btrfs’         # Debian