Useful bash commands (that I keep forgetting)

useful_commands.md

Phred score

python -c 'print ord("<ASCII>")-33'

GIT

git tag

git tag -a v1.2 -m "My message"
git push origin --tags

Google

Download from Google Storage with gsutil

cat file.list | gsutil -m cp -I ./download_dir

VIM

add text at the end of each line in Vim

vip<C-V>$A,<Esc>

"Write with sudo" trick

:w !sudo tee %

BASH

grep tabs

grep -P '\t' file.txt

array subselection

${array[@]0:2} # first two elements

list the contents of a tar.gz file

tar ztvf archive.tar.gz

create tar.gz archive

tar cvzf archive.tar.gz *txt *py

sort in-place

sort filename -o filename

compress from stdin

cat file.csv | gzip > file.csv.gz

connect to server passwordless

ssh-keygen [-t rsa]
ssh-copy-id [-i ~/.ssh/id_rsa.pub] UserName@RemoteServer
ssh-add

print first line from a set of compressed files

find . -name "*.gz" | while read -r file; do zcat -f "$file" | head -n 1; done

send folders to server with rsync

rsync -azP <source> <destination>

unzip single file from archive

unzip -p myarchive.zip path/to/zipped/file.txt > file.txt

install from source with checkinstall for easy unistallation

./configure
make
sudo checkinstall # to install instead of make install
sudo dpkg -r packagename  # to uninstall

combine pdf

pdfunite in1.pdf in2.pdf in3.pdf out.pdf

print the elements of an array separating them with a given character

function join {local IFS="$1"; shift; echo "$*"}
join , a "b c" d # a,b c,d
join / var local tmp # var/local/tmp
FOO=(a b c); join , "${FOO[@]}" # a,b,c

FIND

find files smaller than 1000k and delete

find . -type f -size +1000k -delete

find files modified between specific times

find . -newermt "2013-01-01 00:00:00" ! -newermt "2013-01-02 00:00:00"

AWK

print lines NOT starting with pattern (e.g. "@")

awk '!/^@/'

pass variable to awk

variable="xyz"
awk -v var="$variable" 'BEGIN {print var}'

change output field separator (OFS)

awk '{OFS="\t"; print $1,$2}' file

skip lines

awk 'NR>1 {print}' file
awk 'NR<=100 {print}' file

SED

print if line matches pattern

sed -n "/word/p"

don't print (delete) if line matches pattern

sed "/word/d"

print range of lines

sed -n 10,15p

print specific line

sed "10q;d"

R

command line parsing

args <- commandArgs(trailingOnly= TRUE)

readr: default column type (integer)

read_tsv("file.tsv", col_types=cols(.default = "i"))

stringr: alterative to grepl

str_detect(x, ",")

ggplot2: legend point alpha colors

guides(colour = guide_legend(override.aes = list(alpha = 1)))

ggplot2: lollipop

geom_segment(aes(x=Measure, xend=Measure, y=0, yend=Value))

ggplot2: change legend title

scale_*_*(name="My fancy title")
labs(fill="My fancy title")
labs(color="My fancy title")

ggplot2: add text annotation

annotate("text", x=1, y=1, label="my interesting comment")

ggplot2: add "," to big numbers

scale_x_continuous(labels=scales::comma)

ggplot2: text size for publication

theme(axis.text.x = element_text(angle = 90, hjust = 1, vjust = 0.5), 
      legend.position = "none", 
      axis.title = element_text(size = 7),
      axis.text = element_text(size = 6), 
      legend.title = element_text(size = 6), 
      legend.text = element_text(size = 6), 
      strip.text = element_text(size = 5)
      )

forcats: modify factor levels

gss_cat <- mutate(gss_cat, partyid = fct_recode(partyid,
    "Republican, strong"    = "Strong republican",
    "Republican, weak"      = "Not str republican",
    "Independent, near rep" = "Ind,near rep",
    "Independent, near dem" = "Ind,near dem",
    "Democrat, weak"        = "Not str democrat",
    "Democrat, strong"      = "Strong democrat"
    ))

Python

ternary operator (one-line if else)

a if condition else b

How to fix a locale setting warning from Perl?

• Generate the locale. Generate the locale on the server with sudo locale-gen
• Stop forwarding locale from the client. Do not forward the locale environment variable from your local machine to the server. You can comment out the SendEnv LANG LC_* line in the local /etc/ssh/ssh_config file.
• Stop accepting locale on the server. Do not accept the locale environment variable from your local machine to the server. You can comment out the AcceptEnv LANG LC_* line in the remote /etc/ssh/sshd_config file.
• Set the server locale to English. Explicitly set the locale to English on the server. Add lines to your remote ~/.bashrc or ~/.profile files (https://askubuntu.com/questions/144235/locale-variables-have-no-effect-in-remote-shell-perl-warning-setting-locale-f/144448#144448)

USERS

change user password

passwd (change for current user)

add users to groups

usermod -aG staff dario

set default shell on login

sudo chsh <username> -s /bin/bash