common commands

common_commands.md

#Common Commands

System Utility

• Upgrade all packages with pip

sudo pip freeze --local | grep -v '^\-e' | cut -d = -f 1  | xargs -n1 sudo pip install -U

• Split and Concat files in Shell

split -l 10000 new_data_lower.txt
cat *.txt > merged-file

• Change Mode to Executable

sudo chmod +x countdown.sh
sudo chmod +x new_thisweek.sh

• Kill a process by PID

kill -9 pid

• Print out first/last few lines

head -n 1 file  # change 1 to any number
tail -n 1 file
# if only print out first line
sed -n 1p file
head -n 1 file
awk 'NR==1' file

• Count total number of lines in a file

sed -n '$=' filename
wc -l filename

• Apply a change in a file (e.g., .bash_profile)

source .bash_profile

• VPN PrivateInternetAccess Usernam

p6441849

• remove a directory

rm -r directory

• Change default shell

chsh -s $(which zsh)
chsh -s $(which bash)

Stata

• Plotting coefficient estimates

parmby "xtreg ..., fe r", norestore
eclplot estimate min95 max95 parmseq

//identify nonnumeric strings and drop
gen byte notnumeric = real(twitter_count)==. 
/*makes indicator for obs w/o numeric values*/
tab notnumeric  
/*==1 where nonnumeric characters*/
list  twitter_count  if notnumeric==1  
/*will show which have nonnumeric*/
drop if notnumeric==1

Python

• To get a value of a key in a dictionary

dictionary.get('key', 'return_otherwise')

• String formatting

a = 'John'
b = 'Doe'
'Hi there %s %s. How is it going' % (a,b)  #c-like placeholder
'Hi there {0} {1}. How is it going'.format(a,b) #python method

System

• Get rid of the field codes with Endnote

CMD + Shift + F9 + fn

Git

cd /Users/Hong/Google Drive/website
git add.
git commit -m "message"
git push origin master

Update personal website

cd "/Users/Hong/Google Drive/website"
scp * [email protected]:www  # push all files to the www folder under the asu public address
scp -r files/ [email protected]:www # push the files folder to the files folder under www folder in asu address
scp files/social.png [email protected]:www/files
ssh [email protected]

Update the Hack Font

brew cask install caskroom/fonts/font-hack

Access AWS Server

sudo ssh -i hongkey1.pem [email protected]

scp -i hongkey1.pem  /Users/Hong/Google\ Drive/Research/Freelancer\ Crawler/Final\ Data\ and\ Scripts/Latest/Database/FreelancerDump20160716.sql  [email protected]:~/data/

## upload a directory
sudo scp -r -i hongkey1.pem  /Users/Hong/Google\ Drive/Research/Freelancer\ Crawler/Final\ Data\ and\ Scripts/Latest/Scripts/  [email protected]:~/scripts/

##install mysqldb (mysql-python) on linux
sudo apt-get install libmysqlclient-dev python-dev
sudo ~/anaconda2/bin/pip install mysql-python

MySQL

#import:
mysql > set global max_allowed_packet=100000000

in command line: mysql -u root -p --max_allowed_packet=100m freelancer<e:dump.sql;

#dump:

mysqldump --user=root --password=****** --database gaf > d:dump.sql 

#Grant only select function
GRANT SELECT ON *.* TO 'chen'@'%' IDENTIFIED BY 'isr';