intro to *nix!

gistfile1.md

Intro to Unix and Vim star wars: telnet towel.blinkenlights.nl

Who am I

• 4th year comp sci, cmd line noob

Why command line

• highly productive
• O(1) vs long time for mouse
• Sometimes the only way to get something done. (SysAdmin and deployment.)
• hacking open source, working in the real world, building projects, show fx-team, git!
• lot of history buried in the depths

Why vim

• a piece of history
• Tetris for writing code
• insanely extendable, runs everywhere, especially on that server.
• if you can endure the learning cuve, the bindings will pay back in dividends can use vim bindings in all major ides and text editors.

The plan - whirlwind tour of the basics, we'll see what we get time for in the end

Unix command line Where am I? CD and LS A tour of the Unix file system

• rooted at /, thinking in trees
• how do we know where are? PWD
  • your home direectory by default
• absolute vs relative paths, special . and ..
• dev, usr, etc.

/bin

• mostly important apps like disk space, echo, cat chmod.
• the safe mode subset of bash commands.
• these are binaries, can be invoked like this ./binary

/sbin

• mission critical system binaries.
• lots of stuff re: mounting fs

/dev

• virtual files for each of your devices
• cat stuff into this if you are feeling lucky

/etc

• system wide config
• explore this directory, lots of cool stuff: -passwd all user accounts
• miss with this at your own peril

/etc/passwd

• pass config: name:password:UID:GID:GECOS:directory:shell

/proc

• if on linux boxes, virtual files that let you look into the kernel. kernel devs live by these files.

/usr

• all programs needed by regular users, non system critical stuff

/tmp

• storage of temporary ephemeral data, good place for the system to dump stuff in a panic. Sometimes emptied at boot

/usr/bin

• linux distro unique binaries live here, osx binaries too here. These are binaries that are non kernel specific and what make your distribution unique.
• have fun in here, check out ./yes

/usr/libs

• shared libs for use with /usr/bin

/usr/local

• your apps! for use system wide, apps usually compiles into /usr/local/bin

/usr/share

• shared stuff, images, logos, icons, docs

/var

• stuff that changes a lot, most other data is static on the file system --relatively speaking at least.

/var/log

• all the logs and more. cat around see what's here.

the mighty tilde ~~~ usually set to your home folder, very useful for pathing

The essentials man -k, jjjj kkkk /return mv, rm, cat, ps, wget, curl,

ln - symbolic links and hard links!

• useful for verison changes. Used constantly in build systems to prevent constant renaming of files.

ls -la
-rw-r--r--    1 jtgi  staff   2.5K Aug 28  2014 .zshrc-e
lrwxr-xr-x    1 jtgi  staff    21B Mar 17 11:58 416 -> drive/school/2015/416
drwxr-xr-x    4 jtgi  staff   136B Sep 16 14:05 AndroidStudioProjects
drwx------    4 jtgi  staff   136B Feb 14 15:14 Applications
drwx------+ 114 jtgi  staff   3.8K Mar 17 10:27 Desktop
drwx------+  13 jtgi  staff   442B Jan  5 22:54 Documents
drwx------+ 436 jtgi  staff    14K Mar 17 02:03 Downloads

useful for basics like drive and course folders cs416 ~> /drive/school/2015

ln -s path_to_file symname
ln -s ~/drive/school/2015/416 416

wildcards!

ls *
ls **/*
ls ok.*
ls [[:upper:]]*
ls [A-Z]*

okay, we are ls'ing around, but when's this useful? quite often, make files, static config files that compile c

mv *.java ../
javac *.java
rm *.o <- note this isn't rm * .o cuz lol
rm -rf /

searching - like text? hope so, cs lives for text grep! possibly the world most useful nix command examples

grep "string" <filename>
grep -i "string" <filename> //case insensitive
grep "<regex>" filename
grep -iw "full_string" <filename> //search for a whole word
grep -A <num> "string" file //show num lines after match
grep -B <num> "string" file //show num lines after match
grep -C <num> //surrounding lines
grep -r "cats" * //search all files recursively in cwd
grep -v //negations
grep -c //count the matches
grep -l //display only file names that match
grep '[0-9]\{3\}-[0-9]\{4\}' //phone numbers
alias grep='grep --color=auto'

find: search for files top, htop, ps aux - kill when those pesky pids get away from you

redirection! stdout to file cat - concatenate

cat file1 file2 file3 > superfile
cat *[0-9].log > megalog

ls -l . > superput.txt //truncate and write
ls -l . >> superput.txt //append

stdout - useful output stderr - status or err msgs

bound to file descriptors (kernel allocated identifiers used to index a device table, usually)

stdout fd 1 stderr fd 2

Skipping silly input ls -la . 2> /dev/null //dump this /dev/null as a service, /dev/null jokes

How do these programs get input? The command lines unifying interface stdin, stdout, stderror (again, files that by default are linked to the screen). Incredibly powerful abstraction.

Unix philosophy Perhaps, you're thinking these are boring in on their own..

Enter pipes!

• wc, uniq, sort, grep example

ls /bin | sort | less
ls /usr/bin | sort | less | uniq | wc  (lines, word, byte count)
cat test_in | java challengeA
ps grep | ruby

Life beyond pipes, composing commands with bash:

EXERCISE - Search log file Find all the errors in this file

wget http://jtgi.me/labs/development.log

1. How many are there?

2. What lines are they on?

3. Store the result in a file for later analysis

4. What if we had n log files?

5. Engineers said the problem is too hard to deal with looking at just one line. Now what?

cat development.log | grep -c "error"
cat development.log | grep -n "error" > errors.log
cat *.log | grep -n "error" > errors.log

Bash - wtf is bash? Bash scripting - variables, loops, scripts - EXAMPLE

• I dare you to read man bash
• expansion - prioritizing the dolla bills

echo $((5*2)) //10
echo $USER
echo $TERM

think you know pipes? meet recursive fib with named pipes courtesy of @tavisrudd

mkfifo fib
echo 0 1 1 >fib &
while read i j k; do
  echo $i >&2
  echo $j $k $(($j+$k))
  sleep .4
done <fib >fib

more expansion

ls -l $(which cp) #get binary of `cp`, ls that directory

What is the dollar sign madness, where are these coming from? Variables and your Environment

• just configs set on your bash to let it function

printenv, set
echo $PATH $rvm_version

aliases - simple text replacement on the command name

  alias ubc='ssh [email protected]'
  alias ls='ls -la --color=auto'
  alias ..='cd ..'
  alias yolo='git push -force'
  alias grep='grep --color=auto'
  alias apt-get="sudo apt-get"

So what about the rest of these commands, are they aliases? Magic? We saw that most of them are in /usr/bin are these just preset aliases? Nope. Meet Path: $PATH echo $PATH PATH=$PATH:wow

Path tells the shell where to look when resolving commands. It looks in left to right order, this is important. Sometimes you have multiple versions of the same tool installed, usually your personal local binaries are prioritized.

to execute a binary we run ./binary . is just for current directory Cool, but how do we make binaries? Where do they come from? Let's compile something from source! (no way this will work out)

wget http://downloads.sourceforge.net/project/tmux/tmux/tmux-1.9/tmux-1.9a.tar.gz
tar xzf tmux-1.9a.tar.gz
cd tmux-1.9a.tar.gz
./configure         //creates a make file customized to your hardware config, makes you feel cool
make                //a tool that runs other bash stuff that knows how to build the software, makes your feel more cool and scared

What do we get? a binary! ./tmx cd .. tmux... wat? "no command found"

It's not in our path! Path is just a variable: lets add to it: PATH=$(pwd):$PATH

Now type it --coooool

This process is laborious, but sometimes needed of installing beta builds, patching things etc. for the most part though, this is what package managers do:

Package Managers (linux: apt-get, osx: homebrew) Host repositories of source code, one comand install, setup on path, versioning, security, and much more

install homebrew: ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" <- #terrifying bash expansion

Let's install tree! (brew install tree, sudo apt-get install tree) Let's install cowsay! (brew install cowsay, sudo apt-get install cowsay)

Now close terminal and reopen. Hmm, it's gone. So are the aliases.

enter rc files and dotfiles .*rc files and other dotfiles.

they live in your home directory, check em out with ls -la

| The ‘rc’ suffix goes back to Unix's grandparent, CTSS. | It had a command-script feature called "runcom". Early | Unixes used ‘rc’ for the name of the operating system's | boot script, as a tribute to CTSS runcom.

Tips and tricks backgrounding/foregrounding processes command line fu, stuff that impresses your friends.

• remap your caps lock, srsly when'd you use it last? Its in prime position.
• emacs bindings, ctrle, ctrla, ctrlu, ctrlk, ctrld, ctrln, ctrlf
• previous !!, ctrl-r, search replace prev command
• .bash_history, what was that command again?

from here: you've seen what lives in /usr/bin/, explore stuff, search man pages, break things. brendan gregs incredible slide: http://www.brendangregg.com/Perf/linux_observability_tools.png