SITCON2015-cadre-training-shell-script

slide

--withborder
--center SITCON Cadre Training

--center shell script data analysis



--author Denny Huang
--date today

--newpage
--withborder
--heading Denny Huang



---
- http://me.sitcon.org/denny0223


---
- Chief Coordinator at SITCON 2013, 2014


---
- SITCON
	- http://sitcon.org


---
- TTUCSC
	- http://goo.gl/X086X

--newpage
--heading Survey
--withborder


---
SITCON 2014 participants?


---
Linux / Mac user?


---
Programming?
---
	C / Java / JavaScript / Python ...


---
CLI only?

--newpage
--heading Shell?
--withborder




---
--center user
--center ------------
--center shell
--center ------------
--center kernel
--center ------------
--center hardware


--newpage
--heading Why learn shell?
--withborder



---
--boldon
--center ######  ####### #     # #######
--center #     # #       ##   ## #     #
--center #     # #       # # # # #     #
--center #     # #####   #  #  # #     #
--center #     # #       #     # #     #
--center #     # #       #     # #     #
--center ######  ####### #     # #######
--boldoff

--newpage
--heading sh, bash, zsh ... ?
--withborder

---
--beginshelloutput
$ cat /etc/shells
/bin/sh
/bin/bash
/sbin/nologin
/usr/bin/sh
/usr/bin/bash
/usr/sbin/nologin
/usr/bin/tmux
--endshelloutput

---
- echo $SHELL

--newpage
--withborder




--center Basic knowhow

--newpage
--withborder
--heading The File System Tree


---
- pwd - Print name of current working directory


---
- cd - Change directory

---
- ls - List directory contents

--newpage
--withborder
--heading Manipulating Files And Directories


---
- cp - Copy files and directories

---
- mv - Move/rename files and directories

---
- mkdir - Create directories

---
- rm - Remove files and directories

---
- ln - Create hard and symbolic links

--newpage
--heading Command line format
--withborder

---
--beginshelloutput
$ ls
--endshelloutput

---
--beginshelloutput
$ ls -a
--endshelloutput

---
--beginshelloutput
$ ls --all
--endshelloutput

---
--beginshelloutput
$ ls -a -l
--endshelloutput

---
--beginshelloutput
$ ls -al
--endshelloutput
---

--newpage
--heading How does that work?
--withborder


---
- type - Indicate how a command name is interpreted

---
- which - Display which executable program will be executed

---
- alias - Create an alias for a command

--newpage
--heading System Environment
--withborder


---
$ env
$ printenv


---
$ echo $PATH



--newpage
--heading Arguments
--withborder

---
- argc, argv


--newpage
--heading Ask the man!
--withborder


---
--beginshelloutput
$ man ls
--endshelloutput


---
$ sudo yum install man-pages


--newpage
--heading Help
--withborder


---
--beginshelloutput
$ help pwd
--endshelloutput


---
--beginshelloutput
$ ls --help
--endshelloutput


--newpage
--heading System Environment (cont.
--withborder


---
- PS1
	- http://xta.github.io/HalloweenBash/
- PS2


--newpage
--heading rc-files
--withborder


---
https://github.com/denny0223/rc-files


---
- bashrc


---
- inputrc


--newpage
--heading inputrc
--withborder


---
emacs, vi mode

---
--beginshelloutput
$ set -o vi
--endshelloutput

---
--beginshelloutput
$ man readline
--endshelloutput

---
$ sudo yum install readline-devel

--newpage
--withborder
--heading Redirection


---
STDIN - keyboard


---
STDOUT - screen


---
STDERR - screen

--newpage
--withborder
--heading Redirection (cont.


                                    XXXXX
                           +--------+FD1+--------+
                           |        XXXXX        |
                           |                     v
+XXXXXXXXX     XXXXX   +---+---+              +XXXXXXX
|Keyboard+---->XFD0+---+Process|              |ScreenX
+--------+     XXXXX   +---+---+              +------+
                           |                     ^
                           |        XXXXX        |
                           +--------+FD2+--------+
                                    XXXXX


--newpage
--withborder
--heading Redirection (cont.


---
--beginshelloutput
$ ./a.out < in_file
--endshelloutput

---
--beginshelloutput
$ ./a.out <<.
> First line
> Second line
.
--endshelloutput

---
--beginshelloutput
$ cat in_file | ./a.out
--endshelloutput

--newpage
--withborder
--heading Redirection (cont.

---
--beginshelloutput
$ ./a.out > out_file
--endshelloutput

---
--beginshelloutput
$ ./a.out >> out_file
--endshelloutput

---
--beginshelloutput
$ ./a.out 2> out_file
--endshelloutput

---
--beginshelloutput
$ ./a.out 2>&1
--endshelloutput

---
--beginshelloutput
$ ./a.out &> out_file
--endshelloutput

--newpage
--withborder
--heading Text Processing

---
- cat - Concatenate files and print on the standard output
---
- grep - Print lines matching a pattern
---
- sort - Sort lines of text files
---
- uniq - Report or omit repeated lines
---
- wc - Print newline, word, and byte counts for each file
---
- head - Output the first part of a file
---
- tail - Output the last part of a file
---
- tee - Read from standard input and write to standard output and files

--newpage
--withborder
--heading sed

- stream editor for filtering and transforming text

---
- regex
    - https://www.regex101.com/

---
- s

---
- p

---
- d

--newpage
--withborder
--heading sed tricky

- sehll variable in sed
while read line; do sed -i "/${line}/d" tmpf; done < tmp

--newpage
--withborder
--heading awk

- pattern scanning and processing language

---

--newpage
--withborder
--heading example

- history | awk '{print $2}' | sort | uniq -c | sort -nr | head -n 20

- grep 台南 tn-ks-cfp-meetup.csv | awk -F\" '{print $2}' | sed 's/, /\n/g' | sort | uniq -c | sort -nr

--newpage
--withborder
--heading sed tricky

- $1=$2=""

- -F [][]

- -F $'\t'

--newpage
--heading tmux
--withborder

- .tmux.conf

--newpage
--withborder
--heading Resources


- http://www.gnu.org/software/bash/manual/html_node/index.html

- http://www.tldp.org/LDP/abs/html/index.html

- http://linuxcommand.org/tlcl.php

- https://sites.google.com/a/study-area.org/sa-activity/home/2014-02-22-sa-taipei

- https://sites.google.com/a/study-area.org/sa-activity/home/2013-08-tn

--newpage
--withborder

--center  _____          ____        ______
--center /\  __`\      /|  _ \      /\  _  \
--center \ \ \/\ \     |/\   |      \ \ \L\ \
--center  \ \ \ \ \     \// __`\/\   \ \  __ \
--center   \ \ \\'\\    /|  \L>  <_   \ \ \/\ \
--center    \ \___\_\   | \_____/\/    \ \_\ \_\
--center     \/__//_/    \/____/\/      \/_/\/_/
---

--center THE END!
--center Thanks for your listening