Intro

shell takes commands and passes to the operating system
most popular shell is bash (Bourne Again Shell): a replacement of sh, the origianl Unix shell, which was written by Steve Bourne
motivating examples:

# Copies all HTML files to `destination`,
# but only those that do not exist or are newer
cp -u *.html destination
# Finds the ten most common words in a text file
curl http://composingprograms.com/shakespeare.txt | \
tr -d '[[:punct:]]' | tr ' ' '\n' | sort | uniq -c | sort -nr | head
# Finds all the function definitions in a Python file 
find . -name '*.py' -exec grep -H '^def ' {} \;

explain shell
appreciate geeky jokes: make me a sandwich

who | grep -i blonde | date; cd ~; unzip; touch; strip; finger; mount; gasp; yes; uptime; umount; sleep

powerful suite of commands; examples: man cal date free exit

Navigation and system exploration

Navigation.

cd change directory
ls lists directory contents
pwd prints working directory
files are organized in a tree
/ root directory
. current directory
.. parrent directory
~ home directory
~username username's home directory

Some important folders:

/ root
/bin programs, common binaries
/etc system configurations
/home users' home
/tmp, /var temporary files
/usr user-installed programs

Exploring the system.

ls lists direcotry contents
file determines file type
less view file contents
Many of these commands accept arguments or parameters
Options can be combined: ls -l -a -r can be written as ls -lar
Help --help or -help or -h

Manipulating files and directories

cp copies files and directories
mv moves files and directories
mkdir creates directories
rm removes files and directories
touch create a new empty file or updates its last-modified time stamp
ln create hard and symbolic links

Warning! The above command do not prompt for confirmation It makes it easy to overwrite/delete a file; use the -i flag to overwrite this setting.

Flags:

-i, --interative
-r, --recursive
-u, --update

Wildcards:

Wildcard	Matches
`*`	any character
`?`	any single character
`[characters]`	any
`[!characters]`	any
`[[:class:]]`	any character that is a member of the specified class

Classes are:

[:alnum:]
[:alpha:]
[:digit:]
[:lower:]
[:upper:]

Redirection

Get images from here

Standard input, output, error.

Redirecting standard output > and >>

ls -l /usr/bin
ls -l /usr/bin > ls-output.txt
ls -l ls-output.txt
less ls-output.txt
ls -l /bin/usr > ls-output.txt
> ls-output.txt
ls -l /usr/bin >> ls-output.txt
ls -l /usr/bin >> ls-output.txt

Redirecting standard error 1> stdout and 2> stderr:

ls -l /bin/usr 2> ls-error.txt

Redirecting both standard output and standard error to one file &>

ls -l /bin/usr &> ls-output.txt

cat concatenate files
head outputs the first part of a file
tail outputs the last part of a file
sort sort lines of text
uniq report or omit repeated lines
wc counts newlines, words and bytes for each file
tee read from standard input and write to standard output and files
grep prints lines matching a pattern
sort sorts lines of text
find finds files in a given directory
regex101: Online regex tester and debugger
regexone: Interactive tutorial for regex

Pipelines.

command1 | command2

Dissecting the motivating example:

curl http://composingprograms.com/shakespeare.txt \
| tr -d '[[:punct:]]' | tr ' ' '\n' | sort \
| uniq -c | sort -nr | head

Unix philosophy:

Write programs that do one thing and do it well.
Write programs to work together.
Write programs to handle text streams, because that is a universal interface.

Keyboard shortcuts and miscellany

clear clears the screen
history display the contents of the history list
My favorite: ALT+. -- place the argument of the most recent command on the shell

mv a{,.bk}
!!  # execute last command
sudo !!

Access Rights and Intro to Processes

File system access rights

In your directory, type ls -l (for long listing)
You will see that you now get lots of details about the contents of your directory, similar to the example below:

-rwxrw-r-- 1 marius speed   5538367 Apr 21 11:35 'LectureX.pdf'

Each file (and directory) has associated access rights: in the left-hand column is a 10 symbol string consisting of the symbols d, r, w, x, -, and, occasionally, s or S:

If d is present, it will be at the left hand end of the string, and indicates a directory: otherwise - will be the starting symbol of the string. The 9 remaining symbols indicate the permissions, or access rights, and are taken as three groups of 3.
The left group of 3 gives the file permissions for the user that owns the file (or directory) (marius in the above example);
The middle group gives the permissions for the group of people to whom the file (or directory) belongs (speed in the above example);
The rightmost group gives the permissions for all others.

The symbols r, w, etc., have slightly different meanings depending on whether they refer to a simple file or to a directory.

Access rights on files:

r (or -), indicates read permission (or otherwise), that is, the presence or absence of permission to read and copy the file;
w (or -), indicates write permission (or otherwise), that is, the permission (or otherwise) to change a file;
x (or -), indicates execution permission (or otherwise), that is, the permission to execute a file, where appropriate

Access rights on folders

r allows users to list files in the directory;
w means that users may delete files from the directory or move files into it;
x means the right to access files in the directory. This implies that you may read files in the directory provided you have read permission on the individual files.

Example:

-rw-------
#  means a file that only the owner can read and write, no-one else can read or write and no-one has execution rights (e.g. a mailbox file).

Changing access rights

chmod: changing a file mode. Only the owner of a file can use chmod to change the permissions of a file.

Symbols: u user, g group, o other, a all, r read, w write (and delete), x execute (and access directory), + add permission, - take away permission.

Example:

chmod go-rwx some_file
# removes read write and execute permissions on the file for the group and others. It will leave the other permissions unaffected.
chmod a+rw some_file
# gives read and write permissions on the file to all.

Processes and Jobs

A process is an executing program identified by a unique PID (process identifier). To see information about your processes, with their associated PID and status, type ps.

A process may be

in the foreground,
in the background, or
be suspended.

In general the shell does not return the UNIX prompt until the current process has finished executing. Some processes take a long time to run and hold up the terminal. Backgrounding a long process has the effect that the UNIX prompt is returned immediately, and other tasks can be carried out while the original process continues executing.

Running background processes

To background a process, type an & at the end of the command line. For example, the command sleep waits a given number of seconds before continuing.

sleep 10
# will wait 10 seconds before returning the command prompt. Until the command prompt is returned, you can do nothing except wait.

sleep 10 & 
[1] 27349
# will run sleep in the background.
# the sleep command is typed in by the user; the next line, indicating job number and PID, is returned by the machine.

The & runs the job in the background and returns the prompt straight away, allowing you do run other programs while waiting for that one to finish.

The user is returned a job number (numbered from 1) enclosed in square brackets, together with a PID and is notified when a background process is finished. Backgrounding is useful for jobs which will take a long time to complete (e.g., training a ML system).

Backgrounding a current foreground process

Type sleep 1000
You can suspend the process running in the foreground by typing ^Z, i.e., hold down the [Ctrl] key and type [z]. Then to put it in the background, type bg.

Listing suspended and background processes

When a process is running, backgrounded or suspended, it will be entered onto a list along with a job number. To examine this list, type jobs. An example output could be

[1] Suspended sleep 1000
[2] Running firefox
[3] Running matlab

To restart (foreground) a suspended processes:

# fg %jobnumber

fg 1 
# will bring sleep 1000 back
# typing fg w/o a job number foregrounds the last suspended process

Killing a process

kill: terminate or signal a process.

It is sometimes necessary to kill a process (for example, when an executing program is in an infinite loop). To kill a job running in the foreground, type ^C.

kill jobnumber
# kills a suspended or background process 

sleep 100 &
jobs
# say it is job number 2, for example
kill 2

ps: process status

Alternatively, processes can be killed by finding their process numbers (PIDs) and using kill PID_number.

sleep 100 & 
ps

PID TT S TIME COMMAND
20077 pts/5 S 0:05 sleep 100
21563 pts/5 T 0:00 firefox
21873 pts/5 S 0:25 gedit

# let's kill the sleep 100 process 
kill 20077 
# and check if it worked
ps

If a process refuses to be killed, use the -9 option: kill -9 20077.

Optional

Play with htop: https://codeahoy.com/2017/01/20/hhtop-explained-visually/

danoneata/README.md