Obscure Shell Syntax

obscure_shell_syntax_schumatize_draft1.md

(draft 1) Obscure Shell Syntax

• Like irregular verbs in a foreign language, the most common primitives are optimized for brevity rather than consistency, so often are illogical and idiomatic (you have to "just memorize" them to get the benefit).
• --I'm organizing this by symbol rather than concept since there is a universal closed set of characters, but concepts have subjective implicit meaning (like key bindings)

$

• $'' - verbatim string, no interpretting (useful if you have filenames with non ASCII characters)

  • $"" - verbatim string, no interpretting (except locale)

cat "1    Winner" | awk -F \$'\t' '{print $1,$2}'

• You could think of $ as a dereferencing operation, like C's * for variables.

• $_ - last argument of last command

mkdir `date -I` && cd $_

• ${@}

• ${1}

• There are others that are less useful, but are good for perspective in understanding

  • $?
  • $$ - PID of newly created subshell (useful as a random number)
  • ${1:?"missing arg"}
  • ${TERM+"term is set"} - return value when variable is set
  • ${TERM-"term is not set"} - return value when variable is NOT set
  • echo "${PWD/$HOME/~}" (if you want to use variables in the substitution, the expression must be in double quotes - somewhat obviously)
    • to match from the beginning of the string, add a hash: "${PWD/#HOME/~}"
  • ${PWD:3:5} - characters from positions 3 to 5
  • ${PWD#*/} - remove everything BEFORE the FIRST slash
  • ${PWD##*/} - remove everything BEFORE the LAST slash
  • (doesn't work)${PWD%*/} - remove everything AFTER the FIRST slash
  • (doesn't work)${PWD%%*/} - remove everything AFTER THE LAST SLASH
  • "${!TERM*}" - only in bash, not zsh (indirect expansion)
  • Read about "parameter substitution" for more

!

• You could think of ! as escaping the shell into a meta-shell that accesses the command history

  • For more, read about "word designators" and "event designators"

• !:0

• !:1

• !:2

• !:2-4

• !:2-

• !$ - last argument of last command (do not confuse with $!). Shorthand for !!:$

• !^ - first arg of previous command, excluding the command (same as !:1)

• !ls - last command beginning with ls . Zsh supports tab expansion. Can be useful when a server shell doesn't have history search configured with a key binding.

• !! - same as !:-1

• !? - last command that resulted in an error

• !* all args in previous line, but not the 0th

• !:-5:2 - 2nd arg of 5 commands ago

• !!:gs/foo/bar/ - greedy substitution

• !fi:2 2nd arg of the last command starting with fi

%

• %1

``

• `some_command` is syntactic sugar for eval "some_command"
• Inside double quotes, it will get executed and the result inserted into the string. Inside single quotes, the command itself will remain in the string. (e.g. `cat ~/.sshpass`)

()

• (some_command) is syntacitc sugar foir sh -c "some_command".
  • You can use - inside the brackets to read stdin.
  • (cd /tmp/; zip web.xml my.jar) | (cd /tmp/; unzip -l my.jar)
    • I'm not sure if we can do something similar using xargs, or whether we have to resort to a pipe to a while loop
  • It's a bit like javascript promises actually
  • useful also for suppressing output of a command you want to run in the background using &
• Creates a subshell (variables created will not be valid outside it)
• find | (while read LINE; basename "$LINE"; dirname "$LINE")
  • you can change the delimeter using read -d
• $(some command) can be thought of as an anonymous variable that you assign the output of a command to. It is not a dash standard
• <(some command) can be thought of as an anonymous file that a command writes to, and outside you can read from (not dash compatible). e.g. diff
• >(some command) can be thought of as an anonymous file that a command reads from, and outside you can write to (not dash compatible). Useful writing to multiple files e.g. 2>(), 1>(), but is otherwise the same as | (some command)

{}

For commands:

• Groups commands for associativity only (no subshell). Useful for printing an error message and exiting if a precondition fails. Binding precedence of && and || is identical in shell, not like in OOP.
• semicolon is required for the last (or only) command
• spaces are required after opening and before closing

test -f /tmp/in.txt || { echo "not found" && exit 1; }

test -f

For variables (tldp parameter substitution explains it well):

• $1
• read stdin if no args given

while read line; do; echo $line; done < "${1:-/dev/stdin}"

• ${FILE:=}- return default value and assign
• ${FILE:-/tmp/in.txt} - return d``efault value, but do not assign
  • it will work without the : . See paramter substitution for exact difference.

echo ${username-`whoami`}

echo $PATH:gs/\/home\/sarnobat/~/

echo ${PATH/#$HOME/~}

[]

:

^

• ^foo^bar^ quick substitution (!!:s^foo^bar^) - only substitutes the first occurrence

>

• &>

• 2> - send to stderr

<<

• Multiple heredocs

cat <<AAA <<BBB
a
AAA
b
BBB

<<<

• Zsh only

&

• I wonder if we can use this with xargs
• &1 - shorthand for /dev/stdout
• |& - shorthand for 2>&1 |

\

• at the end of a line, you can think of it as escaping the newline that follows it (that your editor renders graphically but in memory is just another character)
• use builtin command, not alias with the same name

test

• test -z is empty string

• test $1 = 'build'

• test -e exists

• test -f is file

• test -d is dir

• Good for failing fast

• You can use if statements instead of test ... && , but that encourages memory sharing

  • [[...]] && is (also a shorthand) but I find that harder to understand

set

• set

• set -e fail fast (same effect as using && between every statement)

exit

• failure: exit 1 (do not confuse with -1 in OOP)

decl

read

trap

select