Bash string manipulation cheatsheet.

README.md

Bash string manipulation cheatsheet

Assignment
Assign value to variable if variable is not already set, value is returned. Combine with a : no-op to discard/ignore return value.	${variable="value"} : ${variable="value"}
Removal
Delete shortest match of needle from front of haystack.	${haystack#needle}
Delete longest match of needle from front of haystack.	${haystack##needle}
Delete shortest match of needle from back of haystack.	${haystack%needle}
Delete longest match of needle from back of haystack.	${haystack%%needle}
Replacement
Replace first match of needle with replacement from haystack.	${haystack/needle/replacement}
Replace all matches of needle with replacement from haystack.	${haystack//needle/replacement}
If needle matches start of haystack replace with replacement.	${haystack/#needle/replacement}
If needle matches end of haystack replace with replacement.	${haystack/%needle/replacement}
Substitution
If variable not set, return value, else variable.	${variable-value}
If variable not set or empty, return value, else variable.	${variable:-value}
If variable set, return value, else null string.	${variable+value}
If variable set and not empty, return value, else null string.	${variable:+value}
Extraction
Extract length characters from variable starting at position.	${variable:position:length}
Return string length of variable.	${#variable}
Escaping
Single quotes inside a single quoted string.	echo 'Don'\''t break my escape!'
Indirection
Return value of variable name held in indirect, else value.	indirect="apple" apple="fruit" ${!indirect-value}

Reference

• https://tldp.org/LDP/abs/html/string-manipulation.html
• https://tldp.org/LDP/abs/html/parameter-substitution.html
• https://tldp.org/LDP/abs/html/ivr.html
• Special characters:
  • *: https://www.tldp.org/LDP/abs/html/special-chars.html#ASTERISKREF
  • ?: https://www.tldp.org/LDP/abs/html/special-chars.html#WILDCARDQU
• https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_02
• https://www.gnu.org/software/bash/manual/bash.html#Shell-Parameter-Expansion

I'd like to propose a few additions

${variable##*.} -> returns everything after the last . in a string. Handy for getting file extensions

variable="/Users/cbongiorno/rds.json"
echo ${variable##*.}
json

Remove the last character from a string:

echo ${variable%%?}
/Users/cbongiorno/rds.jso

Remove the first character from a string:

echo ${variable##?}
Users/cbongiorno/rds.json

Nice one @chb0github 👍

The last/first char drops can be extended by padding out the ? count too - works nicely.

Maybe you can make this into a repo so others can PR with said additions?

Regex for these string manipulation functions seems to have its own unique standard, not regular or extended like sed and [[ ]]. Where is that cheatsheet?

@BobDodds these methods/functions don't support Regular Expressions. They are effectively glob like patterns, like * and . - sadly not a full regexp implement/engine.

Examples are here: http://tldp.org/LDP/abs/html/string-manipulation.html

@magnetikonline Thank you yes I found where it is explained that if we have shellopt extglob on, then we have a regex-like power but not a BASH_REMATCH or sed extended regex and \1 \2 etc to use in the right hand of ${string/x(y)x/z\1z/}.

Example. and extglob works like this:

      ?(pattern-list)
             Matches zero or one occurrence of the given patterns
      *(pattern-list)
             Matches zero or more occurrences of the given patterns
      +(pattern-list)
             Matches one or more occurrences of the given patterns
      @(pattern-list)
             Matches one of the given patterns
      !(pattern-list)
             Matches anything except one of the given patterns

"extglob implements a subset of ksh extended globs. ksh93 actually has a printf operator to convert between patterns and (AT&T) REs (printf '%P\n' '\[[0-9]\]' gives \[([0-9])\]) – Stéphane Chazelas Aug 28 '13 at 10:03
Hmm, it seems that *[0-9] works in other regex queries (without round brackets). – macieksk Jan 22 at 22:27"

One more thing that we can do...

string=abcdeff ; echo ${string/#a/} ${string/%f/} ${string/%f*/}
bcdeff abcdef abcde

...so weird the "#" working as expected as synonym for '^' in extended regex, but so weirdly the "%" coming before the "f" to glob the same as '$' in sed -E extended regex

Thanks for that @BobDodds - going over that voodoo, you can excuse me if I decide to avoid that going forward 😄 - my brain can only handle/hold the workings of Regexp!

# is for hopscotch at the ground level = beginning.

% is for mickey mouse ears on top = end.

Mickey Mouse club, cult, voodoo: ${string/%f*/} like % is a weird internal func %(){THE END;} which is mickey_mouse_ears() {garbage;}

I may need to meditate on that with my legs crossed like the optic nerve makes an X like #, behind the two eyeballs like % with a nose in the middle. That's not helping. I should just know Y.

Thank you @BobDodds, for shining more confusion on this already confusing topic

❓
String manipulation:
I have a variable called AWS_REGION, it's value get changed every time I deploy my application to different AWS regions.
for example:
AWS_REGION=eu-west-1 // here the value is not fixed.

export SHORT_REGION="${AWS_REGION/us-east-1/ue1}"
export SHORT_REGION="${AWS_REGION/us-east-2/ue2}"
export SHORT_REGION="${AWS_REGION/eu-west-1/ew1}"
export SHORT_REGION="${AWS_REGION/ap-northest-1/apne-1}"

this works, but it's kinda repetitive.

you guys have any suggestions using sed, awk etc...,

I could make those from ${BASH_REMATCH[3]} and ${BASH_REMATCH[3]} and${BASH_REMATCH[4]:0-3:1} in a branching conditional statement after this matchhashes matches into those: for pre in us-east- us-west- eu-east- eu-west- ap-northest- ap-northwst-do for n in 1 2 do AWS_REGION="${pre}$n" [[ $AWS_REGION =~ ^((.).)-(.)([^-0-9]+)-(.)$ ]] && echo ${AWS_REGION}: `  for n in ${!BASH_REMATCH[@]}   do echo -n $n ${BASH_REMATCH[$n]}' '  done  [ -n "${BASH_REMATCH[4]:4}" ] && echo -n quadrant: ${BASH_REMATCH[4]:0-3:1} ` || echo ${#BASH_REMATCH[@]}donedone us-east-1: 0 us-east-1 1 us 2 u 3 e 4 ast 5 1us-east-2: 0 us-east-2 1 us 2 u 3 e 4 ast 5 2us-west-1: 0 us-west-1 1 us 2 u 3 w 4 est 5 1us-west-2: 0 us-west-2 1 us 2 u 3 w 4 est 5 2eu-east-1: 0 eu-east-1 1 eu 2 e 3 e 4 ast 5 1eu-east-2: 0 eu-east-2 1 eu 2 e 3 e 4 ast 5 2eu-west-1: 0 eu-west-1 1 eu 2 e 3 w 4 est 5 1eu-west-2: 0 eu-west-2 1 eu 2 e 3 w 4 est 5 2ap-northest-1: 0 ap-northest-1 1 ap 2 a 3 n 4 orthest 5 1 quadrant: eap-northest-2: 0 ap-northest-2 1 ap 2 a 3 n 4 orthest 5 2 quadrant: eap-northwst-1: 0 ap-northwst-1 1 ap 2 a 3 n 4 orthwst 5 1 quadrant: wap-northwst-2: 0 ap-northwst-2 1 ap 2 a 3 n 4 orthwst 5 2 quadrant: w -Bob Dodds 17.01.2020, 13:24, "Devikiran Khode" <[email protected]>:❓String manipulation:I have a variable called AWS_REGION, it's value get changed every time I deploy my application to different AWS regions.for example:AWS_REGION=eu-west-1 // here the value is not fixed.export SHORT_REGION="${AWS_REGION/us-east-1/ue1}" export SHORT_REGION="${AWS_REGION/us-east-2/ue2}" export SHORT_REGION="${AWS_REGION/eu-west-1/ew1}" export SHORT_REGION="${AWS_REGION/ap-northest-1/apne-1}" this works, but it's kinda repetitive.you guys have any suggestions using sed, awk etc...,—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

Can i ask that string concatenation be included?

For example:

VAR1="Hello,"
VAR2=" World"
VAR3="$VAR1$VAR2"

and for literal strings and variable:

VAR1="Hello, "
VAR2="${VAR1}World"

From the guide here:https://linuxize.com/post/bash-concatenate-strings/

Thanks!

BASH Lookup ${! and $[!NAME*} can be powerful, too. You can use it to make indirect references or calculate

#!/bin/bash

VAR1="abc"
VAR2="def"
REF="VAR2"
def="1234"

$ echo "  \${!VAR*} = '${!VAR*}'"
  ${!VAR*} = 'VAR1 VAR2'
# Lists all variables that start with VAR

$ echo -e "  \${!REF} = '${!REF}'"
  ${!REF} = 'def'
# Because REF is 'VAR2' and the VALUE of VAR2 is 'def'.

$ echo "  \${!VAR1:-NOT FOUND} = '${!VAR1:-NOT FOUND} '"
  ${!VAR1:-NOT FOUND} = 'NOT FOUND '
# because VAR1 is abc which is not defined. However...

$ CALCREF="VAR${1:-2}"; echo -e "  \${CALCREF} = '${CALCREF}':\n  \${!CALCREF:-missing} = '${!CALCREF:-missing}'"
  ${CALCREF} = 'VAR2':
  ${!CALCREF:-missing} = 'def'
# Use the parameter $1 to lookup VAR${1}. If $1 has no value, use '2' so return the value of VAR2. 
# Or if VAR2 not found, return 'missing'
# NOTE that "${!VAR${INDEX}}" will not work - you get `bad substitution`. You have to create the reference variable 
# in one statement and the lookup in the next.
 
$ echo "  \${!VAR2:-NOT FOUND} = '${!VAR2:-NOT FOUND}'"
  ${!VAR2:-NOT FOUND} = '1234'
# because VAR2 = "def" and the value of def is 1234!

Lastly you can do some nesting

$ MYVALUE="${!AREF:-${DEFAULTVAL:-nothing}}";   echo "  \${MYVALUE} = '${MYVALUE}'"
  ${MYVALUE} = 'nothing'

The above means look up AREF (which does not exist. So instead it looks up DEFAULTVAL which also doesn't exist and thus returns nothing.
If AREF contained a valid reference, or DEFAULTVAL had a value those would have been return.

And here is an important point: by always using :- in every variable reference you can safely run a bash script with the '-u' option. -u is useful for finding uses of unassigned variables. In my experience that often happens due to misspellings which can be a booger to find.

BASH Lookup ${! and $[!NAME*} can be powerful, too. You can use it to make indirect references

thanks @nightskyguy - I've actually been needing this for a script recently. Nicely timed. 👍