Skip to content

Instantly share code, notes, and snippets.

@vhbui02
Last active October 16, 2023 15:34
Show Gist options
  • Save vhbui02/c1245b4d2db1447238d086d664c14f2f to your computer and use it in GitHub Desktop.
Save vhbui02/c1245b4d2db1447238d086d664c14f2f to your computer and use it in GitHub Desktop.
[sed command] search and edit from file or stdin #linux

sed take inputs from a file or stdin, which means you can pipingsed ' output of other commands to sed

sed edits line-by-line and in a non-interactive way, that means all editing decisions are determined before running the command.

Syntax

sed [options] command [file-to-edit]

Printing lines

sed 'p' BSD Without p command, sed prints each line by default. Then with p, you've told it to print lines explicitly => each line printed twice.

sed -n 'p' BSD: disable default printing

Address Ranges

Addresses let you target specific parts of a text stream, that could be 1 line or a range of lines sed only performs commands that follow on those lines.

sed -n '1p' BSD: operate on line number 1

sed -n '1,5p' BSD: operate on line number 1 to 5

sed -n '1,+4p' BSD: operate on line number 1, then 4 lines after it.

sed -n '1~2p' BSD: operate on odd-number lines (1, 3, 5, 7, ...)

Deleting Text

Using d command

Note: you no longer need to use -n option since sed will print everything that is not deleted.

sed '1~2d' BSD: delete odd-numbered lines.

REMEMBER: source file is not being affected after the edit. You can say it only affected the output data in terminal.

There are 2 ways to save edits into file:

  • sed '1~2d' BSD > NewBSD.txt: redirect stdout to a file.

  • sed -i.backup '1~2d' BSD: using -i option, which also means perform edits in-place. This will alter the source file.

WARNING: Using the -i will overwrite the original file, so make sure you either confirm your desired edited version first, or create a backup file -i.backup

Substituting Text

sed can search for text patterns using regular expression, and the replace te found text with something else.

SYNTAX: s/<search_string>/<replacement_string>/flag...

s is the substitude command. The three / forward slash are used to seperate different text fields. You can, however, use other characters to delimit the fields in order to make your life easier E.g. inputs are URLs, you wouldn't want to use / as delimiter.

echo "http://www.example.com/index.html" | sed 's.com/index.org/home.'

NOTE: don't forget the final delimiter. People tend to forget it since there aren't any flags (e.g. g) or commands (e.g. d) after s

sed 's/on/forward/g' song.txt: using g flag to mark it change every instances in 1 line, not just the 1st instance.

sed s/on/forward/2 song.txt: using 2 flag to mark it change only for 2nd instance in 1 line.

sed -n 's/on/forward/2p' song.txt: by using -n option and p command, sed only printed the modified lines.

sed 's/SINGING/saying/i' song.txt: using i flag, patterns are now case-insensitive.

Replacing and Referencing Matched Text

Represent the search string

sed 's/^.*at/(&)/' song.txt: using regex to define pattern is start from the beginning of the line to the first encounter of at, this whole exact phrase aren't known in search string, you can use the & to represent the matched text, & only valid in replacement string.

Group different sections of matched text using parenthesis

Every group of search text, which is marked with escaped parenthesis \(something\), each of them can be referenced by an escaped reference number \1, \2, \3, ... The 1st parentheses group can be ref with \1, the 2nd with \2, ...

sed 's/\([^ ][^ ]*\) \([^ ][^ ]*\)/\2 \1/' song.txt: 1st group is the 1st word, 2nd group is the 2nd word.

Recall: * character are zero or more.

Some more docs

Option

-n: supresses automatic printing of pattern space.

-e <script>: add the script to the command to be executed

-f <script_file>: add the contents of the script file to the commands to be executed

-E: use extended regular expressions in the script.

-i[suffix]: edit files in place (makes backup if suffix supplied)

E.g. sed -i.txt 's/old/new/g' myfile.fish, suffix is .txt

Replace all 'old' with 'new' in myfile.fish, and creates a backup of the original file with a .txt extension (if SUFFIX's omitted sed will not create a backup file).

Note: If no -e or -f option is given, the first non-option argument is taken as the sed script to interpret, all remaining arguments are names of input files, if no input files are specified, then the stdin is read

E.g. sed "s/a/A/g" fruits.txt: "s/a/A/g" is "the first non-option argument is taken as the sed script to interpret"

fruits.txt is the name of input file. (if omitted, you have to write the content yourself, remember to use Ctrl+D to exit).

Script

s/abc/def/[g p i I e [w file] [r file]]

g: rplaces all occurrences of the pattern on each line of the input file (if omitted, sed will only replace the first occurrence)

p: print the pattern space (modified lines) if a substitution was made.

If using -n with p: sed only prints modified lines.

If not using -n with p: sed will duplicate modified lines when outputing.

i: match the pattern case-insensitively

I: match the pattern case-sensitively

w file: write to file, write the contents of the pattern space to the specified file after the substitution has been made

r file: read from file, reads the contents of the specified file and appends them to the pattern space after the substitution has been made.

Tab character in sed pattern

Triple backslash:\\\t

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment