Skip to content

Instantly share code, notes, and snippets.

@jpcofr
Last active February 17, 2024 10:34
Show Gist options
  • Save jpcofr/364f8d9b3959e75bd5adfcb34979f934 to your computer and use it in GitHub Desktop.
Save jpcofr/364f8d9b3959e75bd5adfcb34979f934 to your computer and use it in GitHub Desktop.
Task Description Command GNU coreutils/POSIX utils/Unix utils
Create tcsh Alias for readlink alias toclip 'readlink -f \!:1 | to_clipboard' alias, readlink, to_clipboard
Improving alias for copying a path to the clipboard Change the alias name to copy_path_to_clipboard for clarity: alias copy_path_to_clipboard='readlink -f \!:1 | to_clipboard' alias, readlink, to_clipboard
Clean package cache (APT-based systems) sudo apt-get clean apt-get
Prepend filename to each line during concatenation awk '{print FILENAME": "$0}' *something.txt > concatenated_file.txt awk
Identify duplicate files based on SHA-256 hashes, using sort and uniq. cat file_hashes.txt | sort | uniq -d > duplicates.txt cat, sort, uniq, >
Concatenate text files ending with a specific suffix cat *something.txt > concatenated_file.txt cat
Print private keys cat /path/to/privatekey.key cat, less, vim
Compare two files and output 'yes' or 'no' cmp -s file1 file2 && echo "yes" || echo "no" cmp, echo
Check disk space usage df -h df
Check which filesystem home directory is on df -h ~ df
Exclude a directory in du command du -h --max-depth=1 --exclude=~/.snapshot ~ du
Exclude multiple directories in du command du -h --max-depth=1 --exclude=~/.snapshot --exclude=~/another_dir ~ du
List size of files/directories in home directory du -sh ~/* du
Identify large files/directories in home directory du -h --max-depth=1 ~ | sort -hr du, sort
Limit du depth and sort by size du -h --max-depth=1 ~ | sort -hr du, sort
Generate a file with SHA-256 hashes and absolute paths of all files in the directory. find . -type f -exec sh -c 'shasum -a 256 "$0" | awk "{print \$1, \"$(readlink -f "$0")\"}"' {} \; > file_hashes.txt find, shasum, awk, >
Count files in a directory excluding . and .. find . -maxdepth 1 -type f | wc -l find, wc
Find files over a specific size (e.g., 100MB) find ~ -type f -size +100M find
Find files larger than a specific size (e.g., 50MB) find ~ -type f -size +50M -exec ls -lh {} \; find, ls
Find files with a specific pattern and display their absolute path find . -type f -iname "name.*" -exec readlink -f {} \; find, readlink
Delete files and directories recursively except .tgz files find . ! -name "*.tgz" -exec rm -rf {} + find, rm, rmdir
Create a command to open *.pcap files with Wireshark in background find /path/to/directory -name "*.pcap" -exec sh -c 'wireshark "$0" &' {} \; find, sh, wireshark
Delete files recursively except .tgz files find . ! -name "*.tgz" -type f | xargs rm find, xargs
Compress Files with find and zip find /path/to/files -type f -exec sh -c 'f="{}"; zip "output.zip" -j "$f" "$(dirname "$f" | sed "s#.*/##")_$(basename "$f")"' \; find, zip
Compress multiple files into a ZIP archive with specific naming based on main subdirectory find /path/to/files -type f -exec sh -c 'zip -j "output.zip" "$(basename $(dirname "{}"))/$(basename "{}")"' \; find, zip
Search for text using regular expressions grep -E 'pattern' file.txt grep -E
Escape double quotes in grep command grep \" pattern_file.txt grep
Escape open curly brace in grep command grep \{ pattern_file.txt grep
Extract characters from start of line up to the character ':' grep -o '^[^:]*' file.txt grep
Filter file for timestamps within a specific range grep -E '2023-10-31 (0[7-9]|1[0-6]):[0-5][0-9]:[0-5][0-9]' file.txt grep
Filter lines containing specific strings into separate files grep 'A' file.txt > file_A.txt grep 'B' file.txt > file_B.txt grep 'C' file.txt > file_C.txt grep
Search for a specific string in files grep "example xstringx" file1 file2 file3 grep
Search for a string "example" in specific files recursively grep -r "example" --include="file_name" /path/to/directory grep
Search for a string in files without showing filenames grep -h "example xstringx" file1 file2 file3 grep, -h
Convert Memory Size to GB grep MemTotal /proc/meminfo | awk '{print $2 / 1024 / 1024 " GB"}' grep, awk
Test the integrity of a gzip file gzip -t file.tar.gz gzip, -t
Count files in a directory excluding directories ls -lA | grep -v '^d' | wc -l ls, grep, wc
Check available modules module avail module
Load a specific module (e.g., ncdu) module load ncdu module
Check RAM Usage by User's Processes ps -u $USER --no-headers -o rss | awk '{sum+=$1} END {print sum / 1024 " MB"}' ps, awk
Check disk quota usage quota -s quota
Check quota for a specific filesystem quota -s -f /path/to/filesystem quota
Format quota output using awk quota -s | awk '{print "User: "$1, " - Used: "$2, " - Quota: "$3}' quota, awk
Format output into a table using column quota -s | column -t quota, column
Delete a specific directory rm -r /path/to/directory rm
Delete a specific file rm /path/to/file rm
Delete the .snapshot directory rm -rf ~/.snapshot rm
Sort text lines in reverse human-readable order sort -hr sort -hr
Identify duplicate files based on SHA-256 hashes, using sort and awk. sort file_hashes.txt | awk '{if (p==$1) print $0; else p=$1}' > duplicates.txt sort, awk, >
Compress a directory tar -czvf archive_name.tar.gz /path/to/directory tar
Create a tar archive of all files in a directory without compression. tar -cf archive.tar directory_path/* tar
Create a tar archive of files in a directory without including the entire path. tar -cf archive.tar -C directory_path . tar
Create a tar archive of specific files in a directory without including the parent directory path. tar -cf archive.tar -C directory_path file1 file2 file3 tar
Create a tar archive without compression. tar -cf archive.tar files_or_directories tar
List the files inside a tar archive. tar -tf archive.tar tar
Untar (extract) the contents of a tar archive. tar -xf archive.tar tar
Test the integrity of a tar.gz file tar -tzf file.tar.gz tar, -tzf
List files in a .tgz file tar -tzf file.tgz tar, gunzip
Set default permissions to 700 for new files and directories umask 0077 umask
Implement a private mode in the terminal alias private='umask 0077' in .tcshrc or .cshrc umask, alias
Extract a specific HTTP packet as a hex string tcpdump -r your_file.pcap -nn 'tcp port src_port and tcp port dst_port and ip src src_ip and ip dst dst_ip' -c 1 -w - | xxd -p tcpdump, xxd
Get list of packet numbers for every HTTP packet in a trace tshark -r trace.pcap -Y http -T fields -e frame.number tshark
Extract HTTP packet details and write them as hex to a file Script-based command; not a single-line CLI command python, scapy
Generate filter for extracting HTTP packets from a pcap file Script-based command; not a single-line CLI command tshark, scapy
Sorting Files by Size in Descending Order sort -hr file_sizes.txt sort
Searching for Text with Regular Expressions grep -E 'pattern' input.txt grep
Capture correct HTTP flow Script-based command; not a single-line CLI command Wireshark, curl
Generate fragmented and irrecoverable HTTP flows Script-based command; not a single-line CLI command Scapy
Capture network traffic Script-based command; not a single-line CLI command Wireshark
Replay captured network traffic tcpreplay --intf1=<interface> <capture_file> tcpreplay
Extract a RAR archive into a folder with the same name 7z x -o./$(basename "$1" .rar) "$1" 7z x -o./$(basename "$1" .rar) "$1"
Extract a RAR archive into a folder with the same name (without assuming shell) 7z x -o./$(basename "$RAR_FILE" .rar) "$RAR_FILE" 7z x -o./$(basename "$RAR_FILE" .rar) "$RAR_FILE"
Extract and decompress .gz files within a RAR archive 7z x -so "$RAR_FILE" | gzip -d | tar xf - -C "$(basename "$RAR_FILE" .rar)" 7z x -so "$RAR_FILE" | gzip -d | tar xf - -C "$(basename "$RAR_FILE" .rar)"
Decompress every .gz file in the current directory for file in *.gz; do gzip -d "$file"; done for file in *.gz; do gzip -d "$file"; done
Concatenate every .pcap file in the current directory mergecap -w output.pcap *.pcap mergecap -w output.pcap *.pcap
Remove C++ comments from a source file Script-based command; not a single-line CLI command sed
Formatting C++ code including removing comments Script-based command; not a single-line CLI command clang-format
Understanding the using keyword vs typedef in C++ No command
Overview of C++ memory pool implementation No command
We can make this file beautiful and searchable if this error is corrected: Illegal quoting in line 3.
Task Description Command GNU coreutils/POSIX utils/Unix utils
Clean package cache (APT-based systems) sudo apt-get clean apt-get
Prepend filename to each line during concatenation awk '{print FILENAME": "$0}' *something.txt > concatenated_file.txt
awk
Concatenate text files ending with a specific suffix cat *something.txt > concatenated_file.txt cat
Print private keys cat /path/to/privatekey.key cat`, `less`, `vim
Check disk space usage df -h df
Check which filesystem home directory is on df -h ~ df
Exclude a directory in `du` command du -h --max-depth=1 --exclude=~/.snapshot ~ du
Exclude multiple directories in `du` command du -h --max-depth=1 --exclude=~/.snapshot --exclude=~/another_dir ~ du
List size of files/directories in home directory du -sh ~/* du
Find files over a specific size (e.g., 100MB) find ~ -type f -size +100M find
Find files larger than a specific size (e.g., 50MB) find ~ -type f -size +50M -exec ls -lh {} \; find`, `ls
Find files with a specific pattern and display their absolute path find . -type f -iname "name.*" -exec readlink -f {} \; find`, `readlink
Delete files and directories recursively except `.tgz` files find . ! -name "*.tgz" -exec rm -rf {} + find`, `rm`, `rmdir
Create a command to open `*.pcap` files with Wireshark in background find /path/to/directory -name "*.pcap" -exec sh -c 'wireshark "$0" &' {} \; find`, `sh`, `wireshark
Compress multiple files into a ZIP archive with specific naming based on main subdirectory find /path/to/files -type f -exec sh -c 'zip -j "output.zip" "$(basename $(dirname "{}"))/$(basename "{}")"' \; find`, `zip
Search for text using regular expressions grep -E 'pattern' file.txt grep -E
Escape double quotes in `grep` command grep \" pattern_file.txt grep
Escape open curly brace in `grep` command grep \{ pattern_file.txt grep
Extract characters from start of line up to the character ':' grep -o '^[^:]*' file.txt grep
Filter lines containing specific strings into separate files grep 'A' file.txt > file_A.txt` `grep 'B' file.txt > file_B.txt` `grep 'C' file.txt > file_C.txt grep
Search for a specific string in files grep "example xstringx" file1 file2 file3 grep
Search for a string "example" in specific files recursively grep -r "example" --include="file_name" /path/to/directory grep
Search for a string in files without showing filenames grep -h "example xstringx" file1 file2 file3 grep`, `-h
Test the integrity of a gzip file gzip -t file.tar.gz gzip`, `-t
Check available modules module avail module
Load a specific module (e.g., ncdu) module load ncdu module
Check disk quota usage quota -s quota
Check quota for a specific filesystem quota -s -f /path/to/filesystem quota
Delete a specific directory rm -r /path/to/directory rm
Delete a specific file rm /path/to/file rm
Delete the `.snapshot` directory rm -rf ~/.snapshot rm
Sort text lines in reverse human-readable order sort -hr sort -hr
Compress a directory tar -czvf archive_name.tar.gz /path/to/directory tar
Create a tar archive of all files in a directory without compression. tar -cf archive.tar directory_path/* tar
Create a tar archive of files in a directory without including the entire path. tar -cf archive.tar -C directory_path . tar
Create a tar archive of specific files in a directory without including the parent directory path. tar -cf archive.tar -C directory_path file1 file2 file3 tar
Create a tar archive without compression. tar -cf archive.tar files_or_directories tar
List the files inside a tar archive. tar -tf archive.tar tar
Untar (extract) the contents of a tar archive. tar -xf archive.tar tar
Test the integrity of a tar.gz file tar -tzf file.tar.gz tar`, `-tzf
List files in a `.tgz` file tar -tzf file.tgz tar`, `gunzip
Set default permissions to 700 for new files and directories umask 0077 umask
Implement a private mode in the terminal alias private='umask 0077'` in `.tcshrc` or `.cshrc umask`, `alias
Get list of packet numbers for every HTTP packet in a trace tshark -r trace.pcap -Y http -T fields -e frame.number tshark
Extract HTTP packet details and write them as hex to a file Script-based command; not a single-line CLI command python`, `scapy
Generate filter for extracting HTTP packets from a pcap file Script-based command; not a single-line CLI command tshark`, `scapy
Sorting Files by Size in Descending Order sort -hr file_sizes.txt sort
Searching for Text with Regular Expressions grep -E 'pattern' input.txt grep
Capture correct HTTP flow Script-based command; not a single-line CLI command Wireshark`, `curl
Generate fragmented and irrecoverable HTTP flows Script-based command; not a single-line CLI command Scapy
Capture network traffic Script-based command; not a single-line CLI command Wireshark
Replay captured network traffic tcpreplay --intf1=<interface> <capture_file> tcpreplay
Extract a RAR archive into a folder with the same name 7z x -o./$(basename "$1" .rar) "$1" 7z x -o./$(basename "$1" .rar) "$1"
Extract a RAR archive into a folder with the same name (without assuming shell) 7z x -o./$(basename "$RAR_FILE" .rar) "$RAR_FILE" 7z x -o./$(basename "$RAR_FILE" .rar) "$RAR_FILE"
Decompress every .gz file in the current directory for file in *.gz; do gzip -d "$file"; done for file in *.gz; do gzip -d "$file"; done
Concatenate every .pcap file in the current directory mergecap -w output.pcap *.pcap mergecap -w output.pcap *.pcap
Remove C++ comments from a source file Script-based command; not a single-line CLI command sed
Formatting C++ code including removing comments Script-based command; not a single-line CLI command clang-format
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment