Skip to content

Instantly share code, notes, and snippets.

@MarWeUMR
Last active June 5, 2024 15:09
Show Gist options
  • Save MarWeUMR/8118cff897cf4b771a741a8e31880f44 to your computer and use it in GitHub Desktop.
Save MarWeUMR/8118cff897cf4b771a741a8e31880f44 to your computer and use it in GitHub Desktop.
CLI Cheat-Sheet

How to find the java home dir?

This is especially useful for nix installed jdks.

java -XshowSettings:properties -version 2>&1 > /dev/null | grep 'java.home'

How to list the disk usage sorted by size

du -sh /* | sort -rh

How to grep for file contents in a directory and only list the file names with a hit

 grep -rl "my-search-string" /etc/

How to extend an existing LVM with the space of a new disk

# vgs 
# lvs

pvcreate /dev/sdb
vgextend vg_openlab /dev/sdb
lvextend -l +100%FREE /dev/vg_openlab/lv_root
resize2fs /dev/vg_openlab/lv_root

Simple way to find files by name

# use sudo if files may be under permission protected directories
# 2> /dev/null removes errors like 'do not have permission' while traversing the filesystem
find / -name my_file.txt 2> /dev/null

How to get the latest entry for a set of directories with random numbers in the name

This one is really useful in cases like the following. Imagine a cloudera cluster where tons of processes are running and multiple instances have been run. This is what you could be faced with:

[root@sl-019895 ~]# ls /var/run/cloudera-scm-agent/process/

The result:

1546336091-ranger-RANGER_ADMIN-SetupRangerCommand                                       1546338486-atlas-ATLAS_SERVER
1546336099-oozie-OOZIE-SERVER-createdbtables                                            1546338531-atlas-ATLAS_SERVER
1546336185-ranger-RANGER_TAGSYNC                                                        1546338545-atlas-ATLAS_SERVER
1546336187-ranger-RANGER_USERSYNC                                                       1546338560-atlas-ATLAS_SERVER
1546336189-ranger-RANGER_ADMIN                                                          1546338575-atlas-ATLAS_SERVER
1546336236-ranger-RANGER_ADMIN-SetupPluginServices                                      1546338592-atlas-ATLAS_SERVER
1546336321-oozie-OOZIE-SERVER-upload-sharelib                                           1546338606-atlas-ATLAS_SERVER-InitializeAtlasRole
1546336390-oozie-OOZIE_SERVER                                                           1546338615-atlas-ATLAS_SERVER
1546336394-hue-HUE_SERVER                                                               1546338660-atlas-ATLAS_SERVER-InitializeAtlasRole
1546336396-hue-KT_RENEWER                                                               1546338698-atlas-ATLAS_SERVER-InitializeAtlasRole
1546336398-hue-HUE_LOAD_BALANCER                                                        1546338706-atlas-ATLAS_SERVER
1546336425-atlas-ATLAS_SERVER-InitializeAtlasRole                                       1546338744-atlas-ATLAS_SERVER-InitializeAtlasRole
1546336474-atlas-ATLAS_SERVER                                                           1546338752-atlas-ATLAS_SERVER
1546336739-ranger-RANGER_TAGSYNC                                                        1546338766-atlas-ATLAS_SERVER
1546336741-ranger-RANGER_USERSYNC                                                       1546338814-atlas-ATLAS_SERVER

Now how to get the current Atlas process dir? Utilizing ls with its flags 1dtr and a catch all * on the pathname, we can have one result per line, sorted and restricted to directories. With tail -1 we take the last (most recent in this case) entry, which would correspond the the process dir we were looking for.

The final command comes together like this:

export ATLAS_PROCESS_DIR=$(ls -1dtr /var/run/cloudera-scm-agent/process/*ATLAS_SERVER | tail -1)

This allows to work with the ugly directory path much more conveniently.

How to find the process ID for a given port?

lsof -i :<port_number>

How to fix E: Release file for http://archive.ubuntu.com/ubuntu/dists/bionic-updates/InRelease is not valid yet (invalid for another 9h 14min 16s). Updates for this repository will not be applied. error

This might happen if some systemtime issues occur.

# example with a dockerfile
RUN apt-get -o Acquire::Max-FutureTime=86400 update -y && apt-get install -y ...
#           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

How to list (and set) available java version

update-java-alternatives --list
update-java-alternatives --set [JDK/JRE name e.g. java-8-oracle]

How to create a new image from a running container?

Comes in handy, if, for example, you need to develop an ansible script and need different stages.

docker commit <container id or name>

# now the image needs a name
docker tag <IMAGE_ID> <IMAGE_NAME>

# now a container can be started with:
docker run <IMAGE_NAME> 

How to find the path, where systemd configs are located?

systemctl show -P FragmentPath cron-daily.service
# result: /lib/systemd/system/cron-daily.service

How to generate an overview of a current projects working dir including file contents?

The main idea here is to provide a quick and easy way to dump the project into a LLM. NOTE: ssh -Y might be necessary if using it on a remote machine.

#!/bin/bash

# Default search path is the current working directory
SEARCH_PATH="$(pwd)"

# Default list of file names/types and directories to ignore
IGNORE_LIST=(".DS_Store" "LICENSE" "*.lock" "__pycache__" "__init__.py" "ruff.toml" "*.nix")

# Function to join array elements
join_by() {
	local IFS="$1"
	shift
	echo "$*"
}

# Parse command line arguments
while getopts ":e:" opt; do
	case ${opt} in
	e)
		# Add custom ignore patterns from -e option
		IGNORE_LIST+=("$OPTARG")
		;;
	\?)
		echo "Invalid option: $OPTARG" 1>&2
		exit 1
		;;
	:)
		echo "Invalid option: $OPTARG requires an argument" 1>&2
		exit 1
		;;
	esac
done
shift $((OPTIND - 1))

# Check if a path argument is provided
if [ "$#" -eq 1 ]; then
	SEARCH_PATH="$1"
elif [ "$#" -gt 1 ]; then
	echo "Usage: $0 [-e pattern_to_ignore] [path_to_search]" 1>&2
	exit 1
fi

# Convert IGNORE_LIST to fd exclude patterns
EXCLUDE_PATTERNS=$(join_by , "${IGNORE_LIST[@]}")

# Execute fd with the constructed exclude patterns and search path
TREE_OUTPUT=$(fd --exclude "{$EXCLUDE_PATTERNS}" . "$SEARCH_PATH" | tree -a --fromfile)

# Copy the tree output to the clipboard
echo "$TREE_OUTPUT" | xclip -selection clipboard
# For macOS, use: echo "$TREE_OUTPUT" | pbcopy

# Extract file paths from the tree output
FILE_PATHS=$(fd --exclude "{$EXCLUDE_PATTERNS}" . "$SEARCH_PATH")

# Initialize an empty string to hold the file contents
FILE_CONTENTS=""

# Loop through each file path, read its contents, and append to FILE_CONTENTS
while IFS= read -r file_path; do
	# Check if the path is a file and not a directory
	if [ -f "$file_path" ]; then
		# Append the separator and file contents to FILE_CONTENTS
		printf "\n\n==============\n\nFile Contents for:\n%s\n\nvvvvvvvvvvvvvv\n\n\n" "$file_path" >>temp_file

		cat "$file_path" >>temp_file
		FILE_CONTENTS+=$(<temp_file)
		true >temp_file
	fi
done <<<"$FILE_PATHS"

FILE_CONTENTS+="$TREE_OUTPUT"

echo "$FILE_CONTENTS"

# Copy the file contents to the clipboard
echo "$FILE_CONTENTS" | xclip -selection clipboard

rm temp_file
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment