Skip to content

Instantly share code, notes, and snippets.

@khuongduybui

khuongduybui/hoard.md

Last active February 24, 2025 16:39
Show Gist options
  • Save khuongduybui/abec54d9df99b9395c27f58a79f2dd89 to your computer and use it in GitHub Desktop.
Save khuongduybui/abec54d9df99b9395c27f58a79f2dd89 to your computer and use it in GitHub Desktop.
Download ZIP
hoard.md
Raw
hoard.md

WSL

Recover unused cached memory

echo 1 > /proc/sys/vm/drop_caches

Export in .vhdx format

wsl.exe --shutdown
cd ~
wsl.exe --export Ubuntu --vhd .\Pengwin\backups\ubuntu.vhdx

Import in .vhdx format

cd ~
wsl.exe --import Ubuntu Ubuntu .\Pengwin\backups\ubuntu.vhdx --vhd

Export in .tgz format (legacy)

cd ~
wsl.exe --export Ubuntu .\Pengwin\backups\ubuntu.tgz

Import in .tgz format (legacy)

cd ~
wsl.exe --import Ubuntu Ubuntu .\Pengwin\backups\ubuntu.tgz

Docker

Start dockerd with start-stop-daemon

sudo start-stop-daemon -b --exec $(which dockerd) --start -- -G $(whoami)

Delete all images

docker rmi -f $(docker images -a -q)

AWS

EC2

Add public key

See doc here: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/describe-keys.html#retrieving-the-public-key

mkdir -p ~/.ssh
curl http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key >> ~/.ssh/authorized_keys

MSI

Infrastructure

Run ansible-playbook against a host

# pip install ansible
AWS_PROFILE=${pod} ansible-playbook -i ansible/inventories/${pod}/hosts.yml --limit ${host}.ec2.${pod}.activeeye.com ansible/site.yml

Prune a host from SFT

ssh jenkins2.ec2.hawk.activeeye.com sftpruner --project ${pod} --hostname ${host}.ec2.${pod}.activeeye.com --ip ${ip}

Backend

Syslog in S3 read bucket

DATE=yyyy/mm/dd/hh/MM/
CUSTOMER_ID=
SYSLOG_ID=
aws s3 ls s3://activeeye-syslog-read-wasp/$CUSTOMER_ID/$SYSLOG_ID/$DATE
aws s3 cp s3://activeeye-syslog-read-wasp/$CUSTOMER_ID/$SYSLOG_ID/$DATE/... | gzip -d | jq

Read an ingest file

aws --profile=${pod} s3 cp ${s3_path} - | gzip -d | jq -r .Data.body.body | base64 -d | jq

Cancel AERSS secret rotation

python $backend/src/aerss-password-rotator/secret_scope.py <pod> AERSS/<client_module_id> list_versions
python $backend.src/aerss-password-rotator/secret_scope.py <pod> AERSS/<client_module_id> remove_label --version-id <version_id> --label AWSPENDING

Deploy a lambda version across all known pods

build/lambda/production.sh ${app} v${version} wasp
build/lambda/production.sh ${app} v${version} lion
build/lambda/production.sh ${app} v${version} bear
build/lambda/production.sh ${app} v${version} seal
build/lambda/production.sh ${app} v${version} wren
build/lambda/production.sh ${app} v${version} orca
build/lambda/production.sh ${app} v${version} wolf

Access the database

ssh -N -L 5432:activeeye-prod.cluster-cgsvjbtsik1e.us-east-1.rds.amazonaws.com:5432
PGPASSWORD=$(aws rds generate-db-auth-token --hostname activeeye-prod.cluster-cgsvjbtsik1e.us-east-1.rds.amazonaws.com --port 5432 --region us-east-1 --username "[email protected]" --profile wasp-administrator)
psql -h activeeye-prod.cluster-cgsvjbtsik1e.us-east-1.rds.amazonaws.com -p 5432 "[email protected] dbname=activeeye sslmode=verify-full sslrootcert=./global-bundle.pem"

Report Runner

Run test reports in wasp

ssh reportrunner1.ec2.wasp.activeeye.com

sudo su - reports

export AWS_REGION=`curl -s 169.254.169.254/latest/meta-data/placement/availability-zone | sed 's/.$//'`
cd /opt/report-runner/current
python3.9 report_scheduler/report_scheduler.py -config /etc/report-runner/report-runner -test-reports

Run monthly reports in lion

Take a look at this secret in lion arn:aws:secretsmanager:us-east-1:754700948275:secret:reportrunner/cfg-b5Qbyy Append your email to the monthly_email_recipients field (it's a CSV).

ssh reportrunner1.ec2.lion.activeeye.com

sudo su - reports

export AWS_REGION=`curl -s 169.254.169.254/latest/meta-data/placement/availability-zone | sed 's/.$//'`
cd /opt/report-runner/current
python3.9 soc_monthly_metrics.py -config /etc/report-runner/report-runner.cfg -customers deltarisks

Factory API

Obtain the auth token for v1

set TOKEN (curl -s -X POST -H "X-Amz-Target: AWSCognitoIdentityProviderService.InitiateAuth" -H "Content-Type: application/x-amz-json-1.1" -d '{"AuthParameters": { "USERNAME": "factory1", "PASSWORD": "..." }, "AuthFlow": "USER_PASSWORD_AUTH", "ClientId": "6urmkc7gcaopu336crs4q32tuj"}' "https://cognito-idp.us-east-1.amazonaws.com" | jq -r .AuthenticationResult.AccessToken)
set FAPI_VERSION v1
set FAPI_ENDPOINT https://api.provisioning.activeeye.com

Obtain the auth token for v2 and later

set TOKEN ...  # see https://dash.op.activeeye.com/fapi/v2 or https://dash.op-test.activeeye.com/fapi/v2
set FAPI_VERSION v2  # MEP 3.0 and before
set FAPI_VERSION v3  # MEP 3.1 and before
set FAPI_ENDPOINT https://api.provisioning.activeeye.com  # lion / bear / seal / wren / wolf
set FAPI_ENDPOINT https://provisioning.op-test.activeeye.com  # wasp / orca
set ESN $(uuidgen)
set ESN $(show_esn | cut -d'-' -f3)
set ESN ...

Commonly used actions

curl -H "Authorization: Bearer $TOKEN" "$FAPI_ENDPOINT/$FAPI_VERSION/whoami"  # available since v2
curl -H "Authorization: Bearer $TOKEN" "$FAPI_ENDPOINT/$FAPI_VERSION/get_health_check?region=us"  # lion / bear / wren
curl -H "Authorization: Bearer $TOKEN" "$FAPI_ENDPOINT/$FAPI_VERSION/get_health_check?region=ca"  # seal
curl -H "Authorization: Bearer $TOKEN" "$FAPI_ENDPOINT/$FAPI_VERSION/get_health_check?region=gov"  # wolf
curl -H "Authorization: Bearer $TOKEN" "$FAPI_ENDPOINT/$FAPI_VERSION/get_health_check?region=stagecom"  # wasp
curl -H "Authorization: Bearer $TOKEN" "$FAPI_ENDPOINT/$FAPI_VERSION/get_health_check?region=stagegov"  # orca
curl -H "Authorization: Bearer $TOKEN" "$FAPI_ENDPOINT/$FAPI_VERSION/get_health_check?region=dev"  # hawk
curl -H "Authorization: Bearer $TOKEN" -X POST "$FAPI_ENDPOINT/$FAPI_VERSION/create_customer_key?cust_id=<pod>-<customer_short_code>"  # for virtual onboarding script
curl -H "Authorization: Bearer $TOKEN" -X POST "$FAPI_ENDPOINT/$FAPI_VERSION/create_device?cust_id=aeye&esn=$ESN&son=11111111&region=us"  # lion / bear / wren
curl -H "Authorization: Bearer $TOKEN" -X POST "$FAPI_ENDPOINT/$FAPI_VERSION/create_device?cust_id=aeye&esn=$ESN&son=11111111&region=ca"  # seal
curl -H "Authorization: Bearer $TOKEN" -X POST "$FAPI_ENDPOINT/$FAPI_VERSION/create_device?cust_id=aeye&esn=$ESN&son=11111111&region=gov"  # wolf
curl -H "Authorization: Bearer $TOKEN" -X POST "$FAPI_ENDPOINT/$FAPI_VERSION/create_device?cust_id=aeye&esn=$ESN&son=11111111&region=stagecom"  # wasp
curl -H "Authorization: Bearer $TOKEN" -X POST "$FAPI_ENDPOINT/$FAPI_VERSION/create_device?cust_id=aeye&esn=$ESN&son=11111111&region=stagegov"  # orca
curl -H "Authorization: Bearer $TOKEN" -X POST "$FAPI_ENDPOINT/$FAPI_VERSION/create_device?cust_id=aeye&esn=$ESN&son=11111111&region=dev"  # hawk (pimepafmst)
curl -H "Authorization: Bearer $TOKEN" "$FAPI_ENDPOINT/$FAPI_VERSION/register_device?lsn=$LSN"
curl -H "Authorization: Bearer $TOKEN" "$FAPI_ENDPOINT/$FAPI_VERSION/get_csh_status?lsn=$LSN"
curl -H "Authorization: Bearer $TOKEN" -X POST "$FAPI_ENDPOINT/$FAPI_VERSION/complete_device_build?lsn=$LSN"
curl -H "Authorization: Bearer $TOKEN" -X DELETE "$FAPI_ENDPOINT/$FAPI_VERSION/delete_device?lsn=$LSN&delete_csh_record=true"

BitBucket

List all projects

msi-init
curl -K iap.header https://bitbucket.mot-solutions.com/rest/api/1.0/projects

AERSS

Retrieve CMEP default passwords

aws --profile=op secretsmanager get-secret-value --secret-id=dev/jenkins/cmep_admin | jq -r .SecretString

Retrieve password for logs fetched via op-dash

aws --profile=op secretsmanager get-secret-value --secret-id=aerss/logs | jq -r .SecretString | jq -r .password

Fetch logs

as root on AERSS

/opt/Motorola/ssp/ssp_log_collector.py get
# --host if you want to include host logs
# --passwd=YWJjZAo= if you want a password (base64 encoded)
# --upload if you want to push to CSH

cp /ssp_disks/ssp_app/log_collector/*.xz /home/admin
chown admin:admin /home/admin/*.xz

on host

scp -o PreferredAuthentications=password "[email protected]:~/*.xz" ~/

as admin on AERSS

rm -f /home/admin/*.xz

as root

/opt/Motorola/ssp/ssp_log_collector.py rm

Manual Switchover

as root

systemctl start rollback_trigger
systemctl status nubis_initializer
systemctl status rollback_timer

Get Serial Number / Service Tag

as root

dmidecode -t system

Restart container count metrics

as root

systemctl restart fluent-bit-metrics

Dashboard

Add a user to SSO

When connected to CORP network on MSI-imaged laptop. GROUP=activeeye-<pod>-users

net group <GROUP> <CORE_ID> /add /domain
# prereq
sudo apt install -y samba-common-bin
# add member
GROUP=
CORE_ID=
ADMIN_CORE_ID=
ADMIN_PASSWORD=
# check if member is in group
net rpc group members $GROUP --server=ds.mot.com -U $ADMIN_CORE_ID%$ADMIN_PASSWORD | grep -i $CORE_ID
# if not, add
net rpc group addmem $GROUP $CORE_ID --server=ds.mot.com -U $ADMIN_CORE_ID%$ADMIN_PASSWORD

Tenable

Install a Nessus agent

curl -H "X-Key: ${TENABLE_LINKING_KEY}" "https://cloud.tenable.com/install/agent?name=$(hostname)&groups=$(hostname | cut -d. -f3)" | sudo bash -

Check Nessus agent status

sudo /opt/nessus_agent/sbin/nessuscli agent status

PowerShell

Edit command in external editor

Set-PSReadLineKeyHandler -Chord Alt+e -Function ViEditVisually
$env:VISUAL='notepad'

Cannot use VSCode due to PowerShell/PSReadLine#3214.

Ubuntu / Debian

Manage repository keys

Ubuntu doesn’t want you to add all the signature keys in the single /etc/apt/trusted.gpg file. It suggests using a separate file that are located in the /etc/apt/trusted.gpg.d directory.

It’s the same mechanism it uses for the sources list where external repository sources are listed in their own file under /etc/apt/sources.list.d instead of keeping everything under the /etc/apt/sources.list file. It makes managing the external repos a bit easier.

This means that instead of using the apt-key in this fashion:

curl -sS https://download.spotify.com/debian/pubkey_5E3C45D7B312C643.gpg | sudo apt-key add -

You should use it like this:

curl -sS https://download.spotify.com/debian/pubkey_5E3C45D7B312C643.gpg | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/spotify.gpg

Release upgrade

Note that on Ubuntu Server, the new LTS release isn't made available to do-release-upgrade until its first point release, such as 22.04.1. This usually comes a few months after the initial release date.

If you don't see an available release, add the -d option to upgrade to the development release.

sudo do-release-upgrade [-d]

Reboot to UEFI

sudo systemctl reboot --firmware-setup

Laptop Close-Lid Behaviors

If you look into the content of the file /etc/systemd/logind.conf, you’ll see three different types of default settings for the laptop lid closing.

  • HandleLidSwitch: When the laptop is on battery power
  • HandleLidSwitchExternalPower: When the laptop is plugged into a power outlet
  • HandleLidSwitchDocked: When the laptop is connected to a docking station

If you want, you can change the value of those parameters to one of these as per your preference:

  • lock: lock when lid is closed
  • ignore: do nothing
  • poweroff: shutdown
  • hibernate: hibernate when lid is closed

You can either edit the /etc/systemd/logind.conf file and uncomment the said settings and change their value, or you create a new file in /etc/systemd/logind.conf.d directory. Create this directory if it doesn’t exist.

Podman rootless

We need the newuidmap and newgidmap binaries, which can be obtained with

sudo apt install -y uidmap

Getting local IP addresses

Various ways to get local IP addresses without ip of ifconfig.

cat /proc/net/fib_trie
hostname -I
networkctl status
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment