devops_cheatsheet.md

Table of Contents

• Shell
• Python
• Node
• Anaconda
• Git
• Javascript DOM
• SQL statement
• Postgres DB
• Mac Facts
• VS Code
• Zed
• Ubuntu
• Homebrew
• ADB
• Docker
• Dockerfile
• Kubernetes
• Argocd
• Batocera

Shell

# show contents in .zshrc without comments and empty lines
# -v, --invert-match select non-matching lines
~$ cat ~/.zshrc | grep -v "^#" | grep -v "^$"

# unzip a Java pkg to get all the contents in Mac
~$ pkgutil --expand-full OpenJDK11U-jre_x64_mac_hotspot_11.0.3_7.pkg target_folder

Python

# intialize venv
# the first `venv` is a standard module included starting from Python 3.3.
# the second `venv` is simply a folder created to hold third-party dependencies.
test_project ~$ python -m venv venv
test_project ~$ pip install flask
test_project ~$ pip freeze > requirements.txt

# enable and disable virtual environment
test_project ~$ source venv/bin/activate
test_project ~$ deactivate

Node

Updating modules in package.json to the latest versions.

test_project ~$ cp package.json package.json.bk
test_project ~$ npm install --no-package-lock --no-save npm-check-updates
test_project ~$ ./node_modules/.bin/ncu -u
test_project ~$ mv package.json.bk package.json

Anaconda

# activate conda
~$ source ~/anaconda3/bin/activate
~$ conda init

# lists all Python versions available
~$ conda search python

# lists all envs
~$ conda env list

# activate an env, say py37
~$ conda activate py37

# create an env<py27> and install a Python version<2.7> for that
~$ conda create --name py27 python=2.7

# 在docker中运行anaconda
# 并把当前目录映射到docker中的/opt/notebooks文件夹
~$ docker run -i -t -p 8888:8888 -v $(pwd):/opt/notebooks continuumio/anaconda3 /bin/bash -c "\
conda install jupyter --quiet && \
jupyter lab \
--notebook-dir=/opt/notebooks --ip='*' --port=8888 \
--no-browser --allow-root"

Git

# show git log in a nicer way
~$ git log --all --graph --oneline

# unstage last changes
~$ git reset file_name

# uncommit last changes
~$ git reset --soft HEAD~1 

# uncommit + unstage last changes
~$ git reset --mixed HEAD~1

# uncommit + unstage + delete last changes, nothing left
~$ git reset --hard HEAD~1

# prune/cleanup the local references to remote branch
~$ git remote prune origin

# show remote
~$ git ls-remote --get-url

# change remote
~$ git remote set-url origin https://github.wdf.sap.corp/I516530/sonar-scanner-cli-docker

# clean local branches
~$ git remote prune origin

# save changes temporarily
~$ git stash
~$ git stash pop

Javascript DOM

// 1.
document.querySelectorAll('a.storylink')    // return a `NodeList`
document.querySelector('a.storylink')       // return the first matched `Element`
document.getElementById('hnmain')           // return `Element`
document.getElementsByClassName('storylink')// return `HTMLCollection`
 
// 2.
// The `NodeList` returned by document.querySelectorAll() is not live.
 
// 3. Array.from() is a unified way to iterate `NodeList` and `HTMLCollection`
const cs = document.getElementsByClassName('storylink')
Array.from(cs).forEach(x => console.log(x))
const as = document.querySelectorAll('a.storylink')
Array.from(as).forEach(x => console.log(x))
 
// 4. get object's type
//   http://javascriptweblog.wordpress.com/2011/08/08/fixing-the-javascript-typeof-operator/
const toType = function(obj) {
  return ({}).toString.call(obj).match(/\s([a-zA-Z]+)/)[1].toLowerCase()
}

SQL statement

-- group by
SELECT comment_type, Count(*), Max(created_at) FROM comments GROUP BY comment_type; 

-- child query
SELECT *
FROM   jobs 
WHERE  work_contract_id IN (SELECT id 
                            FROM   work_contracts 
                            WHERE  video_project_id = 52195); 

Postgres DB

# check client version
~$ psql --version
 
# connect to Protgres DB
~$ psql --host=127.0.0.1 --port=5432 --username=postgres --dbname=vaas_development --password
 
# use database
postgres=# \c vaas_development
 
# l = show databases
vass_development=> \l
 
# dt = show tables
vass_development=> \dt
 
# check version
select version();

# get all databases in the server
select datname from pg_database;

# get current DB in use
select current_database();

# list the talbes in current DB.
select table_name from information_schema.tables where table_schema = 'public' order by table_name;
 
# check sql execution time
select pid, usename, query, age(clock_timestamp(), query_start) from pg_stat_activity where state != 'idle';

Ubuntu

# check service status
~$ service postgresql status
~$ service redis status

# check port is occupied by which process
~$ lsof -i :3808

Mac Facts

• macOS 14.5 Sonoma
• macOS 11.* Big Sur
• macOS 10.15.*, with code name Catalina.
• macOS 10.14.*, with code name Mojave.
• Software must be 64-bit from Catalina.
• Xcode is an integrated development environment(IDE) for for developing software for macOS, iOS, iPadOS, watchOS and tvOS.
• Xcode 11.* is the latest stable version.
• The Command Line Tools Package is a small self-contained package separately from Xcode. It consists of the macOS SDK and command-line tools such as gcc, clang etc.
• You could have multiple Xcode installed, just need to pick a new location.
• You could use xcode-select to switch between the different Xcode versions.

# install command line tool
~$ xcode-select --install

# print path
~$ xcode-select -p
# => /Library/Developer/CommandLineTools

# check tools installed
~$ ls /Library/Developer/CommandLineTools/usr/bin
# => gcc, clang, python, swift, make etc

# check macOS SDKs installed
~$ ls /Library/Developer/CommandLineTools/SDKs
# => MacOSX10.15.sdk and MacOSX10.14.sdk

VS Code

Key binding.

Command + i             # inline copilot
Control + Command + i   # copilot chat view
Shift + Command + e     # vscode explorer
Shift + Command + p     # command palette

Control + `             # terminal
Control + 1             # editor 1
Control + #             # editor #

Control + p             # search file

Zed

Command + B             # 开关左侧Panel
Command + R             # 开关右侧Panel (AI Assistant Pannel)
Command + J             # 开关底部Panel

Command + Shift + p     # 打开command palette
Command + Shift + v     # 预览markdown格式的内容

Homebrew

# check installed packages
~$ brew leaves

# check package details
~$ brew info dnsmasq

# check installed path of nvm(or any other apps)
~$ brew --prefix nvm

ADB

# list all devices connected
~$ adb devices

# enter into shell of device
~$ adb shell

# list all packages
16th:/ pm list package

# list only 3rd party packages
16th:/ pm list packages -3

# check App details, e.g. installed time, permission granted, location etc.
16th:/ dumpsys package package_name

# grant permission
16th:/ pm grant sg.gov.tech.bluetrace android.permission.CAMERA

# revoke permission
16th:/ pm revoke sg.gov.tech.bluetrace android.permission.CAMERA

# start App with package name
16th:/ monkey -p com.meizu.safe 1

# uninstall specific package. *No* need of root priviledge
16th:/ pm uninstall --user 0 com.meizu.media.music

# install back the package which been uninstalled before
16th:/ cmd package install-existing com.meizu.media.music

# get android id
16th:/ settings get secure android_id

# use logcat to check App's logs
# from the output, can filter `Displayed` to find out appPackage and appActivity
16th:/ logcat

# execute command inside devices
~$ adb shell ls

Docker

# build docker image
~$ docker build -t test_controller  -f controller.Dockerfile .

# start docker container
# --rm: remove container when it exits
~$ docker run -td --rm test_controller

# start docker container, but override original entrypoint and keep it running
# it needs to follow `docker run --entrypoint <command> <docker-image> <param1> <param2> ...` format
~$ docker run --entrypoint tail test_controller -f /dev/null

# list all docker containers
~$ docker ps -a

# ssh to docker container
~$ docker exec -it <docker container id> bash

# remove all images without at least one container associated to them
~$ docker image prune -a

# remove all stopped containers
~$ docker container prune

# remove all volumes not used by at least one container
~$ docker volume prune

# remove unused data
~$ docker system prune

# list all docker networks
~$ docker network ls

# check containers inside a network
~$ docker network inspect <network id>

# with --build-arg, you can refer to the values in dockerfile by `ARG SOME_VARIABLE`
~$ docker build --build-arg SOME_VARIABLE=a_value -t test_controller  -f controller.Dockerfile .

Dockerfile

# LTS version by default
FROM ubuntu

RUN apt update
RUN apt install -y curl git-core vim unzip iproute2

Using Entrypoint as command, while CMD as default arguments.

E.g.

# ...
ADD fortuneLoop.js /bin/fortuneLoop.js
ENTRYPOINT ["node", "/bin/fortuneLoop.js"]
CMD ["10"]

Then could

# run directly as default setting
~$ docker run -it <image>

# or passing in arguments
~$ docker run -it <image> 15

Overriding both command and arguments in Kubernetes (mostly cases only need to override arguments)

apiVersion: v1
kind: Pod
spec:
  containers:
    - image: some/image
      command: ["node", "/bin/command"]
      args: ["arg1"]

Kubernetes

After some experience, I've found MicroK8s is way better than minikube for experimentation.

# get current default namespace.
# if the output is empty, the default namespace is `default`.
~$ kubectl config view --minify --output 'jsonpath={..namespace}'

# management multiple clusters
# 1. `mkdir ~/.kube/clusters` and move all the config files in `clusters` folder.
# 2. Put following line to `.zshrc`. The awk part is to replace new line to ':'.
# export KUBECONFIG=$(find ~/.kube/clusters -type f | awk '{printf "%s:",$0} END {print ""}')
~$ kubectl config get-clusters
~$ kubectl config use-context minikube
~$ kubectl config delete-cluster minikube

# set default namespace
~$ kubectl config set-context --current --namespace=<name>
# get contexts for all clusters
~$ kubectl config get-contexts

# ssh to pod
~$ kubectl exec -it <pod-name> -- bash

# port forward
~$ kubectl port-forward <pod-name> <local-port>:<pod-port>

# add local image into cache to use later in k8s.
~$ minikube image load docker_image
~$ minikube image list
~$ minikube image rm docker_image

Argocd

~$ argocd login argocd-apirule.c-5707fc8.stage.kyma.ondemand.com:443 --username admin
~$ argocd cluster list
~$ argocd proj list

# place where defined initial admin password(base64 encoded).
~$ kubectl -n argocd get secret argocd-initial-admin-secret -o yaml

# argocd-cm is the default place to define users.
~$ kubectl -n argocd get configmap argocd-cm -o yaml

# argocd-rbac-cm is the default place to define roles.
~$ kubectl -n argocd get configmap argocd-rbac-cm -o yaml

Batocera

我买的零刻SER8用来安装Batocera。电脑上的两个C口都不能接显示器。

键盘使用的是狼蛛F75，注意：Fn + F7 才是真正的F7键（使用F7键进入启动项选择）。键盘最右侧的Delete键是进入BIOS的键（不需要配合Fn键）。可以使用Google搜索keyboard test online，找到可以查看按键的网页。

使用8bitdo N30 Pro的Xinput连接Batocera的蓝牙。手柄连接设备时，说明书上写的是start+x键，但是按的时候，要先按住x，再按start。有时Batocera搜索不到设备，应该是Batocera的问题，可以把蓝牙关闭再打开。

使用Batocera的U盘安装系统时，依然会出现连接不上WIFI的情况，是由于我家的WIFI名字中有特殊字符。装好系统后的WIFI模块不会有这个问题。

System Settings -> Developer Options -> Swap function A and B. 这个可以把手柄按键设置成switch的模式，即B确定，A退出。

关于ROM的游戏格式：

1. GameCube/Wii -> RVZ. Being 100% reversible to ISO.
2. Disc Based Games -> CHD. E.g. PS1 & 2, Sega Dreamcast, Sega Saturn. Being 100% reversible to ISO.