Stana mathsigit

function docker_container_ip() {
  container_ids=$(docker ps -aq)
  for cid in ${container_ids}
  do
    info=$(docker inspect -f 'name={{.Name}}  ip={{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' ${cid})
    echo -e "$cid \t $info"
  done
}

如何在 kafka connector 的 Restful Server 建立 https

1.使用 openssl 建立 SSL key, 整個 script 如下

#!/bin/bash
#Step 1
keytool -keystore kafka.server.keystore.jks -alias localhost -validity 365 -genkey
#Step 2
openssl req -new -x509 -keyout ca-key -out ca-cert -days 365
keytool -keystore kafka.server.truststore.jks -alias CARoot -import -file ca-cert

Something about Pyenv On MacOS

How to install pyenv

Installing pyenv

#Via homebrew
brew update
brew install pyenv

Intellij on jetbrains - Ultimate feature

How to setup, to see how-can-pyspark-be-called-in-debug-mode - stackoverflow

Virtualenv

Commands

Create virtualenv

virtualenv -p ${PYTHON_VERSION} ${ENV_FOLDER_PATH}

#For instance

Minikube

Install

macOS

Before you begin

To check if virtualization is supported on macOS, run the following command on your terminal.

Spark On K8s

Requirement

spark 2.4.7 or 3.0.1
minikube 1.15.3 or 1.19
kubectl 1.19

Checking the compatibility matrix of Kubernetes Client: https://github.com/fabric8io/kubernetes-client/blob/master/README.md#compatibility-matrix

The Best of the Best Practices (BOBP) Guide for Python

A "Best of the Best Practices" (BOBP) guide to developing in Python.

In General

Values

"Build tools for others that you want to be built for you." - Kenneth Reitz
"Simplicity is alway better than functionality." - Pieter Hintjens