blockspacer/Istio_cpp.md

Last active October 11, 2022 11:29

Star (1) You must be signed in to star a gist
Fork (0) You must be signed in to fork a gist

Select an option

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/blockspacer/0237dcdea981f78f8c3ea80b8b1d037d.js"></script>
Save blockspacer/0237dcdea981f78f8c3ea80b8b1d037d to your computer and use it in GitHub Desktop.

Download ZIP

Istio C++

Raw

Istio_cpp.md

About Docker

https://github.com/katopz/best-practices#docker

About Kubernetes

pods are the base unit of operation for Kubernetes. Pods are the most visible, viable, and ephemeral units that comprise one or more tightly coupled containers. Kubernetes does not operate at the level of containers. There can be multiple pods in a single server node and data sharing easily happens in between pods. Kubernetes automatically provision and allocate pods for various services. Each pod has its own IP address and shares the localhost and volumes.
Labels are typically the metadata that is attached to objects, including pods.
Services offer a low-overhead way to route all kinds of service requests to a set of pods to accomplish the requests. Read:
https://docs.google.com/document/d/1E03-g0h3MgFlohXqPNYCjl7Pv_CA_l0zhD6K3MZ8O0M/edit?usp=sharing
https://www.freecodecamp.org/news/learn-kubernetes-in-under-3-hours-a-detailed-guide-to-orchestrating-containers-114ff420e882/
https://neuvector.com/network-security/advanced-kubernetes-networking/
https://layer5.io/landscape/
https://www.tfir.io/2019/05/17/whats-the-right-ingress-controller-for-my-kubernetes-environment/

About Service Mesh

https://www.slideshare.net/dbryant_uk/microxchg-2018-what-is-a-service-mesh-do-i-need-one-when-developing-cloud-native-microservices

About Cloud-enabled apps

The currently running massive and monolithic applications get modernized and migrated to cloud environments to reap the distinct benefits of the cloud paradigm

About Cloud-native apps

This is all about designing, developing, debugging, delivering, and deploying applications directly on cloud environments by intrinsically leveraging the non-functional capabilities of cloud environments

About microservices architecture (MSA)

Microservices are the architecture paradigm wherein a monolithic (silo) application is decomposed into small tiny micro applications which are packaged and deployed independently. Read:

Microservice design principles

High cohesion among services: A microservice should have one single focus and the sole responsibility for that action. It should not change as a result of other related services. Services should be easily rewritable so that we can achieve scalability, reliability, and flexibility. It should handle a single business function and domain-specific functionality.
Autonomous service: A service should independently handle its work without the help of any other services. It should not be tightly integrated with any other service; it should remain loosely coupled in nature. By autonomous, we mean that a microservice should not change because of the external components with which it interacts. Autonomous services honor contracts and interfaces. They should be stateless, independently changeable, independently deployable, backwards compatible, and they should support concurrent development.
Business domain-centric service: Each individual service should perform or represent a single business function. This could be a calculation of sales, tax, income tax, or any other function related to a specific area. Each service should bound or define its scope. Business-centric code can help to provide more cohesion and make services more responsive to handle any changes in the domain or business logic requirements.
Resilience: Resilience is a standard these days when providing a service to a customer. Failure to provide resilience may result in another endpoint not providing a response to your microservices. Designing your service in microformat helps to overcome failure. Our service should register itself during startup and de-register itself upon failure. This should be part of a dynamic discovery service, such as the auto-creation of a queue or the auto-removal of the queue in a message queue. There could be a number of problems or exceptions that a network-based service could encounter. It should be able to handle delays and the unavailability of another service.
Observable service or functionality: Observability is another important design principle while working on distributed microservices. When a complex interconnected service breaks, it can take hours or days to isolate issues. We should design our services in such a way that we can check the health of any service by either showing its status on a health page or by sending it to a central logging service such as Splunk, Logstash, syslogd, Logentries, Datadog, or Sumo Logic. Observability is required to support reliable, scalable, and cost-effective services and metrics to scale up, metrics to scale down, and metrics to alert the team. This kind of monitoring and logging needs to be located at a central place. In a containerized environment, auto deployment should be able to auto-detect when a deployment fails so that it can be rolled back quickly to an older running version. Observability can be related to CPU usage, memory usage, network input/output metrics, disk metrics, the number of connections to a service, and so on. All these metrics are easily available and measurable through tools such as Check_MK, Nagios, New Relic, AppDynamics, StatsD, and Graphana.
Observability not only helps in terms of providing a technical solution but also so that we can identify business decision-making, like the sales of a specific service or the returns for a specific product. Automation: Microservices also create challenges for an operation team with regard to deployment, verifying functionality, and performing various types of testing. There is now a wide range of automation tools available on market that can easily be integrated to achieve automated deployment, verification, testing, failure, and rollbacks. Some of the famous tools are Jenkins; Teamcity; Bamboo; Git workflow plugins; GitLab CI/CD; UI test tools, such as Selenium, PhantomJS, Nightwatch, BrowserStack; and many more. One important point here is that while Docker changed the container market when it was developed, it was hard to implement in a production environment, where a complete stack is required to maintain it as a production-level service. There wasn't much clarity with regard to monitoring or deployment. After that, Google released Borg in the form of Kubernetes and changed the container market again by providing easy deployment and rollback options with easy service and routing functionalities that were perfect for production-grade deployments.

NOTE:

Microservices prefer letting each service manage its own database The key point it that the services should have no knowledge of each other's underlying database. https://github.com/katopz/best-practices/blob/master/best-practices-for-building-a-microservice-architecture.md#service-essentials-2
Services communicate using either synchronous protocols such as HTTP/REST or asynchronous protocols such as AMQP.
Services can be developed and deployed independently of one another.
In the microservices world, the network is your biggest point of failure. In production you will fail if too much connections between. https://medium.com/@oprearocks/blasphemy-multiple-microservices-shared-database-f525025a8a81
'micro' part doesn't mean replace every class with a network service, but componentize a monolithic application into sensibly sized components, each one dealing with an aspect of your program.
use In-memory service data

Install curl with https support

git clone https://github.com/bagder/curl.git
# requires https://askubuntu.com/a/826891
sudo apt-get build-dep curl
cd curl
./buildconf
./configure --with-ssl --disable-shared
make -j8
# sudo make uninstall
sudo make install 
curl -V
# Features: ... SSL ... TLS-SRP ...

Install VirtualBox

Follow https://www.virtualbox.org/wiki/Downloads

Install docker

Follow https://phoenixnap.com/kb/how-to-install-docker-on-ubuntu-18-04

Or under proxy, see https://gist.github.com/blockspacer/893b31e61c88f6899ffd0813111b3e41

Install kubectl

Follow https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl-on-linux

curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin/kubectl
kubectl version --client

sudo apt-get update && sudo apt-get install -y apt-transport-https
curl -s http://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubectl

Install kubeadm

Follow https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/

curl -SSL https://dl.k8s.io/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubeadm > ./kubeadm
chmod a+rx kubeadm
sudo mv kubeadm /usr/bin/kubeadm
kubeadm version

Under proxy - setup Kubernetes The Hard Way

If under proxy - See https://wiki.christophchamp.com/index.php?title=Kubernetes/the-hard-way

Install Minikube

Follow https://kubernetes.io/docs/setup/learning-environment/minikube/#installation (For Windows: https://meteatamel.wordpress.com/2018/02/14/minikube-on-windows/)

curl -Lo minikube http://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 \
  && chmod +x minikube
sudo install minikube /usr/local/bin
minikube status
minikube stop
# OR minikube delete
minikube start --alsologtostderr --kubernetes-version v1.12.10 --memory=12288 --cpus=2 --disk-size 25GB --vm-driver virtualbox \
  --extra-config='apiserver.enable-admission-plugins=LimitRanger,NamespaceExists,NamespaceLifecycle,ResourceQuota,ServiceAccount,DefaultStorageClass,MutatingAdmissionWebhook' \
  --extra-config=apiserver.authorization-mode=RBAC \
  --insecure-registry='localhost' \
  --insecure-registry='127.0.0.1' \
  --insecure-registry "192.168.39.0/24"
  
# OPTIONAL: open dashboard
minikube addons enable dashboard && kubectl get pods --all-namespaces | grep dashboard && sleep 15 && minikube dashboard

kubectl config use-context minikube

NOTE: minikube start --alsologtostderr --memory=16384 --cpus=4 --disk-size 25GB with 16384 MB of memory and 4 CPUs. 16384 MB is sufficent to run Istio and bookinfo.

# (Optional, recommended) If you want minikube to provide a load balancer for use by Istio, you can use the minikube tunnel feature. Run this command in a different terminal, because the minikube tunnel feature will block your terminal to output diagnostic information about the network:

minikube tunnel

# Sometimes minikube does not clean up the tunnel network properly. To force a proper cleanup:

minikube tunnel --cleanup

NOTE: Use may want to run minikube delete before minikube start

NOTE: you can set proxy by --docker-env:

minikube start --alsologtostderr --kubernetes-version v1.12.10 --memory=12288 --cpus=2 --disk-size 25GB --vm-driver virtualbox \
  --extra-config='apiserver.enable-admission-plugins=LimitRanger,NamespaceExists,NamespaceLifecycle,ResourceQuota,ServiceAccount,DefaultStorageClass,MutatingAdmissionWebhook' \
  --extra-config=apiserver.authorization-mode=RBAC \
  --insecure-registry='localhost' \
  --insecure-registry='127.0.0.1' \
  --insecure-registry "192.168.39.0/24" \
  --docker-env http_proxy=http://172.17.0.1:3128 \
  --docker-env https_proxy=http://172.17.0.1:3128 \
  --docker-env no_proxy=192.168.99.0/24,$(minikube ip),localhost,127.0.0.*,10.*,192.168.*,*.corp1.ru,*.corp.ru \
  --docker-env HTTP_PROXY=http://172.17.0.1:3128 \
  --docker-env HTTPS_PROXY=http://172.17.0.1:3128 \
  --docker-env NO_PROXY=192.168.99.0/24,$(minikube ip),localhost,127.0.0.*,10.*,192.168.*,*.corp1.ru,*.corp.ru
  
# add corp certs to ~/.minikube/files/etc/ssl/certs
cp -r /etc/ssl/certs/* ~/.minikube/files/etc/ssl/certs
cp -r /usr/share/ca-certificates/* ~/.minikube/files/etc/ssl/certs
# cert files must have 750 permission
sudo chmod 750 ~/.minikube/files/etc/ssl/certs/*.crt

Now you can ssh into minikube and change daemon.json according to your proxy

minikube ssh
sudo mkdir -p /etc/docker
# NOTE: can't set "insecure-registries" due to `--insecure-registry` minikube arg
sudo tee /etc/docker/daemon.json <<-'EOF'
{
    "dns": ["127.0.0.53", "10.8.13.11", "10.8.13.12", "8.8.4.4", "8.8.8.8", "10.8.13.11", "10.8.13.12"],
    "registry-mirrors":["https://docker.mirrors.ustc.edu.cn"],
    "log-driver": "json-file",
    "log-opts": {
      "max-size": "50m",
      "max-file": "3"
    }
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker
# test pull under minikube
# docker pull grafana/grafana:6.4.3
exit

Enable minikube addons

# see http://rastko.tech/kubernetes/2019/01/01/minikube-on-mac.html
minikube addons enable ingress

# Use `--insecure-registry='192.168.39.0/24'`
# see https://minikube.sigs.k8s.io/docs/tasks/docker_registry/
minikube addons enable registry

Prepere minikube for istio

Follow on linux https://istio.io/docs/setup/platform-setup/minikube/

# monitor memory usage in minikube
minikube ssh -- sudo top

Install istioctl

Follow on linux https://istio.io/docs (For Windows: https://github.com/MicrosoftDocs/azure-docs/blob/master/articles/aks/istio-install.md#windows)

curl -L https://git.io/getLatestIstio | sh -
cd istio*
echo 'export PATH=$(pwd)/bin:$PATH' >> ~/.bashrc
# OR 
# sudo cp ./bin/istioctl /usr/local/bin/istioctl
# sudo chmod +x /usr/local/bin/istioctl
istioctl version

Istio releases http://gcsweb.istio.io/gcs/istio-release/releases/1.4.3/

Apply istio profile

Follow https://istio.io/docs/setup/install/istioctl/

istioctl manifest apply --skip-confirmation
# OR
# istioctl manifest apply --set values.global.mtls.enabled=true,values.security.selfSigned=false --set values.global.controlPlaneSecurityEnabled=true

# see https://istio.io/docs/setup/getting-started/
kubectl get svc -n istio-system

# ensure corresponding Kubernetes pods are deployed and have a STATUS of Running
kubectl get pods -n istio-system

Make sure that all istio-system pods are running or completed before continuing. This can take several minutes when starting the pods for the first time. Be patient.

enable automatic sidecar injection:

kubectl label namespace default istio-injection=enabled

if you want to exclude a specific pod from getting istio sidecar injected, add this to Deployment kind

metadata:
  annotations:
  sidecar.istio.io/inject: "false"

Add istioctl to PATH

(see https://github.com/IBM/istio101/blob/master/workshop/exercise-2/README.md or https://docs.google.com/document/d/1Qo8o5C4UpGwMF7Mg02kLTaU4-xCSfJjLcnIFNveMEEA/edit?usp=sharing)

for i in install/kubernetes/helm/istio-init/files/crd*yaml; do kubectl apply -f $i; done
kubectl apply -f install/kubernetes/istio-demo.yaml
kubectl get svc -n istio-system
kubectl get pods -n istio-system

NOTE: How to build docker images which will be seen by Kubernetes directly without having to push them anywhere

just run

# NOTE: Later, when we no longer wish to use the Minikube host, we can undo this change by running: eval $(minikube docker-env -u)
eval $(minikube docker-env)

and now you can build docker images which will be seen by Kubernetes directly without having to push them anywhere.

Open istio dashboards

istioctl dashboard   controlz    # Open ControlZ web UI
istioctl dashboard   envoy       # Open Envoy admin web UI
istioctl dashboard   grafana     # Open Grafana web UI
istioctl dashboard   jaeger      # Open Jaeger web UI
istioctl dashboard   kiali       # Open Kiali web UI
istioctl dashboard   prometheus  # Open Prometheus web UI
istioctl dashboard   zipkin      # Open Zipkin web UI

See https://istio.io/docs/reference/commands/istioctl/#istioctl-dashboard

NOTE: How to run Jaeger Dashboard

# in Istio >= 1.4.x
istioctl dashboard jaeger

OR if Istio < 1.4.x:

kubectl port-forward -n istio-system $(kubectl get pod -n istio-system -l app=jaeger -o jsonpath='{.items[0].metadata.name}') 16686:16686

URL to Open Jaeger: http://localhost:16686

NOTE: How to run Grafana Dashboard

# in Istio >= 1.4.x
istioctl dashboard grafana

OR if Istio < 1.4.x:

kubectl -n istio-system port-forward $(kubectl -n istio-system get pod -l app=grafana -o jsonpath='{.items[0].metadata.name}') 3000:3000 &

URL to open Grafana: http://localhost:3000/dashboard/db/istio-mesh-dashboard

NOTE: How to run Prometheus Dashboard

# in Istio >= 1.4.x
istioctl dashboard prometheus

OR if Istio < 1.4.x:

kubectl -n istio-system port-forward $(kubectl -n istio-system get pod -l app=prometheus -o jsonpath='{.items[0].metadata.name}') 9090:9090 &

URL to open Prometheus: http://localhost:9090

NOTE: How to run Kiali

Kiali is Istio’s dashboard and this is one of the coolest features in 1.4.x: To open the Kiali dashboard you no longer need to execute complicated port-forwarding commands, simply type

# in Istio >= 1.4.x
istioctl dashboard kiali

OR if Istio < 1.4.x:

Run the following command to install Kiali:

bash <(curl -L http://git.io/getLatestKialiKubernetes)

Note: For some reason the script didn't work for me. I had to replace one line:

get_downloader
github_api_url="https://api.github.com/repos/kiali/kiali/releases/latest"
kiali_version_we_want="v0.15.0"

To launch Kiali you need the IP address and NodePort:

minikube ip
kubectl get svc -n istio-system kiali --output 'jsonpath={.spec.ports[*].nodePort}'

URL to open Kiali: https://[minikube-ip]:[kiali-nodeport]/kiali

Run `istioctl verify-install`

istioctl verify-install

Dev tools

Install nodejs

# sudo apt remove node npm
# sudo rm /usr/local/bin/node # must be removed
# sudo rm /usr/local/bin/npm # must be removed
# sudo rm -rf /usr/local/lib/node_modules # must be removed
OS_ARCH=x64 # $(uname -m)
NODE_V=v10.18.1
wget https://nodejs.org/dist/$NODE_V/node-$NODE_V-linux-$OS_ARCH.tar.gz
tar -xvf node-$NODE_V-linux-$OS_ARCH.tar.gz
cd node-$NODE_V-linux-$OS_ARCH
sudo cp -R * /usr/local/
sudo chown -R $USER /usr/local/lib/node_modules
cd -
# npm install npm -g # optional
node -v
npm -v

Under proxy: you may want to configure ~/.npmrc like so https://stackoverflow.com/a/36929934

# NOTE: you may want to use NODE_TLS_REJECT_UNAUTHORIZED=0 under proxy during `npm install`
NODE_TLS_REJECT_UNAUTHORIZED=0 \
  HTTP_PROXY=http://127.0.0.1:8088 \
  HTTPS_PROXY=http://127.0.0.1:8088 \
    npm install \
      --unsafe-perm binding

Install protobuf from sources https://developers.google.com/protocol-buffers/docs/downloads and (if exists) remove old protobuf version apt-get remove libprotobuf-dev

NOTE: it is better to clone https://github.com/grpc/grpc/ repo and build protobuf from grpc/third_party/protobuf

Tested with GRPC_RELEASE_TAG=v1.26.x

python -V # Python 2.7 or newer
sudo apt-get install autoconf automake libtool curl make g++ unzip
git clone https://github.com/protocolbuffers/protobuf.git
cd protobuf
git submodule update --init --recursive
./autogen.sh
./configure --prefix=/usr
make
make check
sudo make install
sudo ldconfig # refresh shared library cache.
protoc --version

Install Protocol Buffers for Go https://github.com/golang/protobuf#installation

Install grpc (requres protobuf) https://github.com/grpc/grpc/blob/master/BUILDING.md

Install Helm https://helm.sh/docs/intro/install/

mkdir /tmp/helm

cd /tmp/helm

helm_version=v3.0.2-linux-amd64

wget https://get.helm.sh/helm-$helm_version.tar.gz
 
tar zxvf helm-$helm_version.tar.gz
 
sudo mv linux-amd64/helm /bin/helm

cd -

helm version

helm repo add stable https://kubernetes-charts.storage.googleapis.com

Install Tiller (the Helm server-side component) into the Kubernetes cluster

see https://github.com/ruzickap/k8s-istio-demo#install-helm

Install Rook Operator (Ceph storage for k8s):

see https://github.com/ruzickap/k8s-istio-demo#install-rook

Install ElasticSearch, Kibana, FluentBit

see https://github.com/ruzickap/k8s-istio-demo#install-elasticsearch-kibana-fluentbit

Check prerequisites

function _out() {
  echo "$(date +'%F %H:%M:%S') $@"
}

function checkPrerequisites() {
    MISSING_TOOLS=""
    git --version &> /dev/null || MISSING_TOOLS="${MISSING_TOOLS} git"
    curl --version &> /dev/null || MISSING_TOOLS="${MISSING_TOOLS} curl"
    which sed &> /dev/null || MISSING_TOOLS="${MISSING_TOOLS} sed"
    docker -v &> /dev/null || MISSING_TOOLS="${MISSING_TOOLS} docker"
    unzip -version &> /dev/null || MISSING_TOOLS="${MISSING_TOOLS} unzip"
    kubectl version --client=true &> /dev/null || MISSING_TOOLS="${MISSING_TOOLS} kubectl"
    minikube version &> /dev/null || MISSING_TOOLS="${MISSING_TOOLS} minikube"
    if [[ -n "$MISSING_TOOLS" ]]; then
      _out "Some tools (${MISSING_TOOLS# }) could not be found, please install them first"
      exit 1
    else
      _out You have all necessary prerequisites installed
    fi
    if ! kubectl describe namespace default | grep istio-injection=enabled > /dev/null ; then
       _out "Istio automatic sidecar injection needs to be enabled. See documentation/SetupLocalEnvironment.md"
    fi   
}

checkPrerequisites

Create app

https://blogs.vmware.com/opensource/2019/04/16/implementing-grpc-web-istio-envoy/ https://github.com/lucperkins/colossus/blob/1a487d9768d80094939a8171645227b64fd76b67/userinfo/userinfo-server.cc https://grpc.io/blog/state-of-grpc-web/ https://venilnoronha.io/seamless-cloud-native-apps-with-grpc-web-and-istio & https://github.com/venilnoronha/grpc-web-istio-demo https://docs.google.com/document/d/1Qo8o5C4UpGwMF7Mg02kLTaU4-xCSfJjLcnIFNveMEEA/edit?usp=sharing & https://github.com/saturnism/istio-by-example-java https://github.com/grpc/grpc/tree/master/examples/cpp/load_balancing https://istiobyexample.dev/grpc/

Deploy the app to minikube

https://github.com/IBM/istio101/blob/master/workshop/exercise-3/README.md https://meteatamel.wordpress.com/2018/04/24/istio-101-with-minikube/

https://dzone.com/articles/service-mesh-with-istio-on-kubernetes-in-5-steps

Deploy the app to GKE

https://medium.com/@at_ishikawa/getting-started-to-develop-a-web-application-on-gke-with-istio-for-grpc-web-1d84e084fb01

TODO: RBAC

https://rinormaloku.com/authorization-in-istio/

TODO: CI/CD

https://github.com/castlemilk/kubernetes-cicd/tree/8505444338567cfaad5ab61496ad14da4f578657

TODO: read gameontext

https://gameontext.gitbooks.io/gameon-gitbook/content/microservices/

TODO: vagrant

https://github.com/gameontext/gameon/blob/master/Vagrantfile#L62

TODO: read CoolStore

https://vietnam-devs.github.io/coolstore-microservices/#coolstore-website

TODO: websockets

TODO: Running on Google Kubernetes Engine (GKE)

See https://github.com/GoogleCloudPlatform/microservices-demo#option-2-running-on-google-kubernetes-engine-gke

blockspacer commented Feb 26, 2020 •

edited

Loading

Author

Search query: "keycloak" "grant_type=authorization_code" "redirect_uri"

https://ria.ua.pt/bitstream/10773/23555/1/Disserta%C3%A7%C3%A3o.pdf

The Authorization Code Grant consists of the following calls (these examples came from the OAuth2 spec): https://community.apigee.com/articles/37103/saml2-vs-jwt-understanding-oauth2.html
keycloak "response_type=code"
https://www.stefaanlippens.net/oauth-code-flow-pkce.html
istio K8S keycloak
https://github.com/geoffroyvergne/devops/tree/5a887ad5c05300e3c5d087c38337c27b0afcdb23/k8s/deployments/keycloak
istio keycloak
https://github.com/kameshsampath/istio-keycloak-demo/blob/master/openshift-files/keycloak.yaml
BOOK
Securing the Perimeter: Deploying Identity and Access Management with Free Open Source Software

export MY_IP=$(ip route get 8.8.8.8 | sed -n '/src/{s/.*src *\([^ ]*\).*/\1/p;q}')

# see https://www.ivonet.nl/2015/05/23/Keycloak-Docker/

# Data volume
docker pull busybox:1.31.1
docker run --name postgres-data -v /var/lib/postgresql/data busybox:1.31.1 true

docker pull postgres:9.6.17
# Postgres coupled to the datavolume
(docker rm postgres_9 || true)
docker run -d --name postgres_9 -p 15432:5432 --volumes-from postgres-data -e POSTGRES_DB=keycloak_db -e POSTGRES_DATABASE=keycloak_db -e POSTGRES_USER=keycloak -e POSTGRES_PASSWORD=password -e POSTGRES_ROOT_PASSWORD=password postgres:9.6.17
docker logs -f postgres_9

# CHECK:
# sudo add-apt-repository "deb https://apt.postgresql.org/pub/repos/apt/ trusty-pgdg main"
# sudo apt-get update
# sudo apt install postgresql-client-common postgresql-client-10 -y
# psql -h localhost -p 15432 -U keycloak keycloak_db
# Ctrl+D

cd to certs dir

# MUST EXIST
file httpbin.example.com.crt
file httpbin.example.com.crt

# Keycloak server image linking to the postgres image
docker pull jboss/keycloak:9.0.0
docker run --name keycloak-data -v /var/lib/keycloak/data busybox:1.31.1 true
export MY_IP=$(ip route get 8.8.8.8 | sed -n '/src/{s/.*src *\([^ ]*\).*/\1/p;q}')
(docker rm keycloak || true)
docker run --rm -it -p 8443:8443 -p 8080:8080 --name keycloak --memory="512m" --link postgres_9 --volumes-from keycloak-data \
    -e PROXY_ADDRESS_FORWARDING=true \
    -e DB_VENDOR=postgres -e DB_ADDR=$MY_IP:15432  -e DB_PORT=15432 \
    -e DB_DATABASE=keycloak_db -e DB_SCHEMA=public \
    -e DB_USER=keycloak -e DB_PASSWORD=password \
    -e KEYCLOAK_USER=admin -e KEYCLOAK_PASSWORD=admin_pass \
    -e KEYCLOAK_HTTPS_PORT=8443  -e KEYCLOAK_HTTP_PORT=8443 \
    -e KEYCLOAK_HOSTNAME=keycloak.example.com \
    -v httpbin.example.com.crt:/etc/x509/https/tls.crt \
    -v httpbin.example.com.crt:/etc/x509/https/tls.key \
    jboss/keycloak:9.0.0

        # Uncomment the line below if you want to specify JDBC parameters. The parameter below is just an example, and it shouldn't be used in production without knowledge. It is highly recommended that you read the PostgreSQL JDBC driver documentation in order to use it.
        # -e JDBC_PARAMS="ssl=true"

Add to host /etc/hosts file:
```bash
sudo -- sh -c -e "echo '127.0.0.1 keycloak.example.com' >> /etc/hosts";
sudo -- sh -c -e "echo '127.0.0.1 example.com' >> /etc/hosts";
cat /etc/hosts

Add to minikube /etc/hosts file:

export MY_IP=$(ip route get 8.8.8.8 | sed -n '/src/{s/.*src *\([^ ]*\).*/\1/p;q}')
minikube ssh -- sudo tee -a /etc/hosts <<EOF
127.0.0.1       localhost
127.0.1.1       minikube
127.0.0.1 example.com
$MY_IP keycloak.example.com
EOF
minikube ssh -- sudo cat /etc/hosts
minikube ssh -- ping keycloak.example.com

Open https://keycloak.example.com:8443/auth/admin/

Set sslRequired in realm settings

Navigate to https://keycloak.example.com:8443/auth/admin/master/console/#/realms/master/token-settings
Default Signature Algorithm RS256
Set Access Token Lifespan 60 min
Set Client login timeout 60 min

Navigate to https://keycloak.example.com:8443/auth/admin/master/console/#/realms/master/clients
~~OLD~~TODO~set up a client "pkce-test" (in the "master" realm) with access type "public"~~~
set up a client "pkce-test" (in the "master" realm) with access type "confidential"
set protocol to openid-connect
set Root URL to app url, like http://dev.local/
enable options Direct Access Grants and Authorization in the Settings section of newly created client. Also Access Type should be set to confidential
(to be able to use PKCE) and the "http://dev.local/*" as valid redirect URIs (which is a required field).
NOTE: replace dev.local with you app url
set Secret available under Credentials tab from Client section as "client_secret" in configmap

create a user "john_username" in https://keycloak.example.com:8443/auth/admin/master/console/#/realms/master/users with non-temporary password (credentials tab).

provider = "https://keycloak.example.com:8443/auth/realms/master"
username = "john_username"
password = "j000hn_pass" (not Temporary)
Remove Update password from Required User Actions

TODO: Create code verifier and challenge as in https://www.stefaanlippens.net/oauth-code-flow-pkce.html#PKCE-code-verifier-and-challenge

Open https://keycloak.example.com:8443/auth/realms/master/.well-known/openid-configuration to see endpoints

Open https://keycloak.example.com:8443/auth/realms/master/protocol/openid-connect/certs
copy it and replace " with "
set as "jwks" in configmap (yaml file, on istio side)

Follows
https://medium.com/@robert.broeckelmann/openid-connect-authorization-code-flow-with-red-hat-sso-d141dde4ed3f
https://gist.github.com/ataube/2dd3632eeb4f0b46286c87005dc20c74
http://www.janua.fr/keycloak-access-token-verification-example/
https://qiita.com/rururu_kenken/items/5a7b94146cf0a2eb537d
https://dzone.com/articles/oauth-20-authorisation-code-grant

Now lets test Keycloak Server OIDC URI Endpoints https://www.keycloak.org/docs/latest/server_admin/index.html#keycloak-server-oidc-uri-endpoints

Run in shell
echo "https://keycloak.example.com:8443/auth/realms/master/protocol/openid-connect/auth?response_type=code&client_id=pkce-test&state=38610846-c64d-4fd2-8d9-b5fa2f976298&redirect_uri=https://keycloak.example.com:8443/authorization-code/callback&scope=openid profile User"
Open new in incognito browser window url generatedfrom command above

change master to realm name, redirect_uri, client_id, e.t.c.

username = "john_username"
password = "j000hn_pass" (must be not Temporary from step above)

NOTE: state is generated.
TODO: provide state generator and validator

example response:
http://localhost:4200/authorization-code/callback?session_state=a4930924-fbc5-40cc-9e47-a5b0d4d357f4&code=cfe46307-d7d3-420f-b438-c5cbeccb4f13.a4930924-fbc5-40cc-9e47-a5b0d4d357f4.2e7a2cba-9b0e-4840-93db-bbd29999d0dc

replace CODE_FROM_REQUEST below with answer to request above, just copy text after &code=

NOTE: If the redirect_uri was included in the authorization request, this value must be the same as the value used in that request.

export CODE_FROM_REQUEST=e4c91079-2c06-4157-a12d-c981dea7e6c3.0aa5bab3-a6e2-4d0a-babf-274adc241bd8.2aa47e2c-9dde-4b95-95d4-17c59ace967c

NOTE: client from keycloak settings

export CLIENT_SECRET=199c35ad-8e4b-4546-8e18-6f0d158eb364
curl -X POST -H "Content-Type: application/x-www-form-urlencoded" -d "grant_type=authorization_code&client_id=pkce-test&client_secret=$CLIENT_SECRET&code=$CODE_FROM_REQUEST&redirect_uri=https://keycloak.example.com:8443/authorization-code/callback&scope=openid profile User&client_id=pkce-test&client_secret=$CLIENT_SECRET" https://keycloak.example.com:8443/auth/realms/master/protocol/openid-connect/token

TODO: user must provide client_secret or not?

Example response:

{
"access_token": "eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldU ...",
"expires_in": 300,
"refresh_expires_in": 1800,
"refresh_token": "eyJhbGciOiJIUzI1NiIsInR5cCIgOiAiSld ...",
"token_type": "bearer",
"not-before-policy": 0,
"session_state": "04c1e2e4-e6a2-460f-841a-83ef532b01d8",
"scope": "email profile"
}

Decode the access_token in jwt.io
Open keycloak.example.com:8443/auth/realms/master/protocol/openid-connect/certs
VERIFY SIGNATURE with Public key from rsa-generated (Realm Settings -> Keys)

NOTE: replace rsa-generated in production


Run keycloak under https

Follows
https://stackoverflow.com/questions/59446436/keycloak-keystore-and-java-keystore-with-https-redirect-loop

Google for `keycloak Free SSL Certificates from Let's Encrypt`

Generate certs as in https://stackoverflow.com/a/58801128

Run keycloak as in https://stackoverflow.com/a/58801128 , provide `KEYCLOAK_HOSTNAME`, `KEYCLOAK_HTTPS_PORT`, `KEYCLOAK_HTTP_PORT` and volumes
```bash
// ...
 -p 8443:8443 --name keycloak --memory="512m" \
    -e KEYCLOAK_USER=admin -e KEYCLOAK_PASSWORD=admin_pass \
    -e KEYCLOAK_HTTPS_PORT=8443  -e KEYCLOAK_HTTP_PORT=8443 \
    -e KEYCLOAK_HOSTNAME=keycloak.example.com \
-v /<path>/tls.crt:/etc/x509/https/tls.crt \
  -v /<path>/tls.key:/etc/x509/https/tls.key \
// ...

Open /auth/admin/
Navigate to /auth/admin/master/console/#/realms/master/clients
Set sslRequired to true in realm settings

PostgreSQL database that is hosted outside of both of the Keycloak servers.

Follow
https://homelab.blog/blog/security/keycloak-part-2-setting-up-keycloak/
https://homelab.blog/blog/devops/Istio-OIDC-Config/

Deploy istio with custom certs

Google "istio letsencrypt httpsRedirect"

Follow
https://homelab.blog/blog/devops/Istio-OIDC-Config/
https://istio.io/blog/2019/custom-ingress-gateway/
https://github.com/stefanprodan/istio-gke/blob/master/docs/istio/05-letsencrypt-setup.md
https://istio.io/docs/tasks/security/citadel-config/plugin-ca-cert/
https://istio.io/docs/tasks/traffic-management/ingress/ingress-certmgr/
https://istio.io/docs/ops/configuration/security/root-transition/
https://istio.io/docs/tasks/security/citadel-config/ca-namespace-targeting/
https://istio.io/docs/tasks/security/authentication/https-overlay/
https://itsmetommy.com/2019/10/10/kubernetes-istio-cert-manager-gke/

kubectl create secret generic cacerts -n istio-system --from-file=samples/certs/ca-cert.pem \
    --from-file=samples/certs/ca-key.pem --from-file=samples/certs/root-cert.pem \
    --from-file=samples/certs/cert-chain.pem

istioctl manifest apply --set values.global.mtls.enabled=true,values.security.selfSigned=false --set values.global.controlPlaneSecurityEnabled=true

# CAUTION!
 kubectl delete secret istio.default

blockspacer commented Feb 28, 2020 •

edited

Loading

Author

TBD:
Istio with OpenID, validation on app-side

https://github.com/IBM/cloud-native-starter/blob/master/documentation/DemoAuthentication.md

https://www.dangtrinh.com/2019/08/end-user-authentication-with-istio-and.html
https://keycloak.discourse.group/t/end-user-authentication-with-istio-and-keycloak/390

blockspacer commented Mar 2, 2020 •

edited

Loading

Author

https://github.com/SaifRehman/ICP-Airways
https://github.com/vietnam-devs/coolstore-microservices
https://github.com/EdwinVW/pitstop
https://github.com/GoogleCloudPlatform/microservices-demo
https://github.com/selloween/docker-keycloak-letsencrypt
https://www.infracloud.io/service-mesh-demo-istio-sockshop/

blockspacer/Istio_cpp.md

About Docker

About Kubernetes

About Service Mesh

About Cloud-enabled apps

About Cloud-native apps

About microservices architecture (MSA)

Microservice design principles

Install curl with https support

Install VirtualBox

Install docker

Install kubectl

Install kubeadm

Under proxy - setup Kubernetes The Hard Way

Install Minikube

Enable minikube addons

Prepere minikube for istio

Install istioctl

Apply istio profile

Add istioctl to PATH

NOTE: How to build docker images which will be seen by Kubernetes directly without having to push them anywhere

Open istio dashboards

NOTE: How to run Jaeger Dashboard

NOTE: How to run Grafana Dashboard

NOTE: How to run Prometheus Dashboard

NOTE: How to run Kiali

Run istioctl verify-install

Dev tools

Install Tiller (the Helm server-side component) into the Kubernetes cluster

Install Rook Operator (Ceph storage for k8s):

Install ElasticSearch, Kibana, FluentBit

Check prerequisites

Create app

Deploy the app to minikube

Deploy the app to GKE

TODO: RBAC

TODO: CI/CD

TODO: read gameontext

TODO: vagrant

TODO: read CoolStore

TODO: websockets

TODO: Running on Google Kubernetes Engine (GKE)

blockspacer commented Feb 26, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

NOTE: client from keycloak settings

Uh oh!

blockspacer commented Feb 28, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

blockspacer commented Mar 2, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Run `istioctl verify-install`

blockspacer commented Feb 26, 2020 •

edited

Loading

blockspacer commented Feb 28, 2020 •

edited

Loading

blockspacer commented Mar 2, 2020 •

edited

Loading