Skip to content

Instantly share code, notes, and snippets.

@mattfenwick

mattfenwick/KPNG-windows-hacking.md

Last active August 29, 2022 17:33
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save mattfenwick/ce1b93dcc701183ade2e793119b3480f to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save mattfenwick/ce1b93dcc701183ade2e793119b3480f to your computer and use it in GitHub Desktop.
Download ZIP
Kubernetes resources
Raw
KPNG-windows-hacking.md 
git clone [email protected]:kubernetes-sigs/sig-windows-dev-tools.git
cd sig-windows-dev-tools

vagrant destroy --force
make all

vagrant ssh controlplane

  # now inside linux

  # wget https://storage.googleapis.com/pub/gsutil.tar.gz
  # tar xfz gsutil.tar.gz -C $HOME
  # export PATH=${PATH}:$HOME/gsutil

  wget https://storage.googleapis.com/jayunit100/kpng-2-21

  chmod u+x ./kpng-2-21
  ./kpng-2-21 kube --kubeconfig=/home/vagrant/.kube/config to-api

vagrant ssh winw1

  # now inside windows

  curl.exe -o .\hns.psm1 https://raw.githubusercontent.com/microsoft/SDN/master/Kubernetes/windows/hns.psm1

  ipmo .\hns.psm1

  New-NetFirewallRule -Name OverlayTraffic4789UDP -Description "Overlay network traffic UDP" -Action Allow -LocalPort 4789 -Enabled True -DisplayName "Overlay Traffic 4789 UDP" -Protocol UDP -ErrorAction SilentlyContinue

  New-HNSNetwork -Type Overlay -AddressPrefix "192.168.255.0/30" -Gateway "192.168.255.1" -Name "External" -SubnetPolicies @(@{Type = "VSID"; VSID = 9999; }) -AdapterName $vxlanAdapter -Verbose

  curl.exe -o kpng-windows-2-21-2.exe http://storage.googleapis.com/jayunit100/kpng-windows-2-21-2.exe

  $env:KUBE_NETWORK = "External"

  .\kpng-windows-2-21-2.exe local --api=10.20.30.10:12090 to-winkernel
Raw
kube-resources.md

Prereqs

  1. you have a recent version of kubectl installed
  2. you have access to a kubernetes cluster and your kubectl is configured to talk to the cluster

Kube resources

Check your client and server version: "kubectl version"

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"20+", GitVersion:"v1.20.4-dirty", GitCommit:"e87da0bd6e03ec3fea7933c4b5263d151aafd07c", GitTreeState:"dirty", BuildDate:"2021-03-15T10:03:32Z", GoVersion:"go1.16.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"52c56ce7a8272c798dbc29846288d7cd9fbae032", GitTreeState:"clean", BuildDate:"2020-06-03T04:00:21Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

Note: If your client is older than your server, please upgrade your client.
See the kubernetes docs for more information on client/server skew:

kubectl is supported within one minor version (older or newer) of kube-apiserver.

Check available resources: "kubectl api-resources"

$ kubectl api-resources 
NAME                              SHORTNAMES   APIVERSION                        NAMESPACED   KIND
bindings                                       v1                                true         Binding
componentstatuses                 cs           v1                                false        ComponentStatus
configmaps                        cm           v1                                true         ConfigMap
endpoints                         ep           v1                                true         Endpoints
events                            ev           v1                                true         Event
limitranges                       limits       v1                                true         LimitRange
namespaces                        ns           v1                                false        Namespace
nodes                             no           v1                                false        Node
persistentvolumeclaims            pvc          v1                                true         PersistentVolumeClaim
persistentvolumes                 pv           v1                                false        PersistentVolume
pods                              po           v1                                true         Pod
podtemplates                                   v1                                true         PodTemplate
replicationcontrollers            rc           v1                                true         ReplicationController
resourcequotas                    quota        v1                                true         ResourceQuota
secrets                                        v1                                true         Secret
serviceaccounts                   sa           v1                                true         ServiceAccount
services                          svc          v1                                true         Service
mutatingwebhookconfigurations                  admissionregistration.k8s.io/v1   false        MutatingWebhookConfiguration
validatingwebhookconfigurations                admissionregistration.k8s.io/v1   false        ValidatingWebhookConfiguration
customresourcedefinitions         crd,crds     apiextensions.k8s.io/v1           false        CustomResourceDefinition
apiservices                                    apiregistration.k8s.io/v1         false        APIService
controllerrevisions                            apps/v1                           true         ControllerRevision
daemonsets                        ds           apps/v1                           true         DaemonSet
deployments                       deploy       apps/v1                           true         Deployment
replicasets                       rs           apps/v1                           true         ReplicaSet
statefulsets                      sts          apps/v1                           true         StatefulSet
tokenreviews                                   authentication.k8s.io/v1          false        TokenReview
localsubjectaccessreviews                      authorization.k8s.io/v1           true         LocalSubjectAccessReview
selfsubjectaccessreviews                       authorization.k8s.io/v1           false        SelfSubjectAccessReview
selfsubjectrulesreviews                        authorization.k8s.io/v1           false        SelfSubjectRulesReview
subjectaccessreviews                           authorization.k8s.io/v1           false        SubjectAccessReview
horizontalpodautoscalers          hpa          autoscaling/v1                    true         HorizontalPodAutoscaler
cronjobs                          cj           batch/v1beta1                     true         CronJob
jobs                                           batch/v1                          true         Job
certificatesigningrequests        csr          certificates.k8s.io/v1beta1       false        CertificateSigningRequest
leases                                         coordination.k8s.io/v1            true         Lease
endpointslices                                 discovery.k8s.io/v1beta1          true         EndpointSlice
events                            ev           events.k8s.io/v1beta1             true         Event
ingresses                         ing          extensions/v1beta1                true         Ingress
nodes                                          metrics.k8s.io/v1beta1            false        NodeMetrics
pods                                           metrics.k8s.io/v1beta1            true         PodMetrics
ingressclasses                                 networking.k8s.io/v1beta1         false        IngressClass
ingresses                         ing          networking.k8s.io/v1beta1         true         Ingress
networkpolicies                   netpol       networking.k8s.io/v1              true         NetworkPolicy
runtimeclasses                                 node.k8s.io/v1beta1               false        RuntimeClass
poddisruptionbudgets              pdb          policy/v1beta1                    true         PodDisruptionBudget
podsecuritypolicies               psp          policy/v1beta1                    false        PodSecurityPolicy
clusterrolebindings                            rbac.authorization.k8s.io/v1      false        ClusterRoleBinding
clusterroles                                   rbac.authorization.k8s.io/v1      false        ClusterRole
rolebindings                                   rbac.authorization.k8s.io/v1      true         RoleBinding
roles                                          rbac.authorization.k8s.io/v1      true         Role
priorityclasses                   pc           scheduling.k8s.io/v1              false        PriorityClass
csidrivers                                     storage.k8s.io/v1                 false        CSIDriver
csinodes                                       storage.k8s.io/v1                 false        CSINode
storageclasses                    sc           storage.k8s.io/v1                 false        StorageClass
volumeattachments                              storage.k8s.io/v1                 false        VolumeAttachment

Key resources

Basic resources which are important to have a good working grasp of:

pod

$ kubectl get pods --all-namespaces

These are where your programs run, inside of containers, and define a network namespace and filesystem. Many other resources work with pods to manage them and add capabilities to them.

namespace

Many resources are namespaced -- that is, they live in namespaces.

$ kubectl get ns
NAME                 STATUS   AGE
cim-local            Active   95m
default              Active   97m
ingress-nginx        Active   96m
kube-node-lease      Active   97m
kube-public          Active   97m
kube-system          Active   97m
local-path-storage   Active   97m
metrics              Active   95m

Use -n $NS or --namespace $NS to access a namespaced resource:

$ kubectl get pod -n $NS $POD

node

A node is a VM or physical machine

$ kubectl get nodes -o wide
NAME                 STATUS   ROLES    AGE   VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE           KERNEL-VERSION     CONTAINER-RUNTIME
kind-control-plane   Ready    master   96m   v1.18.2   172.20.0.5    <none>        Ubuntu 20.04 LTS   5.10.25-linuxkit   containerd://1.3.3-14-g449e9269
kind-worker          Ready    <none>   95m   v1.18.2   172.20.0.3    <none>        Ubuntu 20.04 LTS   5.10.25-linuxkit   containerd://1.3.3-14-g449e9269
kind-worker2         Ready    <none>   95m   v1.18.2   172.20.0.4    <none>        Ubuntu 20.04 LTS   5.10.25-linuxkit   containerd://1.3.3-14-g449e9269

service

Kubernetes uses Services to provide networking capabilities so that pods can easily talk to other pods over the network. Pods are assigned IP addresses, but as pods are ephemeral and horizontally scalable, services also provide:

  • convenient, stable DNS names (which don't change even if the underlying pods or IP addresses do change)
  • load balancing across multiple pods
$ kubectl get service -A

configmap

From the docs:

A ConfigMap is an API object used to store non-confidential data in key-value pairs. Pods can consume ConfigMaps as environment variables, command-line arguments, or as configuration files in a volume.

For example, a file that needs to be injected into a pod/container can first be created as a configmap, then volume-mounted into a pod/container.

$ kubectl get configmap -A

secret

From the docs:

Kubernetes Secrets let you store and manage sensitive information, such as passwords, OAuth tokens, and ssh keys. Storing confidential information in a Secret is safer and more flexible than putting it verbatim in a Pod definition or in a container image.

ingress

Ingress manages incoming traffic at the boundary of your cluster. From the docs:

An API object that manages external access to the services in a cluster, typically HTTP. Ingress may provide load balancing, SSL termination and name-based virtual hosting.

Ingress objects typically cooperate with an ingress controller, of which nginx is a commonly used example.

$ kubectl get ingress -A

job

Jobs are used to run pods that perform tasks and finish.

$ kubectl get jobs -A

deployment

Deployments are used to run pods that act as servers, staying up indefinitely.

Since pods are ephemeral (i.e. a pod can crash or be rescheduled to a different node), deployments act to ensure that the desired state is maintained regardless of disruptions. For example, if a pod crashes, the deployment will bring up a new one.

$ kubectl get deployments -A

API Version

The API version of resources can change with kube server version as the resources mature. For example, ingress started out in extensions/v1beta1, moved on to networking.k8s.io/v1beta1, and with kube 1.19 is now in networking.k8s.io/v1.

Verbs

Verbs that work with resources include create, get, patch, edit, and delete (CRUD):

kubectl create secret -n $NS abc
kubectl get secret -n $NS abc
kubectl patch secret -n $NS abc ...(more args)...
kubectl edit secret -n $NS abc
kubectl delete secret -n $NS abc

Yaml files

Yaml files (or json files) can be used to create and update resources.

Create a file named ingress.yaml:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: abc
  namespace: def
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: my-service
          servicePort: 8080
        path: /
        pathType: ImplementationSpecific

Then run kubectl create -f ingress.yaml, and finally kubectl get ingress abc -n def -o yaml.

Extra flags

Passing -o json or -o yaml to a kubectl get shows additional, machine-readable information about objects:

kubectl get pods -n kube-system kindnet-8dfw9 -o yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: "2021-07-16T14:32:40Z"
  generateName: kindnet-
  labels:
    app: kindnet
    controller-revision-hash: 794498c448
    k8s-app: kindnet
... etc. ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment