Skip to content

Instantly share code, notes, and snippets.

@fosterseth

fosterseth/awx_dev_kind.md

Last active November 10, 2023 23:12
Show Gist options
  • Save fosterseth/081c05248975048beb55858def010866 to your computer and use it in GitHub Desktop.
Save fosterseth/081c05248975048beb55858def010866 to your computer and use it in GitHub Desktop.
Download ZIP
AWX development on Kind
Raw
awx_dev_kind.md

AWX development on kind

Features

  • Uses your locally checked out code. No need to build and upload custom awx or awx-operator images.
  • Live reloading. Making changes to your local awx files will trigger services to restart in the deployment to reflect code changes.
  • The kind cluster runs in a single docker container. You can easily stop, pause, and restart this single container to bring up the entire deployment.
  • This deployment method is useful for API, UI and operator work

Requirements

kind https://kind.sigs.k8s.io/docs/user/quick-start/#installation

kubectl https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/

ansible (via pip)

ansible-runner (via pip)

(Optional)

stern https://github.com/stern/stern/releases

Instructions

  1. checkout local copies of the awx and awx-operator repos
    • github.com/ansible/awx.git
    • github.com/ansible/awx-operator.git
  2. from awx repo, make Dockerfile.kube-dev
    • this will run an ansible-playbook to generate the following files
    • ❗ you should only need to do this step once 
       _build_kube_dev/
 ├── supervisor_rsyslog.conf
 ├── supervisor_task.conf
 └── supervisor_web.conf

🚧 the following steps should be run from the awx-operator repo 🚧

  1. create kind-cluster.yml
    • ❗ "hostPath" should point to your local awx repo
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  extraMounts:
  - hostPath: /home/sbf/awx
    containerPath: /awx_devel
  extraPortMappings:
  - containerPort: 30080
    hostPort: 30080
  1. create awxdev.yml
---
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
spec:
  service_type: nodeport
  nodeport_port: 30080
  development_mode: True
  extra_settings:
  - setting: CSRF_TRUSTED_ORIGINS
    value:
      - https://localhost:3001
  - setting: RECEPTOR_RELEASE_WORK
    value: "False"
  - setting: LOG_AGGREGATOR_LEVEL
    value: "'DEBUG'"
  - setting: AWX_CLEANUP_PATHS
    value: "False"
  image: ghcr.io/ansible/awx_kube_devel
  image_version: devel
  image_pull_policy: Always
  no_log: False
  1. run the following commands from the awx-operator repo
kind create cluster --config kind-cluster.yml
kubectl create ns awx
kubectl config set-context --current --namespace=awx
make install
kubectl apply -f awxdev.yml
make run

Alternatively, if you aren't making awx-operator changes, you can just deploy an already built operator image

kind create cluster --config kind-cluster.yml
kubectl create ns awx
kubectl config set-context --current --namespace=awx
IMG=quay.io/ansible/awx-operator:devel make deploy
kubectl apply -f awxdev.yml
  1. While operator is running, open another terminal and run watch kubectl get pods to see the pods come online

Verify installation

$ kubectl get pods
NAME                        READY   STATUS    RESTARTS   AGE
awx-postgres-13-0           1/1     Running   0          9h
awx-task-5cb744975c-2fmvs   4/4     Running   0          6h3m
awx-web-576b8d88bb-vqtgx    3/3     Running   0          6h3

A healthy deployment should have those pods in a "Running" state

View logs

stern -A awx to see logs from all of the containers

Username and password

http://localhost:30080/api/v2 should be up and running (note the http, not https)

admin password can be obtained by running

kubectl get secret awx-admin-password -o jsonpath="{.data.password}" | base64 --decode ; echo

After you log in

Once you log in, verify that the Demo Job Template is created. If so, it is now safe stop the operator from running by hitting Control+C in the terminal window

Getting UI to work

By default the UI will not work, as ghcr.io/ansible/awx_kube_devel does not come pre-baked with the static UI files. When the awx-web container starts, it will collect the static files from your locally checked out awx/ui/build/ and copy them to /var/lib/awx/public/static/ inside of the container. This location is where nginx is configured to serve static content from.

It might be best to build inside of the task container since it comes installed with the correct npm and nodejs libraries.

kubectl exec -it deploy/awx-task -c awx-task -- bash
cd /awx_devel
make ui-devel

This will take around 5-10 minutes to complete

Then delete the web pods to allow new ones to come up.

kubectl delete pod -l app.kubernetes.io/name=awx-web

The UI should now be working!

For UI development (with live reloading), you probably want to start the development server

TARGET='http://127.0.0.1:30080' npm --prefix awx/ui start

🚧 Use 127.0.0.1 instead of localhost 🚧

POST requests won't work due to cross site request forgery (CSRF). Add the proxy server to the CSRF_TRUSTED_ORIGINS django setting. You may need to re-apply awxdev.yml and delete your web pod (to recreate them).

in awxdev.yml

extra_settings:
- setting: CSRF_TRUSTED_ORIGINS
  value:
    - https://localhost:3001

Useful aliases and commands

kubectl and docker commands can get cumbersome to type out, so here are aliases I put into my .bashrc file to make development easier

kubectl

alias kubemig='kubectl exec -it deploy/awx-task -c awx-task -- awx-manage showmigrations'
alias kubegetall='kubectl get all -n awx'
alias kubens='kubectl config set-context --current --namespace=awx'
alias kubetask='kubectl exec -it deploy/awx-task -c awx-task -- bash -c "source /var/lib/awx/venv/awx/bin/activate; bash"'
alias kubeweb='kubectl exec -it deploy/awx-web -c awx-web -- bash -c "source /var/lib/awx/venv/awx/bin/activate; bash"'
alias kubeee='kubectl exec -it deploy/awx-task -c awx-ee -- bash'

docker

alias dockerpause='docker pause `docker ps -q`'
alias dockerps='docker ps --format "table {{.ID}}\t{{.Image}}\t{{.Command}}\t{{.RunningFor}}\t{{.Names}}"'
alias dockerstart='docker start tools_awx_1 tools_postgres_1'
alias dockerstop='docker stop `docker ps -aq`'
alias dockerunpause='docker unpause `docker ps -q`'

Scaling up deployment

kubectl scale deploy/awx-web --replicas=2

kubectl scale deploy/awx-task --replicas=2

Tips

Use kind delete cluster to tear down everything. You can start at step 5 at this point to re-create the cluster.

Use kubectl delete ns awx to delete all of the AWX resources, but keep the kind cluster up and running. This is useful for redeploying the app from scratch, without destroying the cluster. You can simply start at step 5 make install.

For crashbackoff pod errors, use kubectl describe pod and see if the image was able to pull successfully. Another common reason for this error is that you didn't run the make Dockerfile.kube-dev command before deploying. You can run it now and kubectl delete your task and web pods to allow them to come back online.

You can delete the web and task containers without much consequence. They will recreate very quickly.

Raw
awxmeshingress3.yml
apiVersion: awx.ansible.com/v1alpha1
kind: AWXMeshIngress
metadata:
name: awxmeshingress3
namespace: awx
spec:
deployment_name: awx
@fosterseth
Copy link
Author

to deploy internal hop node,

kubectl apply -f awxmeshingress3.yml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment