Skip to content

Instantly share code, notes, and snippets.

@rcarrata

rcarrata/appgw_aro.md

Created July 18, 2023 08:32
Show Gist options
  • Save rcarrata/e53424a5e9741dfbdf07ca9042d4735e to your computer and use it in GitHub Desktop.
Save rcarrata/e53424a5e9741dfbdf07ca9042d4735e to your computer and use it in GitHub Desktop.
Download ZIP
Application GW Load Balancer in ARO
Raw
appgw_aro.md

Application GW Load Balancer in ARO

  • Set environment variables useful for this guide:
export NAMESPACE=aro-agw
export AZR_CLUSTER=aro-$USER
export AZR_RESOURCE_LOCATION=eastus
export AZR_RESOURCE_GROUP=aro-$USER-rg
export AppGW_CIDR="10.0.10.0/23"
export AppGW_SUBNET="Ingress-subnet"
export ARO_VNET_NAME="aro-$USER-vnet"
export APPGW_DOMAIN="aro-httpd.aroplay.openshiftdemo.dev"
export DNS_ZONE_NAME="aroplay.openshiftdemo.dev"
export AppGW_PIP="AppGW-pip"
export AZR_DNS_RESOURCE_GROUP="mobb-dns"
export EMAIL=[email protected]
  • Create Subnet for AppGW:
az network vnet subnet create \
  --resource-group $AZR_RESOURCE_GROUP \
  --vnet-name $AZR_CLUSTER-aro-vnet-$AZR_RESOURCE_LOCATION \
  --name $AppGW_SUBNET \
  --address-prefixes $AppGW_CIDR \
  --service-endpoints Microsoft.ContainerRegistry

NOTE: due to the Subnets from AppGW and ARO share the same VNet is not needed to add a peering. If you want to split between two different VNets instead of Subnets, please be remember adding the VNet peering between them.

  • Create a static public IP address for the Application Gateway LB:
az network public-ip create \
  --resource-group $AZR_RESOURCE_GROUP \
  --name $AppGW_PIP \
  --allocation-method Static \
  --sku Standard
  • Create a Web Application Firewall (WAF) policy for the Application Gateway:
az network application-gateway waf-policy create \
  --resource-group $AZR_RESOURCE_GROUP \
  --name AppGW-WAF-Policy-$USER
  • Creates an Application Gateway with the specified configurations, including the WAF policy, public IP, and subnet:
# Create the Application Gateway with the WAF policy
az network application-gateway create \
  --name "AppGW-aro-$USER" \
  --location $AZR_RESOURCE_LOCATION \
  --resource-group $AZR_RESOURCE_GROUP \
  --capacity 1 \
  --priority 1 \
  --sku WAF_v2 \
  --http-settings-cookie-based-affinity Disabled \
  --public-ip-address $AppGW_PIP \
  --vnet-name $AZR_CLUSTER-aro-vnet-$AZR_RESOURCE_LOCATION \
  --subnet $AppGW_SUBNET \
  --waf-policy AppGW-WAF-Policy-$USER
  • Generates SSL certificates using Let's Encrypt's Certbot tool:
export SCRATCH_DIR=/tmp/scratch
mkdir -p $SCRATCH_DIR

certbot certonly --manual \
  --preferred-challenges=dns \
  --email $EMAIL \
  --server https://acme-v02.api.letsencrypt.org/directory \
  --agree-tos \
  --manual-public-ip-logging-ok \
  -d "*.$APPGW_DOMAIN" \
  --config-dir "$SCRATCH_DIR/config" \
  --work-dir "$SCRATCH_DIR/work" \
  --logs-dir "$SCRATCH_DIR/logs"

NOTE: don't close or interrupt this process, we will finish after the dns challenge in Azure.

  • Open a second terminal and paste the DNS_Challenge (and remember to export again the variables from the beggining):
export DNS_CHALLENGE="xxxx"
  • Adds a TXT record to the Azure DNS zone for the ACME challenge:
az network dns record-set txt add-record \
  --resource-group $AZR_DNS_RESOURCE_GROUP \
  --zone-name $DNS_ZONE_NAME \
  --record-set-name "_acme-challenge.aro-httpd" \
  --value "$DNS_CHALLENGE"
  • Wait up to 5mins (maybe more) until the TXT record propagates and check the DNS resolution from the ACME challenge within Azure DNS. Check that the dig output matches with the DNS Challenge shown before by the certbot command:
dig +short TXT _acme-challenge.$APPGW_DOMAIN

  • Return to the previous terminal and finish the generation of the ACME certificate for our ARO example App

  • Certificate Bundle: Concatenate the generated SSL certificates into a bundle file and export it as a PKCS12 file.

PFX_PASS="mypa55w0rd"

cat $SCRATCH_DIR/config/live/$APPGW_DOMAIN/fullchain.pem $SCRATCH_DIR/config/live/$APPGW_DOMAIN/privkey.pem > $SCRATCH_DIR/config/live/$APPGW_DOMAIN/gw-bundle.pem

openssl pkcs12 -export -out $SCRATCH_DIR/config/live/$APPGW_DOMAIN/gw-bundle.pfx -in $SCRATCH_DIR/config/live/$APPGW_DOMAIN/gw-bundle.pem
  • NOTE: if you're in a rush and you don't want to wait until the acme challenge is validated and the ACME cert issued, you can use openssl to generate a self-signed certificate and continue with the rest of the process:
openssl genrsa -out aro-appgw.key 2048
openssl req -key aro-appgw.key -new -out aro-appgw.csr
openssl x509 -signkey aro-appgw.key -in aro-appgw.csr -req -days 365 -out aro-appgw.crt
openssl pkcs12 -inkey aro-appgw.key -in aro-appgw.crt -export -out aro-appgw.pfx
  • Deletes the TXT record created for the ACME challenge:
az network dns record-set txt delete \
  --resource-group $AZR_DNS_RESOURCE_GROUP \
  --zone-name $DNS_ZONE_NAME \
  --name "_acme-challenge.aro-httpd"
  • Retrieve the public IP address of the Application Gateway:
AGW_PIP=$(az network public-ip show -g $AZR_RESOURCE_GROUP --name $AppGW_PIP --query ipAddress -o tsv)
  • Update the DNS record with the Application Gateway's public IP address:
az network dns record-set a add-record \
--resource-group $AZR_DNS_RESOURCE_GROUP \
--zone-name $DNS_ZONE_NAME \
--record-set-name "$(echo $APPGW_DOMAIN | sed 's/\..*//')"  \
--ipv4-address $AGW_PIP
  • Verify the DNS resolution for the updated domain:
dig +short $APPGW_DOMAIN
  • Creates an HTTPS listener for the Application Gateway:
az network application-gateway ssl-cert create \
  --resource-group $AZR_RESOURCE_GROUP \
  --gateway-name "AppGW-aro-$USER" \
  --name gw-bundle \
  --cert-file $SCRATCH_DIR/config/live/$APPGW_DOMAIN/gw-bundle.pfx \
  --cert-password $PFX_PASS
  • In the Listeners section, create a new HTTPS listener (portal):
Listener name: aro-route-https-listener
Frontend IP: Public
Port: 443
Protocol: HTTPS
Http Settings - choose to Upload a Certificate (upload the PFX file from earlier)
Cert Name: gw-bundle
PFX certificate file: gw-bundle.pfx
Password: $PFX_PASS (what you used when creating the PFX file)
Additional settings 
   - Multi site: aro-httpd.aroplay.openshiftdemo.dev

Note: You can also create multiple listeners - one per site and re-use the certificate and select basic site

  • Create a new backend pool (cli):
az network application-gateway address-pool create \
  --gateway-name "AppGW-aro-$USER" \
  --resource-group $AZR_RESOURCE_GROUP \
  --name aro-routes \
  --servers $APPGW_DOMAIN
  • Create a new backend HTTP setting (portal):

In the HTTP settings section, create a new HTTP setting: HTTP settings name: aro-route-https-settings Backend protocol: HTTPS Backend port: 443 Use well known CA certificat: Yes (if you used one; otherwise upload your CA cer file) Override with new host name: Yes Choose: Pick host name from backend target

  • Define a rule for each website/api (cli):
az network application-gateway rule create \
  --gateway-name "AppGW-aro-$USER" \
  --resource-group $AZR_RESOURCE_GROUP \
  --name aro-app-https-rules \
  --http-listener aro-route-https-listener \
  --address-pool aro-routes \
  --http-settings aro-route-https-settings \
  --priority 2

Exposing HTTPD App

  • Create new project for testing app:
oc new-project aro-appgw
  • Deploy a httpd server K8s Deployment and expose using a K8s Service:
oc create deployment aro-httpd --image registry.redhat.io/rhel8/httpd-24 --port 8080
oc expose deployment aro-httpd
  • Add the Edge Route with the hostname as the "aro-httpd.$DOMAIN"
oc create route edge --service=aro-httpd --hostname=aro-httpd.$DOMAIN
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment