DNS Behavior with APIM + VNet + Private DNS + Custom DNS

Summary

After introducing Private DNS zones into a VNet-integrated environment with custom DNS servers, DNS resolution behavior changes due to zone precedence and authority rules.

This can result in previously working DNS lookups failing, not because DNS is broken, but because resolution is being intercepted earlier in the lookup chain.

What Changed

Before (Working State)

Workload (APIM / VM)
   → Custom DNS (10.x.x.x)
   → Upstream DNS (Azure or Internet)
   → Public resolution succeeds

After Adding Private DNS Zone

Workload
   → Custom DNS
   → Private DNS zone match
       → If record exists: success (private IP)
       → If record missing: NXDOMAIN (fail)

Key Behavior

Private DNS zones are authoritative for matching domains.

If a query matches a private zone, Azure does not fall back to public DNS.
This is especially true for non-privatelink zones.
Missing records result in hard failures.

Why This Broke Things

The private DNS zone intercepts queries earlier.
Previously, queries flowed to upstream DNS.
Now, queries stop at the private zone.
With no fallback, resolution fails.

Important Distinction

Zone Type	Fallback Behavior
`privatelink.*`	Optional fallback, if supported and enabled
Non-`privatelink`	No fallback

Root Cause Pattern

The issue typically occurs when all three are present:

Custom DNS servers (10.x.x.x)
Private DNS zones linked to the VNet
No fallback or forwarding strategy

Recommended DNS Configuration

1. Conditional Forwarding for Private Zones

Configure custom DNS servers to forward:

privatelink.azure-api.net          → 168.63.129.16
privatelink.blob.core.windows.net  → 168.63.129.16
privatelink.database.windows.net   → 168.63.129.16

This ensures Azure-managed private endpoints resolve correctly.

2. Default Forwarding

All other queries should resolve publicly:

. (all domains) → 168.63.129.16

Alternative upstream resolvers:

8.8.8.8
1.1.1.1

3. Avoid Overriding Public Zones

Do not create private DNS zones for:

azure-api.net
blob.core.windows.net
database.windows.net

These override public DNS entirely and require full manual record management.

Target Resolution Flow

Client (APIM / VM)
   ↓
Custom DNS

IF domain = privatelink.*
   → Azure DNS (168.63.129.16)
   → Private IP

ELSE
   → Public DNS
   → Public IP

Optional: Azure Firewall DNS Proxy Pattern

Client → Firewall (DNS Proxy)
           ↓
     Forward to 168.63.129.16

Benefits:

Centralized DNS control
Consistent resolution across VNets
Reduced configuration drift

Verification and Troubleshooting

1. Basic DNS Resolution Tests

Linux or macOS

nslookup google.com
nslookup <apim-name>.azure-api.net
nslookup <storage-account>.blob.core.windows.net

dig google.com
dig <apim-name>.azure-api.net

Windows Command Prompt

nslookup google.com
nslookup <apim-name>.azure-api.net

PowerShell

Resolve-DnsName google.com
Resolve-DnsName <apim-name>.azure-api.net

2. Test Against Specific DNS Server

Force query to your custom DNS:

nslookup google.com 10.82.8.4
nslookup <apim-name>.azure-api.net 10.82.8.4

Test Azure DNS directly:

nslookup google.com 168.63.129.16

3. Separate DNS from Connectivity

If DNS works but connectivity fails:

curl -I https://google.com
curl -I https://<apim-name>.azure-api.net

4. Azure CLI Checks

List Private DNS Zones

az network private-dns zone list -o table

Show Records in a Zone

az network private-dns record-set a list \
  --zone-name privatelink.azure-api.net \
  --resource-group <rg> \
  -o table

Check VNet Links

az network private-dns link vnet list \
  --zone-name privatelink.azure-api.net \
  --resource-group <rg> \
  -o table

5. NIC, DNS, and Effective Route Checks

Show NIC DNS Settings

az network nic show \
  --name <nic-name> \
  --resource-group <rg> \
  --query "dnsSettings"

Show Effective Route Table

az network nic show-effective-route-table \
  --name <nic-name> \
  --resource-group <rg>

6. Common Failure Patterns

Symptom	Likely Cause
Public domains fail to resolve	Missing default forwarder
Private endpoints resolve, public fail	DNS forwarding issue
DNS resolves but requests fail	Firewall or routing issue
Intermittent failures	Split DNS or inconsistent forwarding

Key Takeaways

Private DNS zones override, not augment, DNS resolution.
Non-privatelink zones have no fallback behavior.
Custom DNS must explicitly handle:
- Private resolution via conditional forwarding
- Public resolution via default forwarders

Mental Model

Private DNS introduces an authoritative layer. If it matches a query, it must return a valid record or the lookup fails.

timmyreilly/private_dnz_zone_problems.md

Select an option

No results found