Mac-OS-X-DNS-Mystery.md

I’m connected via ethernet and DHCP, the DHCP provided nameserver is 10.0.0.111 and running dnsmasq for example.com to resolve to 10.0.0.110 when on the local network, while the ip for example.com on the open internet is 91.65.182.25.

Why would Mac OS X, after resetting the DNS cache first resolve the domain correctly, and then after a few minutes somehow get the public DNS ip?

ping, Safari, etc. are all affected. There is nothing going on in the 360 seconds wait time.

> sudo killall -HUP mDNSResponder && ping -c 3 example.com && sleep 360 && ping -c 3 example.com
Password:
PING example.com (10.0.0.110): 56 data bytes
64 bytes from 10.0.0.110: icmp_seq=0 ttl=64 time=0.795 ms
64 bytes from 10.0.0.110: icmp_seq=1 ttl=64 time=1.598 ms
64 bytes from 10.0.0.110: icmp_seq=2 ttl=64 time=0.830 ms

--- example.com ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.795/1.074/1.598/0.371 ms
PING home.jan.io (91.65.182.25): 56 data bytes
64 bytes from 91.65.182.25: icmp_seq=0 ttl=64 time=1.180 ms
64 bytes from 91.65.182.25: icmp_seq=1 ttl=64 time=3.704 ms
64 bytes from 91.65.182.25: icmp_seq=2 ttl=64 time=2.567 ms

--- home.jan.io ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 1.180/2.484/3.704/1.032 ms

There's two probable causes here:

• the device 'example.com' is periodically advertising that its address is 91.65.182.25 via NAT-PMP.
• Your DNS configuration is broken.

I did some quick searching and can't find many useful ways to discover nat-pmp state from the osx command line. If this is indeed the problem, disabling options like 'port mapping' or 'upnp' on the target device might help.

Thinking about it though, its much more likely that your DNS config is broken. Especially if you are using a name which is also a zone apex, there are more places than you'd expect that A records might exist for it. This is why debugging resolution issues with the right tools is essential. Using 'ping' involves the entire client OS name resolution stack, while using 'dig' focuses on DNS responses.

Are you sure that osx is only using 10.0.0.111 as a resolver? Is that nameserver authoritative for 'example.com'? Do you have it configured with both internal and external views?

what's the response you get for the following commands?
dig example.com ns
dig example.com ns @10.0.0.111

@mattghall thank you for looking into this!

the device 'example.com' is periodically advertising that its address is 91.65.182.25 via NAT-PMP.

no NAT-PNP anywhere on the box that is example.com locally. And I disabled uPnP on the router/default gateway here, to see if that “poisons” the cache.

Your DNS configuration is broken.

That is entirely possible!

The scenario is: I have a web service running on machine A in the LAN on the .110 ip address. I want to be able to access the service from inside the LAN ad outside the LAN with the same name. For that to work, I have example.com point to my router’s (.1 LAN ip) WAN ip 91.65.182.25 via my public DNS service, and port forwarding, so I get access from the outside, that’s all working fine.

Now I also want to access the service within the LAN under example.com, but the router (.1) doesn’t do the port forwarding when the call is coming from INSIDE THE HOUSE (and no way to set that up). So via https://gist.github.com/janl/fa8379fbd890282205e7 I set up dnsmasq on my other .111 box (it’s just FreeBSD jails, I don’t have too many machines here :) with DHCP, disabled DHCP on my router, and made the .111 DHCP send out the nameserver of .111 for DHCP clients. That also works nicely. The default gateway remains .1, not .111.

That is, my authoritative nameserver for example.com hands out then WAN ip, but I’m using dnsmasq locally to hijack that. FWIW, it worked for a while (few weeks), and stopped now (maybe after restarting the El Capitan MacBook Pro here)

My example.com is just a CNAME for my-other-example.com (for dyndns reasons, although It changes only every couple of years). Now reading through this, I’ll try to make dnsmasq respond to my-other-example.com with the same LAN ip as example.com, maybe that helps :)

As to your questions:

> dig example.com ns

; <<>> DiG 9.8.3-P1 <<>> example.com ns
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 4309
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;example.com.       IN  NS

;; ANSWER SECTION:
example.com.    86400   IN  CNAME   my-other-example.com.

;; AUTHORITY SECTION:
my-other-example.com.           2978    IN  SOA ns1.mydnsprovider.net. hostmaster.mydnsprovider.com. 2015111500 86400 7200 3600000 3600

;; Query time: 107 msec
;; SERVER: 10.0.0.111#53(10.0.0.111)
;; WHEN: Wed Nov 18 20:34:24 2015
;; MSG SIZE  rcvd: 146

> dig example.com ns @10.0.0.111

; <<>> DiG 9.8.3-P1 <<>> example.com ns @10.0.0.111
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 31531
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 0

;; QUESTION SECTION:
;example.com.       IN  NS

;; ANSWER SECTION:
example.com.    86398   IN  CNAME   my-other-example.com.

;; AUTHORITY SECTION:
my-other-example.com.           2976    IN  SOA ns1.mydnsprovider.net. hostmaster.mydnsprovider.com. 2015111500 86400 7200 3600000 3600

;; Query time: 13 msec
;; SERVER: 10.0.0.111#53(10.0.0.111)
;; WHEN: Wed Nov 18 20:34:26 2015
;; MSG SIZE  rcvd: 146

Reading the man page of mDNSResponder it reads that it's also responding to multicast DNS queries if it knows an answer. Assuming there is another mDNSResponder in your local broadcast domain which does not use 10.0.0.111 as resolver but a public resolver, then your mDNSResponder might send a multicast query for example.com and get's an answer from the peer mDNSResponder quicker than from 10.0.0.111.

Just a theory though... it does not easily explain why the resolution works at the first attempt.

@maxheadroom That sounds plausible.

So far it looks like dnsmasq-ing my-other-example.com might have done the trick.

Hm. I've never seen a nameserver answer a NS query with a CNAME. This suggests that you're most likely trying things that don't work in DNS, and your nameserver (dnsmasq?) is doing broken things trying to make sense of your config. Can you paste the zonefile for 'example.com'?

@mattghali I don’t have a zone file, but example.com is a CNAME for my-other-example.com and more specifically it is really foo.example.com as a CNAME of bar.my-other-example.com — the DNS hoster just has a web interface for DNS, so I can’t really screw up the zone file.

that said, dnsmasqing my-other-example.com totally did the trick, thanks everyone! <3