Fun with DNS Caching on a VMWare vCSA

I ran into a weird problem a few nights ago during a change that took a while to resolve so I thought it was worth posting about.

I was tasked with changing the IP address of a few ESXi servers as they were migrated into a new network segment. Simple enough.

I made the necessary changes on the ESXi hosts, updated DNS, and then removed them from the inventory of vCenter (wasn’t fussed about losing any historical data). I waited a short while and then attempted to add them back in using the appropriate FQDN. Didn’t work. Bugger.

vCenter was complaining about being unable to connect to the host. I was able to add them back in using the IP address though. So I can’t connect with the FQDN or DNS host name but IP address works OK. Hmmm sounds like a DNS problem…..

To cut a long story short, after some Google Fu I discovered that the vCSA has an internal DNS service (running dnsmasq) that seems to aggressively cache lookups far beyond the TTL of the A record. My vCSA was trying to connect to the ESXi host with the old IP address.

To fix it up I had to SSH onto the vCSA as root and run the following to clear the DNS cache:

systemctl restart dnsmasq

This didn’t have any ill effects on the infrastructure so safe to do anytime.

You can check that it has restarted successfully by running the following:

systemctl status dnsmasq

and then a ping to the host will let you see if it has updated.

Leave a Reply

Your email address will not be published. Required fields are marked *