How DNS happens

DNS (Domain Name Service) has been around since the first time someone tried a name instead of an IP to get somewhere on the Internet.  Which translates to only a few years younger than the first network. What happens when you type ‘google.ca’ isn’t stunningly complex, it’s really no more than your machine looking up the number of that domain and then sending you there.  The really interesting part is how much we rely on it and how it’s embedded into nearly everything.  In fact the Internet would grind to a halt in about 5 minutes without it and the fact it is made of millions of simple text files is a minor marvel.

So why the news on DNS?  My home internet (to my server) went out a few days ago and it is one of a chain of DNS servers that maintain a few domains.  It also exposed an vulnerability in my redundancy which I had thought covered, turns out some companies know less of DNS than I do and thier system weren’t capable of taking the added load.  Thus the DNS entries began to expire and the scramble was on.  Long story short, I managed to get a new DNS system up in a few hours and migrate things to a more stable platform.  The whole system is better than ever and more fault tolerant. This meant some email outages for a time and though email can recover, the deliveries where later than expected.

What I learned yet another software system that doesn’t do what it claims and I didn’t even get an error the system wasn’t working as expected.  No way to check and the only way to discover the flaw was to create the problem it was meant to protect against, seems a rather hard way to test a system.  I’ll have to break down and learn to use Unix on the command line and stop relying on a GUI that tries to hide thier failures behind pretty icons.

I’ve never been a proponent of ‘hard testing’ where one creates the disaster to check the recovery system.  My reasoning being if things are other than planned (see Murphy’s law) you’ll have created a problem you do not have the solution for (or your recovery plan would have worked).

So I’ve learned a few new tricks, found a useful service for DNS replication and for one day of annoyance managed to ‘hard test’ my failover system.  Now I just have to get my own regular Internet connection back, thank the tech gods for cell phone tethering 🙂