Using git to deploy DNS changes and treating DNS like code (Part 1: Getting Started)

[tankerkiller125]

The overview

Where I work I'm the only IT guy, so anything I can do to automate things and make my life easier is a huge boost to productivity. Automating our External DNS changes was one of the very first things I did when we started using CI/CD to automate things. In this guide I want to share what I've learned and hopefully help someone else do the same.

And let's face it, when faced with the idea of dealing with DNS we pretty much all look at it like:

Benefits

Deploying DNS like it's code and using Git allows for a ton of benefits. Notably you can allow other people to create pull requests with DNS changes (say the marketing department), automate the deployment, revert changes if they fail, and you have a log of every change made and why.

On top of all those benefits, because dnscontrol is using JavaScript you can get really fancy with your records if you need to, or you can keep it stupid simple. Personally I recommend keeping it simple, and so does the stackexchange team.

How it works

The simple explanation of how all of this works is that you use a program developed by the stackexchange team called dnscontrol, a git server, CI/CD service and a DNS provider that dnscontrol supports. Below is a diagram of how everything gets' deployed once we finish with all the parts of this tutorial.

graph LR A[Local Git Repo] -->|Push to Git| B(DNS Repo) B --> D{Git Service} subgraph CI/CD Vendor D[Test DNS changes are valid] D -->E[Deploy changes to DNS Vendor] end E -->F{DNS Vendor\nPublishes} F -->G[Changes published to internet]

Getting started

To get started make sure that your DNS vendor is supported by dnscontrol

Installation

For Mac users, you can use one of the following commands:

brew install dnscontrol
sudo port install dnscontrol

Then you need to download the latest release of dnscontrol for your platform.

Once you have the latest release, either install it (Ubuntu, RedHat, MacOS) or extract it to the folder you plan to work from (Windows, FreeBSD, other linux)

Create the credentials file

From here you'll want to create a creds.json file in the working folder. It will contain the authentication information for various providers. Below is an example of a couple popular vendors:

{
    "cloudflare": {
        "TYPE": "CLOUDFLAREAPI",
        "apitoken": "token",
        "accountid": "accountid"
    },
    "none": {"TYPE": "NONE"},
    "r53_main": {
        "TYPE": "ROUTE53",
        "DelegationSet": "optional-delegation-set-id",
         "KeyId": "your-aws-key",
         "SecretKey": "your-aws-secret-key",
         "Token": "optional-sts-token"
    },
    "azuredns_main": {
         "TYPE": "AZURE_DNS",
         "SubscriptionID": "AZURE_SUBSCRIPTION_ID",
         "ResourceGroup": "AZURE_RESOURCE_GROUP",
         "TenantID": "AZURE_TENANT_ID",
         "ClientID": "AZURE_CLIENT_ID",
         "ClientSecret": "AZURE_CLIENT_SECRET"
     }
}

You should take a look at the documentation for your specific vendor to see which API scopes are required and what permissions your tokens require. I have not documented that here.

"YOUR KEEPING SECRETS IN GIT!" I here you say? Indeed, with the file as I posted it you would indeed be keeping credentials in git, and that's a horrible practice that should never ever be done. So how do we fix it? Amazingly it's stupidly simple to fix that issue! Simple replace the string where the token goes with an environment variable like so:

{
    "cloudflare": {
        "TYPE": "CLOUDFLAREAPI",
        "apitoken": "$CF_APITOKEN",
        "accountid": "$CF_ACCOUNTID"
    },
    "none": {"TYPE": "NONE"}
}

And just like that we can now use the environment variables CF_APITOKEN and CF_ACCOUNTID to pass credentials. Most likely via secure secrets in the CI/CD solution of our choice.

Once you have the credentials configured in the creds.json file and your environment variables set, you can run dnscontrol check-creds <providername> to make sure that your credentials are working properly before continuing. It should spit out all the domains it has access too. As an example:

PS C:\Users\tankerkiller125\DNSTutorial> .\dnscontrol.exe check-creds cloudflare
sysadmins.zone

Creating a draft of the DNS records

To create a quick draft of all the DNS Zones on your account into a single file you can run:
dnscontrol get-zones --format=js --out=draft.js <provider> - all in my case I ran dnscontrol get-zones --format=js --out=draft.js cloudflare - all

This will output a file called draft.js that will contain all of your already existing DNS records in a kind of messy way, but in theory it would work immediately. However if your using CNAME records in root (Cloudflare users) you will need to update those CNAMEs in the file to ALIAS. As an example:

D("sysadmins. Zone", REG_NONE,
	DnsProvider(DSP_CLOUDFLARE),
	DefaultTTL(1),
	// NOTE: CNAME at apex may require manual editing.
	CNAME('@', 'someothersite.tld.', CF_PROXY_ON),
	CNAME('www', 'sysadmins.zone.', CF_PROXY_ON),
        ...
)

Will become:

D("sysadmins. Zone", REG_NONE,
	DnsProvider(DSP_CLOUDFLARE),
	DefaultTTL(1),
        ALIAS('@', 'someothersite.tld.', CF_PROXY_ON),
	CNAME('www', 'sysadmins.zone.', CF_PROXY_ON),
        ...
)

Once you've fixed this issues, and the file looks correct to you, you can save the file as dnscontrol.js

Pushing Changes

OK, you've now configured your credentials and ensured they work, you've pulled down your already existing DNS records, you've now corrected the comments the program generated. How do we push a change?

Pushing a change is incredibly simple, using one of our examples above, let's add a assets sub-domain to sysadmins.zone.

D("sysadmins. Zone", REG_NONE,
	DnsProvider(DSP_CLOUDFLARE),
	DefaultTTL(1),
        ALIAS('@', 'someothersite.tld.', CF_PROXY_ON),
	CNAME('www', 'sysadmins.zone.', CF_PROXY_ON),
        CNAME('assets', 'bucket.s3.amazonaws.com.`)
)

You'll note that for this record, I did not include the special CF_PROXY_ON parameter, this means that Cloudflare will not proxy this subdomain. This is a Cloudflare only property, other DNS providers may have different optional parameters' you can find in the dnscontrol documentation for that provider.

Now that you've added the domain to the JavaScript file, we can now run dnscontrol check to verify that we didn't make any mistakes. But crap! We did! The mistake here is that dnscontrol requires all CNAME and ALIAS records to contain a period at the end. Adjusting the record to point to bucket.s3.amazonaws.com. and re-running dnscontrol check to verify and we're all good.

Now that we've checked our file is correct we can now preview the changes we're making by running dnscontrol preview this will compare the live DNS settings with the JavaScript file and then display the difference.

And finally once we're satisfied that everything is working as expected, we can run dnscontrol push to push the changes to the DNS provider and have the DNS records updated globally live.

What's next?

I have two more parts planned for this tutorial series, the very next one will be deploying DNS using Github, Gitlab, and Azure DevOps (the three popular ones in companies). The third and final guide will be on using the advanced features and functionality of dnscontrol to clean up the JS files a bit.

Part 2: Deploying DNS with CI/CD is now live!
Part 3: Advanced Control is now live!

[tankerkiller125]

I've now published part 2, the original post has been updated to point to it.

[tankerkiller125]

I've published part 3 now. Original post and part 2 now have links to it.

[katos]

Thanks for the guide tankerkiller125 - This looks great!
I'll be giving this a try for my lab environment as well as my own personal sites