Getting the IP addresses of gmail MX servers – via Super User – dig isn’t enough
Posted by jpluimers on 2017/03/06
I needed the current IP-addresses of the gmail MX server (don’t ask the details; but it has to do with the brain-dead TP-LINK ER5120 configuration possibilities).
This is the command I finally used:
dig @8.8.8.8 +short MX gmail.com | sed "s/^[0-9]* //g" | sed "s/.$//" | xargs -I {} dig @8.8.8.8 +short {} | uniq | sort
Basically it’s a three stage sequence which had to work on OS X as well as Linux using a bash shell:
- Use the Google DNS servers (either 8.8.8.8 or 8.8.4.4)
- Get the FQDNs of MX records of gmail.com which are the mail servers for GMail.
- Translate these in IPv4 addresses
- Filter into a distinct list (just in case entries are duplicate: they aren’t yet, but might be)
The basics of the above are about using dig to get short (or terse) answers with as little (but still to the point) information as possible.
I got to this from the “verbosity” table (results are the same in Mac OS X as well) that I extended from osx – How do I get the mx records for a domain from *nix command line? – Super User (thanks Gordon Davisson and bortzmeyer):
dig +short MX mydomain.com
host -t mx mydomain.com
nslookup -q=mx mydomain.com
dig -t mx mydomain.com
Stripping the MX to just the FQDNs
The MX query returns the order of the MX and a terminating dot:
$ dig @8.8.8.8 +short MX gmail.com
10 alt1.gmail-smtp-in.l.google.com.
20 alt2.gmail-smtp-in.l.google.com.
30 alt3.gmail-smtp-in.l.google.com.
5 gmail-smtp-in.l.google.com.
40 alt4.gmail-smtp-in.l.google.com.
So we want to remove that with sed:
$ dig @8.8.8.8 +short MX gmail.com | sed "s/^[0-9]* //g" | sed "s/.$//"
gmail-smtp-in.l.google.com
alt2.gmail-smtp-in.l.google.com
alt1.gmail-smtp-in.l.google.com
alt3.gmail-smtp-in.l.google.com
alt4.gmail-smtp-in.l.google.com
Getting the IPv4 addresses from the MX FQDNs
From here, we want the IPv4 addresses of the FQDNs which is basically like this dig query:
$ dig @8.8.8.8 +short gmail-smtp-in.l.google.com
74.125.136.26
That needs to be repeated for all FQDNs, so xargs comes in handy. In this case there is no need for the -0 parameter as there is no separator problem.
As potentially, duplicate entries can be returned, we filter the xargs dig result with uniq:
$ dig @8.8.8.8 +short MX gmail.com | sed "s/^[0-9]* //g" | sed "s/.$//" | xargs -I {} dig @8.8.8.8 +short "{}" | uniq | sort
173.194.72.27
173.194.72.27
173.194.73.27
64.233.189.27
74.125.136.27
The result isn’t stable enough
It appears that in practice, these IPv4 addresses aren’t too stable over time to fill an exclusion table with individual ones in the ER5120 interface as over a few days, the changed, sometimes even during the day:
Same day afternoon Same day morning Previous day afternoon Previous day morning 173.194.72.27
173.194.72.27
173.194.73.27
64.233.189.27
74.125.136.27173.194.72.26
74.125.136.27
74.125.200.27
74.125.23.27
74.125.28.27173.194.72.27
173.194.79.27
74.125.130.27
74.125.130.27
74.125.136.27173.194.72.27
64.233.188.27
74.125.136.27
74.125.200.27
74.125.28.27
In retrospect, I should have checked the TTLs of both the MX records and the IPv4 addresses where you can see the first is less than an hour and the second less than 2 minutes:
$ dig +nocmd +noall +answer +ttlid MX gmail.com
gmail.com. 2983 IN MX 30 alt3.gmail-smtp-in.l.google.com.
gmail.com. 2983 IN MX 20 alt2.gmail-smtp-in.l.google.com.
gmail.com. 2983 IN MX 5 gmail-smtp-in.l.google.com.
gmail.com. 2983 IN MX 10 alt1.gmail-smtp-in.l.google.com.
gmail.com. 2983 IN MX 40 alt4.gmail-smtp-in.l.google.com.
$ dig +nocmd +noall +answer +ttlid gmail-smtp-in.l.google.com.
gmail-smtp-in.l.google.com. 114 IN A 173.194.65.26
Apparently – and not surprisingly – Google uses a pool of MX systems distributing the load using short lived DNS entries even though that isn’t always the best way to load balance.
So back to the drawing board and asking the question “Is there a list of MX subnets available?”.
And there is: Google IP address ranges – Google Apps Administrator Help
That page explains how to get them using nslookup: basically get the SPF records, then go from there.
The dig example for this is:
$ dig @8.8.8.8 TXT +short _spf.google.com
"v=spf1 include:_netblocks.google.com include:_netblocks2.google.com include:_netblocks3.google.com ~all"
We’re after the various netblocks, so we need to cut some parts of the string and split it. Since sed on Mac OS X does not understand inserting newlines, this comes down to a combination of multiple regex replace expressions in one perl -pe (print-loop; program as expression) command:
$ dig @8.8.8.8 TXT +short _spf.google.com | perl -pe 's/^"v=spf1 include://g ; s/ ~all"$//g ; s/ include:/\n/g'
_netblocks.google.com
_netblocks2.google.com
_netblocks3.google.com
I later simplified it using tr:
$ dig @8.8.8.8 TXT +short _spf.google.com | tr ' ' '\n' | grep "^include:" | sed 's/include://g'
_netblocks.google.com
_netblocks2.google.com
_netblocks3.google.com
The next is to get the invidivual IPv4 records for each netblock with a statement like this:
$ dig @8.8.8.8 TXT +short _netblocks.google.com
"v=spf1 ip4:64.18.0.0/20 ip4:64.233.160.0/19 ip4:66.102.0.0/20 ip4:66.249.80.0/20 ip4:72.14.192.0/18 ip4:74.125.0.0/16 ip4:108.177.8.0/21 ip4:173.194.0.0/16 ip4:207.126.144.0/20 ip4:209.85.128.0/17 ip4:216.58.192.0/19 ip4:216.239.32.0/19 ~all"
Each block is in CIDR notation so can contain many IP-addresses. You can also use whois to lookup the CIDRs from the names.
This must be processed in a similar way as the first SPF logic:
$ dig @8.8.8.8 TXT +short _netblocks.google.com | tr ' ' '\n' | grep "^ip4:" | sed 's/ip4://g'
Using xargs, the full command then becomes this:
$ dig @8.8.8.8 TXT +short _spf.google.com | tr ' ' '\n' | grep "^include:" | sed 's/include://g' | xargs -I {} dig @8.8.8.8 TXT +short {} | tr ' ' '\n' | grep '^ip4:' | sed 's/ip4://g' | uniq | sort
108.177.8.0/21
172.217.0.0/19
173.194.0.0/16
207.126.144.0/20
209.85.128.0/17
216.239.32.0/19
216.58.192.0/19
64.18.0.0/20
64.233.160.0/19
66.102.0.0/20
66.249.80.0/20
72.14.192.0/18
74.125.0.0/16
So GMail uses 13 blocks of IP addresses. There are many in them!
Anyway: since the TP-LINK ER5120 doesn’t understand CIDR, it needs netmasks. This could be done by creating bash functions mask2cdr and cdr2mask, but that’s a bit too convoluted right now.
There is a great ipcalc tool by Krischan Jodies which is available on most Linux distributions (like for OpenSuSE or in homebrew) and online at http://jodies.de/ipcalc.
$ dig @8.8.8.8 TXT +short _spf.google.com | tr ' ' '\n' | grep "^include:" | sed 's/include://g' | xargs -I {} dig @8.8.8.8 TXT +short {} | tr ' ' '\n' | grep '^ip4:' | sed 's/ip4://g' | uniq | sort | xargs -I {} ipcalc {} | egrep "Address:|Netmask:"
Address: 108.177.8.0 01101100.10110001.00001 000.00000000
Netmask: 255.255.248.0 = 21 11111111.11111111.11111 000.00000000
Address: 172.217.0.0 10101100.11011001.000 00000.00000000
Netmask: 255.255.224.0 = 19 11111111.11111111.111 00000.00000000
Address: 173.194.0.0 10101101.11000010. 00000000.00000000
Netmask: 255.255.0.0 = 16 11111111.11111111. 00000000.00000000
Address: 207.126.144.0 11001111.01111110.1001 0000.00000000
Netmask: 255.255.240.0 = 20 11111111.11111111.1111 0000.00000000
Address: 209.85.128.0 11010001.01010101.1 0000000.00000000
Netmask: 255.255.128.0 = 17 11111111.11111111.1 0000000.00000000
Address: 216.239.32.0 11011000.11101111.001 00000.00000000
Netmask: 255.255.224.0 = 19 11111111.11111111.111 00000.00000000
Address: 216.58.192.0 11011000.00111010.110 00000.00000000
Netmask: 255.255.224.0 = 19 11111111.11111111.111 00000.00000000
Address: 64.18.0.0 01000000.00010010.0000 0000.00000000
Netmask: 255.255.240.0 = 20 11111111.11111111.1111 0000.00000000
Address: 64.233.160.0 01000000.11101001.101 00000.00000000
Netmask: 255.255.224.0 = 19 11111111.11111111.111 00000.00000000
Address: 66.102.0.0 01000010.01100110.0000 0000.00000000
Netmask: 255.255.240.0 = 20 11111111.11111111.1111 0000.00000000
Address: 66.249.80.0 01000010.11111001.0101 0000.00000000
Netmask: 255.255.240.0 = 20 11111111.11111111.1111 0000.00000000
Address: 72.14.192.0 01001000.00001110.11 000000.00000000
Netmask: 255.255.192.0 = 18 11111111.11111111.11 000000.00000000
Address: 74.125.0.0 01001010.01111101. 00000000.00000000
Netmask: 255.255.0.0 = 16 11111111.11111111. 00000000.00000000
Now my pluimers.com mail isn’t forwarded to the gmail servers any more through the Ziggo network (which is has their network segments blacklisted all the time because they don’t fight SPAM hard enough).
–jeroen
PS: if you don’t have dig, or don’t like commandlines, then you can use the online dig service, for example using the _netblocks query.
The Wiert Corner - irregular stream of stuff 
Getting your local IPv4 addresses, netmasks and CIDRs « The Wiert Corner – irregular stream of stuff said
[…] SoI was a bit premature when I wrote about “This could be done by creating bash functions mask2cdr and cdr2mask, but that’s a bit too convoluted right now” in Getting the IP addresses of gmail MX servers – via Super User – dig isn’t enough. […]