SPF Permanent Error: Too many DNS lookups

Question

We have noticed that a lot of our emails are falsely flagged as spam. Upon reading online, it seems like a good way to solve this issue is to add an SPF record into the DNS, so we added a TXT record with this content:

v=spf1 a mx ip4:162.123.189.010 include:_spf.google.com include:bluehost.com ~all

Bluehost is our host provider, 162.123.189.010 is our VPS IP address from blue host, and _spf.google.com is needed because we send/receive email using GMail.

After running a test on Google's MX tester, we got the following error:

The SPF string can not be parsed, do you have any typos in it?
Decision    permanent error in processing
Explanation SPF Permanent Error: Too many DNS lookups
Record  v=spf1 a mx ip4:162.123.189.010 include:_spf.google.com include:bluehost.com ~all

Does anyone have any idea what the issue is?

Answer 1

We were also having trouble with the Bluehost default SPF-record, and the other solutions here unfortunately didn't work for us.

In our case we had a permerror of too many DNS lookups with 17 of 10 allowed. This was with the dafault Bluehost SPF record of v=spf1 a mx ptr include:bluehost.com ?all.

However, Bluehost have an alternative SPF record that did the trick for us. They describe it here: https://www.bluehost.com/help/article/dns-spf. From the article:

Using SPF Records with Shared and Cloud Hosting

Account Manager Accounts

The outgoing email server for these accounts does not use proxy IPs from unifiedlayer.com. Instead, the outgoing email server for Account Manager accounts is proxied to websitewelcome.com. Email is not transmitted from unifiedlayer.com proxy IPs, hence websitewelcome.com should be used in the SPF record configuration rather than brand-specific URLs.

And the SPF record they suggest is as follows:

v=spf1 a mx include:websitewelcome.com ~all

This record worked for us and prevented the permerror of too many DNS lookups, although I have to add that we didn't need additional SPF-includes.

Answer 2

"SPF Permanent Error: Too many DNS lookups" is a very specific problem. Your record is too big and SPF checkers will refuse to perform enough DNS queries to determine if something passed SPF.

The SPF spec allows at most 10 DNS lookups. Your SPF record has 17.

RFC 4408 § 10.1 – Processing Limits states:

SPF implementations MUST limit the number of mechanisms and modifiers that do DNS lookups to at most 10 per SPF check, including any lookups caused by the use of the "include" mechanism or the "redirect" modifier. If this number is exceeded during a check, a PermError MUST be returned. The "include", "a", "mx", "ptr", and "exists" mechanisms as well as the "redirect" modifier do count against this limit. The "all", "ip4", and "ip6" mechanisms do not require DNS lookups and therefore do not count against this limit. The "exp" modifier does not count against this limit because the DNS lookup to fetch the explanation string occurs after the SPF record has been evaluated.

Your SPF record has four lookups before traversing the inclusions, including your a and mx:

v=spf1 a mx ip4:162.123.189.010 include:_spf.google.com include:bluehost.com ~all

Google's SPF

Google has three DNS lookups for three collections of CIDRs that it blesses:

_spf.google.com (+3 lookups)

v=spf1 include:_netblocks.google.com include:_netblocks2.google.com
  include:_netblocks3.google.com ~all

_netblocks.google.com

v=spf1 ip4:35.190.247.0/24 ip4:64.233.160.0/19
  ip4:66.102.0.0/20 ip4:66.249.80.0/20 ip4:72.14.192.0/18 ip4:74.125.0.0/16
  ip4:108.177.8.0/21 ip4:173.194.0.0/16 ip4:209.85.128.0/17
  ip4:216.58.192.0/19 ip4:216.239.32.0/19 ~all

_netblocks2.google.com

v=spf1 ip6:2001:4860:4000::/36
  ip6:2404:6800:4000::/36 ip6:2607:f8b0:4000::/36 ip6:2800:3f0:4000::/36
  ip6:2a00:1450:4000::/36 ip6:2c0f:fb50:4000::/36 ~all

_netblocks3.google.com

v=spf1 ip4:172.217.0.0/19 ip4:172.217.32.0/20
  ip4:172.217.128.0/19 ip4:172.217.160.0/20 ip4:172.217.192.0/19
  ip4:108.177.96.0/19 ip4:35.191.0.0/16 ip4:130.211.0.0/22 ~all"

Bluehost's SPF

The SPF record for bluehost.com is too large (its SPF record fails Google's MX tester on its own):

bluehost.com (5 lookups before further traversal)

v=spf1 include:spf2.bluehost.com include:_spf.qualtrics.com
  include:_spf.google.com include:_spf.salesforce.com
  include:sparkpostmail.com -all

spf2.bluehost.com (+0)

v=spf1 ip4:66.147.240.0/20 ip4:69.89.16.0/20 ip4:74.220.192.0/19
  ip4:67.222.32.0/19 ip4:70.40.192.0/19 ip4:67.20.64.0/18 ip4:173.254.0.0/17
  ip4:50.87.0.0/16 ip4:69.195.64.0/18 -all

_spf.qualtrics.com (+0)

v=spf1 ip4:139.60.152.0/22 ip4:162.247.216.0/22 ip4:54.186.193.102/32
  ip4:52.222.73.120/32 ip4:52.222.73.83/32 ip4:52.222.62.51/32
  ip4:52.222.75.85/32 ?all

(see above for _spf.google.com's +3 lookups, though redundant lookups don't add to your total)

_spf.salesforce.com (+1 using an SPF macro with the IP address)

v=spf1 exists:%{i}._spf.mta.salesforce.com -all

sparkpostmail.com is a redirection and then another exists macro and a pile of pointers (+6, wow)

v=spf1 redirect=_spf.sparkpostmail.com
v=spf1 exists:%{i}._spf.sparkpostmail.com include:_netblocks.sparkpostmail.com
  ptr:sparkpostmail.com ptr:spmta.com ptr:flyingenvelope.com ~all

Danger! That sparkpost.com inclusion pulls in some ptr entries, which are arguably insecure, using a deprecated and "strongly discouraged" SPF mechanism (that's a direct quote from the spec).

_netblocks.sparkpostmail.com was pulled in by the previous record (+0)

v=spf1 ip4:147.253.208.0/20 ip4:192.174.80.0/20 ~all

Bluehost used to use SendGrid, who actually knows what they're doing (their SPF record has no additional lookups), but apparently they have traded SendGrid for SparkPost, who (based on their six extra lookups plus the insecure ptr entries) does not.

Since that totals 12 (13 with include:bluehost.com), you cannot include Bluehhost's SPF.

Bluehost's own suggested SPF record (and its default for all customers) is similarly broken (with 16 lookups, including an easily forged ptr).

Solution for Bluehost: a trimmed and safer SPF record

📢 Hello, Bluehost. I tweeted this at you. This section is just for you.

Bluehost could fix this with the following SPF records in place of their current one:

bluehost.com (7)

v=spf1 a include:spf2.bluehost.com include:_spf.google.com
  include:_netblocks.sparkpostmail.com ~all

Though note that I had to downgrade include:sparkpostmail.com (7 + 6 = 13, too large, plus that includes the dangerous ptr records) to just its netblocks (7 + 0 ≤ 10). Bluehost needs to yell at SparkPost or go back to SendGrid. spf2.bluehost.com is unchanged from its current state and should be the only inclusion necessary for Bluehost customers.

(I'd use the IP for the A record to skip a lookup, but it changes so often that it looks like fast flux.)

Bluehost should suggest customers include just spf2.bluehost.com for all of the Bluehost services (assuming they're involved in sending mail). See the next section for how to advise Bluehost customers.

Solution for Bluehost customers

As noted in the previous section, start with this base (3 lookups):

v=spf1 a mx include:spf2.bluehost.com ~all

The final ~all ("soft failure") indicates mail recipients should be mildly dubious of —yet still deliver— mail that fails SPF. Set up DMARC to figure out what works and what is missed on the road to DMARC p=reject (which will block all forged mail).

You'll have to add any hosted email or Email Service Provider(s) you use, plus any other hosts that you want to authorize to send mail on behalf of your domain.

In the case of this question, I see an explicit IP address and hosted mail by Google, so you'll need:

v=spf1 ip4:162.123.189.10 a mx include:spf2.bluehost.com
  include:_spf.google.com ~all

Your total DNS lookup count is now seven and therefore your SPF is valid.

Answer 3

"SPF Permanent Error: Too many DNS lookups" is a common type of SPF permanent error. This happens when you have more than 10 DNS lookups in your SPF record.

SPF imposes the 10-DNS-lookup limit to mitigate DDoS attacks.

You can use any online SPF checker to check your SPF record and make sure it doesn't exceed that limit.

However, if your SPF record does exceed the limit, SPF authentication returns the permanent error mentioned above, which is in turn interpreted (in DMARC or otherwise) as fail. This means that the email can fail authentication and be moved to the spam folder. If no further action is taken, this will have a negative impact on your email deliverability.

To fix the too-many-DNS-lookup issue, you can use a service like DMARCLY's Safe SPF feature to automatically "flatten" your SPF record, so that it never exceeds the limit.

For more information on this, check out this post: Why SPF Authentication Fails

Answer 4

The most obvious problem is that leading 0 in your IP address, which makes it invalid. A minor issue is that it's considered best practice to put literal IPs first, as they are faster for receivers to evaluate. Give this a try:

v=spf1 ip4:162.123.189.10 a mx include:_spf.google.com include:bluehost.com ~all

Rather than using google's checker, I'd recommend Scott Kitterman's site, which is more likely to be accurate (Scott is one of the authors of the SPF spec), and spotted this exact problem.