nixos.org domain resolution failures

**Affected service** Nameserver authoritative for nixos.org (ns1.gandi.net). **Describe the issue** On some networks, the nameserver authoritative for nixos.org returns SERVFAIL in reply to queries of `nixos.org`. Subdomains like `cache.nixos.org` and `hydra.nixos.org` are unaffected, though. **System information** I tested from 3 different ISPs, 2/3 are affected: 1. Iliad/Free SAS, affected ``` $ mtr ns1.gandi.net wes -> ns1.gandi.net (173.246.100.2) Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. 194.149.186.30 2.7% 297 2.3 2.9 2.1 4.3 0.4 2. i3dnet.mix-it.net 0.0% 297 3.0 4.0 2.2 44.1 3.5 3. ns-1-a.gandi.net 0.0% 297 2.9 3.0 2.1 3.9 0.4 $ host -4 -t A nixos.org ns1.gandi.net Using domain server: Name: ns1.gandi.net Address: 173.246.100.2#53 Aliases: Host nixos.org not found: 2(SERVFAIL) ``` 2. Vodafone IT, affected ``` $ mtr ns1.gandi.net maxwell -> ns1.gandi.net (173.246.100.2) Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. net-x-y-z-t.cust.vodafonedsl.it 0.0% 5 12.6 14.0 12.6 16.3 1.9 2. (waiting for reply) 3. (waiting for reply) 4. vodafone-it-gw-mlk.cw.net 0.0% 5 16.7 17.1 16.7 17.4 0.3 5. 195.59.1.25 0.0% 5 16.2 16.5 15.4 19.6 1.8 6. ae12-xcr1.mlu.cw.net 0.0% 5 16.8 19.7 16.5 29.4 5.5 7. ae15-xcr1.mlu.cw.net 0.0% 5 15.6 16.8 15.5 20.5 2.1 8. ns-1-a.gandi.net 0.0% 5 16.8 16.9 16.8 17.3 0.2 $ host -4 -t A nixos.org ns1.gandi.net Using domain server: Name: ns1.gandi.net Address: 173.246.100.2#53 Aliases: Host nixos.org not found: 2(SERVFAIL) ``` 3. Telecom Italia, unaffected ``` $ mtr ns1.gandi.net edge -> ns1.gandi.net (173.246.100.2) Packets Pings Host Loss% Snt Last Avg Best Wrst StDev 1. (waiting for reply) 2. 172.17.185.16 0.0% 15 7.3 7.1 6.9 7.7 0.2 3. 172.17.185.26 0.0% 15 8.0 7.5 7.1 8.0 0.3 4. 172.19.184.86 0.0% 15 8.8 8.6 8.3 8.9 0.2 5. 172.19.177.20 0.0% 14 8.5 11.9 8.1 55.6 12.6 6. 195.22.192.144 0.0% 14 8.8 10.8 8.1 40.7 8.6 7. 195.22.210.120 0.0% 14 25.5 28.3 25.4 60.2 9.2 8. 195.22.210.69 0.0% 14 26.6 26.3 26.0 26.6 0.2 9. et20-1.core-2.csd5.gandi.net 0.0% 14 25.9 25.8 25.6 26.1 0.2 10. ns-1-a.gandi.net 0.0% 14 26.6 26.3 26.1 26.6 0.2 $ host -4 -t A nixos.org ns1.gandi.net Using domain server: Name: ns1.gandi.net Address: 173.246.100.2#53 Aliases: nixos.org has address 63.176.8.218 nixos.org has address 35.157.26.135 ``` This is a trace from my recursive nameserver for a `nixos.org` A query: <details> ``` 259 nixos.org: Wants DNSSEC processing, auth data required by query for A 288 nixos.org: No valid SOA found for ., which is the best match for nixos.org 291 QM nixos.org: Step0 Not cached 293 QM nixos.org: Checking if we have NS in cache for 'nixos.org' 299 QM nixos.org: NS (with ip, or non-glue) in cache for 'nixos.org' -> 'ns-161-c.gandi.net', within bailiwick: 0, not in cache / did not look at cache 305 QM nixos.org: NS (with ip, or non-glue) in cache for 'nixos.org' -> 'ns-61-b.gandi.net', within bailiwick: 0, not in cache / did not look at cache 309 QM nixos.org: NS (with ip, or non-glue) in cache for 'nixos.org' -> 'ns-177-a.gandi.net', within bailiwick: 0, not in cache / did not look at cache 319 QM nixos.org: We have NS in cache for 'nixos.org' (flawedNSSet=0) 322 QM nixos.org: Step1 Ancestor from cache is nixos.org 325 QM nixos.org: Step2 New child nixos.org 327 QM nixos.org: Step3 Going to do final resolve 328 nixos.org: Wants DNSSEC processing, auth data required by query for A 337 nixos.org: No valid SOA found for ., which is the best match for nixos.org 341 nixos.org: No cache hit for 'nixos.org|A', trying to find an appropriate NS record 343 nixos.org: Checking if we have NS in cache for 'nixos.org' 346 nixos.org: NS (with ip, or non-glue) in cache for 'nixos.org' -> 'ns-161-c.gandi.net', within bailiwick: 0, not in cache / did not look at cache 350 nixos.org: NS (with ip, or non-glue) in cache for 'nixos.org' -> 'ns-61-b.gandi.net', within bailiwick: 0, not in cache / did not look at cache 354 nixos.org: NS (with ip, or non-glue) in cache for 'nixos.org' -> 'ns-177-a.gandi.net', within bailiwick: 0, not in cache / did not look at cache 359 nixos.org: We have NS in cache for 'nixos.org' (flawedNSSet=0) 367 nixos.org: Cache consultations done, have 3 NS to contact 391 nixos.org: Nameservers: ns-177-a.gandi.net(0.01ms), ns-61-b.gandi.net(0.01ms), ns-161-c.gandi.net(6.49ms) 426 nixos.org: Trying to resolve NS 'ns-177-a.gandi.net' (1/3) 436 ns-177-a.gandi.net: Nameserver ns-177-a.gandi.net IPs: 173.246.100.178(0.01ms) 456 nixos.org: Resolved 'nixos.org' NS ns-177-a.gandi.net to: 173.246.100.178 462 nixos.org: Trying IP 173.246.100.178:53, asking 'nixos.org|A' 210481 nixos.org: ns-177-a.gandi.net (173.246.100.178) returned a Server Failure, trying sibling IP or NS 210497 nixos.org: Trying to resolve NS 'ns-61-b.gandi.net' (2/3) 210507 ns-61-b.gandi.net: Nameserver ns-61-b.gandi.net IPs: 213.167.230.62(0.01ms) 210515 nixos.org: Resolved 'nixos.org' NS ns-61-b.gandi.net to: 213.167.230.62 210520 nixos.org: Trying IP 213.167.230.62:53, asking 'nixos.org|A' 416681 nixos.org: ns-61-b.gandi.net (213.167.230.62) returned a Server Failure, trying sibling IP or NS 416694 nixos.org: Trying to resolve NS 'ns-161-c.gandi.net' (3/3) 416703 ns-161-c.gandi.net: Nameserver ns-161-c.gandi.net IPs: 217.70.187.162(6.44ms) 416711 nixos.org: Resolved 'nixos.org' NS ns-161-c.gandi.net to: 217.70.187.162 416716 nixos.org: Trying IP 217.70.187.162:53, asking 'nixos.org|A' 740944 nixos.org: ns-161-c.gandi.net (217.70.187.162) returned a Server Failure, trying sibling IP or NS 740956 nixos.org: Failed to resolve via any of the 3 offered NS at level 'nixos.org' 740963 nixos.org: Failed (res=-1) 740967 QM nixos.org: Step3 Final resolve: Server Failure/0 ``` </details> Note: IPv6 and AAAA are similarly affected.