When too many PTR RRs are returned nmap's parallel reverse DNS lookup fails as the register is truncated for not fitting into UDP and no fallback to TCP is made. The system-resolver works flawlessly there though.

Nmap should instead try to fallback in such a situation to doing a slower TCP request. This will more likely become a worse problem as signed DNS records become more common.