Feedback & list of issues I am facing

[mansoorr123]

Greetings!!
Thanks for this wonderful tool.
I am facing following issues while using this tool:

1. The tool is not scanning all domains (when given list of 10000 domains hardly scans 1000 of them which I verified by grep "QUESTION SECTION" in output file generated with -o F switch).
   Although above problem is solved for now by using -s 1000(default is 10000) so I think you should change the default value to 1000.
2. No explicit option to grep Domain having NXDOMAIN status since it is required for finding domains vulnerable to subdomain takeover(like Azure).
   Although above problem can be solved by :
   ./massdns -q -t A -r /opt/resolvers.txt domain_list.txt -o F -w output_massdns.txt
   cat output_massdns.txt | grep NXDOMAIN -A 7 | grep CNAME
   But it is not efficient.
3. Getting different result when using -o F and then -w and only using -w (without any output flag)
   Although result should come same.So please tell me if -o F append some extra details which are drop when we are not using it.
4. Include Option which do below operation:
   Take list of domains and output only those domain names whose IP can be resolved.
   (Should also exclude those domains which have status NO ERROR but only have CNAME and no IP)
   Since this list can be passed further to httprobe(or CURL) or other tools which do some active scanning.(As list fetched above will assure that the domains' IP recored exist).

[SbIm]

For your second issue, in my test results, -t A option will output both A record and NXDOMAIN + CNAME results, the latter one can be recognized as subdomain takeover feature.

And for a big domains list, I got different result every time, maybe I have the same issue -s 1000

[mzpqnxow]

@mansoorr123 for #4, this is now supported by using the feature implemented in #72. You can use --ignore multiple times to ignore all answers except A, AAAA and CNAME

The other problems you are having are probably because you're being too aggressive and either overloading the recursive resolver or the authoritative NS with the SOA for the domain(s) you are enumerating. You should try a smaller -s value. You should also stick to public recursive resolvers that are intentionally open to the public and resourced adequately. Using random resolvers from lists across the Internet is likely to give you bad results

For your second issue.. and probably all of your issues... why are you using -o F if you're going to consume the data programmatically? You should be using -o J which outputs NDJSON. This is very, very easy to parse efficiently using any scripting language, including jq which you can use on the command line to do very advanced and efficient processing of the output

[mansoorr123]

@mansoorr123 for #4, this is now supported by using the feature implemented in #72. You can use --ignore multiple times to ignore all answers except A, AAAA and CNAME

The other problems you are having are probably because you're being too aggressive and either overloading the recursive resolver or the authoritative NS with the SOA for the domain(s) you are enumerating. You should try a smaller -s value. You should also stick to public recursive resolvers that are intentionally open to the public and resourced adequately. Using random resolvers from lists across the Internet is likely to give you bad results

For your second issue.. and probably all of your issues... why are you using -o F if you're going to consume the data programmatically? You should be using -o J which outputs NDJSON. This is very, very easy to parse efficiently using any scripting language, including jq which you can use on the command line to do very advanced and efficient processing of the output

Thanks for your comment. At the time of writing this issue NDJSON format contains very less fields (like STATUS of domain was absent). Now I have migrated to NDJSON format.

[blechschmidt]

The -s parameter should be chosen depending on your bandwidth and the number of used resolvers. I might reconsider changing the default value if I receive more feedback that a default value of 10,000 is too large.

Concerning the second issue, 8cb5723 now introduces a --filter option which allows to only output packets matching a specific response code (i.e. it can be used to filter NXDOMAIN responses).

The third issue might arise from the randomness used to select resolvers or from rate limits being enforced causing some resolvers not to reply anymore. It shouldn't have to do anything with the output format.