tumi8 / goscanner Goto Github PK

When using the TLS scanner and generating fingerprints, the final fingerprint csv is produced with the correct header labels - the ip and server_name column are switched. Doing some digging there is a high volume of failures - particularly with CDNs as the scanning is using the ip rather than the server name. I believe this is because the massdns query supplied with the readme gives the input the incorrect way around. When using a csv input with server_name, ip I see high failures, but good success when the input is ip, server_name. However switching the order means the fingerprint wont build. It appears the intermediate hosts.csv.tmp.sorted file is being incorrectly produced:

id,ip,port,server_name,client_hello
1,chanel.com.,443,23.57.76.60,random9057

which is effected the downstream building of the fingerprints? Any chance someone can take a look please? (I'm using this for some university research, thanks!)

edit:
To elaborate the only success i was seeing was :
SUCCESS,"[""tls: invalid certificate chain: x509: cannot validate certificate for 104.18.144.174 because it doesn't contain any IP SANs""]

To fix it locally i generated the dns output:
bin/massdns -r lists/resolvers.txt domains.txt -q -o J | jq -r '[.name, .data.answers[-1].data] | @csv' | tr -d '"' | awk -F, '{print $2","$1}' | awk -F, '$1!=""' > domains.csv

And made a small change in the generate_fingerprints codebase:
prepOneHeader := fmt.Sprintf(echo "id,ip,port,server_name,client_hello" > %s, hostsSortedFile)

Which is giving a much higher volume of success now.

The code makes use of https://github.com/jteeuwen/go-bindata to compile IANA's TLS version data into Go source code.

While the repo above still exists, it has a dependency on golang.org/x/tools/cmd/vet, which no longer exists.

Hence, the compile chain breaks.