add ffufplus

• Add a docker environment for the Laravel app (lightweight and easier to setup for the install script)

Install.sh

• subjack

• Added Laravel application and dependencies in the install script

• Updated the nginx config file to fit the Laravel app

• Added native Golang and changed subfinder usage from Docker to Golang

• Added RVM and Ruby, which makes Aquatone usable without Docker.

full commit info: 7f0babe

Describe the bug
It looks like that the loop for nmap is using a wrong list? At https://github.com/x1mdev/ReconPi/blob/master/recon.sh#L131 the file with host names is used for scanning ports. But in that list there could be a lot of hosts with the same ip (all subdomains). That would take a long time to scan the same host.

To Reproduce
Run the scan against a wildcard DNS domain. All subdomains are valid and producing an IP

Expected behavior
Use the list of "$IPS"/"$domain"-ips.txt of line https://github.com/x1mdev/ReconPi/blob/master/recon.sh#L123 In there are the unique IP numbers of all the subdomain hosts.

Screenshots
None

Desktop (please complete the following information):
Linux black-pearl 4.19.75-v7+ #1270 SMP Tue Sep 24 18:45:11 BST 2019 armv7l GNU/Linux (RPI 3B+)

Additional context
I think that port scanning is not depending on (v)host names. So therefore I would think that only the unique IP address are enough for port scanning. Could be wrong....

Create the index.html that will host the scan results

Describe the bug
When running an fresh ReconPi installation into an Raspberry Pi 2 + Ubuntu Server 20.04, Nuclei can't run because of missing template folder.
I already downloaded manually those Templates into /home/user/tools but ReconPi seems not find it.

To Reproduce
Steps to reproduce the behavior:

1. Just install a fresh ReconPi and run it.

Screenshots

GFSCAN="$RESULTDIR/gfscan"

: 'Use gf to find secrets in meg file'
startGfScan(){
startFunction "Gf scan in meg files"
cd "$SUBS"/meg
for i in gf -list;
do
gf ${i} > "$GFSCAN"/"${i}".txt
[[ -s "$GFSCAN"/"${i}".txt ]] || rm "$GFSCAN"/"${i}".txt
done
cd -
}

startGfScan # add function after startMeg to run

Describe the bug
Installation script throws fatal error when the machine runs out of memory, both in Raspbian and HypriotOS.

To Reproduce
Steps to reproduce the behavior:

1. Install fresh Raspbian OR HypriotOS
2. curl or git clone the installation script.
3. run the installation script
4. somewhere in the middle of the installation, a fatal error: runtime: out of memory is thrown.

Expected behavior
The installation script should go well in RP1B.

Screenshots
Screenshot (error from half of the picture)

Desktop (please complete the following information):

• OS: Raspbian Bullseye AND HypriotOS
• Browser: none.

What is the recommended way to update the script?
"Git pull" or "Rm -rf" and fresh install?

The grep function needs to be a bit more specific, it does grep $URL >> domains.txt and if there is an empty subdomain like .sub.domain.tld it will give an error:

[+] Subfinder scan complete, checking which domains resolve..
host: '' is not in legal name syntax (unexpected end of input)
[+] Resolved domains written to resolveddomains.txt

It still writes the output but it's a bit messy

The dashboard is pretty basic right now.

the recon scan already has IP addresses stored from the scanned targets.

• Adjust the API file (domains.json) to include IP addresses.
• Edit the API and frontend from https://github.com/x1mdev/subdomainDB to accept more data.

Docker doesn't seem to like all images for RPi's

After the automated installation of Golang the Go tools are not installed, go, command not found.

When the device is rebooted after installation the tools can be installed..

Move error when you run multiple scans to one target.

Nuclei missing tamplates for basic-detacions and brute-force

AND

Nuclei missing part of the path to the template

v2.0 release merge

Describe the bug
running recon on a tld I get a traceback and an error telling me it cannot import module named 'tld' even though it is a fresh install and followed the easy install method.

To Reproduce
Steps to reproduce the behavior:

1. Install Hypriot image (version 1.10.0)
2. Login to PI
3. install using curl -L https://raw.githubusercontent.com/x1mdev/ReconPi/master/install.sh | bash
4. recon domain

Expected behavior
I expect to get no errors

Screenshots
image

Desktop (please complete the following information):

• OS: Hypriot image (version 1.10.0)
• Version 1.10.0

Additional context
I followed the instructions listed here
https://x1m.nl/posts/recon-pi/

Describe the bug

Httprobe has no -prefer-https parameter.

Expected behavior

Screenshots

Additional context
Httprobe change to fprobe

I manage to fix this error when change the Nmap command in naabu2nmap.sh to:

echo "[+] Getting the basics..";
sudo apt-get update -y;
sudo apt-get upgrade -y;

This could hose people's setups :(

Aways get this error from Gf Scan, any ideas how to fix it?

Can you add to the project option for Discord webhook?

After adding the GitHub API key I am having this error:

"unable to cache TLDs in file /usr/local/lib/python3.7/dist-packages/tldextract/.tld_set: [Errno 13] Permission denied: '/usr/local/lib/python3.7/dist-packages/tldextract/.tld_set'"

So after take a look at /usr/local/lib/python3.7/dist-packages/tldextract/ folder the actual name of the .tld_set file was .tld_set_snapshot so I have to chmod +775 the file and rename it to .tld_set and now its OK

subfinder brute force is taking a LONG time

maybe skip bruteforce

cat "$SUBS"/hosts | parallel -j 5 --bar --shuf gobuster dir -u {} -t 50 -w wordlist.txt -l -e -r -k -q -o "$DIRSCAN"/"$sub".txt
/home/admin/go/bin/gobuster dir -u "$line" -w "$WORDLIST"/wordlist.txt -e -q -k -n -o "$DIRSCAN"/out.txt

the wordlist.txt is not copied to the wordlists folder found in the resultdir anywhere in the recon.sh script

• output needs to be .json
• .json file needs to be converted to subdomainDB API format
• Extra recon.sh function to make the API POST request

Idea

Maybe a good idea to start the dashboard during install.sh to avoid errors when running recon.sh for a second or third time.

The runDashboard() function within recon.sh now starts up a new dashboard instance via Docker on port 4000 every time you run the script, so the port will become an issue.

Instead of starting the dashboard application it should just make the API request with the json output from subfinder.

Check the way subfinder uses host file in the docker cmd, can be used for resolved-domains.txt as well I guess

: 'Run MassDNS on the given domains'
runMassDNS()
{
	echo -e "[$GREEN+$RESET] Starting MassDNS now!"

	#This doesn't work yet because I need to find a way to get the resolved-domains.txt from the host to docker.
	docker run -it massdns -r lists/resolvers.txt -t A -o S -w resolved-domains.txt > $ROOT/$1/massdns.txt

	echo -e "[$GREEN+$RESET] Done!"
}

Naabu has a new version - v.2 maybe that's the problem

screenshot failed: signal: trace/breakpoint trap

Can it be used without Raspberry Pi
Additional context
Add any other context or screenshots about the feature request here.

Made a new repo for the Dashboard: https://github.com/x1mdev/ReconPiDB

When it is done it can be easily git cloned and installed via install script.

the domains.json file is not getting the right input data, not really sure what is going on there

https://github.com/x1mdev/ReconPi/blob/master/recon.sh

: 'Convert domains.txt to json (subdomainDB format) + make POST API request with output from subfinder'
convertDomainsFile()
{
	echo -e "[$GREEN+$RESET] Converting $GREEN$ROOT/$1/domains.txt$RESET to an acceptable $GREEN.json$RESET file.."
	cat $ROOT/$1/domains.txt | grep -P "([A-Za-z0-9]).*$1" >> $ROOT/$1/domains.json
	echo -e "{\\n\"domains\":"; jq -MRs 'split("\n")' < domains.json | sed -z 's/,\n  ""//g'; echo -e "}"
	
	# TODO: Post request to dashboard - work in progress
	#curl -X POST -H "Content-Type: application/json" -H "X-Hacking: is Illegal!" -d "@domains.json" http://127.0.0.1:4000/api/domain/:domain

}

@ebelties any ideas?

notifyDiscord() {
startFunction "Trigger Discord Notification"
takeover="$(cat $SUBS/takeovers | wc -l)"
totalsum=$(cat $SUBS/hosts | wc -l)
intfiles=$(cat $NUCLEISCAN/*.txt | wc -l)
nucleiCveScan="$(cat $NUCLEISCAN/cve.txt)"
nucleiFileScan="$(cat $NUCLEISCAN/files.txt)"

/usr/local/bin/discord.sh --webhook-url="$WEBHOOK" --text "Found '$totalsum' live hosts for '$domain'" --username "Recon Bot"

/usr/local/bin/discord.sh --webhook-url="$WEBHOOK" --text "Found '$intfiles' interesting files using nuclei" --username "Recon Bot"

/usr/local/bin/discord.sh --webhook-url="$WEBHOOK" --text "Found '$takeover' subdomain takeovers on '$domain'" --username "Recon Bot"

/usr/local/bin/discord.sh --webhook-url="$WEBHOOK" --text "CVEs found for $domain: \n $nucleiCveScan" --username "Recon Bot"

/usr/local/bin/discord.sh --webhook-url="$WEBHOOK" --text "Files for $domain: \n $nucleiFileScan" --username "Recon Bot"

echo -e "[$GREEN+$RESET] Done."

}

Dependencies:
https://github.com/ChaoticWeg/discord.sh

I'm doing some test runs with https://github.com/smiegles/subdomainDB to see if i can port it to local network with Nginx

Describe the bug
While running the installation script the go tools are not getting installed.

To Reproduce
Steps to reproduce the behavior:

1. Cloned the repo into a new droplet/VPS. git clone https://github.com/x1mdev/ReconPi.git
2. goto the cloned repo and run the installation script cd ReconPi && ./install.sh

Expected behavior
The go tools must be installed successfully when we run the installation script.

Screenshots

Used digital ocean droplet/VPS

• OS: Debian
• Version: 10 x64
• Other details: 2 GB Memory / 1 Intel vCPU

Additional context
More details on go install are here https://go.dev/doc/go-get-install-deprecation