tomnomnom / gf Goto Github PK

I think I have installed this correctly as it isn't giving any error but after running it doesn't give any output.
So would like to know what can be the issue.

Unexpected error while executing something like echo $string | gf urls as part of the bash script that need sudo privileges (e.g. massdns). The simplest example would be:

Below is the content in my file.
there is a JWT token which is base64 encoded.

Cache-Control: max-age=0
< Set-Cookie: _ua={"session_id":"b24a5ce4-bd5a-4375-99a5-c5eb78c524c9","session_time_ms":1585593146162}; path=/; httponly
< Set-Cookie: jwt-session=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJkYXRhIjp7ImNzcmYtc2VjcmV0Ijoi77-9Tlxu77-9VSrvv71cdTAwMTNcdTAwMDfvv71T77-977-977-977-9XHUwMDEx77-9XHUwMDEx77-9Rlx1MDAxMu-_ve-_vSDvv70877-977-9McmiMSJ9LCJpYXQiOjE1ODU1OTMxNDYsImV4cCI6MTU4NTY3OTU0Nn0.Wkn8UFPcVJIk0pV1jQI8YqeI2FJstbpXq2UMa7B0tDU; path=/; expires=Tue, 31 Mar 2020 18:32:26 GMT; httponly
< X-Content-Type-Options: nosniff

when I use gf, it shows only half of the base64 encoded output. that is if there is any ' - ' in base64 format then gf regex stops there and outputs until that.

Is that a bug or expected result?
Any help is appreciated!

I am thinking that I might create a pull request that would allow use of backends such as ripgrep as a backend rather than grep.

Is this something you would be interested in?

Obviously, some of the flags are different when using a different tool. I propose changing the json files from this:

{
	"flags": "-oriahE",
	"pattern": "https?://[^\"\\'> ]+"
}

to something more like this:

{	"rg": {
		"flags": "-olIN --binary",
		"pattern": "https?://[^'\"]+"
	},
	"grep": {
		"flags": "-oriahE",
		"pattern": "https?://[^\"\\'> ]+"
	}
}

Note, I haven't tested the rg command above but it should give you an idea of how it could work.

For each json file, it would do the first that it could. The key for each section would be the command to run. In this case, it would try to use rg, if that wasn't available it would try grep.

This way, if someone wanted to use silver surfer or something else, they could.

I am not sure when I will get around to implementing this although it probably won't be for a couple of weeks.

Do you have any requests or suggestions? I am mostly looking for feedback on the proposed json format change.

fatal: 'xss' does not appear to be a git repository
fatal: Could not read from remote repository.

Is Possible Running gf for windows?

Current rules:

{
    "flags": "-hrioaE",
    "patterns": [
        "[a-z0-9.-]+\\.s3\\.amazonaws\\.com",
        "[a-z0-9.-]+\\.s3-[a-z0-9-]\\.amazonaws\\.com",
        "[a-z0-9.-]+\\.s3-website[.-](eu|ap|us|ca|sa|cn)",
        "//s3\\.amazonaws\\.com/[a-z0-9._-]+",
        "//s3-[a-z0-9-]+\\.amazonaws\\.com/[a-z0-9._-]+"
    ]
}

If I have a text file that contains:

S3 bucket: bucketname1.s3.amazonaws.com/xxxxx
S3 bucket: https://s3.amazonaws.com/bucketname2/xxxxx
S3 bucket: http://s3-us-east-2.amazonaws.com/bucketname3/xxxxx
S3 bucket: http://bucketname4.s3-ca-central-1.amazonaws.com/xxxxx

The current rule returns 3 buckets:

$ gf s3-buckets dummy.txt 
bucketname1.s3.amazonaws.com
//s3.amazonaws.com/bucketname2
//s3-us-east-2.amazonaws.com/bucketname3

By adding a single + to the second rule:

"[a-z0-9.-]+\\.s3-[a-z0-9-]+\\.amazonaws\\.com",

I get all of them:

$ gf s3-buckets dummy.txt 
bucketname1.s3.amazonaws.com
//s3.amazonaws.com/bucketname2
//s3-us-east-2.amazonaws.com/bucketname3
bucketname4.s3-ca-central-1.amazonaws.com

hello guys , hope someone here will help to resolve this issue :
i use kali linux and i unable to install gf with :

• go install github.com/tomnomnom/gf@latest && cp ~/go/bin/gf /usr/local/bin/ && gf -h && mkdir ~/.gf && git clone https://github.com/Sherlock297/gf_patterns.git && cd gf_patterns/ && cp *.json ~/.gf && gf -list
  cp: cannot stat '/root/go/bin/gf': No such file or directory

• go install github.com/tomnomnom/gf@latest
  ( gf is not installed )

then i try :

• go get -u github.com/tomnomnom/gf
  ( the some issue i got )

let's say we done like this:

echo/curl 'http://example.com/assets/app.js' | gf strings

now, it should only look for strings in app.js file. but gf will search for strings all over the system.

After gf runs for a period of time, it will treat the input file as a binary file, and grep can usually be solved by using the -a option. But how can gf avoid this problem

Just ran the regular installation process (go build && mv gf /usr/bin/ && cat zsh-completion.zsh >> ~/.zshrc && source ~/.zshrc) just fo find out gf wasn't working due to an alias I did not set up.

If you have oh-my-zsh and the git plugin activated (which I believe actually comes activated by default), then this plugin has an alias for gf command, for git fetch.

Console output:

$ gf -list
error: unknown switch `l'
usage: git fetch [<options>] [<repository> [<refspec>...]]
   or: git fetch [<options>] <group>
   or: git fetch --multiple [<options>] [(<repository> | <group>)...]
   or: git fetch --all [<options>] 
... ... ... 

I think there's a mode were missed and very useful, which is "--color" mode.

Since go get is now deprecated:

New installation guide is required, since go install is not installing examples folder and gf patterns cannot work without them.

Suggest a -l/--list flag to list all pattern names; could then be used for the autocomplete script.

Would you be interested in a PR that allowed multiple flags? For example, it might be useful to specify include or exclude flags which don't have the short hand arguments.

Having flags like

{
    "flags": "-HnroE --include={whatever}"
    ....
}

This doesn't work as it is given to the grep command as a single argument.

I am not sure what the best way to do this is. We could spit the flags key by the space character and pass each to grep, or we could make the flags json property an array like 'patterns'.

Greetings ,
I am able to install with the command of go get, but after installing it neither gives any error nor shows the file. And on terminal it says gf is not a directory neither a file. Why is it so? How can I fix it. I fixed this issue in some other tools by downloading its binary but since this tool doesn't have binary uploaded. So, what should I do, btw I did all the steps from installing to editing and till refreshing .bashrc ...Yet nothing worked.

go install -v github.com/tomnomnom/gf@latest

Hello @tomnomnom

awesome lib! Super useful and applicable to so many things!

I was wondering what you think about the idea to allow for folders in ~/.gf/ - this way one could organize the files better. If I would know golang, I would give it a try, but I'm too new in the game.

Peter

Im using Oh-my-zsh shell. Installed gf. But autocomplete is not working
Commands i used,
go get -u github.com/tomnomnom/gf

Added these lines in .zshrc,

export GOPATH="/Users/gopikrishna/go"
alias gf="$GOPATH/bin/gf"
source $GOPATH/src/github.com/tomnomnom/gf/gf-completion.zsh

when i hit tab after gf command it shows _gf:1: command not found: gf

Any help is appreciated!

I'm running zprezto (similar to oh-my-zsh) and autocomplete isn't working.

I've tried parsing multiple times, hupping the shell, etc.

But it just says, "no such pattern".

Actually go get -u is no longer supported, so I tried to install the package using go install github.com/tomnomnom/gf@latest. But after installing the package, there is no directory src/github.com/tomnomnom/gf/examples in $GOPATH.
So I can't complete installation and now can't use gf.

I have reached this error whenever i tried tab to auto complete
touch test.txt|gf gf: command not found touch |gf gf: command not found

I am pretty sure that i have copy the gf list to ~/.gf
also add source of auto complete to ~/.zshrc file

#Auto complete for gf source ~/go/pkg/mod/github.com/tomnomnom/[email protected]/gf-completion.zsh

Hello

Whenever I include gf in any one liner , I always get no pattern match error.
I have given two one liners below where I am getting this error.
gau $1 | gf lfi | qsreplace "/etc/passwd" | xargs -I % -P 25 sh -c 'curl -s "%" 2>&1 | grep -q "root:x" && echo "VULN! %"'
export LHOST="http://localhost"; gau $1 | gf redirect | qsreplace "$LHOST" | xargs -I % -P 25 sh -c 'curl -Is "%" 2>&1 | grep -q "Location: $LHOST" && echo "VULN! %"'

Could you please tell me why it is happening ?

Best Regards,

Ashish

Suggest a -f flag to force saving when the pattern file exists

gf -list return empty !! how to install patterns on Windows ??

why it's called gf I know g is for grep but what about the f

root@kali:~# source ~/.bashrc
bash: /src/github.com/tomnomnom/gf/gf-completion.bash: No such file or directory
bash: /src/github.com/tomnomnom/gf/gf-completion.bash: No such file or directory
bash: /src/github.com/tomnomnom/gf/gf-completion.bash: No such file or directory
bash: /src/github.com/tomnomnom/gf/gf-completion.bash: No such file or directory

If one wants to use this tool to debug a code base, it would be helpful to find the debugger flag in a JavaScript/NodeJs framework.

I know I can fork and add whatever patterns I like but I want to know if this pattern is actually useful or if it could just make the program run slower overall.

What do you think, @tomnomnom?

It'd be nice to have -dump option or something like that so you can see what the actual grep command being run is. It'd be much easier for tweaking and then overwriting

example:
cat urls | gf xss sqli ssrf | anew output.txt

Hello there !
The output produced by GF is like this

$cat gf-file.txt

• https://example.com/invite?id=3454&ref=sedrt
• https://example.com/invite?id=686&ref=folfjg
• https://example.com/0auth?cat_id=78978&mode=null
• https://example.com/0auth?cat_id=89778&mode=false
• https://example.com/invite?id=767&ref=shfh

But i want gf to print like this as output.
$cat i-wish.txt

• https://example.com/invite?id=3454&ref=sedrt
• https://example.com/0auth?cat_id=78978&mode=null

Anyway to do this.

When running some script like

find /var/www/jsrecon/links/  -maxdepth 3 -name "waybackurl*"  -print0 | while IFS= read -r -d '' file; do gf potential $file  ; done 

you will find instead of using the input file as target , the current direcotry when running the script will be considered as target .

I think it is because of

	files := flag.Arg(1)
	if files == "" {
		files = "."
	}

in main.go

This bug wasted me more than 4 hours, hope it can get fixed

root@kali:~# go get -u github.com/tomnomnom/gf
package encoding/json: unrecognized import path "encoding/json" (import path does not begin with hostname)
package errors: unrecognized import path "errors" (import path does not begin with hostname)
package flag: unrecognized import path "flag" (import path does not begin with hostname)
package fmt: unrecognized import path "fmt" (import path does not begin with hostname)
package os: unrecognized import path "os" (import path does not begin with hostname)
package os/exec: unrecognized import path "os/exec" (import path does not begin with hostname)
package os/user: unrecognized import path "os/user" (import path does not begin with hostname)
package path/filepath: unrecognized import path "path/filepath" (import path does not begin with hostname)
package strings: unrecognized import path "strings" (import path does not begin with hostname)

_PLease Help __

Use the latest version command if you face the error:

go install github.com/tomnomnom/gf@latest

In resource-constrained environments we sometimes cannot afford to install full-blown Go + build packages with it: good practice is to have pre-packaged binary releases for each platform like https://github.com/projectdiscovery/httpx/releases.

See https://www.digitalocean.com/community/tutorials/how-to-build-go-executables-for-multiple-platforms-on-ubuntu-16-04#step-4-building-executables-for-different-architectures for the process of creating binary releases.

You can include this in a GitHub action workflow, by leveraging something like the https://github.com/marketplace/actions/go-release-binaries action which automates the process.

I am by no means a go developer or I would submit a pull request / prototype for this request.

It would be nice to have a way to pass -A# or -B# in the gf binary and/or encoding a field in the .json object.

Main use case is making the search more visibly identifiable, i.e. adjacent headers or other multi-line entries that might be relevant or helpful in recon.

@tomnomnom
I have the last version installed with go install github.com/tomnomnom/gf@latest , found the example and the zsh script folder and added just like before (i am installing in a new eviromment), but even with the sources in my .zshrc and everything like it was before it still not working the gf -list it's blank and when trying to run something it returns : no such pattern

Any help?

As much as I wish I knew how to do this, I know you do!

https://github.com/foospidy/GrepBugsRules/blob/master/rules.json

Would it be easy to incorporate the regex rules in this json file easily? I assume gron could help!

In the installation manual it said that we would have a directory like go/src/github.com...... but even after installing it with go get I don't have this directory created, I cannot install this tool can u help?