Hi ,

can someone help me with the below error. i have downloaded the code and complied in my windows machine.

Below is the error when i run the compiled exe with token.

Thanks
Jeff

Automated vulnerability scans of dependencies need to be enabled and plumbed into CircleCI. Currently Synk is being used but that is not fully automated as they don't support automated GitHub scans, only local scans that then push results.

Submit the tool for inclusion in BlackArch

I get Failed to load signatures file default.yml: open default.yml: no such file or directory when running on an org. I have no problem with the help command.

Describe the bug
For VMs with just 2 logical CPUs, the code will run for 20 seconds and just die thereafter. It might be better to remove the +2 and just use the total number of logical number of CPUs.

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

Is your feature request related to a problem? Please describe.
Signatures should be split into a different repo that can be versioned and updated independently

Describe the solution you'd like

Describe alternatives you've considered
N/A

Additional context
N/A

The current thinking is to use gRPC for communications between the major functions such as Output, Discovery and Detection, and each of their components. This will allow a new component such as a specific language parser or target such as Perforce, SVN, etc to be written in any supported language and versioned and managed independently of the main tool.

The configuration on the terminal side would be managed by Viper, and the command interface by Cobra. At some point, we may want a web interface for the input, especially if we go down the road of using this as a deployable service in a pipeline.

The output will remain terminal and web-based for the foreseeable future as both of these options have very strong use cases.

There is still a lot of planning and thinking that needs to be done from a mile high standpoint and then the implementation details are where the devil lays. Please feel free to comment on this thread as time goes on. At some point, this may be captured in a real document but for now threads make it easy to track comments and feedback.

@michenriksen @codeEmitter @plasticuproject @spekulatius I have tagged you as you either reach out directly to me or have a lot of contributions in the original project. None of this is set in stone but it moves the project leaps and bounds ahead for both blue and red team usage while still maintaining the ease of use for the core functionality.

Is your feature request related to a problem? Please describe.
Multiple Go routines

Describe the solution you'd like
Right now, there's only one process that does the analysis and serving of files sequentially. It would be great to have two separate go routines that:

Does periodic analysis given a specified time interval (every 1 minute, 5 minutes, etc.)
Serves up the analysis result and the corresponding UI
If this is implemented, it might be good to have +3 on GOMAXPROCS in gitrob/core/session.go.

func (s *Session) InitThreads() {
if *s.Options.Threads == 0 {
numCPUs := runtime.NumCPU()
s.Options.Threads = &numCPUs
}
runtime.GOMAXPROCS(*s.Options.Threads + 3) // thread count + main + web server + analysis
}.

Describe alternatives you've considered
N /A

Additional context
N/A

Before any code can be written a functional spec that lays out the requirements and short term goals of the project needs to be written. This spec must include all functional and non-functional requirements as well as any use cases that have been conceived either through. experience or comment.

Should target Centos 6/7

The Makefile needs to be refactored to be project-specific and also less hardcoded of paths. This is currently dependent upon the finished refactoring due to package names and code paths.

@codeEmitter ^ does your code support this? Thoughts?

The maintainability of the codebase should be uplifted to make contributing easier and writing tests more effective.

Describe the bug
If the github token is malformed, is not provided, or is not authorized then the tool will panic with an unrelated error message. Turning on the debugging will show the user that they have gotten a 401 response. This should be fixed to either require a token and if not provided fail fast with an apprioate message or simply give a message describing the problem and how to fix it.

To Reproduce
Steps to reproduce the behavior:
Run the tool with no token or a bad token

Expected behavior
The tool should give an error message saying either the token was not found or the token was not valid

Desktop (please complete the following information):

• OS: Kali, Windows 10, OSX
• Version: 2.0.0-beta

Additional context
N/A

Is your feature request related to a problem? Please describe.
Implement high entropy checks

Describe the solution you'd like
The ability to check for high entropy strings within a file

Describe alternatives you've considered
N/A

Additional context
N/A

Is your feature request related to a problem? Please describe.
The user should be able to exclude test/demo repos from a scan

Describe the solution you'd like
There should be some method that allows a user to either exclude the repos from a scan or exclude them from the results.

Describe alternatives you've considered
N/A

Additional context
N/A

The web sever should be a standalone package for the sake of portability and code organization.

Is your feature request related to a problem? Please describe.
There should be OS specific packages available that can be installed using a package manager

Describe the solution you'd like
A set of packages for the following:

• Kali
• Debian/Ubuntu
• CentOS
• ParrotOS
• Arch/Blackarch
  Describe alternatives you've considered
  A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

When you use the flag -help the current version should be displayed to make bug reports easier to triage and allow users to know if they need to update.

Is your feature request related to a problem? Please describe.
There should exist the ability to scan a single repo

Describe the solution you'd like
The ability to pass in a single repo or a list of repos to be scanned

Describe alternatives you've considered
N/A

Additional context
N/A

Submit the tool to be included in Kali

github.com/N0MoreSecr3ts/[email protected]: parsing go.mod:
module declares its path as: wraith
but was required as: github.com/N0MoreSecr3ts/wraith

Describe the bug
Default values are being displayed instead of actual results

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

Is your feature request related to a problem? Please describe.
We should have a makefile that is capable of performing build, release, and testing tasks rather than separate shell scripts.

Describe the solution you'd like
The use of a makefile is more generic and scalable than some shell scripts. It is also easier to add and modify targets. This will provide for easier maintenance and also conforms to normal practices when compiling code such as C/C++/Go.

Describe alternatives you've considered
N/A

Additional context
N/A

Is your feature request related to a problem? Please describe.
It would be good to show the commit date in the findings output

Describe the solution you'd like
Being able to show the commit date in the findings output would be useful to determine the context surrounding the commit.

Describe alternatives you've considered
N/A

Additional context
N/A

The release process needs to be updated to be as automated as possible and OS packages should be built and submitted to the maintainers of the various repo's.

Is your feature request related to a problem? Please describe.
It would be much easier to read all the flags from a standard environment file or config file.

Describe the solution you'd like
The ability to read all flags from an environment file or config file will go along ways towards making this tool more modular. It also sets the groundwork for being able to run this as a service allowing a user to simply POST an url or repo to the tool and have it run in an automated fashion.

Describe alternatives you've considered
N/A

Additional context
N/A

Code review to ensure there are no glaring holes that could affect a system

Each session should generate usable metrics to aide in performance tunning.

• Start time
• End time
  -Total run time
• Number of repos
• Number of files

The overall design and architecture for the tool needs to be described. How will it be written to meet the functional specifications and goals of the project.

Submit the tool for inclusion in ParrotOS

Describe the bug
If no targets are given the tool will panic

To Reproduce
Steps to reproduce the behavior:

1. Execute the tool with no targets

Expected behavior
A warning that no targets have been provided and then exiting gracefully

Desktop (please complete the following information):

• OS: All
• Version: 0.0.1

Additional context
This is related to #7

@funkwhatyouheard

This function should work. You may still need to add the flags to the command line. The bits should still be in the season struct as well. I don't remember. I wrote this for another version and just didn't get around to re-implementing it here. Like I said it should work but at the least it is a starting point.

There are bound to be some variables that don't line up of fields that need to be changed or eliminated. I wrote this over 2 years ago. Any questions feel free to ping me here or we can talk it out through slack.

// summaryOutput will spit out the results of the hunt along with performance data
func summaryOutput(sess *Session) {

	// alpha sort the findings to make the results idempotent
	if len(sess.Findings) > 0 {
		sort.Slice(sess.Findings, func(i, j int) bool {
			return sess.Findings[i].SecretID < hunt.Findings[j].SecretID
		})
	}

	if sess.JSONOutput {
		if len(sess.Findings) > 0 {
			b, err := json.MarshalIndent(sess.Findings, "", "    ")
			if err != nil {
				fmt.Println(err)
				return
			}
			c := string(b)
			if c == "null" {
				fmt.Println("{}")
			} else {
				fmt.Println(c)
			}
		} else {
			fmt.Println("{}")
		}
	}

	if sess.CSVOutput {
		w := csv.NewWriter(os.Stdout)
		defer w.Flush()
		header := []string{
			"FilePath",
			"Line Number",
			"Action",
			"Description",
			"RuleId",
			"Finding List",
			"Repo Owner",
			"Repo Name",
			"Commit Hash",
			"Commit Message",
			"Commit Author",
			"File URL",
			"Secret ID",
			"Grover Version",
			"Rules Version",
		}
		w.Write(header)

		for _, v := range sess.Findings {
			line := []string{
				v.FilePath,
				v.LineNumber,
				v.Action,
				v.Description,
				v.Ruleid,
				v.Comment,
				v.RepositoryOwner,
				v.RepositoryName,
				v.CommitHash,
				v.CommitMessage,
				v.CommitAuthor,
				v.FileURL,
				v.SecretID,
				v.GroverVersion,
				v.RulesVersion,
			}
			w.Write(line)
		}
	}

The ability to check for updates via a flag should be implemented.

The codebase should be refactored to make it easier for people to build and commit to. RPC/GRPC could be looked at as well to implement plugins for extending the usage to various targets and methods of discovery. Re-organizing the code into multiple packages based on function or target, for example, web server, git scanning, perforce scanning, discovery methods such as regex, syntax tables, etc.

Is your feature request related to a problem? Please describe.
The ability to pause and resume scans would be great, especially in larger projects

Describe the solution you'd like
Be able to pause a scan and save its current state to be picked up at a later point in time.

Describe alternatives you've considered
N/A

Additional context
N/A

Is your feature request related to a problem? Please describe.
The documentation for the project needs to be uplifted and expanded

Describe the solution you'd like
New documentation site, with live demo of the various features and use cases

Describe alternatives you've considered
N/A

Additional context
N/A

When you search through a commit you are not actually finding all the secrets.

1. If a file is moved to a different path within the repo in a later commit it will not be searched in the earlier commits. Wraith will throw a file not found error because it is not fetching the commits, it is simply scanning them based on the file list from the top level commit.

2. There are duplicate findings being displayed. In part this is due to how the commit history is being scanned. Beyond that I am not sure the root cause yet.

3. Findings with a line number of 0 are being displayed. This is related to 2 but I am not sure again what the root cause is, though the zero is being set via hard coding for some reason.

4. The secret ID's are not unique. Due to how the ID's are being calculated, even across commits the same secret in the same file should have a unique secret ID. I am suspecting this is also related to how we are searching through the commit history and the duplicated findings.

@codeEmitter

Is your feature request related to a problem? Please describe.
The codebase needs to be refactored and updated.

Describe the solution you'd like
The codebase should be refactored to make it easier for people to build and commit to. There are several avenues that could be explored. The use of Cobra and Viper could be implemented for a more robust and plugable tool. This would also fix a few issues with the flags and environment variables as well as give the tool the ability to read from a config file.

There is also a simpler package, cli, that could be used. It is not a full-featured as the other two but it does lend itself to a somewhat simpler codebase, though not as modular.
Describe alternatives you've considered
N/A

Additional context
N/A

Currently, when you run gitrob with no arguments, it will throw an error about needing a token. This is fine but it should also display the help.

Create a Debian/Ubuntu package

Is there a way to scan entire GitHub to find all the repo/code which contains a specific word?

Is your feature request related to a problem? Please describe.
Can we save the state of a previous scan and the next one loads in that state and only does a search on the delta of the repo

Describe the solution you'd like
Only search on the delta of the scan

Describe alternatives you've considered
N/A

Additional context
N/A

Unit/Integration tests need to be written. This issue will remain open and can be referenced for multiple pull requests. This is more of a meta issue. Individual issues do not need to be opened for any tests that are written.

All tests should be compatible with Go test, beyond that testify is the preferred test package. Existing tests in either go convey(not maintained anymore) or vanilla go test should be converted to ensure stability and ease of maintenance.

When a user types --help They should get a list of possible flags but also a set of examples usages.

go get github.com/mattyjones/gitrob
go: downloading google.golang.org/appengine v1.0.0

github.com/michenriksen/gitrob/core

../Desktop/GO/pkg/mod/github.com/michenriksen/[email protected]+incompatible/core/github.go:43:5: cannot use user.ID (type *int64) as type *int in field value
../Desktop/GO/pkg/mod/github.com/michenriksen/[email protected]+incompatible/core/github.go:73:11: cannot use repo.ID (type *int64) as type *int in field value
../Desktop/GO/pkg/mod/github.com/michenriksen/[email protected]+incompatible/core/github.go:105:73: cannot use member.ID (type *int64) as type *int in field value
getting error during installation

Is your feature request related to a problem? Please describe.
Scan local Git repos

Describe the solution you'd like
The ability to scan local or offline git repos that a user man have

Describe alternatives you've considered
N/A

Additional context
N/A