unkl4b / gitminer Goto Github PK

{
"resource": "/e:/Gitminer/GitMiner/gitminer-v2.0.py",
"owner": "python",
"code": "anomalous-backslash-in-string",
"severity": 4,
"message": "Anomalous backslash in string: '\s'. String constant might be missing an r prefix.",
"source": "pylint",
"startLineNumber": 32,
"startColumn": 1,
"endLineNumber": 32,
"endColumn": 1
}

Traceback (most recent call last):
File "gitminer-v2.0.py", line 67, in
GitMiner().start()
File "gitminer-v2.0.py", line 39, in init
self.cookie = headers.parseCookie(self.args.cookie)
File "/root/GitMiner/config/headers.py", line 29, in parseCookie
values = line.strip().split("=")[1]
IndexError: list index out of range
请问这是出了什么问题

Hi Team,

I have tried to install these tools to do some manual checks on leakages of API key or any data for some organization for personal purposes, But while executing requirements.txt, we are prompted with version not supported, Then I have installed those dependencies via Python3 & pip3.

media/Tools/GitMiner# ls
config core Dockerfile gitminer-v2.0.py init.py LICENSE README.md requirements.txt
root@xx:/media/Tools/GitMiner# pip3 install requirements.txt
ERROR: Could not find a version that satisfies the requirement requirements.txt (from versions: none)
ERROR: No matching distribution found for requirements.txt

Please let me know, how to overcome these issues.

root@localhost:~/GitMiner# python git_miner.py --query 'filename:configuration extension:php "public password" in:file' -m joomla

██████╗ ██╗████████╗███╗ ███╗██╗███╗ ██╗███████╗██████╗
██╔════╝ ██║╚══██╔══╝████╗ ████║██║████╗ ██║██╔════╝██╔══██╗
██║ ███╗██║ ██║ ██╔████╔██║██║██╔██╗ ██║█████╗ ██████╔╝
██║ ██║██║ ██║ ██║╚██╔╝██║██║██║╚██╗██║██╔══╝ ██╔══██╗
╚██████╔╝██║ ██║ ██║ ╚═╝ ██║██║██║ ╚████║███████╗██║ ██║
╚═════╝ ╚═╝ ╚═╝ ╚═╝ ╚═╝╚═╝╚═╝ ╚═══╝╚══════╝╚═╝ ╚═╝ v1.1
Automatic search for GitHub.

• Autor: Danilo Vaz a.k.a. UNK

• Blog: http://unk-br.blogspot.com

• Github: http://github.com/danilovazb

• Gr33tz: l33t0s, RTFM

  +[WARNING]------------------------------------------+
  | DEVELOPERS ASSUME NO LIABILITY AND ARE NOT |
  | RESPONSIBLE FOR ANY MISUSE OR DAMAGE CAUSED BY |
  | THIS PROGRAM |
  +---------------------------------------------------+

+[PAGE: 1/100]-----------------------------------------+
| [USER]: @monghimng
| [LINK]: http://github.com/monghimng/cmc/raw/d9a332d2cc22fc0b17ec9e93f8e5b917be382c83/configuration.php
Traceback (most recent call last):
File "git_miner.py", line 226, in
GitMiner().start()
File "git_miner.py", line 223, in start
self.parseSearch(HTML.content)
File "git_miner.py", line 179, in parseSearch
print("| [" + frescurinha.OKBLUE + "LAST INDEXED" + frescurinha.ENDC + "]: %s" % last_indexed[number_link])
IndexError: list index out of range

Why is there no data after the run is complete

Are you seriously putting the screeshot without processing the website address in the picture?

Hi,

I was wondering if you know a way how to sort all the data from the output.txt results of "-q filename:mup.json' -o output.txt -m mup" so it could read it in the following format:

HOST USER PASSWORD

right now it looks like (output.txt):

{"host": {"1": " "some.host.name",\r", "2": " false,\r"}, "password": {"1": " "somepassword@.!",\r"}}},{"@DTL625":

Thanks

I'm getting the following error when using your docker setup:

python3: can't open file 'git_miner.py': [Errno 2] No such file or directory

Reproduction steps:

docker build -t gitminer .
docker run -it gitminer -h

when running script
python git_miner.py -q 'filename:wp-config extension:php FTP_HOST in:file' -m wordpress -o result.txt

| [USER]: @cesarbgvieira
| [LINK]: http://github.com/cesarbgvieira/ProjectosAngola/raw/baaa4af8cdf7e075b39c494d88d93822b04ccd8c/citic/wp-config.php
| [LAST INDEXED]: Oct 10, 2015
| [CONTAIN]: "root" IN LINE: 22
| [PARAM FOUND]:
| -------------> FTP_USER
| -------------> César Vieira
Traceback (most recent call last):
  File "git_miner.py", line 223, in <module>
    GitMiner().start()
  File "git_miner.py", line 220, in start
    self.parseSearch(HTML.content)
  File "git_miner.py", line 194, in parseSearch
    self.nextPage(prox_page)
  File "git_miner.py", line 73, in nextPage
    self.parseSearch(HTML.content)
  File "git_miner.py", line 194, in parseSearch
    self.nextPage(prox_page)
  File "git_miner.py", line 73, in nextPage
    self.parseSearch(HTML.content)
  File "git_miner.py", line 194, in parseSearch
    self.nextPage(prox_page)
  File "git_miner.py", line 73, in nextPage
    self.parseSearch(HTML.content)
  File "git_miner.py", line 194, in parseSearch
    self.nextPage(prox_page)
  File "git_miner.py", line 73, in nextPage
    self.parseSearch(HTML.content)
  File "git_miner.py", line 178, in parseSearch
    self.parseCode(code_boladao)
  File "git_miner.py", line 157, in parseCode
    self.saveOutput("| -------------> %s\n" % result_code[i])
  File "git_miner.py", line 66, in saveOutput
    arquivo.write(text)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 18: ordinal not in range(128)

in page 5/15 (Mars the 3rd 2016)

I ran GitMiner and it outputs nothing, just a small message with every example (in the Readme):

 + Autor: Danilo Vaz a.k.a. UNK
 + Blog: http://unk-br.blogspot.com
 + Github: http://github.com/danilovazb
 + Gr33tz: l33t0s, RTFM

 +[WARNING]------------------------------------------+
 | DEVELOPERS ASSUME NO LIABILITY AND ARE NOT        |
 | RESPONSIBLE FOR ANY MISUSE OR DAMAGE CAUSED BY    |
 | THIS PROGRAM                                      |
 +---------------------------------------------------+


+[PAGE: 1/1]-----------------------------------------+

I don't know why it worked last month and today it doesn't work.
Maybe Github made some changes on his search page.

Hello,

Thanks for releasing new version, the older one is not working recently.
However, I've followed the installation guideline with below steps:

1. Git clone the latest repos
2. Update the requirement.txt with (lxml, requests, argparse, json, re), looks like it's missing in code
3. pip3 install -r requirements
4. Execute the sample command: python3 gitminer-v2.0.py -q 'filename:wp-config extension:php FTP_HOST in:file ' -m wordpress -c <my_cookies>
   5 Got below error message:
   Traceback (most recent call last): File "gitminer-v2.0.py", line 11, in <module> from core.Parser import Parser File "/home/scumblr/GitMiner/core/Parser.py", line 6, in <module> from core.sendRequest import requestPage File "/home/scumblr/GitMiner/core/sendRequest.py", line 3, in <module> from core import Parser ImportError: cannot import name 'Parser'

Please help me, am I missing something?

Thanks.

Its not showing nay result

Great tool!

However I had to apply the following patch to make it work:

59c59
<         self.cookie = {'user_session':self.args.cookie}
---
>         self.cookie = dict(cookies_are=self.args.cookie)

Best regards,

Gwen

with the command : $ python git_miner.py -q 'filename:wp-config extension:php FTP_HOST in:file' -m wordpress -o result.txt

 ██████╗ ██╗████████╗███╗   ███╗██╗███╗   ██╗███████╗██████╗
██╔════╝ ██║╚══██╔══╝████╗ ████║██║████╗  ██║██╔════╝██╔══██╗
██║  ███╗██║   ██║   ██╔████╔██║██║██╔██╗ ██║█████╗  ██████╔╝
██║   ██║██║   ██║   ██║╚██╔╝██║██║██║╚██╗██║██╔══╝  ██╔══██╗
╚██████╔╝██║   ██║   ██║ ╚═╝ ██║██║██║ ╚████║███████╗██║  ██║
 ╚═════╝ ╚═╝   ╚═╝   ╚═╝     ╚═╝╚═╝╚═╝  ╚═══╝╚══════╝╚═╝  ╚═╝ v1.1
 Automatic search for GitHub.

 + Autor: Danilo Vaz a.k.a. UNK
 + Blog: http://unk-br.blogspot.com
 + Github: http://github.com/danilovazb
 + Gr33tz: l33t0s, RTFM

 +[WARNING]------------------------------------------+
 | DEVELOPERS ASSUME NO LIABILITY AND ARE NOT        |
 | RESPONSIBLE FOR ANY MISUSE OR DAMAGE CAUSED BY    |
 | THIS PROGRAM                                      |
 +---------------------------------------------------+


Traceback (most recent call last):
  File "git_miner.py", line 223, in <module>
    GitMiner().start()
  File "git_miner.py", line 216, in start
    HTML = self.accessWeb(url_acesso)
  File "git_miner.py", line 198, in accessWeb
    if " find any code matching" in acc.text:
AttributeError: 'Response' object has no attribute 'text'

Same with other commands

/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:334: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:132: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning

Any help would be very appreciated,

Thanks.

[root@202-39-176-98 GitMiner]# python gitminer-v2.0.py --query 'filename:configuration extension:php "public password" in:file' -m joomla -c pAAAhPOma9jEsXyLWZ-16RTTsGI8wDawbNs4
Traceback (most recent call last):
File "gitminer-v2.0.py", line 67, in
GitMiner().start()
File "gitminer-v2.0.py", line 39, in init
self.cookie = headers.parseCookie(self.args.cookie)
File "/tmp/GitMiner/config/headers.py", line 29, in parseCookie
values = line.strip().split("=")[1]
IndexError: list index out of range

My operating system is centos6. Is it supported?

Hi,

Your script is very helpful in searching for my company sensitive leakage on github.com, recently I face the problem that when I run the script as a thread or put it in a crontab for example:

python git_miner.py --query 'filename:configuration extension:php "public password" in:file' -m joomla -c pAAAhPOma9jEsXyLWZ-16RTTsGI8wDawbNs4 &

It's randomly terminated without an error message, I don't know if it is terminated by github.com or not? Do you have any idea?

Thanks.

Hey, I'm writing my Bachelor Thesis about "finding similarities in code from GitHub" . For this Bachelor Thesis I need a lot of code. Now I saw that your tool could help me to find a lot of code but all searches I try to type into your GitMiner Program didn't find any results. These are the searches I've typed in:
python git_miner.py -q 'extension:h #pragma omp in:file' -o result.txt

python git_miner.py -q 'extension:hpp #pragma omp in:file' -o result.txt

python git_miner.py -q 'extension:c #pragma omp in:file' -o result.txt

python git_miner.py -q 'extension:cpp #pragma omp in:file ' -o result.txt
python git_miner.py -q 'filename:* extension:h #pragma omp in:file ' -o result.txt
python git_miner.py -q 'filename:* extension:hpp #pragma omp in:file ' -o result.txt
python git_miner.py -q 'filename:* extension:c #pragma omp in:file ' -o result.txt
python git_miner.py -q 'filename:* extension:cpp #pragma omp in:file ' -o result.txt
python git_miner.py -q 'filename:test extension:h #pragma omp in:file ' -o result.txt

In his Program he's using the coockie from his own Github account, where can I find it?
Please help me than otherwise I have to open every github page and copy the code and download it.

Kind regards

Matthias Maltes

What do I put in it? I just don't know, Send me a picture or just describe it specificly where I can get it.