system install (as a module) support and setup.py for easier installation about theharvester HOT 24 CLOSED

@blshkv would you want to work on this and submit a pull request?

from theharvester.

I noted that there is a new sqlite database in use. The DB should be generated in homedir ~/.theHarverster
from scratch without any need in a template. The wordlist can be installed into a shared directory.

from theharvester.

@blshkv would you want to create an install.sh for this? There is currently a dockerfile and you can also git clone it. Sorry for the late response!

from theharvester.

no, that would be a wrong approach. It has to be setup.py which should list the required dependencies and define system location of all files. The tool should also support system-wide installation, i.e. installed under root into /usr/share or /usr/lib{32,64} and a regular user should be able to run it. That would also require to move all write operations (database, outputs) from a current directory into
a current user ~/.theHarvester or similar.

from theharvester.

Sorry, I don't have bandwidth for development of other tools. Just a simple bug reporting (like this issue) and maintaining hundreds of packages takes a fair amount of time which I could afford.

from theharvester.

Hi All, I have updated the documentation for the dependencies but it needs to be merge and when you run the tool for the first time it creates the sqlite DB

from theharvester.

@blshkv My changes have been merged can you take a look at the README.md please?

from theharvester.

Not sure what to look for. Does the tool support system install using setup.py?
No.

Will it only from the current folder? Yes.
So nothing changed

from theharvester.

from theharvester.

I have tried it, still long way to go.

setup.py is a python file, which usually tells you that the module/package you are about to install has been packaged and distributed with Distutils, which is the standard for distributing Python Modules.

This allows you to easily install Python packages. Often it's enough to write:

$ python setup.py install

and the module will install itself.

https://docs.python.org/3/installing/index.html#installing-index

from theharvester.

@blshkv what do you think of the PR for setup.py?

from theharvester.

I have tried to convert the harvester into a proper python module but this is a major re-design and I'm unable to test all functions. As I said, I have no time to rewrite it, however I will be happy to test your progress. I can also help with minor fixes, once the direction of the development is clear.

from theharvester.

Please review the new setup.py file that was added. Would like to close out this ticket.

from theharvester.

Tested and failed. Have you tested at all?
The current setup.py doesn't install wordlists folder and theHarvester.py.
I copied the binary into /usr/bin manually but the tool failed to start:

$ theHarvester.py 
Traceback (most recent call last):
  File "/usr/bin/theHarvester.py", line 3, in <module>
    from discovery import *
  File "/usr/lib64/python3.6/site-packages/discovery/shodansearch.py", line 3, in <module>
    from shodan import exception
ModuleNotFoundError: No module named 'shodan'

You need to convert it into python module, redirect all outputs into user's dir etc . Still long way to go

from theharvester.

Regarding the wordlists, please use the following example:
derv82/wifite2#102
They used data_files function for this purpose

from theharvester.

@blshkv please try now I tested it after the most recent commit and it seems to be working :)

from theharvester.

worldlist files are installed into an unusual location: /usr/worldlist now, all other issues are not fixed

from theharvester.

FYI, you can debug it with the following command:
python3 setup.py install --user and inspect ~/.local/lib64/python3.6

from theharvester.

#229
I have started to redesign the tool here (spent like 2 hours on it), let's see if you agree with it.

from theharvester.

https://github.com/vulnersCom/getsploit/pull/14/files

from theharvester.

@blshkv if you are satifised with the changes that have happened feel free to close :)

from theharvester.

@NotoriousRebel thank you for accepting the patch. That would be the first stage, I can install the tool as a system package now (using root account). The second stage is to redirect all output to a userdir, (something like https://github.com/vulnersCom/getsploit/pull/14/files which I mentioned before) and redirect all output to ~/.theHarvester so a regular user would be able to run it without touching system files

from theharvester.

@blshkv Would you agree that this issue could be closed now? As I think we now for fill the subject of this thread and open a new thread to do with the other bits and pieces that you have mentioned.

from theharvester.

Ok, sure. Thanks for accepting my patch once again

from theharvester.