Comments (26)
mschwager
commented on August 23, 2024
1
Just a heads up, I've officially released 1.2.1: https://github.com/mschwager/fierce/releases/tag/1.2.1
š
from fierce.
blshkv
commented on August 23, 2024
Based on the FHS, the directory should be something like /usr/share/fierce/lists and the tool should be able to find it.
from fierce.
blshkv
commented on August 23, 2024
FYI, https://stackoverflow.com/questions/4519127/setuptools-package-data-folder-location
Option 1: Install as package data
Option 2: Install to fixed location
from fierce.
mschwager
commented on August 23, 2024
What commands did you use to install fierce?
Typically it's installed via pip with the PyPI package: https://pypi.python.org/pypi/fierce
I haven't tried using setup.py before, but we may be able to debug it. The find_subdomain_list_file function looks in, 1.) the current directory, then 2.) in the directory it was installed as a package in. Note that the package includes the files under the data_files attribute in https://github.com/mschwager/fierce/blob/master/setup.py.
Alternatively, you could just specify the full path of the list file you're trying to use, like:
$ fierce --subdomain-file /usr/lists/default.txt ...
Does that work?
from fierce.
blshkv
commented on August 23, 2024
I already mentioned that I installed the tool using setup.py.
As you can see, I'm have ported the tool to Pentoo (Gentoo) and had to patch it in order it to work by default. We report about such problems to upstream usually so that you can fix it once and forever.
I think the expected logic (1 current 2 installed) did not work for me. Please double check.
+ sed -i -e "s|os.path.dirname(__file__)|\"/usr/share/fierce/\"|" fierce.py || die
+ sed -i -e "s|'lists', data|'share/fierce/lists', data|" setup.py || die
P.S. Please help to reply faster so I would remember about it.
from fierce.
mschwager
commented on August 23, 2024
This project is just a hobby of mine, so I maintain it as I have the time and motivation.
I'm curious about the exact command you ran with setup.py because it works for me in an Alpine Docker container:
$ git clone https://github.com/mschwager/fierce.git
$ cd fierce
$ python3 setup.py install
$ fierce --domain facebook.com --wide
NS: b.ns.facebook.com. a.ns.facebook.com.
SOA: a.ns.facebook.com. (69.171.239.12)
Zone: failure
Wildcard: failure
Found: 0.facebook.com. (157.240.18.36)
...
So I'm curious why this works for most flavors of Linux, but not Gentoo.
Regardless, I've pushed a new commit (7c666e4) that hopefully fixes your issue. Could you pull down the latest code and see if it works as you'd expect?
from fierce.
blshkv
commented on August 23, 2024
somehow, files under "lists/*" are not longer getting installed. Any ideas?
from fierce.
mschwager
commented on August 23, 2024
It appears as though Python has some severe deficiencies when using data_files OR package_data:
- pypa/sampleproject#30
- https://stackoverflow.com/questions/7522250/how-to-include-package-data-with-setuptools-distribute
I'll continue to investigate.
It still seems weird to me that python3 setup.py install works for me and not you, though.
from fierce.
blshkv
commented on August 23, 2024
I'm installing it in a sandbox, could it be an issue?
* fierce-1.2.0.tar.gz SHA256 SHA512 WHIRLPOOL size ;-) ... [ ok ]
>>> Unpacking source...
>>> Unpacking fierce-1.2.0.tar.gz to /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/work
>>> Source unpacked in /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/work
>>> Preparing source in /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/work/fierce-1.2.0 ...
* Applying 7c666e452b8a1b26efc72c7f5444b7bfe17af44a.patch ... [ ok ]
>>> Source prepared.
>>> Configuring source in /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/work/fierce-1.2.0 ...
>>> Source configured.
>>> Compiling source in /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/work/fierce-1.2.0 ...
* python3_5: running distutils-r1_run_phase distutils-r1_python_compile
/usr/bin/python3.5 setup.py build
running build
running build_py
copying fierce.py -> /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/work/fierce-1.2.0-python3_5/lib
warning: build_py: byte-compiling is disabled, skipping.
>>> Source compiled.
>>> Test phase [not enabled]: net-analyzer/fierce-1.2.0-r2
>>> Install fierce-1.2.0-r2 into /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/image/ category net-analyzer
* python3_5: running distutils-r1_run_phase distutils-r1_python_install
/usr/bin/python3.5 setup.py install --root=/var/tmp/portage/net-analyzer/fierce-1.2.0-r2/image/_python3.5
running install
running build
running build_py
running install_lib
creating /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/image/_python3.5
creating /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/image/_python3.5/usr
creating /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/image/_python3.5/usr/lib64
creating /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/image/_python3.5/usr/lib64/python3.5
creating /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/image/_python3.5/usr/lib64/python3.5/site-packages
copying /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/work/fierce-1.2.0-python3_5/lib/fierce.py -> /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/image/_python3.5/usr/lib64/python3.5/site-packages
byte-compiling /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/image/_python3.5/usr/lib64/python3.5/site-packages/fierce.py to fierce.cpython-35.pyc
writing byte-compilation script '/var/tmp/portage/net-analyzer/fierce-1.2.0-r2/temp/tmp7lj1v3qa.py'
/usr/bin/python3.5 -O /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/temp/tmp7lj1v3qa.py
removing /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/temp/tmp7lj1v3qa.py
writing byte-compilation script '/var/tmp/portage/net-analyzer/fierce-1.2.0-r2/temp/tmpjloatwf8.py'
/usr/bin/python3.5 -OO /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/temp/tmpjloatwf8.py
removing /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/temp/tmpjloatwf8.py
running install_egg_info
running egg_info
creating fierce.egg-info
writing requirements to fierce.egg-info/requires.txt
writing entry points to fierce.egg-info/entry_points.txt
writing fierce.egg-info/PKG-INFO
writing top-level names to fierce.egg-info/top_level.txt
writing dependency_links to fierce.egg-info/dependency_links.txt
writing manifest file 'fierce.egg-info/SOURCES.txt'
reading manifest file 'fierce.egg-info/SOURCES.txt'
writing manifest file 'fierce.egg-info/SOURCES.txt'
Copying fierce.egg-info to /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/image/_python3.5/usr/lib64/python3.5/site-packages/fierce-1.2.0-py3.5.egg-info
running install_scripts
Installing fierce script to /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/image/_python3.5/usr/lib/python-exec/python3.5
* python3_5: running distutils-r1_run_phase distutils-r1_python_install_all
>>> Completed installing fierce-1.2.0-r2 into /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/image/
* Final size of build directory: 332 KiB
* Final size of installed tree: 80 KiB
ecompressdir: bzip2 -9 /usr/share/doc
from fierce.
mschwager
commented on August 23, 2024
Hmm, maybe. What type of sandbox are you using?
Also, it looks like you're trying to install fierce-1.2.0.tar.gz:
...
>>> Unpacking fierce-1.2.0.tar.gz to /var/tmp/portage/net-analyzer/fierce-1.2.0-r2/work
...
Where does that file come from? I've been distributing .whl packages, not .tar.gz at https://pypi.python.org/pypi/fierce.
If you can obtain the wheel distribution you should be able to install that with pip via:
$ /usr/bin/python3.5 -m pip install fierce-1.2.0-py3-none-any.whl
I suspect the .tar.gz installation may not be working correctly because this package has been created to be distributed as a wheel file, not a source distribution. So package_data and data_files may not work as expected.
Also, previously you mentioned "FHS":
Based on the FHS, the directory should be something like /usr/share/fierce/lists and the tool should be able to find it.
What's "FHS"?
Thanks for helping me debug this :)
from fierce.
mschwager
commented on August 23, 2024
Okay, I was finally able to reproduce! It'll be much easier to debug now!
from fierce.
blshkv
commented on August 23, 2024
ok, good to hear.
P.S.
- It is Gentoo sandbox;
- We do not use .WHL files because we compile it from sources, tar.gz is the source file from github "release" section obviously.
- FHS is Filesystem Hierarchy Standard, but it is not entirely relevant to python. You are working in the right direction: just use standard python installer (setup.py file)
from fierce.
mschwager
commented on August 23, 2024
I had to revert back the change from data_files -> package_data because it broke my binary wheel distributions. However, adding a MANIFEST.in file with the subdomain lists included fixed the issue in my testing. Can you confirm?
from fierce.
blshkv
commented on August 23, 2024
Can you post the updated patch please?
from fierce.
mschwager
commented on August 23, 2024
Just pushed: https://github.com/mschwager/fierce/releases/tag/1.2.1-rc0
from fierce.
blshkv
commented on August 23, 2024
Ok, so we are back to FHS discussion. The files are getting installed into /usr/lists/* which is wrong.
Could you place it them somewhere /usr/share/fierce/lists/* instead?
from fierce.
mschwager
commented on August 23, 2024
I'm sorry to hear that installation is not working on your machine. However, I'm unable to reproduce your issue on standard Linux distributions. All distros I've tested have installed the files in a predictable place:
Ubuntu 16.04:
$ find / -name default.txt
/usr/local/lib/python3.5/dist-packages/fierce-1.2.0-py3.5.egg/lists/default.txt
Debian 8:
$ find / -name default.txt
/usr/local/lib/python3.4/dist-packages/fierce-1.2.0-py3.4.egg/lists/default.txt
Centos 7:
$ find / -name default.txt
/usr/lib/python3.6/site-packages/fierce-1.2.0-py3.6.egg/lists/default.txt
Further, running fierce after setup.py build and setup.py install works as expected.
If you'd like to debug your specific environment further and post a pull request I'd be happy to review it. If not, I'd recommend you follow the official installation procedure and install via the wheel file.
from fierce.
blshkv
commented on August 23, 2024
Please re-open the issue, here is the latest log:
ebuild fierce-9999.ebuild manifest clean install
>>> Creating Manifest for /data/pentoo/pentoo/net-analyzer/fierce
>>> Unpacking source...
* Fetching https://github.com/mschwager/fierce.git ...
git fetch https://github.com/mschwager/fierce.git +HEAD:refs/git-r3/HEAD
git symbolic-ref refs/git-r3/net-analyzer/fierce/0/__main__ refs/git-r3/HEAD
* Checking out https://github.com/mschwager/fierce.git to /var/tmp/portage/net-analyzer/fierce-9999/work/fierce-9999 ...
git checkout --quiet refs/git-r3/HEAD
GIT update -->
repository: https://github.com/mschwager/fierce.git
at the commit: ce8e08f0941993c8980ee2ee839a8874c06155b9
>>> Source unpacked in /var/tmp/portage/net-analyzer/fierce-9999/work
>>> Preparing source in /var/tmp/portage/net-analyzer/fierce-9999/work/fierce-9999 ...
>>> Source prepared.
>>> Configuring source in /var/tmp/portage/net-analyzer/fierce-9999/work/fierce-9999 ...
>>> Source configured.
>>> Compiling source in /var/tmp/portage/net-analyzer/fierce-9999/work/fierce-9999 ...
* python3_5: running distutils-r1_run_phase distutils-r1_python_compile
/usr/bin/python3.5 setup.py build
running build
running build_py
copying fierce.py -> /var/tmp/portage/net-analyzer/fierce-9999/work/fierce-9999-python3_5/lib
warning: build_py: byte-compiling is disabled, skipping.
>>> Source compiled.
>>> Test phase [not enabled]: net-analyzer/fierce-9999
>>> Install fierce-9999 into /var/tmp/portage/net-analyzer/fierce-9999/image/ category net-analyzer
* python3_5: running distutils-r1_run_phase distutils-r1_python_install
/usr/bin/python3.5 setup.py install --root=/var/tmp/portage/net-analyzer/fierce-9999/image/_python3.5
running install
running build
running build_py
running install_lib
creating /var/tmp/portage/net-analyzer/fierce-9999/image/_python3.5
creating /var/tmp/portage/net-analyzer/fierce-9999/image/_python3.5/usr
creating /var/tmp/portage/net-analyzer/fierce-9999/image/_python3.5/usr/lib64
creating /var/tmp/portage/net-analyzer/fierce-9999/image/_python3.5/usr/lib64/python3.5
creating /var/tmp/portage/net-analyzer/fierce-9999/image/_python3.5/usr/lib64/python3.5/site-packages
copying /var/tmp/portage/net-analyzer/fierce-9999/work/fierce-9999-python3_5/lib/fierce.py -> /var/tmp/portage/net-analyzer/fierce-9999/image/_python3.5/usr/lib64/python3.5/site-packages
byte-compiling /var/tmp/portage/net-analyzer/fierce-9999/image/_python3.5/usr/lib64/python3.5/site-packages/fierce.py to fierce.cpython-35.pyc
writing byte-compilation script '/var/tmp/portage/net-analyzer/fierce-9999/temp/tmpmv0d75gv.py'
/usr/bin/python3.5 -O /var/tmp/portage/net-analyzer/fierce-9999/temp/tmpmv0d75gv.py
removing /var/tmp/portage/net-analyzer/fierce-9999/temp/tmpmv0d75gv.py
writing byte-compilation script '/var/tmp/portage/net-analyzer/fierce-9999/temp/tmpg5975dkz.py'
/usr/bin/python3.5 -OO /var/tmp/portage/net-analyzer/fierce-9999/temp/tmpg5975dkz.py
removing /var/tmp/portage/net-analyzer/fierce-9999/temp/tmpg5975dkz.py
running install_data
creating /var/tmp/portage/net-analyzer/fierce-9999/image/_python3.5/usr/lists
copying lists/default.txt -> /var/tmp/portage/net-analyzer/fierce-9999/image/_python3.5/usr/lists
copying lists/5000.txt -> /var/tmp/portage/net-analyzer/fierce-9999/image/_python3.5/usr/lists
copying lists/20000.txt -> /var/tmp/portage/net-analyzer/fierce-9999/image/_python3.5/usr/lists
running install_egg_info
running egg_info
creating fierce.egg-info
writing top-level names to fierce.egg-info/top_level.txt
writing entry points to fierce.egg-info/entry_points.txt
writing requirements to fierce.egg-info/requires.txt
writing dependency_links to fierce.egg-info/dependency_links.txt
writing fierce.egg-info/PKG-INFO
writing manifest file 'fierce.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'fierce.egg-info/SOURCES.txt'
Copying fierce.egg-info to /var/tmp/portage/net-analyzer/fierce-9999/image/_python3.5/usr/lib64/python3.5/site-packages/fierce-1.2.0-py3.5.egg-info
running install_scripts
Installing fierce script to /var/tmp/portage/net-analyzer/fierce-9999/image/_python3.5/usr/lib/python-exec/python3.5
* python3_5: running distutils-r1_run_phase distutils-r1_python_install_all
>>> Completed installing fierce-9999 into /var/tmp/portage/net-analyzer/fierce-9999/image/
* Final size of build directory: 448 KiB
* Final size of installed tree: 276 KiB
ecompressdir: bzip2 -9 /usr/share/doc
from fierce.
blshkv
commented on August 23, 2024
I'm reading documentation and here it is:
https://packaging.python.org/tutorials/distributing-packages/
Although configuring package_data is sufficient for most needs, in some cases you may need to place data files outside of your packages. The data_files directive allows you to do that.
...
setuptools allows absolute ādata_filesā paths, and pip honors them as absolute, when installing from sdist. This is not true when installing from wheel distributions. Wheels donāt support absolute paths, and they end up being installed relative to āsite-packagesā
So it looks like you are facing the issue with wheel, pypa/wheel#92
However, it is clear from the documentation that files should not be installed there.
from fierce.
mschwager
commented on August 23, 2024
Okay, what's your purposed solution?
It appears the problem here (among the many problems with Python's packaging system) is the usage of --root. When running install with --root I can reproduce your issue, when running without --root everything is fine.
Further, I re-attempted to include this files with package_data instead of data_files, but that didn't correctly install the files either.
from fierce.
blshkv
commented on August 23, 2024
I'm not a Python expert, but the issue seems with the absolute path based on the description.
You may want to try "os.path.relpath" with os.path.join, line 17 in the setup.py?
from fierce.
mschwager
commented on August 23, 2024
Okay, it appears that package_data wasn't working because the list files weren't under the fierce package. cf0e41e should hopefully fix your issue. I've tested that setup.py install should now work with the --root flag. The latest tag should have the changes: https://github.com/mschwager/fierce/releases/tag/1.2.1-rc1. Can you confirm this fixes your issue?
With that being said, there was a reason I wasn't distributing the fierce.py script under a particular Python package directory. I wanted to keep the script simple to download, and self-sufficient, so users don't have to install the full Python package if they don't want to. E.g. a simple wget of the fierce.py script. In order to ensure this isn't a problem I'm going to sit on these changes for a month or so to let anyone voice a concern. If there are no problems I'll push the changes in a new version on PyPI. You'll still be free to pull down the release-candidate tag, the package just won't be on PyPI yet.
Sound good?
from fierce.
blshkv
commented on August 23, 2024
I have tested the latest git and confirm that the issue is fixed finally
Thank you
from fierce.
blshkv
commented on August 23, 2024
Please excuse me for bothering you with all these unrelated bugs but it seems paid off.
The last project (wifite2) have managed to resolve the issue better. Now they install shared data into /usr/share using data_files once only. Would you consider fixing it in the same way?
Thanks
from fierce.
blshkv
commented on August 23, 2024
ping?
from fierce.
blshkv
commented on August 23, 2024
and ping
from fierce.
Related Issues (20)
- Socket errors with --connect option HOT 1
- Doen't work with cloudflare DNS HOT 3
- ConnectionRefusedError: [Errno 111] Connection refused HOT 1
- NoneType issues when using DNS file HOT 3
- Problem for two subdomains separated by dot with --subdomain-file HOT 3
- Problem with subdomains and wildcard failure HOT 1
- `dns.name.EmptyLabel` stack trace HOT 4
- Timeout from DNS resolver crashes program HOT 3
- Timeout during AXFR HOT 1
- Homebrew formula? HOT 3
- Zone transfer error: REFUSED / NOTAUTH HOT 5
- Traverse expander breaks on IP addresses close to zero HOT 4
- Zone: failure & Wildcard: failure
- Allow later dnspython releases
- problem using fierce against an internal DNS
- Pull
- Zone transfert not working when DNS is badly configured
- TypeError: 'float' object cannot be interpreted as an integer HOT 2
- File 'lists/default.txt' missing HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
š Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ššš
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ā¤ļø Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fierce.