sachin-sankar / swiftshadow Goto Github PK
View Code? Open in Web Editor NEWFree IP Proxy rotator library for python
Home Page: https://sachin-sankar.github.io/swiftshadow/
License: GNU General Public License v3.0
Free IP Proxy rotator library for python
Home Page: https://sachin-sankar.github.io/swiftshadow/
License: GNU General Public License v3.0
Implement a new class that when provided with a url requests the url using a proxy validate the request with a function and if either the check or the request itself fails retries the request with a new proxy.
Python 3.11.3 (main, Apr 7 2023, 00:39:07) [Clang 14.0.7 (https://android.googlesource.com/toolchain/llvm-project 4c603efb0 on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from swiftshadow.swiftshadow import Proxy
[ swiftshadow ] ERROR : No cache found. Cache will be created after update
[ swiftshadow ] INFO : Updating proxies from Proxyscrape
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/swiftshadow/swiftshadow.py", line 134, in <module>
print(a.proxy())
^^^^^^^^^
File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/swiftshadow/swiftshadow.py", line 122, in proxy
if cache.checkExpiry(self.expiry):
^^^^^^^^^^^
AttributeError: 'Proxy' object has no attribute 'expiry'
>>>
hi, how to use this with proxychain or oxdork for example
I do not know how but since I added swiftshadow to my codebase my own logging stopped working.
In the file where I import swiftshadow.swiftshadow I do
logging.getLogger().handlers.clear() # <-- first attempt at a workaround
logging.basicConfig(
filename=script_dir + "/log/mylogfile.log",
level=debug_level,
format=(
"%(asctime)s %(levelname)s "
"%(message)s"
),
datefmt="%Y%m%d %H:%M:%S",
)
logger = logging.getLogger("mymainlogger")
logger.info("I am info logging!") # this usually ends up in mylogfile.log
As mentioned this setup worked fine before using swiftshadow.
Now my mylogfile.log stays empty and all I see is swiftshadow output on stdout.
Any idea how I can make my logger work again?
I tried finding your logger with logging.root.manager.loggerDict by to no avail :(
Hi! I'm having an issue where after installing the module it's not found. It's the only with I'have had that issue.
Using Python 3.10 on Windows 11
(venv) PS E:\Github Projects\last-mass> pip install swiftshadow
Collecting swiftshadow
Downloading swiftshadow-0.2.3-py3-none-any.whl (20 kB)
Requirement already satisfied: requests in e:\github projects\last-mass\venv\lib\site-packages (from swiftshadow) (2.30.0)
Requirement already satisfied: charset-normalizer<4,>=2 in e:\github projects\last-mass\venv\lib\site-packages (from requests->swiftshadow) (3.1.0)
Requirement already satisfied: idna<4,>=2.5 in e:\github projects\last-mass\venv\lib\site-packages (from requests->swiftshadow) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in e:\github projects\last-mass\venv\lib\site-packages (from requests->swiftshadow) (1.26.15)
Requirement already satisfied: certifi>=2017.4.17 in e:\github projects\last-mass\venv\lib\site-packages (from requests->swiftshadow) (2023.5.7)
Installing collected packages: swiftshadow
Successfully installed swiftshadow-0.2.3
Pip show
(venv) PS E:\Github Projects\last-mass> pip list
Package Version
------------------ --------
beautifulsoup4 4.12.2
boto3 1.26.129
botocore 1.29.129
certifi 2023.5.7
charset-normalizer 3.1.0
idna 3.4
jmespath 1.0.1
pip 22.3.1
python-dateutil 2.8.2
requests 2.30.0
s3transfer 0.6.1
setuptools 65.5.0
six 1.16.0
soupsieve 2.4.1
swiftshadow 0.2.3
urllib3 1.26.15
Add documentation on how to use swiftshadow with
Hi! On my local environment works fine, but when I try to run it on my lambda I get this
Function Logs
START RequestId: 5ea8d3a7-8f08-4f9b-b9c0-10a6c3c94cc7 Version: $LATEST
07/05/2023 20:35:43 - [swiftshadow] - ERROR : No cache found. Cache will be created after update
07/05/2023 20:35:43 - [swiftshadow] - INFO : Updating proxies from Proxyscrape
[ERROR] OSError: [Errno 30] Read-only file system: '.swiftshadow.json'
Traceback (most recent call last):
File "/var/task/lambda_function.py", line 13, in lambda_handler
swift = Proxy()
File "/opt/python/swiftshadow/swiftshadow.py", line 49, in __init__
self.update()
File "/opt/python/swiftshadow/swiftshadow.py", line 100, in update
with open(".swiftshadow.json", "w") as file:END RequestId: 5ea8d3a7-8f08-4f9b-b9c0-10a6c3c94cc7
REPORT RequestId: 5ea8d3a7-8f08-4f9b-b9c0-10a6c3c94cc7 Duration: 6597.86 ms Billed Duration: 6598 ms Memory Size: 128 MB Max Memory Used: 65 MB Init Duration: 813.64 ms
According to stackoverflow:
Only /tmp seems to be writable in AWS Lambda.
Is there a way to pass a folder path to your library instance?
Add a admonition note about how it wil take some time to update proxies.
Following error generated while auto rotate
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))
implement a new class QuickProxy
that will get a single proxy as soon as possible
(src-py3.11) vikyw@vikyw-X456UR:~/Desktop/chat-based-financial-gpt/server$ poetry add swiftshadow
Using version ^1.0.0 for swiftshadow
Updating dependencies
Resolving dependencies... (4.4s)
Package operations: 1 install, 0 updates, 0 removals
• Installing swiftshadow (1.0.0): Failed
ChefBuildError
Backend subprocess exited when trying to invoke get_requires_for_build_wheel
Traceback (most recent call last):
File "/home/vikyw/.local/pipx/venvs/poetry/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/home/vikyw/.local/pipx/venvs/poetry/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vikyw/.local/pipx/venvs/poetry/lib/python3.10/site-packages/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
^^^^^^^^^^^^^^^^^^^^^
File "/tmp/tmp94df84ec/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmp/tmp94df84ec/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
self.run_setup()
File "/tmp/tmp94df84ec/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 480, in run_setup
super(_BuildMetaLegacyBackend, self).run_setup(setup_script=setup_script)
File "/tmp/tmp94df84ec/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 311, in run_setup
exec(code, locals())
File "<string>", line 18, in <module>
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'requests'
at ~/.local/pipx/venvs/poetry/lib/python3.10/site-packages/poetry/installation/chef.py:164 in _prepare
160│
161│ error = ChefBuildError("\n\n".join(message_parts))
162│
163│ if error is not None:
→ 164│ raise error from None
165│
166│ return path
167│
168│ def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:
Note: This error originates from the build backend, and is likely not a problem with poetry but with swiftshadow (1.0.0) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "swiftshadow (==1.0.0)"'.```
I am trying to get a proxy server IP for selenium:
proxy = Proxy(autoRotate=True, protocol='https').proxy()
print("Proxy response:", proxy)
--> Proxy response: {'https': ''}
options = webdriver.ChromeOptions()
service = Service(executable_path=chrome_driver_path)
options.add_argument('--proxy-server=%s' % ip)
driver = webdriver.Chrome(service=service, options=options)
When I set the protocol to http, I get an IP in the dictionary. Any ideas what I am doing wrong here?
Thanks!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.