Comments (3)
snooppr
commented on June 15, 2024
requests=2.31.0
urllib3=2.0.1
This code works:
import ssl
import requests
import urllib3
ctx = urllib3.util.create_urllib3_context(ciphers=':HIGH:!DH:!aNULL', cert_reqs=ssl.CERT_NONE)
adapter = requests.adapters.HTTPAdapter()
adapter.init_poolmanager(connections=2, maxsize=2, block=False, ssl_minimum_version=ssl.TLSVersion.TLSv1, ssl_context=ctx)
requests_future = requests.Session()
requests_future.mount('https://', adapter)
print(requests_future.get("https://nhl.ru"))
<Response [404]>
The same code, but does not work for multiprocessing (1 process):
import ssl
import requests
import urllib3
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor, TimeoutError
executor1 = ProcessPoolExecutor(max_workers=1)
ctx = urllib3.util.create_urllib3_context(ciphers=':HIGH:!DH:!aNULL', cert_reqs=ssl.CERT_NONE)
adapter = requests.adapters.HTTPAdapter()
adapter.init_poolmanager(connections=2, maxsize=2, block=False, ssl_minimum_version=ssl.TLSVersion.TLSv1, ssl_context=ctx)
req = requests.Session()
req.mount('https://', adapter)
a = []
b = executor1.submit(req.get, url="https://nhl.ru/", timeout=12)
a.append(b)
for i in a:
print(i.result())
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='nhl.ru', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: DH_KEY_TOO_SMALL] dh key too small (_ssl.c:1125)')))
The same code, but processes are replaced with threads and the code works successfully:
import ssl
import requests
import urllib3
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor, TimeoutError
executor1 = ThreadPoolExecutor(max_workers=1)
ctx = urllib3.util.create_urllib3_context(ciphers=':HIGH:!DH:!aNULL', cert_reqs=ssl.CERT_NONE)
adapter = requests.adapters.HTTPAdapter()
adapter.init_poolmanager(connections=2, maxsize=2, block=False, ssl_minimum_version=ssl.TLSVersion.TLSv1, ssl_context=ctx)
req = requests.Session()
req.mount('https://', adapter)
a = []
b = executor1.submit(req.get, url="https://nhl.ru/", timeout=12)
a.append(b)
for i in a:
print(i.result())
<Response [404]>
urllib3=1.26.18 This code works. or multiprocessing (1 process):
import ssl
import requests
import urllib3
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor, TimeoutError
executor1 = ProcessPoolExecutor(max_workers=1)
requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS += ':HIGH:!DH:!aNULL'
adapter = requests.adapters.HTTPAdapter()
adapter.init_poolmanager(connections=2, maxsize=2, block=False)
req = requests.Session()
req.mount('https://', adapter)
a = []
b = executor1.submit(req.get, url="https://nhl.ru/", timeout=12)
a.append(b)
for i in a:
print(i.result())
<Response [404]>
If you have difficulty migrating to v2.0 or following this guide you can open an issue on GitHub Doc.
How to make the code work after migrating to urllib3 >2?
from urllib3.
snooppr
commented on June 15, 2024
I tested the above code on Android/Termux python3.11 (same error when using multiprocessing, but threads work).
I found a workaround:
$ /etc/ssl/openssl.cnf
CipherString = DEFAULT@SECLEVEL=2 -> CipherString = DEFAULT@SECLEVEL=1
Then multiprocessing does not cause an error.
Any ideas on how to use multiprocessing without editing 'openssl.cnf'? Let me remind you that in previous versions of urllib3 the hack urllib3.util.ssl_.DEFAULT_CIPHERS += ':HIGH:!DH:!aNULL' worked. Now it is broken and the proposed alternative does not work (It would be more correct to say that it only works for threads, not processes.).
broken processes (Replace processes with threads and the code will work (replace the line executor1 = ProcessPoolExecutor(max_workers=2) -> executor1 = ThreadPoolExecutor(max_workers=2)))
import ssl
import urllib3
from urllib3 import PoolManager
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
executor1 = ProcessPoolExecutor(max_workers=2)
ctx = urllib3.util.create_urllib3_context(ciphers=':HIGH:!DH:!aNULL', cert_reqs=ssl.CERT_NONE)
http = urllib3.PoolManager(ssl_context=ctx)
a = []
b = executor1.submit(http.request, "GET", "https://nhl.ru/")
a.append(b)
for i in a:
print(i.result().status)Please respond to the developers, or the problem will be closed due to the impossibility of feedback.
from urllib3.
snooppr
commented on June 15, 2024
solution, use the old version 1x.
from urllib3.
Related Issues (20)
- Provide a Test/Test-Mechanism for #3292 HOT 1
- Move response body handling into HTTP2Response HOT 3
- Add http_version property to BaseHTTPResponse HOT 4
- Ensure HTTP/2 isn't offered for tunnel CONNECT requests HOT 1
- Add HTTP/2 support for forwarding proxies HOT 2
- Add support for streaming HTTP/2 responses
- Allow HTTP/2 connection and socket to be re-used for future requests
- Add support for sending a request body with HTTP/2 HOT 1
- Add support for using HTTP/2 without TLS or prior knowledge HOT 1
- pypy tests often fail in CI with reason "cancelled after 30 minutes"
- Comply with TLS settings mandated for HTTP/2 in RFC 9113
- Upgrade mypy to the latest version in CI HOT 1
- Fix type checking when Zstandard is installed
- Move length_remaining into BaseHTTPResponse HOT 1
- Slow test cases on pypy3.9-7.3.15 on Ubuntu 22.04
- TLS 1.3 Post Handshake Auth no longer working with urllib 2.1.0 when ignoring cert validation HOT 1
- Create a workflow (nox?) for testing Emscripten support locally
- Emscripten support emits an InsecureRequestWarning even when using HTTPS
- Path toward testing with a released version of hypercorn? HOT 3
- Handle HTTP/2 informational responses (1xx) HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from urllib3.