Comments (10)
FTR, there's some good external discussions on the topic:
The latter also contains a recommendation of the PSF Security Developer-in-Residence to use https://truststore.rtfd.io/#using-truststore-with-aiohttp to pre-configure the CA chain. (though their — I made a PR to fix them @ sethmlarson/truststore#139)aiohttp
snippet needs to be fixed to avoid resource warnings / cc @sethmlarson
It also explicitly explains that setting up the cert store is something to be done in applications and not in libraries/frameworks.
Hopefully, PEP 543 resurrection will also get back on track (I noticed that @woodruffw started https://github.com/trailofbits/tlslib.py, and he usually gets the job done) bringing nicer APIs in this field.
from aiohttp.
@Dreamsorcerer yep, that was my thinking. Additionally, we may want to document that it's possible to pass truststore.SSLContext
into our interfaces.
from aiohttp.
@doctorpangloss by the way, you don't need an async def in your helper function since you don't actually do anything async inside. JFYI.
Yes, I thought the same, but that's what fssync
wants, it awaits get_client
! So it goes.
I appreciate all the attention to the issue, thank you. I have also raised the issue with the Windows Containers team.
Ultimately, the reason aiohttp
doesn't bug out on Windows is because people are using desktop Windows interactively, and eventually, they have made a requests
call / curl
call or some similar interaction that has some kind of side effects which allow aiohttp
to successfully connect to many https:// addresses. I don't understand fully the side effects, and have asked folks at Microsoft to clarify. Specifically, Windows curl.exe
seems to somehow activate the specific certificate needed for the specific address requested, which many other URLs will share, and those other URLs sharing that certificate will work with aiohttp
. But access a URL that doesn't have a system activated (unknown side effect) certificate, and it will fail.
To summarize, aiohttp
on a clean, naked, vanilla Windows, desktop or container or otherwise, will always fail to access https://
URLs.
from aiohttp.
certifi
is a hack that the requests maintainers regret having to do. The end-users are responsible for setting up what they trust. Libraries like aiohttp
shouldn't be making assumptions or taking over this responsibility.
from aiohttp.
Yeah, unless you have a specific suggestion of some way we are loading the certificates wrong, then this seems like an issue with the platform.
If you want to use the certifi hack, similar to requests, then that is documented at: https://docs.aiohttp.org/en/stable/client_advanced.html#example-use-certifi
from aiohttp.
If you want to use the certifi hack, similar to requests, then that is documented at: https://docs.aiohttp.org/en/stable/client_advanced.html#example-use-certifi
The documentation has flaws, for example:
By default, Python uses the system CA certificates
This may be true. But requests
does not only use the system CA certificates by default. It ships with the web certificates bundle. My specific suggestion is that aiohttp
should ship with the bundle.
I am using the pattern that I do because other libraries, like fsspec
, want a ClientSession
object. I didn't choose to use aiohttp
, my libraries did.
Libraries like aiohttp shouldn't be making assumptions or taking over this responsibility.
Nobody would be able to install things on Windows or macOS if this strategy were used with requests
, because pip
uses requests
and used to throw this error all the time, for Windows and macOS. The messages were vague - maybe that's another fix, make less vague errors - that they didn't report to bug trackers but on Stack Overflow instead. If any reporting at all.
Many users with aiohttp
installed and used in Python didn't choose to use it. Just like with requests
. When using pip
, nobody chose to use requests
. So they added the certificate bundle so that installing packages would work on Windows without vague errors.
from aiohttp.
@doctorpangloss by the way, you don't need an async def
in your helper function since you don't actually do anything async inside. JFYI.
from aiohttp.
The latter also contains a recommendation of the PSF Security Developer-in-Residence to use https://truststore.rtfd.io/#using-truststore-with-aiohttp to pre-configure the CA chain.
Maybe this link can be added to our docs.
from aiohttp.
Here to give a big ++ to everything @webknjaz said, applications shouldn't be carrying around their own certificate bundles since it's only another layer of headache for operations to keep up-to-date. Delegating to the system is what all other pieces of software do, so Python applications should too.
Pip got mentioned as well, pip already supports using Truststore (recent versions with --use-feature=truststore
) and will soon be enabled by default.
from aiohttp.
Related to #955
To a certain extent, this ticket simply means that aiohttp
will always randomly fail on Windows machines, because it has adopted a stance on SSL certificates - not shipping them - on a platform that does not ship with SSL certificates.
One resolution is shown in the Python code here: microsoft/Windows-Containers#503 (comment) which will essentially use certifi
as your trust store.
Another solution is the equivalent of installing ca-certificates
on Windows which is here: microsoft/Windows-Containers#503 (comment)
Hopefully people who find this ticket with various ssl verify errors will encourage the aiohttp maintainers to reconsider this position from a practical point of view. The only solution right now is using certifi.
from aiohttp.
Related Issues (20)
- https://docs.aiohttp.org/ has been defaced/squatted HOT 1
- aiohttp.org site is down HOT 1
- Server; POST-request; BadStatusLine: 400; an endless stream of exception messages that have no impact on operation.
- Broken timeout system with ws_connect HOT 1
- graceful shutdown order and cleanup contexts HOT 7
- ValueError: I/O operation on closed file in aiohttp payload size property on file uploading using form HOT 1
- Allow reading already retrieved response body after connection is closed
- Support for sending Trailer headers HOT 2
- Separate exception for DNS errors
- Weak Etag can not trigger negotiation caching
- The set request parameters were not used HOT 16
- Cannot make request behind a plain http proxy, aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host xxx ssl:default [None] HOT 1
- Exception with aiohttp Using HTTP Proxy in some cases HOT 1
- aiohttp cannot be used in python 3.14. HOT 4
- Iterate over response data (streaming) without decoding content HOT 1
- With forced TLSv1.3, still TLSv1.0 Client Hello is sent HOT 6
- 3.10.0 and 4.0.0 incorrectly reject wss:// urls HOT 3
- Http query character encoding issues HOT 1
- FastAPI integration HOT 2
- Doc Add ConnectionResetError exception to ClientWebSocketResponse documentation
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aiohttp.