Comments (8)
I made this, to make it a bit more transparent:
ansible/ansible#70806
from ansible-node-exporter.
url lookup and uri
module use very similar code internally. This issue can be workaround by using get_url
module instead of uri
.
Ideally, we would use url lookup, but currently, I have no fix.
from ansible-node-exporter.
@paulfantom that's not it. I tried get_url
also, same error, but less verbose.
I think uri
had the best error, I would vote for changing it, if you are okay with it. The others didn't get me anywhere.
All these mechanisms (lookup('url', ...)
, uri
and get_url
) have one thing in common:
The code seems to use a ~/.netrc
file/config — it was mentioned in various tickets on ansible/ansible
which all related to "400" errors when downloading something from Github. And that happens based on an environment variable:
Still not sure why the configuration for Github is appended to a request to s3.amazonaws.com
. I think the "follow redirect" in one of the Python libs may be buggy. But I wasted already 3 days on this, I am not gonna dig deeper now.
Anyhow, I also found netrc in the internals of the CI server we are using:
https://github.com/drone/drone/blob/5b6a3d8ff4c37283cf37df20d871cc8dfe439565/core/netrc.go
I am gonna roll back all changes and confirm this today and then close this ticket. Not sure if you want a note in the readme since it's been a repeated problem. Or add a link to this comment to all the tickets that people previously opened and went stale? =)
from ansible-node-exporter.
Thanks for the thorough investigation! Great job!
Looks to me that GitHub stores all assets in S3 bucket hence redirections to s3.amazonaws.com
.
It seems to me that this is indeed netrc issue. I wonder if we can forcibly omit it in lookup?
Either way, I think describing this in the documentation would be the best approach here. Let's maybe start a TROUBLESHOOTING.md
doc and link to it from README.md
? It seems like there are at least 2 issues that are related to user environment and are out of scope for the role - netrc and python forking on OSX. I could sync this file across all repos as those issues are common to all. WDYT?
from ansible-node-exporter.
Yes, that's a good plan. Where do you want me to PR the file?
from ansible-node-exporter.
Looks to me that GitHub stores all assets in S3 bucket hence redirections to s3.amazonaws.com.
It seems to me that this is indeed netrc issue. I wonder if we can forcibly omit it in lookup?
I haven't looked but what breaks the redirect is that the code adds the headers from the first request to the second which happens when it follows the location.
Even though the netrc is for "github.com" and not for Amazon's server.
GitHub support said in another ticket to not add the headers. I am "assuming" curl doesn't? To be confirmed (by someone else 🤪).
I haven't checked yet if that's how it's supposed to be or if that's something one can turn off. Seems kinda wild that there's no message that the config is loaded. Maybe I PR that to Ansible.
from ansible-node-exporter.
Where do you want me to PR the file?
In root of this repo.
Even though the netrc is for "github.com" and not for Amazon's server.
Yes, but the module follows redirects (which is necessary here) and at the end requests lands in s3.
I am "assuming" curl doesn't?
It doesn't unless specified with --netrc
.
how it's supposed to be or if that's something one can turn off.
I believe executing export NETRC=
before running ansible should solve this problem.
from ansible-node-exporter.
So, check this though (without netrc). This is just to figure out if this is a bug in Ansible or some library underneath. I am forcing basic-auth with cURL, it uses it against Github, but doesn't use it against the redirect — which is why it works. Look at the output, it contains Authorization: Basic ...
only in the initial request. Or maybe cURL is being smart.
I can't find the RFC, but I think headers etc. are only meant to be used for the first request, not for the redirect. But that's less important here. I'll PR a file about netrc.
❯ curl -v -L --basic https://$GITHUB_TOKEN:[email protected]/prometheus/node_exporter/releases/download/v1.0.1/sha256sums.txt
* Trying 140.82.118.4...
* TCP_NODELAY set
* Connected to github.com (140.82.118.4) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: C=US; ST=California; L=San Francisco; O=GitHub, Inc.; CN=github.com
* start date: May 5 00:00:00 2020 GMT
* expire date: May 10 12:00:00 2022 GMT
* subjectAltName: host "github.com" matched cert's "github.com"
* issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert SHA2 High Assurance Server CA
* SSL certificate verify ok.
* Server auth using Basic with user '57bf82e4bef88274a1d8f4db5b2fc08017b1bcbf'
> GET /prometheus/node_exporter/releases/download/v1.0.1/sha256sums.txt HTTP/1.1
> Host: github.com
> Authorization: Basic redacted
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 302 Found
< Date: Wed, 22 Jul 2020 15:14:39 GMT
< Content-Type: text/html; charset=utf-8
< Transfer-Encoding: chunked
< Server: GitHub.com
< Status: 302 Found
< Vary: X-PJAX
< Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/9524057/333d8080-afed-11ea-87b7-18fcef58bd32?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200722%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200722T151439Z&X-Amz-Expires=300&X-Amz-Signature=3130b8649de1412087cef557dbec349c0574fd6d3968148e0eda1965b34d13d3&X-Amz-SignedHeaders=host&actor_id=0&repo_id=9524057&response-content-disposition=attachment%3B%20filename%3Dsha256sums.txt&response-content-type=application%2Foctet-stream
< Cache-Control: no-cache
< Set-Cookie: logged_in=no; domain=.github.com; path=/; expires=Thu, 22 Jul 2021 15:14:39 GMT; secure; HttpOnly; SameSite=Lax
< Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
< X-Frame-Options: deny
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Expect-CT: max-age=2592000, report-uri="https://api.github.com/_private/browser/errors"
< Content-Security-Policy: default-src 'none'; base-uri 'self'; block-all-mixed-content; connect-src 'self' uploads.github.com www.githubstatus.com collector.githubapp.com api.github.com www.google-analytics.com github-cloud.s3.amazonaws.com github-production-repository-file-5c1aeb.s3.amazonaws.com github-production-upload-manifest-file-7fdce7.s3.amazonaws.com github-production-user-asset-6210df.s3.amazonaws.com cdn.optimizely.com logx.optimizely.com/v1/events wss://alive.github.com; font-src github.githubassets.com; form-action 'self' github.com gist.github.com; frame-ancestors 'none'; frame-src render.githubusercontent.com; img-src 'self' data: github.githubassets.com identicons.github.com collector.githubapp.com github-cloud.s3.amazonaws.com *.githubusercontent.com; manifest-src 'self'; media-src 'none'; script-src github.githubassets.com; style-src 'unsafe-inline' github.githubassets.com; worker-src github.com/socket-worker.js gist.github.com/socket-worker.js
< Vary: Accept-Encoding, Accept, X-Requested-With
< Vary: Accept-Encoding
< X-GitHub-Request-Id: E23B:60C0:8665526:C34A05E:5F1857DF
<
* Ignoring the response-body
* Connection #0 to host github.com left intact
* Issue another request to this URL: 'https://github-production-release-asset-2e65be.s3.amazonaws.com/9524057/333d8080-afed-11ea-87b7-18fcef58bd32?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200722%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200722T151439Z&X-Amz-Expires=300&X-Amz-Signature=3130b8649de1412087cef557dbec349c0574fd6d3968148e0eda1965b34d13d3&X-Amz-SignedHeaders=host&actor_id=0&repo_id=9524057&response-content-disposition=attachment%3B%20filename%3Dsha256sums.txt&response-content-type=application%2Foctet-stream'
* Trying 52.216.251.92...
* TCP_NODELAY set
* Connected to github-production-release-asset-2e65be.s3.amazonaws.com (52.216.251.92) port 443 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: C=US; ST=Washington; L=Seattle; O=Amazon.com, Inc.; CN=*.s3.amazonaws.com
* start date: Nov 9 00:00:00 2019 GMT
* expire date: Mar 12 12:00:00 2021 GMT
* subjectAltName: host "github-production-release-asset-2e65be.s3.amazonaws.com" matched cert's "*.s3.amazonaws.com"
* issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert Baltimore CA-2 G2
* SSL certificate verify ok.
> GET /9524057/333d8080-afed-11ea-87b7-18fcef58bd32?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200722%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200722T151439Z&X-Amz-Expires=300&X-Amz-Signature=3130b8649de1412087cef557dbec349c0574fd6d3968148e0eda1965b34d13d3&X-Amz-SignedHeaders=host&actor_id=0&repo_id=9524057&response-content-disposition=attachment%3B%20filename%3Dsha256sums.txt&response-content-type=application%2Foctet-stream HTTP/1.1
> Host: github-production-release-asset-2e65be.s3.amazonaws.com
> User-Agent: curl/7.64.1
> Accept: */*
>
< HTTP/1.1 200 OK
< x-amz-id-2: 7N+Xf+RQxPFActTLIiC1POm+SzHm4MfX7KopS60gfAeSQPuMo/lyU9Xj+8hklN0lMqkTSNSO1WA=
< x-amz-request-id: 0QFPDGBZ5S6T9R1T
< Date: Wed, 22 Jul 2020 15:14:41 GMT
< Last-Modified: Tue, 16 Jun 2020 13:19:50 GMT
< ETag: "9a23fe5723ccdf8bf897864bc51f6f4a"
< Content-Disposition: attachment; filename=sha256sums.txt
< Accept-Ranges: bytes
< Content-Type: application/octet-stream
< Content-Length: 1789
< Server: AmazonS3
<
eb7feb537a96d518644879f617eaef2c28e9af5878c671c0ba0af11d2c27c791 node_exporter-1.0.1.darwin-386.tar.gz
e51d39ef14f5c6accee158e94b5e324fa6eb647444234a4be3491fbc3983df47 node_exporter-1.0.1.darwin-amd64.tar.gz
734e036a849152b185da2080eb8656c36cde862095a464cb17705ca723ea3929 node_exporter-1.0.1.linux-386.tar.gz
3369b76cd2b0ba678b6d618deab320e565c3d93ccb5c2a0d5db51a53857768ae node_exporter-1.0.1.linux-amd64.tar.gz
017514906922fcc4b7d727655690787faed0562bc7a17aa9f72b0651cb1b47fb node_exporter-1.0.1.linux-arm64.tar.gz
38413100bfb935c59aea088a0af792134b75972eb90ab2bc6cf1c09ad3b08aea node_exporter-1.0.1.linux-armv5.tar.gz
c1d7affbc7762c478c169830c43b4c6177a761bf1d2dd715dbffa55ca772655a node_exporter-1.0.1.linux-armv6.tar.gz
e7f4427a25f1870103588e4968c7dc8c1426c00a0c029d0183a9a7afdd61357b node_exporter-1.0.1.linux-armv7.tar.gz
43335ccab5728b3c61ea7a0977143719c392ce13a90fa0d14169b5c10e8babd0 node_exporter-1.0.1.linux-mips.tar.gz
c0109f2f76628d2e25ea78e39d4b95100079ee859863be1471519b5e85a2fe78 node_exporter-1.0.1.linux-mips64.tar.gz
bcba02058b9ce171b5c3b077f78f371eb7685239f113200d15787c55fb204857 node_exporter-1.0.1.linux-mips64le.tar.gz
85f0a24c07c5d8237caf36a5c68a63958280dab802b5056ff36d75563d5e5241 node_exporter-1.0.1.linux-mipsle.tar.gz
43aa5e72f5068d16eb8d33f6b729186bf558d40ec0c734746b40a16902864808 node_exporter-1.0.1.linux-ppc64.tar.gz
5ae6c772108c877038cd66a761e4ad93edcc8c446120478499412b24e7953146 node_exporter-1.0.1.linux-ppc64le.tar.gz
2f22d1ce18969017fb32dbd285a264adf3da6252eec05f03f105cf638ec0bb06 node_exporter-1.0.1.linux-s390x.tar.gz
7766d78638c2f84d1084a79d8cb5d8f036b7ce375390870d5e709673118d1260 node_exporter-1.0.1.netbsd-386.tar.gz
41cc54f77f860ed19a7b74f132269f810e3c01fbac5320c3fa2e244fa2247d56 node_exporter-1.0.1.netbsd-amd64.tar.gz
* Connection #1 to host github-production-release-asset-2e65be.s3.amazonaws.com left intact
* Closing connection 0
* Closing connection 1
from ansible-node-exporter.
Related Issues (20)
- 'latest' version doesn't work HOT 11
- node_exporter_textfile_dir doesn't enable corresponding collector HOT 1
- Unpack node_exporter binary step fails on cluster HOT 3
- With recent release of node_exporter (1.2.2) the role fails to detect version installed. HOT 7
- Unable to find required 'zipinfo' binary in the path HOT 2
- Allow to install node exporter easily without internet HOT 4
- Create password hash on local machine HOT 3
- Encryption not working for basic authentication HOT 5
- new release? HOT 10
- Support for other RHEL-like packages HOT 1
- Refactor: Unnecessary call to github if node_exporter already installed HOT 1
- Still maintained? HOT 8
- tasks/main.yml wrong parameter in import_tasks: install.yml HOT 1
- Create a new release with changes in master branch HOT 4
- PreventHome works only if mount is directly under /home HOT 1
- Role doesn't work with Ansible 7.0.0 HOT 5
- Crash on get checksum list HOT 1
- Is ansible-node-exporter abandoned? No commits have been made since January 2022 (now December 2022) HOT 5
- Node exporter version 1.5.0 systemd unit unable to start HOT 2
- Example code in the README doesn't generate a CSR file HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ansible-node-exporter.