Comments (13)
It looks like a problem with the Ngxin proxy configuration. I can connect directly to the API service from the server:
$ curl -i http://127.0.0.1:3001/v1/proxy/image/https%3A%2F%2Fraw.githubusercontent.com%2Ftorrust%2Ftorrust-index%2Fdevelop%2Fdocs%2Fmedia%2Ftorrust_logo.png
HTTP/1.1 200 OK
content-type: image/png
content-length: 2687
access-control-allow-origin: *
vary: origin
vary: access-control-request-method
vary: access-control-request-headers
access-control-expose-headers: *
date: Thu, 14 Dec 2023 14:39:21 GMT
Warning: Binary output can mess up your terminal. Use "--output -" to tell
Warning: curl to output it to your terminal anyway, or consider "--output
Warning: <FILE>" to save to a file.
from torrust-compose.
I have enabled the Nginx debug mode with:
server
{
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name index.torrust-demo.com;
error_log /var/log/nginx/error.log debug; <- THIS NEW LINE
server_tokens off;
...
location ^~/api/
{
rewrite ^/api/(.*)$ /$1 break;
try_files $uri @index;
}
...
location @index
{
proxy_pass http://index:3001;
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-XSS-Protection "1; mode=block" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "no-referrer-when-downgrade" always;
add_header Content-Security-Policy "default-src * data: 'unsafe-eval' 'unsafe-inline'" always;
#add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;
# enable strict transport security only if you understand the implications
}
...
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
}
And it seems Nginx is removing one slash in the redirection URL:
2023/12/14 16:53:39 [notice] 20#20: *1 "^/api/(.*)$" matches "/api/v1/proxy/image/https:/raw.githubusercontent.com/torrust/torrust-index/develop/docs/media/torrust_logo.png", client: 2.137.102.79, server: index.torrust-demo.com, request: "GET /api/v1/proxy/image/https%3A%2F%2Fraw.githubusercontent.com%2Ftorrust%2Ftorrust-index%2Fdevelop%2Fdocs%2Fmedia%2Ftorrust_logo.png HTTP/2.0", host: "index.torrust-demo.com", referrer: "https://index.torrust-demo.com/torrent/443c7602b4fde83d1154d6d9da48808418b181b6/ubuntu-2304-desktop-amd64"
2023/12/14 16:53:39 [notice] 20#20: *1 rewritten data: "/v1/proxy/image/https:/raw.githubusercontent.com/torrust/torrust-index/develop/docs/media/torrust_logo.png", args: "", client: 2.137.102.79, server: index.torrust-demo.com, request: "GET /api/v1/proxy/image/https%3A%2F%2Fraw.githubusercontent.com%2Ftorrust%2Ftorrust-index%2Fdevelop%2Fdocs%2Fmedia%2Ftorrust_logo.png HTTP/2.0", host: "index.torrust-demo.com", referrer: "https://index.torrust-demo.com/torrent/443c7602b4fde83d1154d6d9da48808418b181b6/ubuntu-2304-desktop-amd64"
2.137.102.79 - - [14/Dec/2023:16:53:39 +0000] "GET /api/v1/proxy/image/https%3A%2F%2Fraw.githubusercontent.com%2Ftorrust%2Ftorrust-index%2Fdevelop%2Fdocs%2Fmedia%2Ftorrust_logo.png HTTP/2.0" 404 0 "https://index.torrust-demo.com/torrent/443c7602b4fde83d1154d6d9da48808418b181b6/ubuntu-2304-desktop-amd64" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36" "-"
As you can see is using /v1/proxy/image/https:/raw.githubusercontent.com/torrust/torrust-index/develop/docs/media/torrust_logo.png
with https:/r...
instead of https://r...
from torrust-compose.
By enabling the option merge_slashes off;
inside the server
block I get this log limes:
2.137.102.79 - - [14/Dec/2023:17:18:06 +0000] "GET /api/v1/proxy/image/https%3A%2F%2Fraw.githubusercontent.com%2Ftorrust%2Ftorrust-index%2Fdevelop%2Fdocs%2Fmedia%2Ftorrust_logo.png HTTP/2.0" 404 0 "https://index.torrust-demo.com/torrent/edit/443c7602b4fde83d1154d6d9da48808418b181b6" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36" "-"
2023/12/14 17:18:06 [notice] 19#19: *9 "^/api/(.*)$" matches "/api/v1/proxy/image/https://raw.githubusercontent.com/torrust/torrust-index/develop/docs/media/torrust_logo.png", client: 2.137.102.79, server: index.torrust-demo.com, request: "GET /api/v1/proxy/image/https%3A%2F%2Fraw.githubusercontent.com%2Ftorrust%2Ftorrust-index%2Fdevelop%2Fdocs%2Fmedia%2Ftorrust_logo.png HTTP/2.0", host: "index.torrust-demo.com", referrer: "https://index.torrust-demo.com/torrent/edit/443c7602b4fde83d1154d6d9da48808418b181b6"
2023/12/14 17:18:06 [notice] 19#19: *9 rewritten data: "/v1/proxy/image/https://raw.githubusercontent.com/torrust/torrust-index/develop/docs/media/torrust_logo.png", args: "", client: 2.137.102.79, server: index.torrust-demo.com, request: "GET /api/v1/proxy/image/https%3A%2F%2Fraw.githubusercontent.com%2Ftorrust%2Ftorrust-index%2Fdevelop%2Fdocs%2Fmedia%2Ftorrust_logo.png HTTP/2.0", host: "index.torrust-demo.com", referrer: "https://index.torrust-demo.com/torrent/edit/443c7602b4fde83d1154d6d9da48808418b181b6"
It seems that now it's using this URL:
/v1/proxy/image/https://raw.githubusercontent.com/torrust/torrust-index/develop/docs/media/torrust_logo.png
It adds two slashes but It's not working because this URL:
does not work. hHe URL of the image has to be URL encoded in the PATH.
from torrust-compose.
It seems I have to change the Nginx config:
- https://stackoverflow.com/q/28684300/3012842
- https://trac.nginx.org/nginx/ticket/727
- https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass
Because:
Characters which are not allowed to be passed unescaped are escaped after URI transformation. The ":" and "/" characters are allowed within URI (though may have special meaning in some cases).
It seems I have to use the variable $request_uri
which contains the original unescaped values.
from torrust-compose.
ChatGTP suggested to do this:
location /api/ {
# Capture the part after '/api/' and store it in a variable
if ($request_uri ~* "^/api/(.+)") {
set $captured_path $1;
}
# Strip off the '/api/' part for internal processing
rewrite ^/api/(.*)$ /$1 break;
# Pass the captured path to the named location
try_files $uri @index;
}
location @index {
# Use the captured path for proxying
proxy_pass http://index:3001/$captured_path$is_args$args;
# ... rest of your configuration
}
from torrust-compose.
Hey @da2ce7 @WarmBeer this seems to be very problematic. Maybe we should find an alternative way to encode URLs inside another URL. What do you thing?
from torrust-compose.
Links:
from torrust-compose.
Relates to: torrust/torrust-index-gui#409
from torrust-compose.
I've tried a lot of different configurations for Nginx but none of them have worked.
- https://trac.nginx.org/nginx/ticket/786?cversion=0&cnum_hist=2
- https://stackoverflow.com/questions/28684300/nginx-pass-proxy-subdirectory-without-url-decoding/37584637#37584637
- https://stackoverflow.com/questions/20496963/avoid-nginx-decoding-query-parameters-on-proxy-pass-equivalent-to-allowencodeds
It seems that if you want to parse and modify the URI, Nginx decodes it. Some options like:
location /api/ {
set $modified_uri $uri;
if ($request_uri ~ ^/api/(.*)$) {
set $modified_uri $1;
}
proxy_pass http://index:3001/$modified_uri;
}
or
location /api {
rewrite ^ $request_uri; # get original URI
rewrite ^/api(/.*) $1 break; # drop /api, put /app
return 400; # if the second rewrite won't match
proxy_pass http://index:3001$request_uri;
}
seems to work but I get this error:
proxy | 2024/02/05 12:58:48 [error] 20#20: *1 no resolver defined to resolve index, client: 172.26.0.1, server: index.torrust-demo.com, request: "GET /api/v1/proxy/image/https%3A%2F%2Fraw.githubusercontent.com%2Ftorrust%2Ftorrust-index%2Fdevelop%2Fdocs%2Fmedia%2Ftorrust_logo.png HTTP/1.1", host: "index.torrust-demo.com"
proxy | 2024/02/05 12:58:48 [debug] 20#20: *1 finalize http upstream request: 502
from torrust-compose.
I'm using this configuration to enable debug
log level in Nginx:
proxy:
image: nginx
container_name: proxy
restart: unless-stopped
networks:
- frontend_network
- backend_network
ports:
- "80:80"
- "443:443"
volumes:
- ./storage/proxy/webroot:/var/www/html
- ./storage/proxy/etc/nginx-conf:/etc/nginx/conf.d
- ./storage/certbot/etc:/etc/letsencrypt
- ./storage/certbot/lib:/var/lib/letsencrypt
- ./storage/dhparam:/etc/ssl/certs
command: [nginx-debug, '-g', 'daemon off;']
logging:
options:
max-size: "10m"
max-file: "10"
depends_on:
- index-gui
- index
- tracker
from torrust-compose.
I finally found a working configuration:
server
{
listen 80;
listen [::]:80;
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
server_name index.torrust-demo.com;
#error_log /var/log/nginx/error.log debug;
location /api/v1/proxy
{
rewrite ^ $request_uri;
rewrite ^/api(/.*) $1 break;
proxy_pass http://index:3001;
}
location ^~ /api/
{
rewrite ^/api/(.*)$ /$1 break;
proxy_pass http://index:3001;
}
location /
{
proxy_pass http://index-gui:3000;
}
location ~ /.well-known/acme-challenge
{
allow all;
root /var/www/html;
}
}
It works, but it has a potential problem. The proxy_pass
in the following location does not include the URL query.
location /api/v1/proxy
{
rewrite ^ $request_uri;
rewrite ^/api(/.*) $1 break;
proxy_pass http://index:3001;
}
CUrrently, it's not a problem because URL are like this:
We do not have any "query" in the image proxy URLs.
from torrust-compose.
It seems this also works:
server
{
listen 80;
listen [::]:80;
root /var/www/html;
index index.html index.htm index.nginx-debian.html;
server_name index.torrust-demo.com;
#error_log /var/log/nginx/error.log debug;
location ^~ /api/
{
rewrite ^ $request_uri;
rewrite ^/api(/.*) $1 break;
proxy_pass http://index:3001;
}
location /
{
proxy_pass http://index-gui:3000;
}
location ~ /.well-known/acme-challenge
{
allow all;
root /var/www/html;
}
}
mergin the two locations, but it does not work for HTTPs using a named location.
from torrust-compose.
I'm going to keep it as it's for the time being. It would be nice to try to simplify the configuration.
from torrust-compose.
Related Issues (5)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from torrust-compose.