Comments (18)
Hey @yquansah, thanks for the above example, I'm able to reproduce
So when you run with multiple nodes, if you connect to node N1 but the upstream is connected to N2, then N1 forwards all traffic to N2
Since Piko forward connects to the server using WebSockets (to wrap the underlying TCP connection and work with a HTTP load balancer), if N1 receives the WebSocket connection that it needs to forward to N2, it falls back to using the HTTP proxy to forward the WebSocket to N2
However by default the HTTP proxy has a timeout of 30 seconds (--proxy.timeout
), meaning N1 closes the connection to N2 after 30 seconds, meaning the end-to-end connection is dropped. So the quick fix is to set --proxy.timeout 0
to disable the timeout, and I'll think about a better way to handle this tomorrow (the timeout should only apply to HTTP requests not hijacked WebSocket connections, as the same issue applies to using the HTTP proxy as normal with a WebSocket client)
from piko.
@condaatje Thats done: ghcr.io/andydunstall/piko:v0.6.1
from piko.
Hey @yquansah, thanks for raising this
I can't think of anything off the top of my head, but I'll play about this this later today to try and reproduce
I will try and dig into the gossip protocol tomorrow
I'd be surprised if it was caused by gossip, which only gossips around what upstreams are connected, but won't (or at least shouldn't) close connections
from piko.
@andydunstall Yeah nice. I am also going to play around with it later this afternoon. Thanks!
from piko.
@andydunstall Really great information here, I appreciate the sleuthing, and it definitely makes sense. We'll go with the proxy timeout for now as you suggested for a quick fix.
from piko.
hey, we gave this a whirl and are getting this error when the container starts:
config: proxy: missing timeout
here is the yaml for the deployment:
30 │ template:
1 │ │ metadata:
2 │ │ │ annotations:
3 │ │ │ │ kubectl.kubernetes.io/restartedAt: "2024-07-30T13:05:09-07:00"
4 │ │ │ creationTimestamp: null
5 │ │ │ labels:
6 │ │ │ │ app: piko
7 │ │ spec:
8 │ │ │ containers:
9 │ │ │ - args:
10 │ │ │ │ - server
11 │ │ │ │ - --config.path
12 │ │ │ │ - /config/server.yaml
13 │ │ │ │ - --config.expand-env
14 │ │ │ │ - --proxy.timeout
15 │ │ │ │ - "0"
perhaps I'm doing something wrong here?
from piko.
config: proxy: missing timeout
Ah sorry I forgot its required, I'll add a patch now
A quick work around is just set a very large timeout (such as --proxy.timeout=24h
)
(Sorry I haven't had much time to fix the underlying issue, I'll try and get to soon)
from piko.
I've merged #151 to allow a timeout of 0, so if you re-build main it should be ok
from piko.
@andydunstall Quick one. Thank you!
from piko.
awesome! could I trouble you for a tagged image @andydunstall ? right now we are pulling piko:v0.6.0
, but would normally not want to pull :latest
in our architecture.
from piko.
yep sure will tag v0.7.0 v0.6.1
from piko.
awesome thanks!
from piko.
by the way - if you'd like to join our alpha (launching today) we'd be honored to have you! especially because we've absolutely loved working with piko so far: https://x.com/hyperbolic_labs/status/1823779096650015026
from piko.
Thanks! I don't have anything I could use a GPU for, but the product looks great!
from piko.
I've merged #155, which only applies --proxy.timeout
to non-websocket (and non-TCP) connections, so you shouldn't need to set --proxy.timeout=0
anymore (sorry it took a while for me to get to)
So I'll close this for now. Let me know if theres anything else I can help with!
from piko.
appreciated!
and this is great - could I trouble you for a v0.6.2
tag?
from piko.
@condaatje sure tagged
from piko.
thanks!
from piko.
Related Issues (10)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from piko.