Comments (18)
I only meant zstd would incur an external dependency because zlib has a std package
from websocket.
from websocket.
We probably do want this. Maybe just add a Compress
method to MessageWriter
to allow compressing messages and transparently decompress compressed messages being read.
from websocket.
Definitely something like #36
Would work just like browsers then which would be nice.
from websocket.
Useful article https://www.igvita.com/2013/11/27/configuring-and-optimizing-websocket-compression/
from websocket.
@coadler this issue is also ripe for work if you're interested.
from websocket.
Though I would suggest you wait for #81 as it is a rather large refactoring of the internals.
from websocket.
I think the move here is to expose zero flags. We should automatically compress text frames larger than X bytes, where X is determined experimentally. My guess is anywhere from 128 -512 bytes. We should also test every level and see which works best. Last time I tested, the lowest level had the most bang for buck.
For binary frames, I'm thinking we shouldn't compress at all as most binary protocols do not compress very well afaik unless you have a lot of duplicate data.
These configuration would work fantastic out of the box for most setups and require zero knobs.
from websocket.
So after some testing, I think the sweet spot is going to be 256 bytes for text. Above that, we should automatically use deflate compression at the lowest level if the peer supports it. Should be good for 99.99% of applications.
For binary messages, we should use a cutoff of 512 bytes. The reason we compress anyway for binary protocols too is I'm guessing that when the message size exceeds 512 bytes, its extremely likely the message can be compressed well.
Should also document this on Conn and link to this issue requesting feedback.
Thoughts @coadler
This should reduce your bandwidth significantly with the discord gateway for your bot.
from websocket.
Disabling compression entirely would be as easy as setting "Sec-WebSocket-Extension"
to ""
on the response or request header, if needed.
from websocket.
I think the deflate extension is not worth support at all.
See my investigation in gorilla/websocket#203 (comment)
It will cause a large memory spike as every flate writer takes up a ridiculous 1.2 megabytes in memory.
from websocket.
If deflate is out of the question, could we potentially still support zlib/zstd? Or would these be best implemented as some sort of wrapper library. For zstd we'd need https://github.com/DataDog/zstd or similar which now brings an external dep and cgo into the mix.
I'm not overly familiar with how websocket compression negotiation works, but for Discord it's picked based on a query param e.g. wss://gateway.discord.gg/?v=6&compress=zlib-stream
. So if this was done inside the library, we'd need a way to manually specify the algorithm within *Conn
.
from websocket.
Would be best as a wrapper lib. I'm happy to link to it from the README.md though. Also, why not https://golang.org/pkg/compress/zlib/?
from websocket.
It doesn't have the same allocation overhead as compress/flate so it should have solid performance.
from websocket.
Ah I wasn't aware there is a difference between the two. There's also https://www.reddit.com/r/golang/comments/be7esc/a_pure_go_port_of_the_zstandard_zstd_decompression/
from websocket.
Another issue with WebSocket compression: https://bugs.chromium.org/p/chromium/issues/detail?id=163882
Looks like browsers will unconditionally compress every message, even smaller ones, thereby increasing their size.
from websocket.
Interestingly, Firefox will compress all payloads but only send the ones smaller than the original payload: https://hg.mozilla.org/mozilla-central/diff/724f0a71d62171da1357e6c1f93453359e54206b/netwerk/protocol/websocket/WebSocketChannel.cpp#l1.277
from websocket.
To summarize:
We do not support the deflate compression extension because Go's compress/flate library is very memory intensive and browsers do not handle WebSocket compression intelligently.
from websocket.
Related Issues (20)
- websocket.Conn.CloseNow() panics in WaitGroup.Wait HOT 2
- failed to get reader: use of closed network connection HOT 3
- Inline part of the full examples HOT 7
- Add custom ping message option to Ping function HOT 1
- failed to WebSocket dial: response body is not a io.ReadWriteCloser: io.nopCloserWriterTo HOT 1
- unsupported permessage-deflate parameter: "client_max_window_bits=15" from client HOT 8
- Update docs and mention explicit close required change in v1.8.11 HOT 16
- panic: sync: WaitGroup is reused before previous Wait has returned HOT 1
- Return net.ErrClosed from Read methods when connection is closed by us HOT 6
- failed: Close received after close HOT 6
- Document wsjson.Read closes the connection with the appropriate close status and error message if bad JSON is read
- Should use path.Match instead of filepath.Match HOT 4
- Compatibility issues HOT 14
- Improve control on close reason
- Support (*http.ResponseController).Hijack() HOT 1
- Use existing JS Websocket
- Setting authorization headers on handshake request HOT 1
- 1008 unexpected data message: When using htmx client
- WebSocket Dial Handshake 403 When Connecting to Server Written Using x/net/websocket HOT 3
- Upgrading broken due to vanity rename HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from websocket.