Giter VIP home page Giter VIP logo

ws's Introduction

ws

GoDoc CI

RFC6455 WebSocket implementation in Go.

Features

  • Zero-copy upgrade
  • No intermediate allocations during I/O
  • Low-level API which allows to build your own logic of packet handling and buffers reuse
  • High-level wrappers and helpers around API in wsutil package, which allow to start fast without digging the protocol internals

Documentation

GoDoc.

Why

Existing WebSocket implementations do not allow users to reuse I/O buffers between connections in clear way. This library aims to export efficient low-level interface for working with the protocol without forcing only one way it could be used.

By the way, if you want get the higher-level tools, you can use wsutil package.

Status

Library is tagged as v1* so its API must not be broken during some improvements or refactoring.

This implementation of RFC6455 passes Autobahn Test Suite and currently has about 78% coverage.

Examples

Example applications using ws are developed in separate repository ws-examples.

Usage

The higher-level example of WebSocket echo server:

package main

import (
	"net/http"

	"github.com/gobwas/ws"
	"github.com/gobwas/ws/wsutil"
)

func main() {
	http.ListenAndServe(":8080", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		conn, _, _, err := ws.UpgradeHTTP(r, w)
		if err != nil {
			// handle error
		}
		go func() {
			defer conn.Close()

			for {
				msg, op, err := wsutil.ReadClientData(conn)
				if err != nil {
					// handle error
				}
				err = wsutil.WriteServerMessage(conn, op, msg)
				if err != nil {
					// handle error
				}
			}
		}()
	}))
}

Lower-level, but still high-level example:

import (
	"net/http"
	"io"

	"github.com/gobwas/ws"
	"github.com/gobwas/ws/wsutil"
)

func main() {
	http.ListenAndServe(":8080", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		conn, _, _, err := ws.UpgradeHTTP(r, w)
		if err != nil {
			// handle error
		}
		go func() {
			defer conn.Close()

			var (
				state  = ws.StateServerSide
				reader = wsutil.NewReader(conn, state)
				writer = wsutil.NewWriter(conn, state, ws.OpText)
			)
			for {
				header, err := reader.NextFrame()
				if err != nil {
					// handle error
				}

				// Reset writer to write frame with right operation code.
				writer.Reset(conn, state, header.OpCode)

				if _, err = io.Copy(writer, reader); err != nil {
					// handle error
				}
				if err = writer.Flush(); err != nil {
					// handle error
				}
			}
		}()
	}))
}

We can apply the same pattern to read and write structured responses through a JSON encoder and decoder.:

	...
	var (
		r = wsutil.NewReader(conn, ws.StateServerSide)
		w = wsutil.NewWriter(conn, ws.StateServerSide, ws.OpText)
		decoder = json.NewDecoder(r)
		encoder = json.NewEncoder(w)
	)
	for {
		hdr, err = r.NextFrame()
		if err != nil {
			return err
		}
		if hdr.OpCode == ws.OpClose {
			return io.EOF
		}
		var req Request
		if err := decoder.Decode(&req); err != nil {
			return err
		}
		var resp Response
		if err := encoder.Encode(&resp); err != nil {
			return err
		}
		if err = w.Flush(); err != nil {
			return err
		}
	}
	...

The lower-level example without wsutil:

package main

import (
	"net"
	"io"

	"github.com/gobwas/ws"
)

func main() {
	ln, err := net.Listen("tcp", "localhost:8080")
	if err != nil {
		log.Fatal(err)
	}

	for {
		conn, err := ln.Accept()
		if err != nil {
			// handle error
		}
		_, err = ws.Upgrade(conn)
		if err != nil {
			// handle error
		}

		go func() {
			defer conn.Close()

			for {
				header, err := ws.ReadHeader(conn)
				if err != nil {
					// handle error
				}

				payload := make([]byte, header.Length)
				_, err = io.ReadFull(conn, payload)
				if err != nil {
					// handle error
				}
				if header.Masked {
					ws.Cipher(payload, header.Mask, 0)
				}

				// Reset the Masked flag, server frames must not be masked as
				// RFC6455 says.
				header.Masked = false

				if err := ws.WriteHeader(conn, header); err != nil {
					// handle error
				}
				if _, err := conn.Write(payload); err != nil {
					// handle error
				}

				if header.OpCode == ws.OpClose {
					return
				}
			}
		}()
	}
}

Zero-copy upgrade

Zero-copy upgrade helps to avoid unnecessary allocations and copying while handling HTTP Upgrade request.

Processing of all non-websocket headers is made in place with use of registered user callbacks whose arguments are only valid until callback returns.

The simple example looks like this:

package main

import (
	"net"
	"log"

	"github.com/gobwas/ws"
)

func main() {
	ln, err := net.Listen("tcp", "localhost:8080")
	if err != nil {
		log.Fatal(err)
	}
	u := ws.Upgrader{
		OnHeader: func(key, value []byte) (err error) {
			log.Printf("non-websocket header: %q=%q", key, value)
			return
		},
	}
	for {
		conn, err := ln.Accept()
		if err != nil {
			// handle error
		}

		_, err = u.Upgrade(conn)
		if err != nil {
			// handle error
		}
	}
}

Usage of ws.Upgrader here brings ability to control incoming connections on tcp level and simply not to accept them by some logic.

Zero-copy upgrade is for high-load services which have to control many resources such as connections buffers.

The real life example could be like this:

package main

import (
	"fmt"
	"io"
	"log"
	"net"
	"net/http"
	"runtime"

	"github.com/gobwas/httphead"
	"github.com/gobwas/ws"
)

func main() {
	ln, err := net.Listen("tcp", "localhost:8080")
	if err != nil {
		// handle error
	}

	// Prepare handshake header writer from http.Header mapping.
	header := ws.HandshakeHeaderHTTP(http.Header{
		"X-Go-Version": []string{runtime.Version()},
	})

	u := ws.Upgrader{
		OnHost: func(host []byte) error {
			if string(host) == "github.com" {
				return nil
			}
			return ws.RejectConnectionError(
				ws.RejectionStatus(403),
				ws.RejectionHeader(ws.HandshakeHeaderString(
					"X-Want-Host: github.com\r\n",
				)),
			)
		},
		OnHeader: func(key, value []byte) error {
			if string(key) != "Cookie" {
				return nil
			}
			ok := httphead.ScanCookie(value, func(key, value []byte) bool {
				// Check session here or do some other stuff with cookies.
				// Maybe copy some values for future use.
				return true
			})
			if ok {
				return nil
			}
			return ws.RejectConnectionError(
				ws.RejectionReason("bad cookie"),
				ws.RejectionStatus(400),
			)
		},
		OnBeforeUpgrade: func() (ws.HandshakeHeader, error) {
			return header, nil
		},
	}
	for {
		conn, err := ln.Accept()
		if err != nil {
			log.Fatal(err)
		}
		_, err = u.Upgrade(conn)
		if err != nil {
			log.Printf("upgrade error: %s", err)
		}
	}
}

Compression

There is a ws/wsflate package to support Permessage-Deflate Compression Extension.

It provides minimalistic I/O wrappers to be used in conjunction with any deflate implementation (for example, the standard library's compress/flate).

It is also compatible with wsutil's reader and writer by providing wsflate.MessageState type, which implements wsutil.SendExtension and wsutil.RecvExtension interfaces.

package main

import (
	"bytes"
	"log"
	"net"

	"github.com/gobwas/ws"
	"github.com/gobwas/ws/wsflate"
)

func main() {
	ln, err := net.Listen("tcp", "localhost:8080")
	if err != nil {
		// handle error
	}
	e := wsflate.Extension{
		// We are using default parameters here since we use
		// wsflate.{Compress,Decompress}Frame helpers below in the code.
		// This assumes that we use standard compress/flate package as flate
		// implementation.
		Parameters: wsflate.DefaultParameters,
	}
	u := ws.Upgrader{
		Negotiate: e.Negotiate,
	}
	for {
		conn, err := ln.Accept()
		if err != nil {
			log.Fatal(err)
		}

		// Reset extension after previous upgrades.
		e.Reset()

		_, err = u.Upgrade(conn)
		if err != nil {
			log.Printf("upgrade error: %s", err)
			continue
		}
		if _, ok := e.Accepted(); !ok {
			log.Printf("didn't negotiate compression for %s", conn.RemoteAddr())
			conn.Close()
			continue
		}

		go func() {
			defer conn.Close()
			for {
				frame, err := ws.ReadFrame(conn)
				if err != nil {
					// Handle error.
					return
				}

				frame = ws.UnmaskFrameInPlace(frame)

				if wsflate.IsCompressed(frame.Header) {
					// Note that even after successful negotiation of
					// compression extension, both sides are able to send
					// non-compressed messages.
					frame, err = wsflate.DecompressFrame(frame)
					if err != nil {
						// Handle error.
						return
					}
				}

				// Do something with frame...

				ack := ws.NewTextFrame([]byte("this is an acknowledgement"))

				// Compress response unconditionally.
				ack, err = wsflate.CompressFrame(ack)
				if err != nil {
					// Handle error.
					return
				}
				if err = ws.WriteFrame(conn, ack); err != nil {
					// Handle error.
					return
				}
			}
		}()
	}
}

You can use compression with wsutil package this way:

	// Upgrade somehow and negotiate compression to get the conn...

	// Initialize flate reader. We are using nil as a source io.Reader because
	// we will Reset() it in the message i/o loop below.
	fr := wsflate.NewReader(nil, func(r io.Reader) wsflate.Decompressor {
		return flate.NewReader(r)
	})
	// Initialize flate writer. We are using nil as a destination io.Writer
	// because we will Reset() it in the message i/o loop below.
	fw := wsflate.NewWriter(nil, func(w io.Writer) wsflate.Compressor {
		f, _ := flate.NewWriter(w, 9)
		return f
	})

	// Declare compression message state variable.
	//
	// It has two goals:
	// - Allow users to check whether received message is compressed or not.
	// - Help wsutil.Reader and wsutil.Writer to set/unset appropriate
	//   WebSocket header bits while writing next frame to the wire (it
	//   implements wsutil.RecvExtension and wsutil.SendExtension).
	var msg wsflate.MessageState

	// Initialize WebSocket reader as previously. 
	// Please note the use of Reader.Extensions field as well as
	// of ws.StateExtended flag.
	rd := &wsutil.Reader{
		Source:     conn,
		State:      ws.StateServerSide | ws.StateExtended,
		Extensions: []wsutil.RecvExtension{
			&msg, 
		},
	}

	// Initialize WebSocket writer with ws.StateExtended flag as well.
	wr := wsutil.NewWriter(conn, ws.StateServerSide|ws.StateExtended, 0)
	// Use the message state as wsutil.SendExtension.
	wr.SetExtensions(&msg)

	for {
		h, err := rd.NextFrame()
		if err != nil {
			// handle error.
		}
		if h.OpCode.IsControl() {
			// handle control frame.
		}
		if !msg.IsCompressed() {
			// handle uncompressed frame (skipped for the sake of example
			// simplicity).
		}

		// Reset the writer to echo same op code.
		wr.Reset(h.OpCode)

		// Reset both flate reader and writer to start the new round of i/o.
		fr.Reset(rd)
		fw.Reset(wr)

		// Copy whole message from reader to writer decompressing it and
		// compressing again.
		if _, err := io.Copy(fw, fr); err != nil {
			// handle error.
		}
		// Flush any remaining buffers from flate writer to WebSocket writer.
		if err := fw.Close(); err != nil {
			// handle error.
		}
		// Flush the whole WebSocket message to the wire.
		if err := wr.Flush(); err != nil {
			// handle error.
		}
	}

ws's People

Contributors

ackratos avatar agnivade avatar alexandear avatar bengadbois avatar cristaloleg avatar dependabot[bot] avatar edwingeng avatar gobwas avatar klauspost avatar magicxyyz avatar maldris avatar matthewmueller avatar navossoc avatar qguv avatar scrivy avatar shazow avatar wfnuser avatar xakep666 avatar xdwart avatar zekelu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ws's Issues

upgrader violate go lint

As described in README, to extend Upgrader, one can add onHeader method. However, the method signature did violate go lint

error should be the last type when returning multiple items

an error occur when send message to chrome client with websocket upgrade by TCP

I did like this as the example say:
func main() {
ln, err := net.Listen("tcp", "localhost:8080")
if err != nil {
log.Fatal(err)
}

for {
	conn, err := ln.Accept()
	if err != nil {
		// handle error
	}
	_, err = ws.Upgrade(conn)
	if err != nil {
		// handle error
	}

	go func() {
		defer conn.Close()

		for {
			header, err := ws.ReadHeader(conn)
			if err != nil {
				// handle error
			}

			payload := make([]byte, header.Length)
			_, err = io.ReadFull(conn, payload)
			if err != nil {
				// handle error
			}
			if header.Masked {
				ws.Cipher(payload, header.Mask, 0)
			}

			// Reset the Masked flag, server frames must not be masked as
			// RFC6455 says.
			header.Masked = false

			if err := ws.WriteHeader(conn, header); err != nil {
				// handle error
			}
			if _, err := conn.Write(payload); err != nil {
				// handle error
			}

			if header.OpCode == ws.OpClose {
				return
			}
		}
	}()
}

}

But I found that when a js client(chrome) received server's response, an error occured: โ€œWebsocket connection to 'ws://127.0.0.1:8800' failed: Could not decode a text frame as UTF-8"

I pretty sure than server response correctly and if it was a go client everything ok.

Then I found than that's because server's response data was too long. But the Max Length that js client could decode correctly is just one key-value pair of json {"keyโ€:"value"}. messages exceed this length will result in that error. And error message differ from the message length. Sometimes the error message is "one or more reserved bits are used"

Package isn't working

Can't even run any of the examples.

# github.com/gobwas/ws
../../../../github.com/gobwas/ws/client.go:249: too many arguments to return
	have (*bufio.Reader, int)
	want (*bufio.Reader)
../../../../github.com/gobwas/ws/server.go:333: multiple-value pbufio.GetReader() in single-value context

I need to handle HTTP & Websocket connection on the same port

I Think Upgrade is breaking the connection which cannot be handled by fasthttp anymore. Any Idea how can i handle it ?

    tcpConfig := tcplisten.Config{
        ReusePort:   false,
        FastOpen:    true,
        DeferAccept: true,
        Backlog:     2048,
    }
    listener, err := tcpConfig.NewListener("tcp4", ":8081")
    if err != nil {
        _LOG.Fatal(_funcName, err.Error())
    }
    // Start Infinite Loop to Accept new Connections
    for {
        conn, err := listener.Accept()
        if err != nil {
            continue
        }

        // Try to Upgrade Connection
        if _, err := ws.Upgrade(conn); err != nil {
            s.LimitUp()
            go func() {
                defer s.LimitDown()
                if err := fasthttp.ServeConn(conn, s.OnHttpRequest); err != nil {
                    _LOG.Error(_funcName, err.Error(), "FastHTTP ServeConn")
                }
            }()
            _LOG.Error(_funcName, err.Error(), "Websocket Upgrade")
            continue
        }

        // Register NetPoll event handler
        // The connDesc (Connection Describer) will be registered in an observer-list.
        if connDesc, err := netpoll.HandleReadOnce(conn); err != nil {
            // If error is non-nil then close the connection and free resources (i.e. File Descriptor, ...)
            _LOG.Error(_funcName, err.Error())
            conn.Close()
        } else {
            wsConn := NewWebsocketConnection(s, conn, connDesc)
            if err := s.poller.Start(connDesc, func(e netpoll.Event) {
                // TODO:: (ehsan) Block for ever ?!!
                s.LimitUp()
                go func() {
                    defer s.LimitDown()
                    wsConn.Receive()
                }()

            }); err != nil {
                log.Println(err.Error())
            }
        }
    }

Sec-WebSocket-Protocol

Hi there!
I'm not exactly sure, but I think this behavior is wrong (or misleading).

Let me explain:
I'm testing the Sec-WebSocket-Protocol, trying to require a custom protocol.

	u := ws.Upgrader{
		Protocol: func(protocol []byte) bool {
			log.Println("Protocol", string(protocol))
			return string(protocol) == "binary"
		},
	}

If my client is using the protocol binary everything works perfectly (as expected).
As soon I change the protocol on my client to binary2 for example, my function Protocol evaluates to false and the client rejects the connection...

The problem happens only when I omit the Sec-WebSocket-Protocol, because the client didn't sent any value, the Protocol func on the server is never called, so the connection is accepted.

The request MAY include a header field with the name |Sec-WebSocket-Protocol|. If present, this value indicates one or more comma-separated subprotocol the client wishes to speak, ordered by preference. The elements that comprise this value MUST be non-empty strings with characters in the range U+0021 to U+007E not including separator characters as defined in [RFC2616] and MUST all be unique strings. The ABNF for the value of this header field is 1#token, where the definitions of constructs and rules are as given in [RFC2616].

Let me know if you need more details.

How can I enforce a subprotocol? The only way I can think about is using the handshake return value on Upgrader.Upgrade and then closing the connection.

Stack based slice does not work.

func ReadHeader(r io.Reader) (h Header, err error) {
	var b [MaxHeaderSize - 2]byte

       // ************ OLD CODE (Go 1.12 does not work!) ************
	//bp := uintptr(unsafe.Pointer(&b))
	//bh := &reflect.SliceHeader{Data: bp, Len: 2, Cap: MaxHeaderSize - 2}
	//bts := *(*[]byte)(unsafe.Pointer(bh))

        // ************ NEW CODE (Go 1.12 work it!) ************
	var bts []byte
	bh := (*reflect.SliceHeader)(unsafe.Pointer(&bts))
	bh.Data = uintptr(unsafe.Pointer(&b))
	bh.Cap = MaxHeaderSize - 2
	bh.Len = 2

	// Prepare to hold first 2 bytes to choose size of next read.
	_, err = io.ReadFull(r, bts)
	if err != nil {
		return
	}
        ....
}```

Using ws.{Reader,Writer} with a persistent JSON encoder/decoder

Hi there, I'm trying to figure out if it's possible to use this library with a persistent JSON encoder and decoder as the messaging interface (without remaking the encoder/decoder with each message).

With other libraries, I'd typically do something like:

func (srv *Server) Join(conn *websocket.Conn) error {	 
	decoder := json.NewDecoder(conn)	
	encoder := json.NewEncoder(conn)	
 	
	for {
		// Decode next message
		msg := Message{...}
		if err := decoder.Decode(&msg); err != nil {
			return err
		}
		...
		if err := encoder.Encode(&Response{...}); err != nil {
			return err
		}
	}
}

Is it possible to do something similar with ws? Every example I find, it's making a new encoder/decoder with each message, which I'm hoping to avoid.

Here's what I have so far:

// Join runs the event loop for a connection, should be run in a goroutine.
func (srv *Server) Join(conn net.Conn) error {
	if _, err := ws.ReadHeader(conn); err != nil {
		return err
	}

	r := wsutil.NewReader(conn, ws.StateServerSide)
	decoder := json.NewDecoder(r)

	w := wsutil.NewWriter(conn, ws.StateServerSide, ws.OpText)
	encoder := json.NewEncoder(w)

	for {
		// Write buffer
		if err = w.Flush(); err != nil {
			return err
		}
		// Prepare for the next message
		if _, err = r.NextFrame(); err != nil {
			return err
		}
		// Decode next message
		msg := Message{}
		if err := decoder.Decode(&msg); err != nil {
			return err
		}
		...
		if err := encoder.Encode(&Response{...}); err != nil {
			return err
		}
	}
}

But I'm getting errors like ErrProtocolControlNotFinal.

Does the design of ws disallow using it in this manner, or am I missing something?

gobwas/pool dependency issue

Right now, dep will resolve gobwas/pool dependency in a way that latest (and used) version is e294a5482d9a8523f8e6b2744ea2185083c838af which is currently 2 commits back. The reason is because it is last tagged version.

Current version of gobwas/ws with older version of gobwas/pool wont compile because of breaking change.

The best solution is to just add tag to latest commit in gobwas/pool.

Getting an error on github.com/gobwas/pool/pbufio with `go1.10.1` and tip

@gobwas I'm getting an error on go1.10.1 and tip (FYI go1.9.5 is passed) travis build https://travis-ci.org/go-aah/aah/jobs/370367917#L627

# github.com/gobwas/pool/pbufio
../../github.com/gobwas/pool/pbufio/pbufio_go19.go:7:35: writerSize redeclared in this block
	previous declaration at ../../github.com/gobwas/pool/pbufio/pbufio_go110.go:7:35
../../github.com/gobwas/pool/pbufio/pbufio_go19.go:13:35: readerSize redeclared in this block
	previous declaration at ../../github.com/gobwas/pool/pbufio/pbufio_go110.go:11:35

Could you please be unblock it? Thank you.

Upgrade Error : Bad Connection Header

Need help

I am using chat application under ws-example and also tried other example applications.

When I try to connect to the websocket directly (not through proxy), getting "Upgrade Error : bad "Connection" header"

I dont want to use proxy in my case. Please suggest

detect client disconnect

i have server, that spawns some external binary and send output to client via websocket.
I'm not read any input from user because external program non-interactive.

How can i detect that user disconnects and stop sending data? naive http handler (skip upgrade part)

   ww := wsutil.NewWriterSize(conn, ws.StateServerSide, ws.OpBinary, 1024)
   cmd := exec.Command("dmesg", "--color=always", "-w")
   cmd.Env = []string{
     "TERM", "xterm-256color",
   }
   ptmx, err := pty.Start(cmd)
   if err != nil {
     log.Printf("%v", err)
     return
   }
   defer ptmx.Close()
   defer cmd.Process.Kill()
   wr := bufio.NewReaderSize(ptmx, 1024)
   for {
     line, err := wr.ReadSlice('\n')
     if err != nil {
       log.Printf("%v\n", err)
       return
     }
     _, err = ww.Write(line)
     if err == nil {
       err = ww.Flush()
     }
     if err != nil {
       log.Printf("%v", err)
       return
     }
   }

Godoc API is wrong about UpgradeHTTP

At the godoc main page:

http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
	  conn, err := ws.UpgradeHTTP(r, w, nil)
  })

Two variables are returned but looks like current source returns 4. It should be:
conn, _, _, err := ws.UpgradeHTTP(r, w, nil)

How to detect idle connection?

Hi,

We are using gobwas/ws in our production server. We have a pool of go routines which will be called on Desc events. The problem is that the only way to detect idle connections is to have a separate routine check connections and close them if needed.

I was wondering if it is the correct way to handle this problem or is there any better solution on this?

wsutil.readData() blocks until desired opcode/frame is received

In wsutil.readData(), whenever a control frame (pings) or unwanted OpCode is received, it performs a "continue" in the for loop to read the next frame (rd.NextFrame()), repeating until a wanted opcode type is received.

For most users using goroutines this is fine as it blocks until the desired data is available.

However when used with epoll this is problematic as it may potentially block. A simple solution would be to return nil,0,nil instead of performing a continue to read the next frame, but that might actually break the expected behavior of this function.

Network Send Overhead

The API calls for a separate write for the header. This results in either an extra TCP package (with a few bytes) or (Nagle) overhead.

ws.Dial() looses frames from server

Suppose we have a server, which accepts connection, upgrades it to WebSocket, and then immediately sends some frame. Current implementation of a client (ws.Dial) use buffer for reading HTTP response, and there is a probability of reading also some (or all) bytes of that frame into buffer. Those bytes will be loosed, because current Dial() expects only HTTP data.

I am going to change the API of the ws.Dialer to handle such cases.

Probably I will do something like this:
- Dial(...) (net.Conn, ws.Response, error)
+ Dial(...) (net.Conn, ws.Response, []ws.Frame, error)

Note that just wrapping returned connection in a way like this:

type wrappedConn struct {
    net.Conn
    r io.Reader
}

func wrap(c net.Conn, p []byte) wrappedConn {
    return wrappedConn{
        c, io.MultiReader(bytes.NewReader(p), c),
    }
}

func (w wrappedConn) Read(p []byte) (int, error) {
    return r.Read(p)
}

; will produce performance degradation for those who using net.Buffers for writing into connection. See this issue for more details.

Please let me know, if someone has a different idea which will help to solve this thing without increasing number of returned values.

connections count

Hi,

I can't to open more then 2600 active connections(
On my mac ulimit about 10000 but it still got error like
Too many files open...

How to increase connections?

Upgrade request with large cookie header causes ws.ReadHeader to return an EOF error

If you handle a Websocket upgrade request with ws.Upgrade(conn io.ReadWriter) and the request has a large cookie header the subsequent call to ws.ReadHeader(r io.Reader) will return EOF because io.ReadFull() will not read any bytes on the connection.

I don't quite know how to write a repro for this but if you set a large cookie on the same domain that you're connecting to you should be able to reproduce it I think.

Intermittent protocol failure and malformed payload

I'm willing to bet this is an oversight on my part, but here goes.

I wrote a light weight websocket connection between two servers for work, with both client and server utilising gobwas/ws.

Intermittently (and much more frequently on windows) I will get a read error "frames from client to server must be masked" on the server, and terminate the connection. However, all writes from my client code use wsutil.WriteClientMessage, so all messages should be masked (assuming my understanding of the library is correct).
I note that this, and some other protocol errors happened early on when I was accidentally reading from the connection in two goroutines simultaneously, though I believe this has been resolved.

I also see the occasional issue where a message payload contains additional bytes (usually ~4) on the front that look like part of the websocket frame data (payload is always JSON in current usages).

I suspect I've overlooked something in one of the systems designs, but at this point I've run out of ideas as to the cause, so I'm appealing for assistance.

Summary:

  • Server intermittently (and more frequently on windows) produces read error "frames from client to server must be masked"
  • client intermittently has additional bytes prefixed to payload, preventing message parsing

the client library I wrote can be seen here:
https://github.com/Maldris/evtWebsocketClient
and a gist of the server connection/communication logic can be seen here:
https://gist.github.com/Maldris/4f62a8b7456fd82c3de6551ef4d8d99c

Is there any thing like gorilla/websocket func (c *Conn) SetReadLimit(limit int64)

when I change from gorilla/websocket to gobwas/ws. I found SetReadLimit() in gorilla/websocket to limit then max size of text or binary frame size, but I found nothing in gobwas/ws like that.

I check the code in gorilla/websocket:

https://github.com/gorilla/websocket/blob/80c2d40e9b91f2ef7a9c1a403aeec64d1b89a9a6/conn.go#L859-L870

I check wsutil code:

ws/wsutil/reader.go

Lines 171 to 194 in 7338e26

r.raw = io.LimitedReader{r.Source, hdr.Length}
frame := io.Reader(&r.raw)
if hdr.Masked {
frame = NewCipherReader(frame, hdr.Mask)
}
if r.fragmented() {
if hdr.OpCode.IsControl() {
if cb := r.OnIntermediate; cb != nil {
err = cb(hdr, frame)
}
if err == nil {
// Ensure that src is empty.
_, err = io.Copy(ioutil.Discard, &r.raw)
}
return
}
} else {
r.opCode = hdr.OpCode
}
if r.CheckUTF8 && (hdr.OpCode == ws.OpText || (r.fragmented() && r.opCode == ws.OpText)) {
r.utf8.Source = frame
frame = &r.utf8
}

likely nothing to limit the size of a frame, so Is there anything about that requirement or could you give me some tips for doing that.

thx~

Looking forward to your response

Hello @gobwas - I'm Jeeva, I'm planning to bring Websocket feature on next release of aah framework. I have read this article and went through your ws codebase.

I'm interested in integrating with aah framework, however you have mentioned in the readme "The library is not tagged as v1.0.0 yet so it can be broken during some improvements or refactoring." I think it is production ready. However I would like to hear from you and also could you please tag it?

Thank you!

when put ws server under nginx, client request throw 505 error?

I start the ws at ws://localhost:9628/v1/ws/agent, when connect directly with following client code, it works ok:

url := `ws://localhost:9628/v1/ws/agent`
conn, _, _, err := ws.DefaultDialer.Dial(context.Background(), url)
if err != nil {
    log.Fatal(err) 
}

But when I put the server as a Nginx (Version: nginx/1.4.6 (Ubuntu)) upstream, and connect ws://localhost:80/v1/ws/agent:

server {
    listen 80;
    location /v1/ws/agent {
        proxy_pass http://localhost:9628;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'Upgrade';
    }
}

The error says that "unexpected HTTP response status: 505". Why this happend?

ARM support

Hi, I want to use your project in a small resource constrained ARM device project, sadly when I try to pull in your library with that context go errors, I assume its due to arm, because when I pull the same set of dependencies with just GOOS=linux set, it does not error.

$ GOOS=linux GOARCH=arm GOARM=7 go get -v -u github.com/gobwas/ws github.com/gobwas/ws/wsutil github.com/mailru/easygo/netpoll
github.com/gobwas/ws (download)
github.com/gobwas/httphead (download)
github.com/gobwas/pool (download)
github.com/mailru/easygo (download)
Fetching https://golang.org/x/sys/unix?go-get=1
Parsing meta tags from https://golang.org/x/sys/unix?go-get=1 (status code 200)
get "golang.org/x/sys/unix": found meta tag main.metaImport{Prefix:"golang.org/x/sys", VCS:"git", RepoRoot:"https://go.googlesource.com/sys"} at https://golang.org/x/sys/unix?go-get=1
get "golang.org/x/sys/unix": verifying non-authoritative meta tag
Fetching https://golang.org/x/sys?go-get=1
Parsing meta tags from https://golang.org/x/sys?go-get=1 (status code 200)
golang.org/x/sys (download)
github.com/gobwas/httphead
github.com/gobwas/pool
golang.org/x/sys/unix
github.com/gobwas/pool/pbufio
github.com/gobwas/pool/pbytes
github.com/gobwas/ws
github.com/mailru/easygo/netpoll
github.com/gobwas/ws/wsutil
# github.com/gobwas/ws/wsutil
..\..\github.com\gobwas\ws\wsutil\writer.go:203: constant 9223372036854775807 overflows int

panic in the production server

Greetings,

We are deploying our chat servers in the production env, the connsrv is the gateway server between clients and internal logic server, which is serving websocket requests , however , we met an unusual panic last day
The panic stack is :

panic: runtime error: makeslice: len out of range

goroutine 5626477 [running]:
aladinfun.com/midware/goim/vendor/github.com/gobwas/ws.ReadFrame(0x7f5be12140c0, 0xc42000ece0, 0x60700, 0x2424ebb535b8385d, 0xfba2344b01, 0x0, 0x0, 0x0, 0x0, 0x0)
 /data/home/user_00/go_workspace/src/aladinfun.com/midware/goim/vendor/github.com/gobwas/ws/read.go:115 +0xd7
aladinfun.com/midware/goim/srv/connsrv.wsReceive(0xe4a5e0, 0xc42000ece0, 0x8, 0x9d4440, 0xc422d8c730, 0xc4231811e0, 0x18)
 /data/home/user_00/go_workspace/src/aladinfun.com/midware/goim/srv/connsrv/conn_ws.go:49 +0xa6
aladinfun.com/midware/goim/srv/connsrv.clientLoop(0xe4a5e0, 0xc42000ece0, 0xc4203e2f80)
 /data/home/user_00/go_workspace/src/aladinfun.com/midware/goim/srv/connsrv/conn_handler.go:49 +0x150
aladinfun.com/midware/goim/srv/connsrv.uriRoute(0xe4a5e0, 0xc42000ece0, 0xc4203e2f80)
 /data/home/user_00/go_workspace/src/aladinfun.com/midware/goim/srv/connsrv/conn_listener.go:85 +0x133
aladinfun.com/midware/goim/srv/connsrv.handleConn(0xe4a5e0, 0xc42000ece0)
 /data/home/user_00/go_workspace/src/aladinfun.com/midware/goim/srv/connsrv/conn_listener.go:69 +0x25f
created by aladinfun.com/midware/goim/srv/connsrv.listenTCP
 /data/home/user_00/go_workspace/src/aladinfun.com/midware/goim/srv/connsrv/conn_listener.go:57 +0x4fd

The location of the panic in read.go is :

func ReadFrame(r io.Reader) (f Frame, err error) {
	f.Header, err = ReadHeader(r)
	if err != nil {
		return
	}

	if f.Header.Length > 0 {
		// int(f.Header.Length) is safe here cause we have
		// checked it for overflow above in ReadHeader.
                //  panic here!! 
		f.Payload = make([]byte, int(f.Header.Length))
		_, err = io.ReadFull(r, f.Payload)
	}

	return
}

However , we don't know how much the frame.length is..

The /data/home/user_00/go_workspace/src/aladinfun.com/midware/goim/srv/connsrv/conn_ws.go:49 is below

45 func wsReceive(conn net.Conn) (data []byte, err error) {
46	if config.SrvCfg.ClientTimeoutR != 0 {
47 		conn.SetReadDeadline(time.Now().Add(time.Duration(config.SrvCfg.ClientTimeoutR) * time.Millisecond))
48 	}
49 	frm, err := ws.ReadFrame(conn)
50 	if err != nil {
51 		xlog.LevelLogfn(logger.DebugLog, "ws ReadFrame failed, %s", err.Error())
52 		return
53 	}
54 	if frm.Header.Masked {
55		ws.Cipher(frm.Payload, frm.Header.Mask, 0)
56	}
57
58	data = frm.Payload
59	// if data, _, err = crypto.Base64Decode(frm.Payload); err != nil {
60	// 	xlog.LevelLogfn(logger.DebugLog, "base64 decode failed, %s", err.Error())
61	// 	return
62	// }
63	xlog.LevelLogfn(logger.TraceLog, "receive buffer: %v", data)
64	return
65 }

in ws.Upgrader, retrieve cookie?

I see the example code given in Readme did check for cookies. But is there anyway to return the cookie value when calling upgrader.Upgrade? Otherwise it would be meaningless to validate the cookie other than prevent unknown requests.

Can't catch a user disconnect event

Hi, its an awesome package, thank you!
But I can't catch a disconnect event.
I try to get OpClose, but its not working...

How to catch it and remove user from list?
thanks

What the correct way to get statusCode for incoming data?

I'm use your library for implement websocket (with Zero-copy upgrade). I have two object - Server and Client.
The Client has handler method Receive which listen data for each connection.

func (ch *Channel) Receive(c chan []byte) {
	log.Println("Read client data")
	msg := make([]wsutil.Message, 0, 4)
        

for {
        msg, err := wsutil.ReadClientMessage(ch.conn, msg[:0])
	if err != nil {
		log.Printf("read message error: %v", err)
		return
	}

	for _, m := range msg {

			if m.OpCode.IsControl() {

				// do something for this ...

			}

			if m.OpCode.IsData() {
				statusCode, reason := ws.ParseCloseFrameData(m.Payload)
				log.Printf("Try get statusCode %q, and reason %q", statusCode, reason)

			}
				
		}
  }
}

For connect to my WebSocket server i use Nginx as reverse-proxy and all works fine exclude one moment.
When on the nginx come proxy_read_timeout event I can't catch it in my code. After it event i get OpContinuation code and my Receive handler go to infinity loop (that load CPU to 100%).
I tried use ws.ParseCloseFrameData function, but payload data has lenght=0 and statusCode=0.

On the other side (web-browser client) gets event 1006 from nginx and browser disconnect properly.

Could you help me. What am I doing wrong?

Question: how/where to save URI parsed during Upgrader.Upgrade() ?

Hi,
Thanks for your blog post and this nice library.

I am considering using it for my own websocket-based application. My app is very similar to your chat server example in your ws-examples repo. One difference though, the user name is supplied by the end user in the URI, during initial connection (websocket upgrade) whilst in your chat server, the user name is a random one generated server side (see randName function).

What I would like is somehow to save in a variable the URI of the HTTP GET request used to upgrade to WS. That variable should be accessible in my app logic, that is outside of the Upgrader.Upgrade.

The signature of Upgrader.Upgrade is Upgrade(conn io.ReadWriter) (hs Handshake, err error). I see 2 possibilities to do what I'd like.

  1. Add a new parameter to Upgrade, something like Upgrade(conn io.ReadWriter, conn_storage map[string]interface{}) and put the info (URI, HTTP Headers etc.) I want to save in that new map
  2. Change the Handshake type and add a new field (in addition to Protocol and Extensions) that would also store the URI at which the handshake was done.

All in all, it boils down to saving the result of some internal parsing already done by the nice upgrade() method.

I believe this feature request could be beneficial to others, so maybe we can come up with something flexible enough? What do you think ?

Thanks

TLS support

Is it possible to use this library over SSL, any attempt to use over ssl returns error in netpool
panic: "could not get file descriptor"

Code example
cer, err := tls.LoadX509KeyPair("./etc/cert.crt", "./etc/cert.key")
if err != nil {
panic(err)
}
config := &tls.Config{Certificates: []tls.Certificate{cer}}
ln, err := tls.Listen("tcp", "0.0.0.0:443", config)
if err != nil {
panic(err)
}
acceptDesc := netpoll.Must(netpoll.HandleListener(
ln, netpoll.EventRead|netpoll.EventOneShot,
))
here on Must panics

on regular sockets works, but on ssl did not manage to make it work in any way :(

tried different methods to get underlying net.Conn in order for File() reflection to work but did not manage to make it happen and could not find single example of this being used

Is this even possible ?

Byte slice overwritten when upgrade

Greetings,

I found that, some websocket client library implementation will split the handshake info into several packets (typically two packets, means there are two tcp send syscalls during the first handshake packet. first line is GET /uri HTTP/1.1 , other part is HOST:some.domain\r\n.......). This will cause the byte slice overwritten by the latter util.go: readLine funciton when the handshake info was delivered from client more than one packet.
The attachment is the test code, you may find the request.uri is not correct .

wstest.zip

How should i deal with message fragmentation?

Hello,for my project i using websockets and protobuf,so the problem is the protobuf's messages isn't self-describing i have to send plain []byte,but your library even with wsutil package returning opcode to me from wsutil.ReadClientData(), the Opcode enumerates state of current frame, the Fin byte instead indicating end of the frame, Fin usage is incapsulated by library ,but Opcode is not, as of these two are working together i have no ideas how it's supposed to work.
P.S You already have code to recieve frames and messages, why not make this func block until the message is completly arriven?

[signal SIGSEGV: segmentation violation code=0x2

unexpected fault address 0x4128c0
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x2 addr=0x4128c0 pc=0x4128c0]

goroutine 3029725 [running]:
runtime.throw(0x8c0392, 0x5)
	/usr/lib/golang/src/runtime/panic.go:605 +0x95 fp=0xc48b1dae38 sp=0xc48b1dae18 pc=0x42d1c5
runtime.sigpanic()
	/usr/lib/golang/src/runtime/signal_unix.go:374 +0x227 fp=0xc48b1dae88 sp=0xc48b1dae38 pc=0x443d07
runtime.newobject(0x8755a0, 0x20)
	/usr/lib/golang/src/runtime/malloc.go:839 fp=0xc48b1dae90 sp=0xc48b1dae88 pc=0x4128c0
vendor/github.com/gobwas/ws/wsutil.NewCipherReader(...)
	/data/jenkins/workspace/src/xxxxxx/vendor/github.com/gobwas/ws/wsutil/cipher.go:21
vendor/github.com/gobwas/ws/wsutil.(*Reader).NextFrame(0xc45c5ad400, 0x1a23a98301020001, 0x20, 0x7f8d484ec060, 0xc46798c5b8)
	/data/jenkins/workspace/src/xxxxxx/vendor/github.com/gobwas/ws/wsutil/reader.go:175 +0x36c fp=0xc48b1daf20 sp=0xc48b1dae90 pc=0x725bec
vendor/github.com/gobwas/ws/wsutil.readData(0x7f8d484ab220, 0xc46798c5b8, 0x301, 0xb3, 0xc0, 0x0, 0x0, 0xc45ffdfc20, 0x24)
	/data/jenkins/workspace/src/xxxxxx/vendor/github.com/gobwas/ws/wsutil/helper.go:258 +0x16a fp=0xc48b1db010 sp=0xc48b1daf20 pc=0x72504a
vendor/github.com/gobwas/ws/wsutil.ReadData(0x7f8d484ab220, 0xc46798c5b8, 0xc48b1db001, 0xc48b1db0b0, 0x410608, 0x84baa0, 0x8ac780, 0x782800, 0x7f8d484ab220)
	/data/jenkins/workspace/src/xxxxxx/vendor/github.com/gobwas/ws/wsutil/helper.go:88 +0x47 fp=0xc48b1db068 sp=0xc48b1db010 pc=0x724b07
vendor/github.com/gobwas/ws/wsutil.ReadClientData(0x7f8d484ab220, 0xc46798c5b8, 0xc46798c5b8, 0x7f8d484ab220, 0xc46798c5b8, 0x0, 0xc45ffdfc20, 0x24)
	/data/jenkins/workspace/src/xxxxxx/vendor/github.com/gobwas/ws/wsutil/helper.go:97 +0x3a fp=0xc48b1db0c0 sp=0xc48b1db068 pc=0x724baa
devserv.acceptWebsocket(0xaf9e40, 0xc4312cd420, 0xc42fcf8200)
	/data/jenkins/workspace/src/xxxxxx/devserv/devserv.go:343 +0x1a9c fp=0xc48b1dbc80 sp=0xc48b1db0c0 pc=0x7828fc
devserv.(*myHttpServerMux).ServeHTTP(0xb34038, 0xaf9e40, 0xc4312cd420, 0xc42fcf8200)
	/data/jenkins/workspace/src/xxxxxx/devserv/devserv.go:85 +0x151 fp=0xc48b1dbd18 sp=0xc48b1dbc80 pc=0x780811
net/http.serverHandler.ServeHTTP(0xc420174340, 0xaf9e40, 0xc4312cd420, 0xc42fcf8200)
	/usr/lib/golang/src/net/http/server.go:2619 +0xb4 fp=0xc48b1dbd48 sp=0xc48b1dbd18 pc=0x6d5284
net/http.(*conn).serve(0xc44a55dcc0, 0xafa680, 0xc46c8cd880)
	/usr/lib/golang/src/net/http/server.go:1801 +0x71d fp=0xc48b1dbfc8 sp=0xc48b1dbd48 pc=0x6d154d
runtime.goexit()
	/usr/lib/golang/src/runtime/asm_amd64.s:2337 +0x1 fp=0xc48b1dbfd0 sp=0xc48b1dbfc8 pc=0x45e001
created by net/http.(*Server).Serve
	/usr/lib/golang/src/net/http/server.go:2720 +0x288

version:v1.0.0
go version:1.10.2
Is the connection broken?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.