docker-archive / go-redis-server Goto Github PK

A Go implementation of the Redis server protocol

License: Apache License 2.0

Go 100.00%

go-redis-server's Introduction

Redis server protocol library

There are plenty of good client implementations of the redis protocol, but not many server implementations.

go-redis-server is a helper library for building server software capable of speaking the redis protocol. This could be an alternate implementation of redis, a custom proxy to redis, or even a completely different backend capable of "masquerading" its API as a redis database.

Sample code

package main

import (
	redis "github.com/dotcloud/go-redis-server"
)

type MyHandler struct {
	values map[string][]byte
}

func (h *MyHandler) GET(key string) ([]byte, error) {
	v := h.values[key]
	return v, nil
}

func (h *MyHandler) SET(key string, value []byte) error {
	h.values[key] = value
	return nil
}

func main() {
	handler, _ := redis.NewAutoHandler(&MyHandler{values: make(map[string][]byte)})
	server := &redis.Server{Handler: handler, Addr: ":6389"}
	server.ListenAndServe()
}

go-redis-server's People

Contributors

Stargazers

Watchers

Forkers

gitor dim wuub iamima wangchll martin-ly koblas catufunwa therealbill leehambley wtmmac alex-docker wudikua clearraining imskyer alienero unclejack cofyc michaelarnaldi robhaswell ralfonso rtoma insionng jmptrader sargun pkdevbox maverick0122 is00hcw robertojrojas richardfreeman tigerzhang qzaidi figoxu zzn01 ppxz2014 rusenask cloudxtreme yiting1122 xiaolushare npaton tang1007 travisby mackjoner platinasystems jharlap madfog dbkaiser rongfengliang clamoriniere zhangciba wxiaowar latermoon kryptblue princessd8251 gallir wleese shamahn xxr3376 tweiker muharremokutan jokosu10 runt18 guangminglion wupeaking goexperts leeparayno prima101112 stefb965 tokopedia mansfield6 andrisetyoo freeliver cluo linecode charsyam hubinix ykumar-rb prettyyjnic sakishum rain168 syaiful6 ruhip acbdef123 liuxuegit kknight99 ganjb0608 han-ian haifeiwu labuladong lntoly chrisgis333 shangm2 damozhiying huwei123 zofuthan charliexp global-localhost global19 global19-atlassian-net go-for-development

go-redis-server's Issues

Memory leak when parse large request

func readArgument(r *bufio.Reader) ([]byte, error) {

    line, err := r.ReadString('\n')
    if err != nil {
        return nil, malformed("$<argumentLength>", line)
    }
    var argSize int
    if _, err := fmt.Sscanf(line, "$%d\r", &argSize); err != nil {
        return nil, malformed("$<argumentSize>", line)
    }

    // I think int is safe here as the max length of request
    // should be less then max int value?
    data, err := ioutil.ReadAll(io.LimitReader(r, int64(argSize)))
    if err != nil {
        return nil, err
    }

    if len(data) != argSize {
        return nil, malformedLength(argSize, len(data))
    }

    // Now check for trailing CR
    if b, err := r.ReadByte(); err != nil || b != '\r' {
        return nil, malformedMissingCRLF()
    }

    // And LF
    if b, err := r.ReadByte(); err != nil || b != '\n' {
        return nil, malformedMissingCRLF()
    }

    return data, nil
}

ioutil.ReadAll will allocate at least 512bytes memery even if we don't need it, when prase a large request (for instance, MGET key1 key2 ... keyn), the memory costs will increase fast ( n * 512bytes).

Force gc will trigger every 2 min, and gc is also a goroutine. if memory increase very fast, even reach the phyiscal memory, gc will not invoke immediately.

Session data handling/select broken

Hi,

How can I store connection related data ? like the current db for each connection.

Currently, spawning two redis-cli console, and selecting a db in the first console, it also affect the second connection.

Any thoughts ? Thanks!

Closing a connection causes panic on a server

panic: runtime error: close of closed channel

goroutine 13 [running]:
github.com/dotcloud/go-redis-server.func·010()
    /home/mg/goworld/src/github.com/dotcloud/go-redis-server/server.go:96 +0xf7
created by github.com/dotcloud/go-redis-server.Serve
    /home/mg/goworld/src/github.com/dotcloud/go-redis-server/server.go:97 +0x34e

goroutine 1 [IO wait]:
net.runtime_pollWait(0x7f028abf1f00, 0x72, 0x0)
    /usr/local/go/src/pkg/runtime/znetpoll_linux_amd64.c:118 +0x82
net.(*pollDesc).WaitRead(0xc20008b080, 0xb, 0xc2000823f0)
    /usr/local/go/src/pkg/net/fd_poll_runtime.go:75 +0x31
net.(*netFD).accept(0xc20008b000, 0x58c9f8, 0x0, 0xc2000823f0, 0xb, ...)
    /usr/local/go/src/pkg/net/fd_unix.go:385 +0x2c1
net.(*TCPListener).AcceptTCP(0xc200000250, 0x42b4c1, 0x7f028ab6ce08, 0x42b4c1)
    /usr/local/go/src/pkg/net/tcpsock_posix.go:229 +0x45
net.(*TCPListener).Accept(0xc200000250, 0xc200000278, 0xc200000058, 0xc200085520, 0x0, ...)
    /usr/local/go/src/pkg/net/tcpsock_posix.go:239 +0x25
github.com/dotcloud/go-redis-server.(*Server).Serve(0xc200082300, 0xc20006f540, 0xc200000250, 0x0, 0x0, ...)
    /home/mg/goworld/src/github.com/dotcloud/go-redis-server/server.go:47 +0x115
github.com/dotcloud/go-redis-server.(*Server).ListenAndServe(0xc200082300, 0xc200082300, 0xc200000058)
    /home/mg/goworld/src/github.com/dotcloud/go-redis-server/server.go:34 +0x169
main.main()
    /home/mg/goworld/src/github.com/mailgun/pulsar/pulsar/main.go:18 +0x247

Support for pipelining?

Pipelined requests seem to hang indefinitely

Tests are broken

gocov test | gocov report
# github.com/dotcloud/go-redis-server
/tmp/gocov687978787/src/pkg/github.com/dotcloud/go-redis-server/parser_test.go:39: cannot use ioutil.NopCloser(strings.NewReader(v)) (type io.ReadCloser) as type net.Conn in function argument:
    io.ReadCloser does not implement net.Conn (missing LocalAddr method)
/tmp/gocov687978787/src/pkg/github.com/dotcloud/go-redis-server/parser_test.go:58: cannot use ioutil.NopCloser(strings.NewReader(p.s)) (type io.ReadCloser) as type net.Conn in function argument:
    io.ReadCloser does not implement net.Conn (missing LocalAddr method)
FAIL    github.com/dotcloud/go-redis-server [build failed]
go test failed: exit status 2
failed to unmarshal coverage data: unexpected end of JSON input
make: *** [coverage] Error 1

A Bug found when execute " hset " command.

127.0.0.1:6389> hset test  good boy 
Could not connect to Redis at 127.0.0.1:6389: Connection refused

there some traceback logs from server-side:

panic: runtime error: assignment to entry in nil map

goroutine 20 [running]:
runtime.panic(0x544d00, 0x631cf3)
        /usr/local/go/src/pkg/runtime/panic.c:279 +0xf5
redis.(*DefaultHandler).Hset(0xc208040140, 0xc2080004c8, 0x7, 0xc208000540, 0x4, 0xc208063200, 0x3, 0x600, 0x0, 0x0, ...)
        /home/openinx/software/go-redis-server-master/src/redis/defaultHandler.go:245 +0x6c3
reflect.Value.call(0x536240, 0x551d78, 0x0, 0x130, 0x55def0, 0x4, 0xc208046080, 0x4, 0x4, 0x0, ...)
        /usr/local/go/src/pkg/reflect/value.go:563 +0x1210
reflect.Value.Call(0x536240, 0x551d78, 0x0, 0x130, 0xc208046080, 0x4, 0x4, 0x0, 0x0, 0x0)
        /usr/local/go/src/pkg/reflect/value.go:411 +0xd7
redis.func·001(0xc208018280, 0x0, 0x0, 0x0, 0x0)
        /home/openinx/software/go-redis-server-master/src/redis/auto.go:94 +0xdbd
redis.(*Server).Apply(0xc20804a140, 0xc208018280, 0x0, 0x0, 0x0, 0x0)
        /home/openinx/software/go-redis-server-master/src/redis/handler.go:39 +0x13d
redis.(*Server).ServeClient(0xc20804a140, 0x7fcf810fd518, 0xc20803a1b8, 0x0, 0x0)
        /home/openinx/software/go-redis-server-master/src/redis/server.go:98 +0x323
created by redis.(*Server).Serve
        /home/openinx/software/go-redis-server-master/src/redis/server.go:50 +0x12f

goroutine 16 [IO wait]:
net.runtime_pollWait(0x7fcf810fd468, 0x72, 0x0)
        /usr/local/go/src/pkg/runtime/netpoll.goc:146 +0x66
net.(*pollDesc).Wait(0xc20802a140, 0x72, 0x0, 0x0)
        /usr/local/go/src/pkg/net/fd_poll_runtime.go:84 +0x46
net.(*pollDesc).WaitRead(0xc20802a140, 0x0, 0x0)
        /usr/local/go/src/pkg/net/fd_poll_runtime.go:89 +0x42
net.(*netFD).accept(0xc20802a0e0, 0x5a5908, 0x0, 0x7fcf810fc2b8, 0xb)
        /usr/local/go/src/pkg/net/fd_unix.go:409 +0x343
net.(*TCPListener).AcceptTCP(0xc20803a1b0, 0x28, 0x0, 0x0)
        /usr/local/go/src/pkg/net/tcpsock_posix.go:234 +0x5d
net.(*TCPListener).Accept(0xc20803a1b0, 0x0, 0x0, 0x0, 0x0)
        /usr/local/go/src/pkg/net/tcpsock_posix.go:244 +0x4b
redis.(*Server).Serve(0xc20804a140, 0x7fcf810fc498, 0xc20803a1b0, 0x0, 0x0)
        /home/openinx/software/go-redis-server-master/src/redis/server.go:46 +0xc6
redis.(*Server).ListenAndServe(0xc20804a140, 0x0, 0x0)
        /home/openinx/software/go-redis-server-master/src/redis/server.go:36 +0x190
main.main()
        /home/openinx/software/go-redis-server-master/src/main/main.go:12 +0x7b

goroutine 19 [finalizer wait]:
runtime.park(0x412eb0, 0x635818, 0x634389)
        /usr/local/go/src/pkg/runtime/proc.c:1369 +0x89
runtime.parkunlock(0x635818, 0x634389)
        /usr/local/go/src/pkg/runtime/proc.c:1385 +0x3b
runfinq()
        /usr/local/go/src/pkg/runtime/mgc0.c:2644 +0xcf
runtime.goexit()
        /usr/local/go/src/pkg/runtime/proc.c:1445

goroutine 21 [runnable]:
redis.func·013()
        /home/openinx/software/go-redis-server-master/src/redis/server.go:68
created by redis.(*Server).ServeClient
        /home/openinx/software/go-redis-server-master/src/redis/server.go:76 +0x140

After review the code, I found a bug from file defaultHandler.go.



func NewDatabase(parent *Database) *Database {
    db := &Database{
        values:   make(HashValue),
        sub:      make(HashSub),
        brstack:  make(HashBrStack),
        children: map[int]*Database{},
        parent:   parent,
    }
    db.children[0] = db
    return db
}

I think you just forget make a map for hvalues :

func NewDatabase(parent *Database) *Database {
    db := &Database{
        values:   make(HashValue),
        hvalues:  make(HashHash),
        sub:      make(HashSub),
        brstack:  make(HashBrStack),
        children: map[int]*Database{},
        parent:   parent,
    }
    db.children[0] = db
    return db
}

AutoHandle + reflect

Instead of having a 'AutoHandler' that reflects and store the methods in a map, wouldn't be easier to provide a default Handler that can be overridden by the user?

docker-archive / go-redis-server Goto Github PK

go-redis-server's Introduction

Redis server protocol library

Sample code

go-redis-server's People

Contributors

Stargazers

Watchers

Forkers

go-redis-server's Issues

Memory leak when parse large request

Session data handling/select broken

Closing a connection causes panic on a server

Support for pipelining?

Tests are broken

A Bug found when execute " hset " command.

AutoHandle + reflect

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent