Giter VIP home page Giter VIP logo

redcon's Introduction

REDCON
GoDoc

Redis compatible server framework for Go

Features

  • Create a Fast custom Redis compatible server in Go
  • Simple interface. One function ListenAndServe and two types Conn & Command
  • Support for pipelining and telnet commands
  • Works with Redis clients such as redigo, redis-py, node_redis, and jedis
  • TLS Support
  • Compatible pub/sub support
  • Multithreaded

This library is also avaliable for Rust and C.

Installing

go get -u github.com/tidwall/redcon

Example

Here's a full example of a Redis clone that accepts:

  • SET key value
  • GET key
  • DEL key
  • PING
  • QUIT
  • PUBLISH channel message
  • SUBSCRIBE channel

You can run this example from a terminal:

go run example/clone.go
package main

import (
	"log"
	"strings"
	"sync"

	"github.com/tidwall/redcon"
)

var addr = ":6380"

func main() {
	var mu sync.RWMutex
	var items = make(map[string][]byte)
	var ps redcon.PubSub
	go log.Printf("started server at %s", addr)
	err := redcon.ListenAndServe(addr,
		func(conn redcon.Conn, cmd redcon.Command) {
			switch strings.ToLower(string(cmd.Args[0])) {
			default:
				conn.WriteError("ERR unknown command '" + string(cmd.Args[0]) + "'")
			case "ping":
				conn.WriteString("PONG")
			case "quit":
				conn.WriteString("OK")
				conn.Close()
			case "set":
				if len(cmd.Args) != 3 {
					conn.WriteError("ERR wrong number of arguments for '" + string(cmd.Args[0]) + "' command")
					return
				}
				mu.Lock()
				items[string(cmd.Args[1])] = cmd.Args[2]
				mu.Unlock()
				conn.WriteString("OK")
			case "get":
				if len(cmd.Args) != 2 {
					conn.WriteError("ERR wrong number of arguments for '" + string(cmd.Args[0]) + "' command")
					return
				}
				mu.RLock()
				val, ok := items[string(cmd.Args[1])]
				mu.RUnlock()
				if !ok {
					conn.WriteNull()
				} else {
					conn.WriteBulk(val)
				}
			case "del":
				if len(cmd.Args) != 2 {
					conn.WriteError("ERR wrong number of arguments for '" + string(cmd.Args[0]) + "' command")
					return
				}
				mu.Lock()
				_, ok := items[string(cmd.Args[1])]
				delete(items, string(cmd.Args[1]))
				mu.Unlock()
				if !ok {
					conn.WriteInt(0)
				} else {
					conn.WriteInt(1)
				}
			case "publish":
				if len(cmd.Args) != 3 {
					conn.WriteError("ERR wrong number of arguments for '" + string(cmd.Args[0]) + "' command")
					return
				}
				conn.WriteInt(ps.Publish(string(cmd.Args[1]), string(cmd.Args[2])))
			case "subscribe", "psubscribe":
				if len(cmd.Args) < 2 {
					conn.WriteError("ERR wrong number of arguments for '" + string(cmd.Args[0]) + "' command")
					return
				}
				command := strings.ToLower(string(cmd.Args[0]))
				for i := 1; i < len(cmd.Args); i++ {
					if command == "psubscribe" {
						ps.Psubscribe(conn, string(cmd.Args[i]))
					} else {
						ps.Subscribe(conn, string(cmd.Args[i]))
					}
				}
			}
		},
		func(conn redcon.Conn) bool {
			// Use this function to accept or deny the connection.
			// log.Printf("accept: %s", conn.RemoteAddr())
			return true
		},
		func(conn redcon.Conn, err error) {
			// This is called when the connection has been closed
			// log.Printf("closed: %s, err: %v", conn.RemoteAddr(), err)
		},
	)
	if err != nil {
		log.Fatal(err)
	}
}

TLS Example

Redcon has full TLS support through the ListenAndServeTLS function.

The same example is also provided for serving Redcon over TLS.

go run example/tls/clone.go

Benchmarks

Redis: Single-threaded, no disk persistence.

$ redis-server --port 6379 --appendonly no
redis-benchmark -p 6379 -t set,get -n 10000000 -q -P 512 -c 512
SET: 941265.12 requests per second
GET: 1189909.50 requests per second

Redcon: Single-threaded, no disk persistence.

$ GOMAXPROCS=1 go run example/clone.go
redis-benchmark -p 6380 -t set,get -n 10000000 -q -P 512 -c 512
SET: 2018570.88 requests per second
GET: 2403846.25 requests per second

Redcon: Multi-threaded, no disk persistence.

$ GOMAXPROCS=0 go run example/clone.go
$ redis-benchmark -p 6380 -t set,get -n 10000000 -q -P 512 -c 512
SET: 1944390.38 requests per second
GET: 3993610.25 requests per second

Running on a MacBook Pro 15" 2.8 GHz Intel Core i7 using Go 1.7

Contact

Josh Baker @tidwall

License

Redcon source code is available under the MIT License.

redcon's People

Contributors

chrisvdg avatar iwanbk avatar oleh-ozimok avatar shankai avatar smallnest avatar thinxer avatar tidwall avatar tomarrell avatar xmikus01 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

redcon's Issues

(empty list or set) problem.

I can't seem to generate (empty list or set) with any of the WriteInt WriteString WriteNull or WriteRaw...

how do I return (empty list or set) like normal redis?

127.0.0.1:6380> lrange a 0 1
(empty list or set)

Race condition using redcon v1.6.0

Hello, there is a race condition in the redcon library used by olric v0.5.0. I was testing my code using go test -race and this warning appeared.

Sample

	olricInstance = config.New("local")
	db, err := olric.New(olricInstance)
	if err != nil {
		return nil, err
	}

	go func() {
		_ = db.Start()
	}()

Output

WARNING: DATA RACE
Read at 0x00c0000360a8 by goroutine 19:
  github.com/tidwall/redcon.(*Writer).Flush()
      /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:651 +0x3e
  github.com/tidwall/redcon.(*conn).Close()
      /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:459 +0x47
  github.com/tidwall/redcon.serve.func1.1()
      /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:338 +0x119
  github.com/tidwall/redcon.serve.func1()
      /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:341 +0x56
  runtime.deferreturn()
      /usr/local/go/src/runtime/panic.go:476 +0x32
  github.com/tidwall/redcon.(*Server).Serve()
      /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:311 +0x155
  github.com/buraksezer/olric/internal/server.(*Server).ListenAndServe()
      /Users/sylvain/go/pkg/mod/github.com/buraksezer/[email protected]/internal/server/server.go:189 +0x784
  github.com/buraksezer/olric.(*Olric).Start.func1()
      /Users/sylvain/go/pkg/mod/github.com/buraksezer/[email protected]/olric.go:324 +0x47
  golang.org/x/sync/errgroup.(*Group).Go.func1()
      /Users/sylvain/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75 +0x86

Previous write at 0x00c0000360a8 by goroutine 37:
  github.com/tidwall/redcon.(*Writer).Flush()
      /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:654 +0xbe
  github.com/tidwall/redcon.handle.func2()
      /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:438 +0x237
  github.com/tidwall/redcon.handle()
      /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:442 +0xb8
  github.com/tidwall/redcon.serve.func2()
      /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:378 +0x47

Goroutine 19 (running) created at:
  golang.org/x/sync/errgroup.(*Group).Go()
      /Users/sylvain/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:72 +0x12e
  github.com/buraksezer/olric.(*Olric).Start()
      /Users/sylvain/go/pkg/mod/github.com/buraksezer/[email protected]/olric.go:323 +0x2a4
  command-line-arguments.EmbeddedOlricConnectionFactory.func3()
      /Users/sylvain/go/src/github.com/darkweak/souin/cache/providers/embeddedOlricProvider.go:86 +0x39

Goroutine 37 (running) created at:
  github.com/tidwall/redcon.serve()
      /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:378 +0x89b
  github.com/tidwall/redcon.(*Server).Serve()
      /Users/sylvain/go/pkg/mod/github.com/tidwall/[email protected]/redcon.go:311 +0x155
  github.com/buraksezer/olric/internal/server.(*Server).ListenAndServe()
      /Users/sylvain/go/pkg/mod/github.com/buraksezer/[email protected]/internal/server/server.go:189 +0x784
  github.com/buraksezer/olric.(*Olric).Start.func1()
      /Users/sylvain/go/pkg/mod/github.com/buraksezer/[email protected]/olric.go:324 +0x47
  golang.org/x/sync/errgroup.(*Group).Go.func1()
      /Users/sylvain/go/pkg/mod/golang.org/x/[email protected]/errgroup/errgroup.go:75 +0x86

Questions about the design of conn.WriteAny

Hi, I have a question about the design concern about the conn.WriteAny function.

If I use the conn.WriteInt and pass a int parameter to it, it will write a integer response to the client.

// WriteInt64 writes a 64-bit signed integer to the client.
func (w *Writer) WriteInt64(num int64) {
    if w.err != nil {
         return
    }
    w.b = AppendInt(w.b, num)
}

However, if I use the conn.WriteAny and pass the same parameter to it, the function will write a string format of integer response which are quoted with a "" to the client.

func AppendAny(b []byte, v interface{}) []byte {
    switch v := v.(type) {
    case SimpleString:
         b = AppendString(b, string(v))
    case SimpleInt:
         b = AppendInt(b, int64(v))
    case SimpleError:
         b = AppendError(b, v.Error())
    case nil:
         b = AppendNull(b)
    case error:
         b = AppendError(b, prefixERRIfNeeded(v.Error()))
    case string:
         b = AppendBulkString(b, v)
    case []byte:
         b = AppendBulk(b, v)
    case bool:
         if v {
	    b = AppendBulkString(b, "1")
         } else {
	    b = AppendBulkString(b, "0")
         }
    case int:
         b = AppendBulkInt(b, int64(v))
    case int8:
         b = AppendBulkInt(b, int64(v))
    case int16:
         b = AppendBulkInt(b, int64(v))
    case int32:
         b = AppendBulkInt(b, int64(v))
    case int64:
         b = AppendBulkInt(b, int64(v))
    ......
    }
}

Can you give me any ideas about the design principles here?

Would pub/sub support be welcome?

Would I be welcome to implement pub/sub and send a pull request to this repository?

I am in need of a pub/sub right now and if a pull request is welcome, I will implement and send it.

error handling differ between redcon & redis 3.0.6/3.2.4

The following simple script shows a difference in behavior between redcon and redis 3.0.6/3.2.4:

`#!/bin/bash

TIMESTAMP=$(TZ=UTC date +"%Y-%m-%dT%H:%M:%S.%3NZ")
HOSTNAME=$(hostname -s)

ROLE=myrole
APPLICATION=myapp
MESSAGE="Ping"
REDISKEY="testkey"
REDIS_SVR="localhost 7379"

PAYLOAD='{ "@fields": { "application": "'${APPLICATION}'", "role": "'${ROLE}'" }, "@message": "'${MESSAGE}'", "@source_host": "'${HOSTNAME}'", "@timestamp": "'${TIMESTAMP}'" }'

(
echo "RPUSH ${REDISKEY} '"${PAYLOAD}"'"
sleep 1s
) | telnet $REDIS_SVR`

redcon, in a sense correctly answers:

-ERR Protocol error: unbalanced quotes in request

redis 3.0.6/3.2.4 however has no gripes with the situation:

:1

export the server.Serve method

sometimes I need to pass the net.Listener to the server but the serve is unexported.
Suggest adding server.Serve method and the net.Listener as the parameter.

Latency is high

Throughput of Redcon seems really good but latency is really high in comparison to Redis with shallow or deep pipelines. Even if you make the Redis commands NOOP.

I created a SET command that only wrote a response to benchmark the I/O performance of redcon versus Redis. Keep in mind Redis is doing key mutations while my benchmark is not.

conn.WriteString("OK")

Redis

redis-benchmark -h server -p 6379 -t set -n 10000000 -P 1 -c 768
====== SET ======
  10000000 requests completed in 101.57 seconds
  768 parallel clients
  3 bytes payload
  keep alive: 1

0.00% <= 2 milliseconds
0.00% <= 3 milliseconds
76.97% <= 4 milliseconds
99.98% <= 5 milliseconds
99.99% <= 6 milliseconds
100.00% <= 7 milliseconds
100.00% <= 8 milliseconds
100.00% <= 8 milliseconds
98455.23 requests per second
redis-benchmark -h server -p 6379 -t set -n 10000000 -P 256 -c 768
====== SET ======
  10000000 requests completed in 8.61 seconds
  768 parallel clients
  3 bytes payload
  keep alive: 1

0.00% <= 8 milliseconds
0.00% <= 9 milliseconds
0.03% <= 10 milliseconds
0.06% <= 78 milliseconds
0.06% <= 79 milliseconds
0.10% <= 80 milliseconds
0.13% <= 81 milliseconds
0.14% <= 82 milliseconds
0.16% <= 84 milliseconds
0.17% <= 85 milliseconds
0.18% <= 86 milliseconds
0.19% <= 87 milliseconds
0.22% <= 88 milliseconds
0.27% <= 89 milliseconds
0.32% <= 90 milliseconds
0.35% <= 91 milliseconds
0.36% <= 92 milliseconds
0.36% <= 96 milliseconds
0.38% <= 97 milliseconds
0.39% <= 98 milliseconds
0.41% <= 99 milliseconds
0.43% <= 100 milliseconds
0.48% <= 101 milliseconds
0.55% <= 102 milliseconds
0.62% <= 103 milliseconds
0.68% <= 104 milliseconds
0.75% <= 105 milliseconds
0.83% <= 106 milliseconds
0.91% <= 107 milliseconds
0.98% <= 108 milliseconds
1.07% <= 109 milliseconds
1.16% <= 110 milliseconds
1.23% <= 111 milliseconds
1.29% <= 112 milliseconds
1.35% <= 113 milliseconds
1.42% <= 114 milliseconds
1.47% <= 115 milliseconds
1.54% <= 116 milliseconds
1.60% <= 117 milliseconds
1.66% <= 118 milliseconds
1.73% <= 119 milliseconds
1.80% <= 120 milliseconds
1.86% <= 121 milliseconds
1.89% <= 122 milliseconds
1.91% <= 128 milliseconds
1.94% <= 129 milliseconds
2.05% <= 130 milliseconds
2.24% <= 131 milliseconds
2.57% <= 132 milliseconds
3.32% <= 133 milliseconds
4.32% <= 134 milliseconds
5.45% <= 135 milliseconds
6.56% <= 136 milliseconds
7.74% <= 137 milliseconds
9.23% <= 138 milliseconds
10.92% <= 139 milliseconds
12.76% <= 140 milliseconds
14.70% <= 141 milliseconds
16.73% <= 142 milliseconds
18.77% <= 143 milliseconds
20.73% <= 144 milliseconds
22.69% <= 145 milliseconds
24.62% <= 146 milliseconds
26.42% <= 147 milliseconds
27.84% <= 148 milliseconds
29.06% <= 149 milliseconds
30.14% <= 150 milliseconds
30.95% <= 151 milliseconds
31.64% <= 152 milliseconds
32.47% <= 153 milliseconds
33.75% <= 154 milliseconds
35.64% <= 155 milliseconds
38.12% <= 156 milliseconds
41.20% <= 157 milliseconds
44.65% <= 158 milliseconds
48.27% <= 159 milliseconds
52.01% <= 160 milliseconds
55.82% <= 161 milliseconds
59.59% <= 162 milliseconds
63.34% <= 163 milliseconds
67.20% <= 164 milliseconds
71.02% <= 165 milliseconds
74.71% <= 166 milliseconds
77.64% <= 167 milliseconds
79.56% <= 168 milliseconds
81.13% <= 169 milliseconds
82.53% <= 170 milliseconds
83.86% <= 171 milliseconds
85.15% <= 172 milliseconds
86.44% <= 173 milliseconds
87.65% <= 174 milliseconds
88.88% <= 175 milliseconds
90.05% <= 176 milliseconds
91.20% <= 177 milliseconds
92.25% <= 178 milliseconds
93.18% <= 179 milliseconds
94.07% <= 180 milliseconds
94.89% <= 181 milliseconds
95.64% <= 182 milliseconds
96.34% <= 183 milliseconds
96.90% <= 184 milliseconds
97.33% <= 185 milliseconds
97.63% <= 186 milliseconds
97.90% <= 187 milliseconds
98.12% <= 188 milliseconds
98.36% <= 189 milliseconds
98.58% <= 190 milliseconds
98.79% <= 191 milliseconds
98.98% <= 192 milliseconds
99.16% <= 193 milliseconds
99.34% <= 194 milliseconds
99.48% <= 195 milliseconds
99.60% <= 196 milliseconds
99.71% <= 197 milliseconds
99.84% <= 198 milliseconds
99.91% <= 199 milliseconds
99.96% <= 200 milliseconds
99.99% <= 201 milliseconds
100.00% <= 201 milliseconds
1161170.38 requests per second

Redcon

redis-benchmark -h server -p 6380 -t set -n 10000000 -P 1 -c 768
====== SET ======
  10000000 requests completed in 90.21 seconds
  768 parallel clients
  3 bytes payload
  keep alive: 1

0.01% <= 2 milliseconds
28.35% <= 3 milliseconds
81.28% <= 4 milliseconds
99.97% <= 5 milliseconds
99.98% <= 6 milliseconds
99.98% <= 260 milliseconds
99.98% <= 261 milliseconds
99.98% <= 262 milliseconds
99.98% <= 263 milliseconds
99.98% <= 264 milliseconds
99.98% <= 265 milliseconds
99.98% <= 266 milliseconds
99.98% <= 267 milliseconds
99.98% <= 268 milliseconds
99.98% <= 269 milliseconds
99.98% <= 270 milliseconds
99.99% <= 271 milliseconds
99.99% <= 272 milliseconds
99.99% <= 273 milliseconds
99.99% <= 274 milliseconds
99.99% <= 275 milliseconds
99.99% <= 864 milliseconds
99.99% <= 865 milliseconds
99.99% <= 866 milliseconds
99.99% <= 867 milliseconds
99.99% <= 868 milliseconds
100.00% <= 869 milliseconds
100.00% <= 870 milliseconds
100.00% <= 871 milliseconds
100.00% <= 872 milliseconds
100.00% <= 873 milliseconds
100.00% <= 873 milliseconds
110853.69 requests per second
redis-benchmark -h server -p 6380 -t set -n 10000000 -P 256 -c 768
====== SET ======
  10000000 requests completed in 4.47 seconds
  768 parallel clients
  3 bytes payload
  keep alive: 1

0.00% <= 2 milliseconds
0.00% <= 9 milliseconds
0.02% <= 10 milliseconds
0.06% <= 11 milliseconds
0.17% <= 12 milliseconds
0.29% <= 13 milliseconds
0.44% <= 14 milliseconds
1.33% <= 15 milliseconds
2.41% <= 16 milliseconds
3.83% <= 17 milliseconds
5.83% <= 18 milliseconds
8.42% <= 19 milliseconds
11.30% <= 20 milliseconds
14.10% <= 21 milliseconds
16.94% <= 22 milliseconds
19.60% <= 23 milliseconds
22.31% <= 24 milliseconds
25.01% <= 25 milliseconds
27.65% <= 26 milliseconds
30.40% <= 27 milliseconds
33.02% <= 28 milliseconds
35.74% <= 29 milliseconds
38.77% <= 30 milliseconds
41.45% <= 31 milliseconds
44.18% <= 32 milliseconds
46.89% <= 33 milliseconds
49.36% <= 34 milliseconds
52.06% <= 35 milliseconds
54.80% <= 36 milliseconds
57.48% <= 37 milliseconds
60.42% <= 38 milliseconds
63.25% <= 39 milliseconds
65.84% <= 40 milliseconds
68.45% <= 41 milliseconds
71.05% <= 42 milliseconds
73.70% <= 43 milliseconds
76.34% <= 44 milliseconds
78.99% <= 45 milliseconds
81.75% <= 46 milliseconds
84.61% <= 47 milliseconds
87.32% <= 48 milliseconds
89.78% <= 49 milliseconds
91.89% <= 50 milliseconds
93.38% <= 51 milliseconds
94.47% <= 52 milliseconds
95.22% <= 53 milliseconds
95.40% <= 54 milliseconds
95.55% <= 55 milliseconds
95.70% <= 56 milliseconds
95.81% <= 57 milliseconds
95.87% <= 58 milliseconds
95.91% <= 59 milliseconds
95.94% <= 60 milliseconds
95.99% <= 61 milliseconds
96.03% <= 62 milliseconds
96.05% <= 63 milliseconds
96.08% <= 64 milliseconds
96.09% <= 253 milliseconds
96.09% <= 259 milliseconds
96.12% <= 260 milliseconds
96.13% <= 261 milliseconds
96.14% <= 262 milliseconds
96.17% <= 263 milliseconds
96.20% <= 264 milliseconds
96.23% <= 265 milliseconds
96.29% <= 266 milliseconds
96.35% <= 267 milliseconds
96.40% <= 268 milliseconds
96.47% <= 269 milliseconds
96.52% <= 270 milliseconds
96.60% <= 271 milliseconds
96.68% <= 272 milliseconds
96.76% <= 273 milliseconds
96.81% <= 274 milliseconds
96.87% <= 275 milliseconds
96.92% <= 276 milliseconds
96.96% <= 277 milliseconds
97.02% <= 278 milliseconds
97.08% <= 279 milliseconds
97.12% <= 280 milliseconds
97.16% <= 281 milliseconds
97.21% <= 282 milliseconds
97.27% <= 283 milliseconds
97.32% <= 284 milliseconds
97.38% <= 285 milliseconds
97.43% <= 286 milliseconds
97.49% <= 287 milliseconds
97.53% <= 288 milliseconds
97.58% <= 289 milliseconds
97.64% <= 290 milliseconds
97.68% <= 291 milliseconds
97.72% <= 292 milliseconds
97.75% <= 293 milliseconds
97.78% <= 294 milliseconds
97.80% <= 295 milliseconds
97.82% <= 296 milliseconds
97.84% <= 297 milliseconds
97.85% <= 298 milliseconds
97.87% <= 299 milliseconds
97.88% <= 300 milliseconds
97.89% <= 301 milliseconds
97.92% <= 302 milliseconds
97.93% <= 303 milliseconds
97.94% <= 304 milliseconds
97.95% <= 305 milliseconds
97.96% <= 306 milliseconds
97.96% <= 307 milliseconds
97.97% <= 308 milliseconds
97.98% <= 309 milliseconds
97.98% <= 310 milliseconds
97.99% <= 311 milliseconds
98.00% <= 312 milliseconds
98.00% <= 313 milliseconds
98.00% <= 315 milliseconds
98.01% <= 316 milliseconds
98.01% <= 317 milliseconds
98.02% <= 318 milliseconds
98.02% <= 320 milliseconds
98.02% <= 321 milliseconds
98.03% <= 322 milliseconds
98.03% <= 323 milliseconds
98.03% <= 326 milliseconds
98.04% <= 327 milliseconds
98.04% <= 328 milliseconds
98.04% <= 830 milliseconds
98.04% <= 832 milliseconds
98.05% <= 834 milliseconds
98.05% <= 835 milliseconds
98.06% <= 836 milliseconds
98.06% <= 837 milliseconds
98.08% <= 838 milliseconds
98.10% <= 839 milliseconds
98.10% <= 840 milliseconds
98.11% <= 841 milliseconds
98.12% <= 842 milliseconds
98.13% <= 843 milliseconds
98.14% <= 844 milliseconds
98.15% <= 845 milliseconds
98.16% <= 846 milliseconds
98.17% <= 847 milliseconds
98.19% <= 848 milliseconds
98.21% <= 849 milliseconds
98.24% <= 850 milliseconds
98.26% <= 851 milliseconds
98.28% <= 852 milliseconds
98.31% <= 853 milliseconds
98.33% <= 854 milliseconds
98.35% <= 855 milliseconds
98.39% <= 856 milliseconds
98.44% <= 857 milliseconds
98.47% <= 858 milliseconds
98.50% <= 859 milliseconds
98.53% <= 860 milliseconds
98.55% <= 861 milliseconds
98.58% <= 862 milliseconds
98.61% <= 863 milliseconds
98.64% <= 864 milliseconds
98.68% <= 865 milliseconds
98.71% <= 866 milliseconds
98.74% <= 867 milliseconds
98.77% <= 868 milliseconds
98.80% <= 869 milliseconds
98.85% <= 870 milliseconds
98.89% <= 871 milliseconds
98.92% <= 872 milliseconds
98.96% <= 873 milliseconds
98.97% <= 874 milliseconds
99.00% <= 875 milliseconds
99.03% <= 876 milliseconds
99.05% <= 877 milliseconds
99.07% <= 878 milliseconds
99.08% <= 879 milliseconds
99.09% <= 880 milliseconds
99.12% <= 881 milliseconds
99.15% <= 882 milliseconds
99.18% <= 883 milliseconds
99.22% <= 884 milliseconds
99.26% <= 885 milliseconds
99.27% <= 886 milliseconds
99.29% <= 887 milliseconds
99.30% <= 888 milliseconds
99.32% <= 889 milliseconds
99.33% <= 890 milliseconds
99.35% <= 891 milliseconds
99.36% <= 892 milliseconds
99.38% <= 893 milliseconds
99.40% <= 894 milliseconds
99.41% <= 895 milliseconds
99.42% <= 896 milliseconds
99.43% <= 897 milliseconds
99.45% <= 898 milliseconds
99.48% <= 899 milliseconds
99.51% <= 900 milliseconds
99.54% <= 901 milliseconds
99.58% <= 902 milliseconds
99.61% <= 903 milliseconds
99.64% <= 904 milliseconds
99.67% <= 905 milliseconds
99.71% <= 906 milliseconds
99.75% <= 907 milliseconds
99.78% <= 908 milliseconds
99.82% <= 909 milliseconds
99.85% <= 910 milliseconds
99.86% <= 911 milliseconds
99.86% <= 913 milliseconds
99.87% <= 915 milliseconds
99.87% <= 916 milliseconds
99.88% <= 918 milliseconds
99.88% <= 919 milliseconds
99.89% <= 920 milliseconds
99.90% <= 921 milliseconds
99.91% <= 922 milliseconds
99.91% <= 924 milliseconds
99.92% <= 927 milliseconds
99.92% <= 928 milliseconds
99.92% <= 930 milliseconds
99.93% <= 931 milliseconds
99.93% <= 932 milliseconds
99.93% <= 934 milliseconds
99.93% <= 935 milliseconds
99.94% <= 936 milliseconds
99.95% <= 937 milliseconds
99.95% <= 938 milliseconds
99.97% <= 939 milliseconds
99.97% <= 940 milliseconds
99.98% <= 941 milliseconds
99.98% <= 942 milliseconds
99.99% <= 943 milliseconds
100.00% <= 943 milliseconds
2236136.00 requests per second

graceful shutdown

Hi,

I wrote a redcon server and I need to support graceful shutdown, from what I observed calling "Close" only closes the server connection and don't wait for in-memory requests to flush.

Is there a recommended way to implement graceful shutdown?

how do u return hscan results?

how to return results like these?

127.0.0.1:6379> hset a c c
(integer) 1
127.0.0.1:6379> hscan a 0
1) "0"
2) 1) "b"
   2) "c"
   3) "c"
   4) "c"

Graceful shutdown

Would be nice for the project I'm working on to have a way to gracefully shutdown a redcon server as we run 1 with TLS and one without.

This is not a priority feature so no idea how long this would take.
@tidwall , will open a pull request when done if interested.

C#: StackExchange.Redis is Unable to connect to REDCON

hi @tidwall

After compiled the example, Go: redigo can connect to the example server, but C#: StackExchange.Redis can't and C#: StackExchange.Redis can connect to tile38 server.

In order to connect to redcon with C#: StackExchange.Redis, what should I do?

what's the max concurrent subscriber redcon can take?

Would like to understand how much memory is allocated for each connection.
If you can share some insights to what to look out for for high concurrent connections and how to improve etc, will be great. Have yet to check out the internal mechanics of the code.

redcon.Serve() never returns.

redcon.serve() only returns if (*Server).done is ever set to true, which is only possible if (*Server).Close() is called.

Given that the use of redcon.Serve(ln net.Listener, ...) unlike any of the other methods exposed in the package assumes that the user is not keeping around a *redcon.Server instance, it wouldn't be possible for the user to force redcon.Serve() to ever return, which is behavior that is desired for e.g. graceful shutdown.

Rather than ignoring errors from net.Listener.Accept() in redcon.serve() unless (*Server).done is set to true, what about breaking early from redcon.serve()'s accept loop if an error is returned by net.Listener.Accept() which indicates that the listener is closed?

For example:

lnconn, err := s.ln.Accept()
if err != nil {
    // .....
    if errors.Is(err, net.ErrClosed) {
        return nil
    }
}

Redis CLI requires `COMMAND` command on connect

While this is not technically a limitation of this library, but as I was implementing a server with redcon I noticed that my local redis-cli client v5.0.7 didn't seem to work with the redcon-based server I made. After doing some digging I noticed that the redis-cli issues a COMMAND command after a connection is established and it expects a response to work. Once I responded to that with redcon the official redis-cli worked fine. Not sure if you want to add this as a not in the README if people attempt to use it and it doesn't work as expected right away.

Discovered a bug

<?php

        $qrediswp = new Redis();
        $qrediswp->connect("127.0.0.1", 6379); //dashboard host, port

//hset "e" "432232157184245456" "123456789"

        $lss_disksize = $qrediswp->hget("e","432232157184245456");
        echo '= '.$lss_disksize.' =';

// it shows :123456789 instead of 123456789.
why is there a ":" in front of the value? it doesnt show this using the real redis server.

        exit;

Tried with this code...

//shows up using this:

                case "hget":

                        if bytes.Compare(cmd.Args[1],[]byte("e"))==0 { //disk space
                                        conn.WriteUint64(uint64(123456789))

                       }

Request for comment on this line

Thank you so much for your work on this tool.

Could you please add a comment to this line?

const maxBufferCap = 262144

Is it to ensure buffer cap is bounded? Curious to understand if you have seen an issue in production that triggered this change.

Thanks!

How to direct transfer of returned {}interface type from redis server.

[A. redis cli] <-> [B. custom redis client redcon] <-> [C. custom redis server redcon] <-> [D. redis server]

When C. received reply interface, i would like this "reply" to be forwarded directly to B. instead of doing an expensive interface{} to []byte conversion then only unserialize []byte to interface{} back on B. How do i get it done?

i realised i'm trying to transfer redigo interface type to another connecting redigo instance. how to transfer interface type directly?

the custom made redis server and custom made redis client (both written in golang) is a bridge to the redis server and redis client. B. does sanitization of data from A. redis-cli and forward to C. which validates the results before returning the data "reply" back without any processing from D to C and then to B but is post processed at B then back to A. The C to B part would like the "reply" to be "unprocessed". How to get it done?

zrange possible? :)

it's great software. will be better if you have a roadmap to mention when you think the possibility is to open up all redis compatibility issues.

been using it for awhile now. it's very good.

Provide Flush() function from the Conn interface

We need a Flush() function on the Conn interface so that the server can flush the response when multiple line of response accumulated. For example

func (r *RedisHandler) Ping(conn redcon.Conn, cmd redcon.Command) {
	conn.WriteString("PONG1")
	conn.WriteString("PONG2")
	conn.WriteString("PONG3")
}

When client sends ping command, only PONG1 returned. Then if client sends something like get something, PONG2 returned.

We need ability to flush the underlying buffer from server side, e.g

func (r *RedisHandler) Ping(conn redcon.Conn, cmd redcon.Command) {
	conn.WriteString("PONG1")
	conn.WriteString("PONG2")
	conn.WriteString("PONG3")
	conn.Flush() // <- we need this here
}

The underlying conn.wr.Flush() has been implemented. We just need to expose it on the Conn interface.

This feature is required when implementing the transaction feature like MULTI/EXEC, which will queue commands and execute at one submit.

Thank you!

TLS support

@tidwall , I have added TLS support on a fork because I needed it for a project I'm working on, is there any interest for that feature to be merged into this repo?

Support for AUTH?

Hello, first of all thank you for creating these suit of project.

I am considering using SummitDB, I could see that redcon already support TLS, but I couldn't grep "AUTH" command. How does redcon handle authentication?

race when close redcon

redcon version: 1.4.4

POC

package main

import (
	"context"
	"log"
	"strings"

	"github.com/go-redis/redis/v9"
	"github.com/tidwall/redcon"
)

func main() {
	redcon := redcon.NewServerNetwork("tcp", "127.0.0.1:12345",
		func(conn redcon.Conn, cmd redcon.Command) {
			switch strings.ToLower(string(cmd.Args[0])) {
			case "set":
				log.Println(string(cmd.Args[1]), string(cmd.Args[2]))
				conn.WriteString("OK")
			default:
				log.Println("cmd not implemented: ", string(cmd.Args[0]))
				conn.WriteError("ERR unknown command '" + string(cmd.Args[0]) + "'")
			}

		}, nil, nil,
	)
	ch := make(chan error)
	go func() {
		redcon.ListenServeAndSignal(ch)
	}()
	if err := <-ch; err != nil {
		log.Fatal(err)
	}

	// create redis client, test set command
	client := redis.NewClient(&redis.Options{
		Addr:    redcon.Addr().String(),
		Network: redcon.Addr().Network(),
	})
	r := client.Set(context.Background(), "tmp", "test", 0)
	if r.Err() != nil {
		log.Fatal(r.Err())
	}
	if err := client.Close(); err != nil {
		log.Fatal(err)
	}

	// close redcon (race)
	if err := redcon.Close(); err != nil {
		log.Fatal(err)
	}
}

// go run -race poc.go

The race occurs when closing redcon while the handle function is not finished (invoking c.wr.Flush, but without lock).

image

memory leaking when connection is reset with go 1.11.x

I built a service using redcon and deployed it in production environment.

One day, for some reason (maybe unstable internal networking), massive connection reset occurred, I observed that reset connections are not released and memory keep leaking.

I recompiled my binary with go 1.10.x and 1.9.x, everything is ok, reset connections are released correctly and no memory leaks.

Problem only occur when compiled with go 1.11.x.

I'd investigate this issue and report any progress later.

How to return "[]interface {}" type result

hi,i use redcon build a proxy,an use go-redis to connect server."keys *" command go-redis give me "[]interface {}" type result,and i want to know ,wich conn.WriteXXX method can return this to client,an how todo .
thx

Support for context.Context

It would be great if there was support for context.Context. So that I could capture os.Signals and kill the server in a nice way.

Type safety

Sometimes we need to guarantee the arguments are of some type (for example GET requires name to be string but what if user put in number?) so how do we check it?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.