Giter VIP home page Giter VIP logo

go-tarantool's Introduction

Go Reference Actions Status Code Coverage Telegram GitHub Discussions Stack Overflow

Client in Go for Tarantool

The package go-tarantool contains everything you need to connect to Tarantool 1.10+.

The advantage of integrating Go with Tarantool, which is an application server plus a DBMS, is that Go programmers can handle databases and perform on-the-fly recompilations of embedded Lua routines, just as in C, with responses that are faster than other packages according to public benchmarks.

Table of contents

Installation

We assume that you have Tarantool version 1.10+ and a modern Linux or BSD operating system.

You need a current version of go, version 1.20 or later (use go version to check the version number). Do not use gccgo-go.

Note: If your go version is older than 1.20 or if go is not installed, download and run the latest tarball from golang.org.

The package go-tarantool is located in tarantool/go-tarantool repository. To download and install, say:

$ go get github.com/tarantool/go-tarantool/v2

This should put the source and binary files in subdirectories of /usr/local/go, so that you can access them by adding github.com/tarantool/go-tarantool to the import {...} section at the start of any Go program.

Build tags

We define multiple build tags.

This allows us to introduce new features without losing backward compatibility.

  1. To run fuzz tests with decimals, you can use the build tag:
    go_tarantool_decimal_fuzzing
    
    Note: It crashes old Tarantool versions.

Documentation

Read the Tarantool documentation to find descriptions of terms such as "connect", "space", "index", and the requests to create and manipulate database objects or Lua functions.

In general, connector methods can be divided into two main parts:

  • Connect() function and functions related to connecting, and
  • Data manipulation functions and Lua invocations such as Insert() or Call().

The supported requests have parameters and results equivalent to requests in the Tarantool CRUD operations. There are also Typed and Async versions of each data-manipulation function.

API Reference

Learn API documentation and examples at pkg.go.dev.

Walking-through example

We can now have a closer look at the example and make some observations about what it does.

package tarantool

import (
	"context"
	"fmt"
	"time"

	"github.com/tarantool/go-tarantool/v2"
	_ "github.com/tarantool/go-tarantool/v2/datetime"
	_ "github.com/tarantool/go-tarantool/v2/decimal"
	_ "github.com/tarantool/go-tarantool/v2/uuid"
)

func main() {
	ctx, cancel := context.WithTimeout(context.Background(), time.Second)
	defer cancel()
	dialer := tarantool.NetDialer{
		Address: "127.0.0.1:3301",
		User: 	 "guest",
	}
	opts := tarantool.Opts{
		Timeout: time.Second,
	}

	conn, err := tarantool.Connect(ctx, dialer, opts)
	if err != nil {
		fmt.Println("Connection refused:", err)
		return
	}

	data, err := conn.Do(
		tarantool.NewInsertRequest(999).Tuple([]interface{}{99999, "BB"})).Get()
	if err != nil {
		fmt.Println("Error:", err)
	} else {
		fmt.Println("Data:", data)
	}
}

Observation 1: The line "github.com/tarantool/go-tarantool/v2" in the import(...) section brings in all Tarantool-related functions and structures.

Observation 2: Unused import lines are required to initialize encoders and decoders for external msgpack types.

Observation 3: The line starting with "ctx, cancel :=" creates a context object for Connect(). The Connect() call will return an error when a timeout expires before the connection is established.

Observation 4: The line starting with "dialer :=" creates dialer for Connect(). This structure contains fields required to establish a connection.

Observation 5: The line starting with "opts :=" sets up the options for Connect(). In this example, the structure contains only a single value, the timeout. The structure may also contain other settings, see more in documentation for the "Opts" structure.

Observation 6: The line containing "tarantool.Connect" is essential for starting a session. There are three parameters:

  • a context,
  • the dialer that was set up earlier,
  • the option structure that was set up earlier.

There will be only one attempt to connect. If multiple attempts needed, "tarantool.Connect" could be placed inside the loop with some timeout between each try. Example could be found in the example_test, name - ExampleConnect_reconnects.

Observation 7: The err structure will be nil if there is no error, otherwise it will have a description which can be retrieved with err.Error().

Observation 8: The Insert request, like almost all requests, is preceded by the method Do of object conn which is the object that was returned by Connect().

Example with encrypting traffic

For SSL-enabled connections, use OpenSSLDialer from the go-tlsdialer package.

Here is small example with importing the go-tlsdialer library and using the OpenSSLDialer:

package tarantool

import (
	"context"
	"fmt"
	"time"

	"github.com/tarantool/go-tarantool/v2"
	_ "github.com/tarantool/go-tarantool/v2/datetime"
	_ "github.com/tarantool/go-tarantool/v2/decimal"
	_ "github.com/tarantool/go-tarantool/v2/uuid"
	"github.com/tarantool/go-tlsdialer"
)

func main() {
	ctx, cancel := context.WithTimeout(context.Background(), time.Second)
	defer cancel()
	dialer := tlsdialer.OpenSSLDialer{
		Address:     "127.0.0.1:3013", 
		User:        "test", 
		Password:    "test", 
		SslKeyFile:  "testdata/localhost.key",
		SslCertFile: "testdata/localhost.crt",
		SslCaFile:   "testdata/ca.crt",
	}
	opts := tarantool.Opts{
		Timeout: time.Second,
	}

	conn, err := tarantool.Connect(ctx, dialer, opts)
	if err != nil {
		fmt.Println("Connection refused:", err)
		return
	}

	data, err := conn.Do(
		tarantool.NewInsertRequest(999).Tuple([]interface{}{99999, "BB"})).Get()
	if err != nil {
		fmt.Println("Error:", err)
	} else {
		fmt.Println("Data:", data)
	}
}

Note that traffic encryption is only available in Tarantool Enterprise Edition 2.10 or newer.

Migration guide

You can review the changes between major versions in the migration guide.

Contributing

See the contributing guide for detailed instructions on how to get started with our project.

Alternative connectors

There are two other connectors available from the open source community:

See feature comparison in the documentation.

go-tarantool's People

Contributors

0x501d avatar alexopryshko avatar ananek avatar askalt avatar better0fdead avatar derekbum avatar differentialorange avatar drewdzzz avatar fl00r avatar funny-falcon avatar grafin avatar grechkin-pogrebnyakov avatar iskandarov-egor avatar kmansoft avatar lenkis avatar ligurio avatar locker avatar mialinx avatar monoflash avatar msiomkin avatar nshy avatar oleg-jukovec avatar oleggator avatar pavemaksim avatar sosiska avatar temoon avatar totktonada avatar vr009 avatar ylobankov avatar ziontab avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

go-tarantool's Issues

v2: soft automatic schema reload

Now that the iproto protocol has sc_schema_id, fix the driver to perform "soft" schema reload.

  • load the current schema automatically when connection is established
  • include sc_schema_id in queries
  • if the response is ER_SCHEMA_CHANGED, reload the schema and re-issue the query.

tarantool/tarantool#1183

Remove log usage from connection.go

Could you remove calls of log.Printf from connection.go?
This is only the file with logging API calls, and it throws messages directly to stdout by default. In some cases, when logging changed to non-standard library, this messages couldn't be interrupted.

InsertTyped (Update, Replace...)

How can i insert typed slice?
like

var tpl2 []tuple
err = conn.SelectTyped(spaceNo, indexNo, 0, 1, IterEq, []interface{}{uint(30)}, &tpl2)

Delete/Update doesn't works on new versions and 1.7.4 server

Hello.
Environment

▶ git log |head -n1
commit 1cf10c4be9c761cfcdc7469a2e7beb7ffff89cd0

▶ docker exec -it tt tarantool --version
Tarantool 1.7.4-0-g927bd9c24
Target: Linux-x86_64-RelWithDebInfo
Build options: cmake . -DCMAKE_INSTALL_PREFIX=/usr/local -DENABLE_BACKTRACE=ON
Compiler: /usr/bin/cc /usr/bin/c++
C_FLAGS: -fexceptions -funwind-tables -fno-omit-frame-pointer -fno-stack-protector -fno-common -fopenmp -msse2 -std=c11 -Wall -Wextra -Wno-strict-aliasing -Wno-char-subscripts -Wno-format-truncation -fno-gnu89-inline
CXX_FLAGS: -fexceptions -funwind-tables -fno-omit-frame-pointer -fno-stack-protector -fno-common -fopenmp -msse2 -std=c++11 -Wall -Wextra -Wno-strict-aliasing -Wno-char-subscripts -Wno-format-truncation -Wno-invalid-offsetof

▶ docker exec -it tt console
connected to unix/:/var/run/tarantool/tarantool.sock
unix/:/var/run/tarantool/tarantool.sock> box.schema.create_space('tmp')
---
- index: []
  on_replace: 'function: 0x41d65200'
  temporary: false
  id: 519
  engine: memtx
  enabled: false
  name: tmp
  field_count: 0
- created
...

unix/:/var/run/tarantool/tarantool.sock> box.space.tmp:create_index('primary')
---
- unique: true
  parts:
  - type: unsigned
    fieldno: 1
  id: 0
  space_id: 519
  name: primary
  type: TREE
...

After it I trying to run simple go-programm

package main

import (
	"log"

	tarantool "github.com/tarantool/go-tarantool"
)

const space = "tmp"
const index = "primary"

func main() {
	conn, err := tarantool.Connect("127.0.0.1:3301", tarantool.Opts{})
	if err != nil {
		log.Fatal("Init ", err)
	}

	_, err = conn.Insert(space, []interface{}{0, "b"})
	if err != nil {
		log.Fatal("Insert ", err)
	}
	_, err = conn.Delete(space, index, 0)
	if err != nil {
		log.Fatal("Delete ", err)
	}
}

And error is

2017/06/10 21:37:11 Delete Invalid MsgPack - packet body (0x14)
exit status 1

In server logs appears line

2017-06-10 18:37:11.027 [1] iproto xrow.cc:206 E> ER_INVALID_MSGPACK: Invalid MsgPack - packet body

Please have a look on this.

Non-obvious behaviour: bug or feaеure?

connection.go: 193
In case of conn.dial() return error and
if conn.opts.Reconnect > 0 but conn.opts.MaxReconnects == 0
we'll stuck in reconnection loop anyway.

Look, when I set MaxReconnects to 0, I want to say: "I don't want to reconnect, go-tarantool",
but go-tarantool start trying to reconnect.

How to use eval with custom args

For example, I have this lua script for determining name of box.space by id:

tnt.Eval("box.space." + spaceIdStr + ".name", []interface{}{})

It seems to be ugly and error prone way. So how can I pass some data to lua and how to use this data as lua varable(s)?

Bug with truncate and tnt 1.7

Here is a bug with Ubuntu 14.04, Tarantool version 1.7.3 (not Tarantool 1.6), Go version go1.7.5 linux/amd64

  1. Begin with a clean empty database. Start the server and execute this script, which is the same as what's in the Connectors chapter:
box.cfg{listen=3301}
box.schema.space.create('examples',{id=999})
box.space.examples:create_index('primary', {type = 'hash', parts = {1, 
'unsigned'}})
box.schema.user.grant('guest','read,write','space','examples')
box.schema.user.grant('guest','read','space','_space')
  1. Create this go program, named example.go:
package main

import (
     "fmt"
     "github.com/tarantool/go-tarantool"
)

func main() {
   opts := tarantool.Opts{User: "guest"}
   conn, err :=
   tarantool.Connect("127.0.0.1:3301", opts)
   if err != nil {
       fmt.Println("Connection refused: %s", err.Error())
   }
   resp, err := conn.Insert(999, []interface{}{99999, "BB"})
   if err != nil {
     fmt.Println("Error", err)
     fmt.Println("Code", resp.Code)
   }
}
  1. Build and run the program:
go build example.go
./example

... Notice that nothing goes wrong, there is no error.

  1. Run the program again:
./example

... Notice that there is an error, "Error Duplicate key exists in unique index", which is correct. So far everything is normal and works as expected.

  1. Back on the server, say this:
box.space.examples:truncate()
  1. Run the program again:
./example

... Notice the message. On my machine it looks like this:

pgulutzan@pgulutzan-TECRA-M10:~/tarantool-go$ ./example
panic: interface conversion: interface is nil, not bool

goroutine 1 [running]:

panic(0x520480, 0xc420066340)
  /usr/local/go/src/runtime/panic.go:500 +0x1a1
github.com/tarantool/go-tarantool.(*Connection).loadSchema(0xc42006f380, 
0x556040, 0xc42006f380)
  /usr/local/go/go-tarantool/src/github.com/tarantool/go-tarantool/schema.go:136 +0xd19
github.com/tarantool/go-tarantool.Connect(0x546ce3, 0xe, 0x0, 0x0, 0x0, 
0x5456d0, 0x5, 0x0, 0x0, 0x0, ...)
  /usr/local/go/go-tarantool/src/github.com/tarantool/go-tarantool/connection.go:207 +0x65a
main.main()
  /home/pgulutzan/tarantool-go/example.go:11 +0xbb

This does not happen if I say "delete" instead of "truncate".

The only way to fix the mess is to destroy the database and start over.

I tried to produce the same problem with Tarantool 1.6 thus:

  • Change the user and password to: 'peter', password 'peter'
  • Change the data type to: 'num'

Result: no error.

Support IPROTO_PUSH

box.session.push() in 1.10 impements a new iproto command, IPROTO_PUSH. Please add support for it.

How can i create a space by go-tarantool?

I want to initialize "spaces" in tarantool database storage from my application before it runs, but i can't find any method for creation. Sorry, but i'm really not found it both in documentation (readme.md) and in source code. How can i do this without some external scripts? Thanks.

Store a rawhide JSON (or binary?)

Hello!

Can I store in Tarantool a rawhide JSON data or a data in binary format?
I want to store in a box a 'history' of remote-application replies in JSON-format for future re-use (e.g. in cases when an 3rd application require a full reply: key, values and JSON's structure)
If I try to convert an incoming data to string (with 'Sprintf') and try to put them in tuple as 'string' result of an operation has been failed (stored only a part of string). Should I prepare a data with some special tricks or can I store a data in a binary? I know than storing of an rawhide' JSON is not a better idea and I know about hazards & adventures in a future :)

Thank you.

Unsafe type assertion

resp := v[0].(Response)

Hi, thanks for the great repo, but I've got a panic using the recent version of go-tarantool client. Here is the stack trace:

collector1 panic: interface conversion: interface {} is *tarantool.Response, not tarantool.Response
collector1
collector1 goroutine 169127 [running]:
collector1 github.com/tarantool/go-tarantool.defaultLogger.Report(0x7, 0xc4208e8100, 0xc420054f50, 0x1, 0x1)
collector1 /home/sites/go/src/github.com/tarantool/go-tarantool/connection.go:74 +0x6ad
collector1 github.com/tarantool/go-tarantool.(*defaultLogger).Report(0xafa830, 0x7, 0xc4208e8100, 0xc420054f50, 0x1, 0x1)
collector1 :1 +0x66
collector1 github.com/tarantool/go-tarantool.(*Connection).reader(0xc4208e8100, 0xc4201ca780, 0xab2b60, 0xc420148028)
collector1 /home/sites/go/src/github.com/tarantool/go-tarantool/connection.go:638 +0x21c
collector1 created by github.com/tarantool/go-tarantool.(*Connection).dial
collector1 /home/sites/go/src/github.com/tarantool/go-tarantool/connection.go:411 +0x641

Sadly, I don't know how to reproduce it, but it is obvious that the client panics because of unsafe type assertion when handling error "connection %s got unexpected resultId". The resp value being passed to conn.opts.Logger.Report() method call is of *Response type whilst in this method there is an attempt to extract Response from it. Please see:

https://github.com/tarantool/go-tarantool/blob/master/connection.go#L638
https://github.com/tarantool/go-tarantool/blob/master/connection.go#L74

I'm using tarantool 1.7.5, golang 1.9 and CentOS 6.9.

Thanks.

Expose Future.wait() and Future.err as part of public API

Hi guys!
We try implement replicaset using this connector.
Replicaset in our understanding is thing that collect multiple connections and can do retry query when one of connection timed out or unavailable.

There is a problem. We make *Async() queries in loop over connection, after we can't understand whether to retry because for this we need to do two things:

  1. Wait for Future will filled.
  2. Check that Future.err equals nil. If it is not, try again on next connection.
for i, conn := range connections {
	futCh := make(chan *tarantool.Future, 1)
	errCh := make(chan error, 1)
	ctx, cancel := context.WithTimeout(context.Background(), queryTimeout)

	go func() {
		fut := query(conn)
		fut.wait()
		if fut.err != nil {
			errCh <- fut.err
			return
		}
		
                futCh <- fut
	}()

	select {
	case <-ctx.Done():
		cancel()
		continue
	case fut := <-futCh:
		cancel()
		return fut, nil
	case err := <-errCh:
		cancel()
		return nil, err
		}
	}
}

Can't insert digits >= 128

Test case:

package main

import (
    "fmt"
    "github.com/tarantool/go-tarantool"
)

func main() {
    spaceNo := uint32(512)
    indexNo := uint32(0)

    client, err := tarantool.Connect("192.168.99.100:3301", tarantool.Opts{
        User: "test",
        Pass: "test",
    })
    if err != nil {
        panic(err)
    }

    cases := map[string]interface{}{
        "uint64": uint64(128),
        "uint32": uint32(128),
        "uint": uint(128),
        "int64": int64(128),
        "int32": int32(128),
        "int": int(128),
    }
    for caseName, val := range cases {
        v := []interface{}{val}
        _, err = client.Insert(spaceNo, v)
        if err != nil {
            fmt.Printf("%s: %s\n", caseName, err.Error())
        } else {
            fmt.Printf("%s: Success\n", caseName)
        }
        client.Delete(spaceNo, indexNo, v)
    }
}

Output:

uint64: Success
uint32: Success
uint: Success
int64: Tuple field 1 type does not match one required by operation: expected NUM (0x8017)
int32: Tuple field 1 type does not match one required by operation: expected NUM (0x8017)
int: Tuple field 1 type does not match one required by operation: expected NUM (0x8017)```

It's msgpack behaviour, it pack small signed int to smallest possible int and tarantool recieve negative value

It's not a problem to use unsigned ints, just all tests in tarantool_test.go using signed

Reconnect options, working?

Not sure if this is a bug or my misunderstanding...

I'm running a test app with these options, artificial values to make the point:

    opts := tarantool.Opts{
        Timeout:       500 * time.Millisecond,
        Reconnect:     100 * time.Second,
        MaxReconnects: 300000,
    }

If I shutdown or SIGKILL the tarantool side of things, and quickly restart (within maybe 1 second, the time it takes me to "arrow up" + "enter") -- then I expect the client to reconnect and keep going.

Instead, my client gets an error from the Tarantool bindings, like this:

        _, err := client.Call("push_ChangeSub", []interface{}{sub_id, dev_id, now})
        if err != nil {
            log.Fatalf("Error calling function: %s", err.Error())
        }

Actual message, happens immediately after I shutdown / kill tarantool:

Error calling function: EOF

Am I misunderstanding what the reconnect options do?

Is there good way to add basic fault tolerance / reconnect?

Should I be calling client.Ping() before every request (seems expensive)?

Reconnect stuck if custom logger with conn.RemoteAddr() is used

When custom logger with conn.RemoteAddr() was used, reconnect process stuck forever. Cause of this - line https://sourcegraph.com/github.com/tarantool/go-tarantool/-/blob/connection.go#L321

Example of the custom logger:

//Report errors from tarantool package
func (t *ttLogger) Report(event tarantool.ConnLogKind, conn *tarantool.Connection, v ...interface{}) {
	switch event {
	case tarantool.LogReconnectFailed:
		reconnects := v[0].(uint)
		err := v[1].(error)
		t.l.WithField("system", "tarantool").
			WithError(err).Errorf("Reconnect (%d) to %s failed\n", reconnects, conn.RemoteAddr())
	case tarantool.LogLastReconnectFailed:
		err := v[0].(error)
		t.l.WithField("system", "tarantool").
			WithError(err).Errorf("Last reconnect to %s failed\n", conn.RemoteAddr())
	case tarantool.LogUnexpectedResultId:
		resp := v[0].(*tarantool.Response)
		t.l.WithField("system", "tarantool").
			Errorf("Connection %s got unexpected resultId (%d) in response\n", conn.RemoteAddr(), resp.RequestId)
	default:
		args := append([]interface{}{event, conn}, v...)
		t.l.WithField("system", "tarantool").
			Errorf("Unexpected event from %s :  (%v)", conn.RemoteAddr(), args)
	}
}

Typed calls without registration

Types registration is somewhat questionable approach, definitely not goish one. Could we have
CallXXX
method with signature like

func (tc *Connection) CallXXX(method string, []interface{}, sucker Decoder) error

instead?
Where

type Decoder interface {
    Decode([]byte) error
}

?

Get a list of 'connectable' instances from the first node

Use-case:
there is a sharded cluster of ~50 tarantools, 12 vshard-routers. User wants to be able to reconnect to another noe smoothly (if the first one goes down). It is not suitable to pass all 50 nodes to connectmesh, moreover - adding a new router requires all client applications restart.

Proposed solution:

connector should receive a list of connectable nodes from the first one it connects to. Exact procedure on tarantool side must be configurable.
this list should replace the one passed initially.
this list should be refreshed periodically in the background

Error decoding float64 when it was received from tarantool as int64

Ниже пример процедуры, тест
Какие есть варианты решить эту проблемы?
Мне пока видится форкнуть и пропатчить msgpack для этого кейса ибо в цепоче тарантул явно меняет тип данных на int

function Procedure(obj) 
    return obj
end
package tarantool

import (
	"intfs"
	"log"
	"reflect"
	"testing"

	msgpack "gopkg.in/vmihailenco/msgpack.v2"

	tarantool "github.com/tarantool/go-tarantool"
)

func TestPoint(t *testing.T) {
	type fields struct {
		c *tarantool.Connection
	}
	type args struct {
		f1 float64
	}

	tests := []struct {
		name    string
		fields  fields
		args    args
		wantErr bool
	}{
		{
			"10",
			fields{
				c,
			},
			args{
				10,
			},
			false,
		},
		{
			"10.0",
			fields{
				c,
			},
			args{
				10.0,
			},
			false,
		},
		{
			"10.1",
			fields{
				c,
			},
			args{
				10.1,
			},
			false,
		},
	}

	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			res := []P{}
			err := tt.fields.c.Call17Typed(
				"Procedure",
				[]interface{}{
					P{tt.args.f1},
				},
				&res,
			)

			if (err != nil || len(res) == 0) != tt.wantErr {
				t.Errorf("error f1=%f, err=%s, len=%d", tt.args.f1, err, len(res))
			}
		})
	}
}

type P struct {
	X float64
}

func init() {
	msgpack.Register(reflect.TypeOf(P{}), encodeP, decodeP)
}

func encodeP(e *msgpack.Encoder, v reflect.Value) error {
	m := v.Interface().(P)

	if err := e.EncodeSliceLen(1); err != nil {
		return err
	}

	// 1
	if err := e.EncodeFloat64(m.X); err != nil {
		return err
	}

	return nil
}

func decodeP(d *msgpack.Decoder, v reflect.Value) error {
	var err error
	var l int

	m := v.Addr().Interface().(*P)

	if l, err = d.DecodeSliceLen(); err != nil {
		return err
	}

	if l != 1 {
		log.Printf("got %d fields", l)
		return intfs.ErrMsgpackNotExpNumFields
	}

	// 1
	m.X, err = d.DecodeFloat64()
	if err != nil {
		return err
	}

	return nil
}

В логе Encoder\Decoder: code %d это информация о том какого типа значение https://github.com/vmihailenco/msgpack/blob/v2/decode_number.go#L161 https://github.com/vmihailenco/msgpack/blob/v2/encode_number.go#L82

test_1       | GOPATH=/go/src/app go test -v \
test_1       | 	-run=TestPoint \
test_1       | 	./src/tarantool/...
test_1       | === RUN   TestPoint
test_1       | === RUN   TestPoint/10
test_1       | 2017/04/12 07:25:10 Encoder: code 203
test_1       | 2017/04/12 07:25:10 Decoder: code 10
test_1       | === RUN   TestPoint/10.0
test_1       | 2017/04/12 07:25:10 Encoder: code 203
test_1       | 2017/04/12 07:25:10 Decoder: code 10
test_1       | === RUN   TestPoint/10.1
test_1       | 2017/04/12 07:25:10 Encoder: code 203
test_1       | 2017/04/12 07:25:10 Decoder: code 203
test_1       | --- FAIL: TestPoint (0.00s)
test_1       |     --- FAIL: TestPoint/10 (0.00s)
test_1       |     	point_test.go:72: error f1=10.000000, err=msgpack: invalid code a decoding float64, len=1
test_1       |     --- FAIL: TestPoint/10.0 (0.00s)
test_1       |     	point_test.go:72: error f1=10.000000, err=msgpack: invalid code a decoding float64, len=1
test_1       |     --- PASS: TestPoint/10.1 (0.00s)
test_1       | FAIL
test_1       | exit status 1
test_1       | FAIL	tarantool	0.012s
test_1       | Makefile:6: recipe for target 'test' failed
test_1       | make: *** [test] Error 1
backend_test_1 exited with code 2

Parsing Call function of client

I'm having issues parsing the result of the Call function of this client.

In the Tarantool console i'm getting this response from a simple select:

unix/:/var/run/tarantool/tarantool.sock> select * from info
---
- metadata:
  - name: CONSENSUSCHANGEID
    type: string
  - name: HEIGHT
    type: integer
  rows:
  - ['initial', 0]

While in the client after creating a function with Lua, so I can later reuse this logic.
I'm getting this as response from the client.Call function.

res.Data
<[]interface {}> (length: 1, cap: 1)
[0]:<interface {}>
data:<[]interface {}> (length: 1, cap: 1)
[0]:<interface {}>
data:<map[interface {}]interface {}> (length: 2)
<interface {}>:<interface {}>
data:<[]interface {}> (length: 2, cap: 2)
[0]:<interface {}>
data:<map[interface {}]interface {}> (length: 2)
<interface {}>:<interface {}>
data:"CONSENSUSCHANGEID"
[1]:<interface {}>
data:<map[interface {}]interface {}> (length: 2)
<interface {}>:<interface {}>
data:"HEIGHT"

It's giving me an array of interfaces of array of interfaces and so on.. to eventually find only the metadata column names in there. Row data is not present.

After printing them with fmt.Print its giving me following correct output:
[[map[metadata:[map[name:CONSENSUSCHANGEID type:string] map[name:HEIGHT type:integer]] rows:[[initial 0]]]]]

So this means the data of the rows is somewhere in there and magically parsed by fmt.Print. Can someone please elaborate how one could parse this into a useable data form?

Question about "unexpected requestId in response"

Hi, guys.
Sometimes I've got a message in logs "tarantool: unexpected requestId (xxx) in response". I have a question: how correctly work with this exception? Should I resend query or reconnect to database?
Do you know a reasons of this issue?

Returned error for request without permissions

When I make a request to the space for which I do not have permission, an error is returned there is no space with name _, however i would like to get something likepermission denied

Сообщение об ошибке не соотвествует ошибке.

Я запустил тарантул, задав box.cfg{listen = "127.0.0.1:3301"} . А в клиенте в Opts задал по ошибке юзера и пароль. Клиент естесвенно не коннектится и выдаёт неверное описание ошибки: client connection is not ready (0x4000) . По нему и по исходникам очень неочевидно, что дело именно в том, что логин и пароль причина ошибки.

Field offset has type uint32

Почему в Select() поле offet uint32 ?

func (conn *Connection) Select(space, index interface{}, offset, limit, iterator uint32, key interface{}) (resp *Response, err error) {
	return conn.SelectAsync(space, index, offset, limit, iterator, key).Get()
}

Хотя оно потом сериализируется в uint64:

func (req *Future) fillIterator(enc *msgpack.Encoder, offset, limit, iterator uint32) {
	enc.EncodeUint64(KeyIterator)
	enc.EncodeUint64(uint64(iterator))
	enc.EncodeUint64(KeyOffset)
	enc.EncodeUint64(uint64(offset))
	enc.EncodeUint64(KeyLimit)
	enc.EncodeUint64(uint64(limit))
}

Можете прояснить.

Complicated request

Hi folks!
I have a question about how to use a select by 4 fields. Two values simple equals and third value between time(uint32) range :
field1val = Xval and field2val = Yval and field3val < Zval and Zval < field4val

Or I should use SQL?
Do this library support SQL or how I can connect to Tarantool using plain SQL? SQL driver from MySQL for example? Any SQL integration examples?

Insert a filled structure by prototype.

Would it be too much trouble for you have a share an example of insert (and upsert) of a filled struct instead a separate interfaces? My procedure of data insertion (upsert) have a look like:

r, err := cnf.Tptr.Upsert("hosts", []interface{}{vhostname,cpuq}, []interface{}{[]interface{}{"=", 1, cpuq})

Can I replace that by a more abstract, where 'StructOne' is a prototype of a tuple and 'StructTwo' is a filled structure? E.g. (it isn't correct and used for mine idea):

r, err := cnf.Tptr.Upsert("hosts", []interface{}{StructOne}, []interface{}{[]interface{}{"=", *StructOne})

Error: Invalid MsgPack - packet body (0x14)

Когда делаешь запрос
Если в базе есть какие данные все нормально,
коннектор отдает данные правильно, без ошибок,
но если данных нет (nil) то получаем

- Error: Invalid MsgPack - packet body (0x14)

С чем это может быть связано?
И CallTyped и CallTyped17 так отвечают

res := &[]model.Event{}
err := a.DB.Conn.CallTyped("get", id, &res)

Как добится чтобы ошибки не было а то она немного путает, должен быть nil везде

Eval script

How I can eval script?
resp, err = client.Eval("box.space.acc:count()", []interface{}{1, []interface{}{"iterator='ALL'"}})
Always return 0 without err.

V2 proposals

Please, write your wishes for v2 version of this library.
Tend to write one wish per comment, so others may vote for them separately.

Support field names in the driver

Add field aliases support to the driver. If the schema is "formatted" with space:format{}, download field names and allow alias-base access to tuple fields.

Reconnection strategy - round-robin between instances

Use-case: tarantool is deployed as a geo-distributed cluster, and when one DC goes down, application must switch to reserve DC with minimal service disruption.

Proposal 1: add built-in reconnect strategy which would take several connection URI's as a config, and would quickly to establish connection to them in a circle, returning the first succesful one.
Edge-case: if all URIs are unavailable, return error.

Proposal 2: implement overridable connection strategy class (as in tarantool-java), and provide proposal 1 as example.

UpsertTyped possibly(?) meaningless

Hi,

I see you've completed adding *Typed variations of several data manipulation methods.

One of those new methods is UpsertTyped.

However, per the docs here:

https://tarantool.org/doc/book/box/box_space.html#box-space-upsert

box.space.upsert() always returns nil

Given this, having an UpsertTyped is possibly

  1. meaningless (res will always remain "nil") and

  2. misleading (a developer who hasn't read the tarantool docs, only go-tarantool docs, might assume that Upsert is supposed to return the just inserted tuple)

result as interface{}

So, I got a response from tarantool as json in interface{} container
and how Unmarshal it? how work with it (cause it can be a variable structure)?
give me an advice please...

Limit in select?

Hi! I am very surprised, why there is no possible to do select query without limit?

Example for inserting tuple with byte slice

I try to write tuple where one fields is byte slice:
client.Insert(TEST_SPACE, []interface{}{"testStr", cdata, "param1"})
cdata - is []byte slice
lua space :

{name = 'KEY',       type = 'string'},
{name = 'CDATA',          type = 'array'},

Err: panic: Tuple field 2 type does not match one required by operation: expected array (0x17) [recovered] panic: Tuple field 2 type does not match one required by operation: expected array (0x17)
Could you please present some example.

something wrong?

go client is not stable?

bug screenshot

  //tarantool
server := ":3301"
opts := tarantool.Opts{
    Timeout:       500 * time.Millisecond,
    Reconnect:     1 * time.Second,
    MaxReconnects: 3,
    User:          "test",
    Pass:          "test",
}
client, err := tarantool.Connect(server, opts)
if err != nil {
    ff("Failed to connect: ", err.Error())
}
resp, err := client.Ping()
ff(resp.Code)
ff(resp.Data)
ff(err)

Failed to connect: client connection is not ready
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x16dd42]

goroutine 1 [running]:
panic(0x766de0, 0xc82000a120)
/usr/local/go/src/runtime/panic.go:464 +0x3e6
github.com/tarantool/go-tarantool.(*Connection).nextRequestId(0x0, 0xc82025f800)

Data race on timeouts

Hello. I use relative small timeouts for some reason and from time to time it occurs.
Debugging my code under race detector I got the following data race few times:

2017-06-25 02:01:11.6975 pub term error: client timeout for request 19628700 (0x4003)
2017-06-25 02:01:11.7933 pub term error: client timeout for request 19628716 (0x4003)
==================
WARNING: DATA RACE
Write at 0x00c420188098 by goroutine 14:
  github.com/tarantool/go-tarantool.(*Connection).timeouts()
      /home/go/src/github.com/tarantool/go-tarantool/connection.go:741 +0x633

Previous read at 0x00c420188098 by goroutine 47:
  github.com/tarantool/go-tarantool.(*Future).send()
      /home/go/src/github.com/tarantool/go-tarantool/request.go:383 +0x89
  github.com/tarantool/go-tarantool.(*Connection).Call17Async()
      /home/go/src/github.com/tarantool/go-tarantool/request.go:334 +0x101
  github.com/tarantool/go-tarantool.(*Connection).Call17()
      /home/go/src/github.com/tarantool/go-tarantool/request.go:113 +0x64
[my code skipped]

Goroutine 14 (running) created at:
  github.com/tarantool/go-tarantool.Connect()
      /home/go/src/github.com/tarantool/go-tarantool/connection.go:248 +0xb14
[my code skipped]

Goroutine 47 (running) created at:
[my code skipped]
==================
2017/06/25 02:01:11 tarantool: unexpected requestId (19628700) in response
2017/06/25 02:01:11 tarantool: unexpected requestId (19628716) in response

using latest code from github (728550a).

Not error yet, but I think it should be checked and fixed.

Proposal to change driver interface to context

Golang has standard library named "context"
https://golang.org/pkg/context/

This library widely used in golang libraries to implement

  • cancellation of calculation tree
  • timeout of calculation tree

Please read about context and how it used.

Golang standard libraries does not use context directly only because keep compatibility of API since golang 1.0.
But in some libraries you can find usage of context, for instance here:
https://golang.org/pkg/database/sql/#Conn.QueryContext

I propose to get rid of DeadlineIO struct and rewrite driver API to context usage.
It's more explicit and more golang-way, than actual implementation

Newbie's question (by issue#18)

Hello!

I want to write little Go-application for statistical processing of data stored in Tarantool. I tried some examples from README and I faced with troubles at the step of data selection. Similar request I found at issue#18 but the proposed solution was not expand for me principles of work with msg.pack. Can I ask about more enhanced CRUD examples?
The essence of the task. I want to select of some records from TarantoolDB, count that and compare some values inside tuples with a data by another sources. The structure of a data in a tuple can be flexible and by fact is undefined.

Thank you!

Предложения по улучшению от Олега Царёва.

Всем приветы
Если вдруг интересно - я посмотрел на гошный драйвер к тарантулу
Он не то, чтобы ужасный, но определенно его можно сделать лучше

Вместо DeadlineIO и прочего в golang есть стандартная штука - https://golang.org/pkg/context/
context передается первым аргументом, через него делают как cancel, так и timeout'ы

Вот пример использования в sql https://golang.org/pkg/database/sql/#Conn.QueryContext

Он не вкручен во всей стандартной библиотеке явно лишь по одной причине - пацаны обещали совместимость с golang 1.0, когда context'ов еще не было
Но вот в http, например, он уже есть, просто неявно - через request.WithContext(...)

В общем, если драйвер перевести на context'ы, то он будет сильно проще для восприятия гошных разработчиков, чем текущий велосипед с DeadlineIO

memory leak

I have an extremely simple program for testing WPS (in this case, 30k). But at the same time I have a memory leak. How can this be fixed?

import uuid "github.com/satori/go.uuid"

func init() {
...
opts := tarantool.Opts{User: conf.User}
	conn, err = tarantool.Connect(conf.URI, opts)
	if err != nil {
		log.Fatalln("Connection refused:", err)
	}
...
}

func main() {
...
	go writeProcess()
...
}

func writeProcess() {
	writeLimiter := rate.NewLimiter(rate.Limit(conf.WPS), 100)
	for {
		go write()
		sleep(writeLimiter)
	}
}

func write() {
	id := uuid.NewV4().String()

	_, err := conn.Insert(conf.Space, []interface{}{id, time.Now().UTC().Unix(), conf.FakeDataStr})
	if err != nil {
		log.Fatalln("Error:", err)
	}
...
}

image
image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.