Giter VIP home page Giter VIP logo

aliyun-tablestore-go-sdk's Introduction

Aliyun tablestore SDK for Go

GitHub version Build Status Coverage Status

关于

  • 此Go SDK基于阿里云表格存储服务 API构建。
  • 阿里云表格存储是阿里云自主研发的NoSQL数据存储服务,提供海量结构化数据的存储和实时访问。

运行环境

  • 推荐使用Go 1.4及以上。

安装方法

GitHub安装

  • 执行命令go get github.com/aliyun/aliyun-tablestore-go-sdk获取远程代码包。
  • 在您的代码中使用import "github.com/aliyun/aliyun-tablestore-go-sdk"引入TableStore Go SDK的包。

贡献代码

  • 我们非常欢迎大家为TableStore Go# SDK以及其他阿里云SDK贡献代码

联系我们

扫码加入TableStore讨论群,和我们直接交流讨论

钉钉群号:23307953

aliyun-tablestore-go-sdk's People

Contributors

actimermao avatar chuanwang66 avatar danielxiaoran avatar doraemonext avatar elya-alibaba avatar gorexlv avatar lqvito avatar qiyuewuyi avatar red-chen avatar thetruechar avatar timeexceed avatar wanttobeamaster avatar waterxjw avatar whybert avatar whyiug avatar yunjinlxp avatar yvxiang avatar zhenghaozzz avatar zhoucan1990 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aliyun-tablestore-go-sdk's Issues

timeline.ColumnMap改为struct,使用强类型

目前,timeline.ColumnMap的定义是:

type ColumnMap map[string]interface{}

运行期,会对value做check。很容易出现类型不匹配导致 MissInUse 错误。

建议,采用struct,定义强类型:

type ColumnMap struct {
	int64Fields   map[string]int64
	bytesFields   map[string][]byte
	stringFields  map[string]string
	float64Fields map[string]float64
}

并辅助一些setter,简化操作:

func (cm *ColumnMap) AddInt64Column(key string, val int64) {
	cm.int64Fields[key] = val
}

func (cm *ColumnMap) AddStringColumn(key string, val string) {
	cm.stringFields[key] = val
}

在某些平台上,SDK丢失了RequestId

map[X-Ots-Requestid:[0005698d-9dce-b347-0cc3-980a09c33ec8] X-Ots-Contenttype:[protocol buffer] X-Ots-Date:[2018-04-11T07:30:42.430654Z] Date:[Wed, 11 Apr 2018 07:30:42 GMT] Connection:[keep-alive] Authorization:[OTS *************] X-Ots-Contentmd5:[TzgBb4fK4cxD8Ldjw4JGHQ==]]

显示通过x-ots-requestid 拿数据失败

Unmaintained and vulnerable dependency

The dependency satori/go.uuid is no longer maintained satori/go.uuid#84 and it's current pinning on v1.2.0 suffers from this vulnerability satori/go.uuid#73.

Suggested fixes include either pinning the latest commit on the master branch of satori/go.uuid where this issue has been fixed satori/go.uuid@75cca53 or move to a maintained replacement such as gofrs/uuid or google/uuid, which also resolves a number of other bugs.

The only change besides changing the pinning would be this line:

--- tunnel/api.go.orig	2021-03-02 16:37:50.890286864 +1100
+++ tunnel/api.go	2021-03-02 16:38:28.083334608 +1100
@@ -163,7 +163,7 @@ func (api *TunnelApi) doRequestInternal(
 		hreq.Header.Set(xOtsHeaderStsToken, api.securityToken)
 		otshead.set(xOtsHeaderStsToken, api.securityToken)
 	}
-	traceId := uuid.NewV4()
+	traceId := uuid.Must(uuid.NewV4())
 	hreq.Header.Set(xOtsHeaderTraceID, traceId.String())
 	otshead.set(xOtsHeaderTraceID, traceId.String())

DescribeTable Panic: runtime error: invalid memory address or nil pointer dereference

goroutine 1 [running]:
github.com/aliyun/aliyun-tablestore-go-sdk/tablestore.(*TableStoreClient).DescribeTable(0xc420396150, 0xc42013fbb8, 0x0, 0x0, 0x2)
/Users/redchen/go/src/github.com/aliyun/aliyun-tablestore-go-sdk/tablestore/api.go:367 +0x461
ots.DescTable(0xb84cc0, 0xc4201eaa40, 0x6, 0x2, 0x8f2b90, 0x15)
/Users/redchen/workspace/go/aliyun-tablestore-cli/src/ots/ots.go:88 +0x110
commands.glob..func11(0xc4202fa1c0)

批量写入时设置了SetReturnPk并没有返回主键

  • go版本: 1.17
  • sdk版本: v5.0.6
  • 代码如下
batchWriteReq := &tablestore.BatchWriteRowRequest{}

for _, data := range datas {
      putRowChange := new(tablestore.PutRowChange)
      putRowChange.TableName = table_im_user_pool
      putPk := new(tablestore.PrimaryKey)
      putPk.AddPrimaryKeyColumn("pk1", "xxxx")
      putPk.AddPrimaryKeyColumnWithAutoIncrement("pk2")
      putRowChange.PrimaryKey = putPk
      putRowChange.AddColumn("col", "xxxxx")
      putRowChange.SetCondition(tablestore.RowExistenceExpectation_IGNORE)
      // 这里设置了
      putRowChange.SetReturnPk()
      batchWriteReq.AddRowChange(putRowChange)
}

// resp内并没有返回主键
resp, err := client.BatchWriteRow(batchWriteReq)

Bug - Panic when query multiple rows by partition key

Two primary keys: ProdId string, CommentId int64 (NOT incremental)

When query multiple rows by ProdId, with Min and Max of CommentId, panic when kicks off query

func GetMultiComments(prodId string) (*tablestore.GetRangeResponse, error) {

fmt.Println("batch get row started")

rangeRowQueryCriteria := &tablestore.RangeRowQueryCriteria{}
rangeRowQueryCriteria.TableName = TableName

startPK := new(tablestore.PrimaryKey)
startPK.AddPrimaryKeyColumn("ProdId", prodId)
startPK.AddPrimaryKeyColumnWithMinValue("CommentId")

endPK := new(tablestore.PrimaryKey)
endPK.AddPrimaryKeyColumn("ProdId", prodId)
endPK.AddPrimaryKeyColumnWithMaxValue("CommentId")

rangeRowQueryCriteria.StartPrimaryKey = startPK
rangeRowQueryCriteria.EndPrimaryKey = endPK
rangeRowQueryCriteria.MaxVersion = 1
rangeRowQueryCriteria.Limit = 1
rangeRowQueryCriteria.Direction = tablestore.FORWARD

getRangeRequest := &tablestore.GetRangeRequest{}
getRangeRequest.RangeRowQueryCriteria = rangeRowQueryCriteria

client := CreateClient()
getRangeResp, err := client.GetRange(getRangeRequest)	//panic

return getRangeResp, err

}

I refer to below SDK example:
https://github.com/aliyun/aliyun-tablestore-go-sdk/blob/master/sample/MultipleRowOperation.go
https://help.aliyun.com/document_detail/52265.htm

使用多元索引查询数据时返回的行数跟总数不一致

image

运行的代码如下:

    searchRequest.SetTableName(tableName)
    searchRequest.SetIndexName(indexName)
    query := &search.GeoDistanceQuery{} // 设置查询类型为GeoDistanceQuery
    query.FieldName = "Col_GeoPoint"
    query.CenterPoint = "5,5"  // 设置中心点
    query.DistanceInMeter = 10000.0 // 设置到中心点的距离条件,不超过10000米
    searchQuery := search.NewSearchQuery()
    searchQuery.SetQuery(query)
    searchQuery.SetGetTotalCount(true)
    searchRequest.SetSearchQuery(searchQuery)
    // 设置返回所有列
    searchRequest.SetColumnsToGet(&tablestore.ColumnsToGet{
        ReturnAll:true,
    })
    searchResponse, err := client.Search(searchRequest)
    if err != nil {
        fmt.Printf("%#v", err)
        return
    }
    fmt.Println("IsAllSuccess: ", searchResponse.IsAllSuccess) // 查看返回结果是否完整
    fmt.Println("TotalCount: ", searchResponse.TotalCount) // 匹配的总行数
    fmt.Println("RowCount: ", len(searchResponse.Rows))

http: panic serving 127.0.0.1:62204: [tablestore] invalid input

场景:1、在实例上创建表 2、go客户端连接实例操作数据。

在Sample代码中,遇到两个问题:

1、插入数据时如果不添加pk主键,不返回内容,也不报错,数据库也没有内容
2、客户端数据类型和实例数据类型不一致会导致客户端崩溃,报以下错误:
[HTTP Server] http: panic serving 127.0.0.1:62204: [tablestore] invalid input goroutine 43 [running]: net/http.(*conn).serve.func1(0xc000294aa0)
例如:主键或者某一列在实例中是int(11),在客户端是int8

这个垃圾SDK到底能不能用

timeline 中的Scan函数,MaxCount参数没用,返回数据也无序。这种基本的问题(几行代码搞定)为何迟迟没人解决??用户怎么使用??

#45
#48

util.go文件缺少按照指定版本更新的函数

PutColumnWithTimestamp
提供一个example

func (rowchange *UpdateRowChange) PutColumnWithTimestamp(columnName string, value interface{}, timestamp int64) {
	column := &ColumnToUpdate{ColumnName: columnName, Value: value}
	column.Timestamp = timestamp
	column.HasTimestamp = true
	rowchange.Columns = append(rowchange.Columns, *column)
}

TmLine.scan 返回数据无序

使用TmLine.scan 获取timeline的数据,发现是无序的。
看了一下SDK的代码timeline/store.go返回的是一个map, timeline.go中的asyncScan直接循环获取map中k,v 导致结果无序

table update hit 'OTSParameterInvalid TimeToLive cannot be 0 or less than -1.' issue

I tried to update MaxVersion without changing TimeToLive's value, but it hit 'OTSParameterInvalid TimeToLive cannot be 0 or less than -1.' issue with the code below.

	fmt.Println("UpdateTableSample started")
	updateTableReq := new(tablestore.UpdateTableRequest)
	updateTableReq.TableName = tableName
	updateTableReq.TableOption = new(tablestore.TableOption)
	updateTableReq.TableOption.MaxVersion = 5

	_, err := client.UpdateTable(updateTableReq)

BatchGetRow bug

当BatchGetRow中某个表的某个主键不存在时,会返回unexpect io end

在ots的文档中说到(https://help.aliyun.com/document_detail/27310.html?spm=5176.doc52265.6.615.NDnOUA):
如果某行不存在或者某行在指定的 columns_to_get 下没有数据,仍然会在 TableInBatchGetRowResponse 中有一条对应的 RowInBatchGetRowResponse,但其 row 下面的 primary_key_columns 和 attribute_columns 将为空。

unexpect io end应该就是读到了空的primary_key_columns 和 attribute_columns。
应该判断一下错误类型,而不是直接返回unexpect io end。

Can not got keyOption because the code not work

// enable it when we support kep option in describe table

/*if key.Option != nil {
keyOption := PrimaryKeyOption(key.Option)
responseTableMeta.SchemaEntry = append(responseTableMeta.SchemaEntry, &PrimaryKeySchema{Name: key.Name, Type: &keyType, Option: &keyOption })
} else {
responseTableMeta.SchemaEntry = append(responseTableMeta.SchemaEntry, &PrimaryKeySchema{Name: key.Name, Type: &keyType })
}
/

TableMeta:

{
"TableMeta": {
"TableName": "auto",
"SchemaEntry": [
{
"Name": "uid",
"Type": 2,
"Option": null
},
{
"Name": "pid",
"Type": 1,
"Option": null
}
]
},
"TableOption": {
"TimeToAlive": -1,
"MaxVersion": 1
},
"ReservedThroughput": {
"Readcap": 0,
"Writecap": 0
},
"StreamDetails": {
"EnableStream": false,
"StreamId": null,
"ExpirationTime": 0,
"LastEnableTime": 0
}
}

CreateTable导致panic,建议做错误检查,返回error,直接运行时panic不妥当

`
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0x7509e5]

goroutine 1 [running]:
github.com/aliyun/aliyun-tablestore-go-sdk/tablestore.(*TableStoreClient).CreateTable(0xc000122000, 0xc0000cbeb0, 0x4, 0x4, 0x5)
D:/GoLang/GoWork/src/github.com/aliyun/aliyun-tablestore-go-sdk/tablestore/api.go:348 +0x965
main.CreateTable(0xc000122000, 0x31, 0x84e146)
D:/GoLang/GoWork/src/local_no_vcs/OTSTest202103040238/main.go:42 +0x5ef
main.main()
D:/GoLang/GoWork/src/local_no_vcs/OTSTest202103040238/main.go:17 +0xad
`

相关的上下文(github.com/aliyun/aliyun-tablestore-go-sdk/tablestore/api.go 348行附近):
`
346 req.ReservedThroughput = new(otsprotocol.ReservedThroughput)
347 req.ReservedThroughput.CapacityUnit = new(otsprotocol.CapacityUnit)
348 req.ReservedThroughput.CapacityUnit.Read = proto.Int32(int32(request.ReservedThroughput.Readcap))
349 req.ReservedThroughput.CapacityUnit.Write = proto.Int32(int32(request.ReservedThroughput.Writecap))

`

支持ORM操作

目前TableStore Golang SDK的操作特别繁琐,代码量较多,请考虑支持ORM框架的实现,比如gorm。

PUTROW BUG - failed to get non-nullable "PrimaryKeys" from PUTRow response

Scenario:
Put a row into TableStore, expect to get the primary keys back since CommentId (2nd PK) is auto incremental int64

Bug: failed to get non-nullable "PrimaryKeys" from PUTRow

Exception: put response: {"ConsumedCapacityUnit":{"Read":0,"Write":1},"PrimaryKey":{"PrimaryKeys":null},"RequestId":"0005a22b-94b8-67ca-8ac0-dc0b38e2f4bc"}

Justification:
according to the api at line 649, https://github.com/aliyun/aliyun-tablestore-go-sdk/blob/master/tablestore/api.go; PrimaryKeys should be returned.
according to doc, https://www.alibabacloud.com/help/doc-detail/27306.htm, The data type after the row is successfully written. Currently, only the primary key can be returned, which is used for the auto-increment function of the primary key column.

Coding Language: GOLANG

type Comment struct {
ProdId string // prodid as partition key
CommentId int64 //row key for each comment
Content string //comments text
}

CreatTable PK meta
meta := new(tablestore.TableMeta)
meta.TableName = "commentTest"
meta.AddPrimaryKeyColumn("ProdId", tablestore.PrimaryKeyType_STRING) // prodid as partition key
meta.AddPrimaryKeyColumnOption("CommentId", tablestore.PrimaryKeyType_INTEGER, tablestore.AUTO_INCREMENT) //row key for each comment
meta.AddDefinedColumn("Content", tablestore.DefinedColumn_STRING) //comments text

method

putRowChange := new(tablestore.PutRowChange)
putRowChange.TableName = "commentTest"

meta := new(tablestore.PrimaryKey)
meta.AddPrimaryKeyColumn("ProdId", c.ProdId)
if c.CommentId > 0 {
	meta.AddPrimaryKeyColumn("CommentId", c.CommentId)
} else {
	meta.AddPrimaryKeyColumnWithAutoIncrement("CommentId")
}
putRowChange.PrimaryKey = meta
putRowChange.SetCondition(tablestore.RowExistenceExpectation_IGNORE)

putRowChange.AddColumn("Content", c.Content)

request := new(tablestore.PutRowRequest)
request.PutRowChange =putRowChange

client := CreateClient()
resp, err := client.PutRow(request)

data, _ := json.Marshal(resp)
fmt.Println("put response: ", string(data))

Add EnsureTable method

It's a common requirement that user may ensure the table exists, if not, EnsureTable should create table automatically, something like Sync in timeline pkg

timeline模型无法跑起来

目前go版这个timeline模型可以正常跑不?
sync表和store表需要自己创建吗?
我按readme要求, 设置了配置文件. 但是无法跑起来.
返回"OTSParameterInvalid Invalid table name:...."错误.

如果需要自己创建表. 相关的主键字段如何设置呢?

util.NewPrimaryKeyColumn时支持的value type太少了

t := reflect.TypeOf(value)
switch t.Kind() {
case reflect.String:
v.Type = otsprotocol.PrimaryKeyType_STRING
case reflect.Int64:
v.Type = otsprotocol.PrimaryKeyType_INTEGER
case reflect.Slice:
v.Type = otsprotocol.PrimaryKeyType_BINARY
default:
panic(errInvalidInput)
}
只支持string/int64/slice,导致如put response或者stream record中携带的primaryKey,用户都没有办法直接用,因为primaryKey中的各个column的value都是interface类型的,用户需要拷贝处理一次才能继续使用,eg:
row := getAStreamRecord() //type *ots.StreamRecord
putChange.PrimaryKey = row.PrimaryKey // panic!
需要改写成
pk := new(ots.PrimaryKey)
pk.AddPrimaryKeyColumn("pk_0", int64(row.PrimaryKey.PrimaryKeys[0].Value.(uint64)))
putChange.PrimaryKey = pk

BatchWriteRow's response lose something important? v1.7.7

func BatchWrite(cli *tablestore.TableStoreClient, tableName string) {
	cs := make(map[string][]tablestore.RowChange)
	for i := 0; i < 200; i++ {
		putRowChange := &tablestore.PutRowChange{TableName: tableName, PrimaryKey: &tablestore.PrimaryKey{}}
		putRowChange.PrimaryKey.AddPrimaryKeyColumn("pk1", strconv.Itoa(i))
		putRowChange.PrimaryKey.AddPrimaryKeyColumnWithAutoIncrement("pk2")
		putRowChange.AddColumn("col_"+strconv.Itoa(1), strconv.Itoa(i))
		putRowChange.AddColumn("col_"+strconv.Itoa(2), int64(i))
		putRowChange.AddColumn("col_"+strconv.Itoa(3), strconv.Itoa(i))
		putRowChange.AddColumn("col_"+strconv.Itoa(4), int64(i))
		putRowChange.AddColumn("col_"+strconv.Itoa(5), strconv.Itoa(i))

		putRowChange.SetCondition(tablestore.RowExistenceExpectation_IGNORE)
		cs[putRowChange.GetTableName()] = append(cs[putRowChange.GetTableName()], putRowChange)
	}

	res, err := cli.BatchWriteRow(&tablestore.BatchWriteRowRequest{RowChangesGroupByTable: cs})
	if err != nil {
		fmt.Println("batch request failed with:", err.Error())
	}
	fmt.Println(res)
}

debug screenshot

image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.