cloudwu / pbc Goto Github PK
View Code? Open in Web Editor NEWA protocol buffers library for C
License: MIT License
A protocol buffers library for C
License: MIT License
查了下protobuf.lua,没找到类似has_xx的api,这让使用者无法区分table中的某个字段是缺省值还是根本就没有(用nil?)
将您的test.lua做了些微改动之后test.lua代码如下:
require "protobuf"
addr = io.open("../../build/addressbook.pb","rb")
buffer = addr:read "*a"
addr:close()
protobuf.register(buffer)
addressbook = {
name = "Alice",
id = 12345,
phone = {
{ number = "1301234567" },
{ number = "87654321", type = "WORK" },
}
}
code = protobuf.encode("tutorial.Person", addressbook)
decode = protobuf.decode("tutorial.Person" , code)
print(decode.name)
print(decode.id)
for _,v in ipairs(decode.phone) do
print("\t"..v.number, v.type)
end
print("=========");
phonebuf = protobuf.pack("tutorial.Person.PhoneNumber number","87654321");
buffer = protobuf.pack("tutorial.Person name id phone", "Alice", 123, { phonebuf });
name,id,phone = protobuf.unpack("tutorial.Person name id phone", buffer);
print(name)
print(id)
for _,v in pairs(phone) do
print(v.number, v.type)
end
输出结果如下:
Alice
12345
1301234567 HOME
Alice
123
nil nil
不知道为什么87654321 WORK会在这个网页上变粗体加大
在使用import时报错,查了下似乎没有import机制,确定下,是否不支持import。
你好!我想问下message嵌套问题,代码跟上一位的一致,encode时一直有错。你所说的打开lua 5.2的宏是可以解决这个问题吗?是的话这个宏在哪呢?
能否用到嵌入式系统中,嵌入式通信这一块是个问题
Hi, Is it possible to include short example of using the lib from Lua for encoding / decoding nested messages? I mean when the message i have to encode has a field of type another message?
Thanks,
Plamen
我们在项目中用了pbc来和客户端交互,在个别情况下,策划可能需要对协议做一些热更新(比如在客户端多显示一个XX值)
pbc有办法将register到环境里的数据删掉重新register一遍么?或者直接更新的方法
内存泄漏发生的位置是test_wmessage函数内对phone的操作, repeated字段。测试了一下,
struct pbc_wmessage * phone = pbc_wmessage_message(msg , "phone");
pbc_wmessage_string(phone , "number", "87654321" , -1);
调用一次没有问题, 接下来, 再次调用pbc_wmessage_string就会出现内存泄漏
==10504== 8 bytes in 1 blocks are definitely lost in loss record 1 of 1
==10504== at 0x402BE68: malloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==10504== by 0x8048871: _pbcM_malloc (in /home/van9ogh/Desktop/mr/tmp/pbc/test/a.out)
==10504== by 0x8051E5A: pbc_wmessage_string (in /home/van9ogh/Desktop/mr/tmp/pbc/test/a.out)
==10504== by 0x8052DEC: test_wmessage (in /home/van9ogh/Desktop/mr/tmp/pbc/test/a.out)
==10504== by 0x8052F18: main (in /home/van9ogh/Desktop/mr/tmp/pbc/test/a.out)
==10504==
==10504== LEAK SUMMARY:
==10504== definitely lost: 8 bytes in 1 blocks
==10504== indirectly lost: 0 bytes in 0 blocks
==10504== possibly lost: 0 bytes in 0 blocks
==10504== still reachable: 0 bytes in 0 blocks
==10504== suppressed: 0 bytes in 0 blocks
[david@host1 lua{master}]$ C_INCLUDE_PATH=/usr/include/lua5.1 make
gcc -O2 -Wall --shared -o protobuf.dll -I../.. -I/usr/local/include -L/usr/local/bin -L../../build pbc-lua.c -lpbc -llua52
pbc-lua.c: In function ‘_env_enum_id’:
pbc-lua.c:80:38: warning: implicit declaration of function ‘_pbcM_sp_query’ [-Wimplicit-function-declaration]
struct _enum *enum_map = (struct _enum *)_pbcM_sp_query(env->enums, enum_type);
^
pbc-lua.c:80:64: error: dereferencing pointer to incomplete type
struct _enum *enum_map = (struct _enum *)_pbcM_sp_query(env->enums, enum_type);
^
pbc-lua.c:86:5: warning: implicit declaration of function ‘_pbcM_si_query’ [-Wimplicit-function-declaration]
int err = _pbcM_si_query(enum_map->name, enum_name, &enum_id);
^
pbc-lua.c:86:38: error: dereferencing pointer to incomplete type
int err = _pbcM_si_query(enum_map->name, enum_name, &enum_id);
^
pbc-lua.c: In function ‘_pattern_pack’:
pbc-lua.c:809:14: warning: unused variable ‘b’ [-Wunused-variable]
luaL_Buffer b;
^
make: *** [all] Error 1
Any pointers?
由于windows下编译不通过,查看原因。
https://github.com/cloudwu/pbc/blob/master/binding/lua/pbc-lua.c
#ifndef _MSC_VER
#include <alloca.h>
#else
#define alloca _alloca
#endif
是否应该改成
#ifndef WINVER
#include <alloca.h>
#else
#define alloca _alloca
#endif
在用MinGW编译pbc与slua合进dll时发生的错误。
hi, cloudwu,
thanks for your code of pbc. Maybe I found a bug in the wmessage.c , when a field is LABEL_OPTIONAL , the output of protobuf bytes may lost the _bitField information,because you don't write this field to output bytes when it has a default value. Just to see the code below:
if (f->label == LABEL_OPTIONAL) {
if (f->type == PTYPE_ENUM) {
if (strncmp(v , f->default_v->e.name, len) == 0 && f->default_v->e.name[len] =='\0') {
return 0;
}
} else if (f->type == PTYPE_STRING) {
if (len == f->default_v->s.len &&
strcmp(v, f->default_v->s.str) == 0) {
return 0;
}
}
}
then, when a user decode this bytes, call the function to check whether this field has been set, got the wrong result.
hi cloud and all,
because of some reason, I port pbc to windows(vs2010), there is no big problem in the process, if anyone want one, mail me. if cloud wanna me to commit it to git, it's my presure.
vincent
message foo {
}
parser报错
比如
message a
{
optional b;
}
这个b
我们的项目使用lua来注册proto,项目自定义的proto文件需要引入descriptor.proto,当用下面这种方式的时候:
import "descriptor.proto"
生成的pb文件可以注册成功,但是在Java上使用时会因为命名空间的检查导致失败;
如果我们使用:
import "google/protobuf/descriptor.proto"
做引用,生成的pb文件总是注册失败。请问有什么方法可以解决么?
message EnterMapNotice
{
required bool is_login = 1;
repeated limit.Datum limit_dt = 2;
}
local enData = protobuf.encode("ic.scene.EnterMapNotice", {is_login = true})
local deData1 = protobuf.decode("ic.scene.EnterMapNotice", enData)
table.insert(deData1.limit_dt, {aaa = 1})
local enData = protobuf.encode("ic.scene.EnterMapNotice", {is_login = true})
local deData2 = protobuf.decode("ic.scene.EnterMapNotice", enData)
print(deData2.limit_dt, deData2.limit_dt[1], deData2.limit_dt[1].aaa)
---这里deData2.limit_dt[1]应该没数据的,被打印出来了
对repeate字段,如果无数据,进行修改后。
会影响后续所有此字段无数据的包。
应该是都指向空的默认table,而非生成新的table的原因。
不管是否可修改,留给其他用户提个醒。
打算引入pbc了, cloudwu出品必属精品, ;-)
项目跑在基于linux的ARM设备上, 貌似交叉编译没什么问题, 还没跑, 不知云风哥protobuf解析会不会有字节序的问题? 还是protoc工具本身的事情 :-)
环境: luajit 2.0.2, windows
1.无法遍历嵌套对象
2.无法对decode的结果encode
3.如果optional字段不编进去,decode得到table却不是nil
message A {
required int64 id = 1;
}
message B{
required int64 id = 1;
optional A a = 2;
local deData = {
id = 1,
a = {id = 1}
}
local enData = protobuf.encode("B", deData)
local deData = protobuf.decode("B", enData)
for k, v in pairs(deData.a) do
print(k, v) --无法遍历
end
protobuf.encode("B", deData) --出错
local deData = {
id = 1
}
local enData = protobuf.encode("B", deData)
local deData = protobuf.decode("B", enData)
print(deData.a) --不是nil
在message A中包含了另外一个message B
在Deocde A的时候不会立即对B进行解析,需要通过访问B中的内容来获取B的完整解析
现在的应用场景是从数据库中取出pb结构,只更改某一个字段,其它不变,然后需要存回数据库,结果encode不支持这种属于中间状态的结构进行存储
不知道是基于什么样的考虑没有提供这样的一个encode方法?
hi, 云风,我尝试测试一下pbc对int64的支持,我修改了test/test.proto如下:
message at {
optional int32 aa = 1;
optional int32 bb = 2;
optional string cc = 3;
optional int64 dd = 4;
}
相应的test/test.c测试test函数改为:
struct pbc_wmessage* w_msg = pbc_wmessage_new(env, "at");
struct pbc_rmessage* r_msg = NULL;
struct pbc_slice sl;
char buffer[1024];
sl.buffer = buffer, sl.len = 1024;
uint32_t ilow = 0xabcdef12, ihigh = 0xabcdef12;
printf("writing:\tlow=%x, high=%x\n", ilow, ihigh);
pbc_wmessage_integer(w_msg, "dd", ilow, ihigh);
pbc_wmessage_buffer(w_msg, &sl);
r_msg = pbc_rmessage_new(env, "at", &sl);
uint64_t d = 0ull;
uint32_t low = 0, high = 0;
low = pbc_rmessage_integer(r_msg, "dd", 0, &high);
printf("read:\tlow=%x, high=%x\n", low, high);
d = ((uint64_t)high)<<32 | (uint64_t)low;
printf("dd=%llx\n", d);
pbc_rmessage_delete(r_msg);
pbc_wmessage_delete(w_msg);
运行的输出为:
writing: low=abcdef12, high=abcdef12
read: low=abcdef12, high=fbcdef12
dd=fbcdef12abcdef12
可以看到dd字段的输入高低32位都是0xabcdef12,而从msg中读取出来的dd字段的打印为dd=fbcdef12abcdef12
,请问下这个是个bug?还是目前不支持int64?
这个接口果然比较好用,比nanopb, protobuf-c好用多了!有个问题,这个是不是线程安全的?
optional int32 的字段如果赋值为0, encode 的时候好像没有把这个字段encode 进去, decode的时候也没有应字段的值
#ifndef _LUA_PBC_EXTRA_H_
#define _LUA_PBC_EXTRA_H_
#if defined(_USRDLL)
#define LUA_EXTENSIONS_DLL _decispec(dllexport)
#else
#define LUA_EXTENSIONS_DLL
#endif
#ifdef __cplusplus
extern "C"{
#endif
#include "lauxlib.h"
int LUA_EXTENSIONS_DLL luaopen_protobuf_c(lua_State *L);
#ifdef __cplusplus
}
#endif
#endif // _LUA_PBC_EXTRA_H_
在链接时找不到符号 luaopen_protobuf_c
请问如何破解?
the message like this:
message Role{
optional int32 fieldA = 1;
}
i got message from server, but server didn't packed 'fieldA' everytime .
so how did know if 'fieldA' exist by lua code ?
在lua中,test.proto
message A{
enum {
B = 1;
};
};
pb:register 'test.pb'后
在lua怎么拿到 B的值
环境是mac osx Yosemite 10.10.4,openresty附带的luajit-2.1。
···
gcc -O2 -fPIC -Wall -shared -o protobuf.so -I../.. -I/usr/local/openresty/luajit/include/luajit-2.1 -L../../build pbc-lua.c -lpbc
Undefined symbols for architecture x86_64:
"_luaL_checkinteger", referenced from:
__rmessage_new in pbc-lua-cc4638.o
__rmessage_integer in pbc-lua-cc4638.o
__rmessage_int32 in pbc-lua-cc4638.o
__rmessage_int64 in pbc-lua-cc4638.o
__rmessage_int52 in pbc-lua-cc4638.o
__rmessage_uint52 in pbc-lua-cc4638.o
__rmessage_real in pbc-lua-cc4638.o
...
"_luaL_checklstring", referenced from:
__env_register in pbc-lua-cc4638.o
__env_type in pbc-lua-cc4638.o
__rmessage_new in pbc-lua-cc4638.o
__rmessage_integer in pbc-lua-cc4638.o
__rmessage_int32 in pbc-lua-cc4638.o
__rmessage_int64 in pbc-lua-cc4638.o
__rmessage_int52 in pbc-lua-cc4638.o
...
"_luaL_checknumber", referenced from:
__wmessage_real in pbc-lua-cc4638.o
__wmessage_int52 in pbc-lua-cc4638.o
__wmessage_uint52 in pbc-lua-cc4638.o
__pattern_pack in pbc-lua-cc4638.o
"_luaL_checktype", referenced from:
__decode in pbc-lua-cc4638.o
"_luaL_error", referenced from:
__env_register in pbc-lua-cc4638.o
__env_type in pbc-lua-cc4638.o
__rmessage_new in pbc-lua-cc4638.o
__rmessage_delete in pbc-lua-cc4638.o
__rmessage_integer in pbc-lua-cc4638.o
__rmessage_int32 in pbc-lua-cc4638.o
__rmessage_int64 in pbc-lua-cc4638.o
...
"_luaL_register", referenced from:
_luaopen_protobuf_c in pbc-lua-cc4638.o
"_lua_call", referenced from:
_push_value in pbc-lua-cc4638.o
"_lua_checkstack", referenced from:
__pattern_unpack in pbc-lua-cc4638.o
"_lua_createtable", referenced from:
__pattern_unpack in pbc-lua-cc4638.o
__decode in pbc-lua-cc4638.o
__gc in pbc-lua-cc4638.o
_decode_cb in pbc-lua-cc4638.o
"_lua_insert", referenced from:
_decode_cb in pbc-lua-cc4638.o
"_lua_isstring", referenced from:
__rmessage_new in pbc-lua-cc4638.o
__pattern_unpack in pbc-lua-cc4638.o
"_lua_isuserdata", referenced from:
__pattern_unpack in pbc-lua-cc4638.o
"_lua_newuserdata", referenced from:
__gc in pbc-lua-cc4638.o
"_lua_objlen", referenced from:
__pattern_pack in pbc-lua-cc4638.o
_decode_cb in pbc-lua-cc4638.o
"_lua_pushboolean", referenced from:
__env_type in pbc-lua-cc4638.o
__pattern_unpack in pbc-lua-cc4638.o
__decode in pbc-lua-cc4638.o
_push_value in pbc-lua-cc4638.o
"_lua_pushcclosure", referenced from:
__gc in pbc-lua-cc4638.o
"_lua_pushinteger", referenced from:
__env_type in pbc-lua-cc4638.o
__rmessage_integer in pbc-lua-cc4638.o
__rmessage_size in pbc-lua-cc4638.o
__wmessage_buffer in pbc-lua-cc4638.o
__pattern_size in pbc-lua-cc4638.o
__pattern_unpack in pbc-lua-cc4638.o
__env_enum_id in pbc-lua-cc4638.o
...
"_lua_pushlightuserdata", referenced from:
__env_new in pbc-lua-cc4638.o
__rmessage_new in pbc-lua-cc4638.o
__rmessage_int32 in pbc-lua-cc4638.o
__rmessage_message in pbc-lua-cc4638.o
__wmessage_new in pbc-lua-cc4638.o
__wmessage_message in pbc-lua-cc4638.o
__wmessage_buffer in pbc-lua-cc4638.o
...
"_lua_pushlstring", referenced from:
__rmessage_int64 in pbc-lua-cc4638.o
__rmessage_string in pbc-lua-cc4638.o
__wmessage_buffer_string in pbc-lua-cc4638.o
__pattern_unpack in pbc-lua-cc4638.o
__pattern_pack in pbc-lua-cc4638.o
_push_value in pbc-lua-cc4638.o
"_lua_pushnumber", referenced from:
__rmessage_int52 in pbc-lua-cc4638.o
__rmessage_uint52 in pbc-lua-cc4638.o
__rmessage_real in pbc-lua-cc4638.o
__pattern_unpack in pbc-lua-cc4638.o
_push_value in pbc-lua-cc4638.o
"_lua_pushstring", referenced from:
__env_type in pbc-lua-cc4638.o
__last_error in pbc-lua-cc4638.o
_push_value in pbc-lua-cc4638.o
"_lua_pushvalue", referenced from:
__decode in pbc-lua-cc4638.o
_decode_cb in pbc-lua-cc4638.o
_push_value in pbc-lua-cc4638.o
"_lua_rawgeti", referenced from:
__pattern_pack in pbc-lua-cc4638.o
_decode_cb in pbc-lua-cc4638.o
"_lua_rawseti", referenced from:
__pattern_unpack in pbc-lua-cc4638.o
_decode_cb in pbc-lua-cc4638.o
"_lua_setfield", referenced from:
__gc in pbc-lua-cc4638.o
_decode_cb in pbc-lua-cc4638.o
"_lua_setmetatable", referenced from:
__gc in pbc-lua-cc4638.o
"_lua_settop", referenced from:
__env_type in pbc-lua-cc4638.o
__pattern_pack in pbc-lua-cc4638.o
_decode_cb in pbc-lua-cc4638.o
"_lua_toboolean", referenced from:
__pattern_pack in pbc-lua-cc4638.o
"_lua_tointeger", referenced from:
__rmessage_string in pbc-lua-cc4638.o
__rmessage_message in pbc-lua-cc4638.o
__pattern_unpack in pbc-lua-cc4638.o
__pattern_pack in pbc-lua-cc4638.o
"_lua_tolstring", referenced from:
__rmessage_new in pbc-lua-cc4638.o
__wmessage_int64 in pbc-lua-cc4638.o
__pattern_unpack in pbc-lua-cc4638.o
__pattern_pack in pbc-lua-cc4638.o
"_lua_touserdata", referenced from:
__env_register in pbc-lua-cc4638.o
__env_type in pbc-lua-cc4638.o
__rmessage_new in pbc-lua-cc4638.o
__rmessage_delete in pbc-lua-cc4638.o
__rmessage_integer in pbc-lua-cc4638.o
__rmessage_int32 in pbc-lua-cc4638.o
__rmessage_int64 in pbc-lua-cc4638.o
...
"_lua_type", referenced from:
__env_type in pbc-lua-cc4638.o
__wmessage_int32 in pbc-lua-cc4638.o
__wmessage_int64 in pbc-lua-cc4638.o
__pattern_pack in pbc-lua-cc4638.o
__decode in pbc-lua-cc4638.o
_decode_cb in pbc-lua-cc4638.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [protobuf.so] Error 1
···
I want to support upgrade my application at runtime, so i need to register a new pb file at runtime, currently, it won't overwrite the old one.
Can you show me what's the best way to implement this? I'm happy to send a patch.
proto 文件中定义有 double 或者 float类型的
在lua中进行赋值的时候 设置int类型 或者 double/float浮点类型 进行encode通过http发送到java后台进行解析的时候解析不出来
我进行在lua中把encode的数据 在decode回来进行打印发现浮点类型加长了好多位 我不知道是不是这样导致java端解析不出数据
求解
Proto file defined in double or float type
In lua assignment set type int or double/float float to encode sent via HTTP to a Java background when parsing resolution not to come out
I conducted in lua to encode the data found in floating-point decode back for printing type lengthened a lot I don't know whether that is in Java can't parse the data
For help
==13968== Invalid write of size 4
==13968== at 0x4249F4: _pbcA_open (array.c:19)
==13968== by 0x428732: pbc_pattern_unpack (pattern.c:835)
==13968== by 0x429472: register_internal (bootstrap.c:275)
==13968== by 0x42980C: _pbcB_init (bootstrap.c:302)
==13968== by 0x425D46: pbc_new (proto.c:32)
note: when I use pbc as a library and link into nginx, it is not segment fault.
发现编码的结果跟protobuf不一致,生成的编码protobuf不能正常解析。
对于消息:
message test{
required string id = 1;
}
赋值:
test.id = "1"
pbc编码结果:
0d 0a 02 31 20
protobuf编码结果:
0a 01 31
云风您好,我在使用你的pbc库时出现了decode失败的问题,我的使用环境是这样的:
proto文件:
package Cookie;
message CookieValue {
message Segments {
optional int32 s_type = 1;
optional int32 s_id = 2;
optional int32 s_ct = 3;
optional int32 s_ttl = 4;
}
repeated Segments seg = 1;
//last bid info
message Pbid{
optional int32 p_id = 1;
optional int32 p_value = 2;
}
repeated Pbid pbid = 2;
//frequency daily
message Daily {
optional int32 daily_id = 1;
message DailyImp {
optional int32 d_id = 1;
optional int32 d_value = 2;
}
repeated DailyImp daily_imp = 2;
}
repeated Daily daily = 3;
//frequency total
message Total {
optional int32 t_id = 1;
optional int32 t_value = 2;
}
repeated Total total = 4;
message Imp {
optional int32 imp_c_id = 1;
optional int32 win = 2;
optional int32 bid = 3;
optional int32 imp_time = 4;
}
repeated Imp imp = 5;
message Clk {
optional int32 clk_id = 1;
optional int32 clk_time = 2;
}
repeated Clk clk = 6;
}
lua代码:
local protobuf = require "protobuf"
local cjson = require "cjson"
addr = io.open("../../build/cookie.pb","rb")
buffer = addr:read "*a"
addr:close()
protobuf.register(buffer)
local CookieValue={
seg={
{s_type=1,s_ct=1415384160,s_ttl=1415384160,s_id=100001}
},
total={
{t_value=1,t_id=100188}
},
pbid={
{p_value=3,p_id=100188},
{p_value=3,p_id=100189},
{p_value=3,p_id=100190}
},
daily={
{daily_id=1125,daily_imp={{d_value=1,d_id=100188}}},
{daily_id=1126,daily_imp={{d_value=1,d_id=100118}}}
},
imp={
{win=2,bid=3,imp_time=8160,imp_c_id=100188},
{win=2,bid=3,imp_time=8155,imp_c_id=100182},
{win=2,bid=3,imp_time=8157,imp_c_id=100181},
}
}
local buffer = protobuf.encode("Cookie.CookieValue", CookieValue)
print('buffer: ',buffer)
print('the len of buffer: ',#buffer)
local t = protobuf.decode("Cookie.CookieValue", buffer)
print('cjson(t): ',cjson.encode(t))
for k,v in pairs(t) do
if type(k) == "string" then
print(k,v)
end
end
for k,v in pairs(t.daily[1]) do
print(k,v)
end
for k,v in pairs(t.pbid[1]) do
print(k,v,type(k),type(v))
end
According to official documentation, https://developers.google.com/protocol-buffers/docs/cpptutorial, official implementation allow compatibility when add/delete optional fields. But pbc don't implement this.
test_optional.lua
:
pb = require "protobuf"
addr = io.open("optional.pb","rb")
buffer = addr:read "*a"
addr:close()
pb.register(buffer)
local buf = pb.encode('Message1', {x=1,y=2,w=3,h=4})
local o = assert(pb.decode('Message2', buf), 'decode failed')
print(o.x)
optional.proto
:
message Message1 {
required int32 x = 1;
required int32 y = 2;
optional int32 w = 3;
optional int32 h = 4;
}
message Message2 {
required int32 x = 1;
required int32 y = 2;
}
The output is expected to be 1
, but gets a decode error.
pattern.c line 752
if (len <= 0) {
return strcmp(pf->defv->s.str, slice->buffer) == 0;
}
when len ==0
would cause core dump
C++:
tutorial::Person person;
person.set_name("Alice");
person.set_id(123)
std::string stringbuffer =person:SerializeAsString();
字符传给lua
Lua:
local file = "person.pb";
protobuf.register_file(file)
result = protobuf.decode("tutorial.Person", stringbuffer)
result一直是false
define PROTO_temp
TAR :=
PROTO :=
$$(TAR).pb : | $(BUILD)
$$(TAR).pb : test/$(1)
protoc -o$$@
endef
The decode api only decode one layer.
For example, a message like this:
message Detail {
...
}
message List {
repeated Detail details;
}
I want to decode a list, i have to manually decode details with a loop:
local list = protobuf.decode('List', buffer)
for _, buf in list.details or {} do
local detail = protobuf.decode(unpack(buf))
end
Is there an api to decode it recursively, or is it possible to implement this using a lua function.
在lua中,test.proto
message A{
enum {
B = 1;
};
};
pb:register 'test.pb'后
在lua怎么拿到 B的值
in function dump_value
pbc_rmessage_integer(m, key, i, NULL); i should be idx ...
I've been trying to make your protobuf working for iOS. At first I couldn't compile on Mac OS X but figured it out by adding dynamic_lookup into CFLAGS. Think you could add this as an option for Mac OS X build
But when I loaded using protobuf.lua in my iOS game (running quick-cocos2dx which is lua) I got this error:
[LUA ERROR] error loading module 'protobuf.c' from file '/usr/local/lib/lua/5.1/protobuf.so':
dlopen(/usr/local/lib/lua/5.1/protobuf.so, 2): no suitable image found. Did find:
/usr/local/lib/lua/5.1/protobuf.so: mach-o, but not built for iOS simulator
Any ideas?
调用luaopen_protobuf_c之后,lua_gettop会返回1,建议加上一个lua_pop的调用,以符合
Lua手册中说的好的编程实践.
Note that the code above is "balanced": at its end, the stack is back to its original configuration. This is considered good programming practice.
gcc -O2 -Wall --shared -o libprotobuf.so -I../.. -I/usr/local/include -L/usr/local/bin -L../../build pbc-lua.c ../../build/libpbc.a -llua
pbc-lua.c: In function ‘_pattern_pack’:
pbc-lua.c:790: warning: unused variable ‘b’
/usr/bin/ld: /tmp/ccA2yIgd.o: relocation R_X86_64_32 against `.rodata' can not be used when making a shared object; recompile with -fPIC
/tmp/ccA2yIgd.o: could not read symbols: Bad value
I already modified /pbc-master/Makefile:
FROM
CFLAGS = -O2
TO
CFLAGS = -O2 -fPIC
Cannot compile on ubuntu 11.10 (x86_64).
Propose following fix:
diff --git a/Makefile b/Makefile
index c71ed25..5b59879 100644
--- a/Makefile
+++ b/Makefile
@@ -39,7 +39,7 @@ define BUILD_temp
$$(TAR).o : | $(BUILD_O)
-include $$(TAR).d
$$(TAR).o : src/$(1)
$(CC) $(CFLAGS) -c -Isrc -I. -o $$@ -MMD $$<
$(CC) $(CFLAGS) -c -Isrc -I. -fPIC -o $$@ -MMD $$<
endef
@@ -55,7 +55,7 @@ define TEST_temp
cd $(BUILD) && $(CC) $(CFLAGS) -I.. -L. -lpbc -o $$(notdir $$@) ../$$<
cd $(BUILD) && $(CC) $(CFLAGS) -I.. -o $$(notdir $$@) ../$$< -L. -lpbc
endef
testLogin.lua 文件内容如下:
require "protobuf"
t = parser.register("MessageType.proto","../../test/aa")
t = parser.register("Scene.proto","../../test/aa")
MessageType.proto 文件内容如下:
package com.proto.msg;
option java_package = "com.proto.msg.common";
option java_outer_classname = "MessageType";
enum CGMessageType{
CG_START_MOVE = 851;
}
Scene.proto 文件内容如下:
package com.proto.msg.scene;
option java_package = "com.proto.msg.scene";
option java_outer_classname = "Scene";
import "MessageType.proto";
message CGStartMove {
required int32 x = 1 ;
required int32 y = 2 ;
}
我尝试了按顺序注册,但还是报错。如果打成一个pb文件的话,实战中某些pb文件会过大,不利于客户端动态更新
使用lua5.1运行testparser.lua脚本:
lua ./testparser.lua
出现 segmentation fault,其他的test脚本以及c的test程序均可正常运行。使用gdb环境执行,得到的输出和调用栈如下:
Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x0000000000000000
0x00007fff887f0403 in strncmp ()
(gdb) bt
#0 0x00007fff887f0403 in strncmp ()
#1 0x000000010020f50b in pbc_wmessage_string (m=0x10011fd80, key=0x100101e48 "type", v=0x10011bef8 "TYPE_STRING", len=11) at wmessage.c:289
#2 0x0000000100201bb9 in _wmessage_string (L=0x100100810) at pbc-lua.c:251
#3 0x000000010004b854 in luaD_precall ()
#4 0x0000000100056091 in luaV_execute ()
#5 0x000000010004b8ff in luaD_call ()
#6 0x000000010004ad45 in luaD_rawrunprotected ()
#7 0x000000010004adb9 in luaD_pcall ()
#8 0x0000000100045623 in lua_pcall ()
#9 0x0000000100001386 in docall ()
#10 0x0000000100001e5b in handle_script ()
#11 0x0000000100002457 in pmain ()
#12 0x000000010004b854 in luaD_precall ()
#13 0x000000010004b8ee in luaD_call ()
#14 0x000000010004ad45 in luaD_rawrunprotected ()
#15 0x000000010004adb9 in luaD_pcall ()
#16 0x000000010004547b in lua_cpcall ()
#17 0x000000010000256f in main ()
(gdb)
其直接崩溃原因是传给strncmp的字符串指针为0x0
我做了一些简单的分析,发现在pbc_wmessage_string这一层m的type是google.protobuf.FieldDescriptorProto
,field的id是5,type是14,我查看了google.protobuf.FieldDescriptorProto
的定义,id为5的字段的确叫做type_name
但是,类型是string,不应该是14。
我没有完整仔细的阅读pbc的全部代码,可能分析并不正确,如果你需要什么额外的信息,请告诉我 :)
BTW:
我仔细看了一下pbc的lua binding代码,有一个疑问,就是,如果用户在decode对象中获取并持有一个sub message的对象,那么其父对象被gc之后,这个子对象应该就会变成野指针吧?例如在test.lua中的一段代码加以演绎:
decode = protobuf.decode("tutorial.Person" , code)
local myphone = decode.phone
decode = nil
print(myphone[1].number
测试代码
// foo.proto
message Foo {
message Bar {
required uint32 value = 1;
};
required Bar bar = 1;
};
// foo.c
struct pbc_wmessage * foo = pbc_wmessage_new(env, "Foo");
struct pbc_wmessage * bar = pbc_wmessage_message(foo, "bar");
pbc_wmessage_integer(bar, "value", 1, 0); // 这里会core掉,因为bar->type == NULL
看了一下代码,问题应该时出在 src/register.c 文件中几次对_register_message的调用时候,参数prefix_sz 填的是 prefix_sz + name_sz + 1
而这里如果prefix_sz = 0 的话,事实上不应该+1,结果导致嵌套message的名字错误
运行几个lua测试文件(test.lua、test2.lua、textparser.lua),结果都是Bus error: 10错误。
通过打日志发现,是protobuf.encode或者protobuf.decode触发的,
再往里跟发现是pbc-lua.c文件里的
static void
new_array(lua_State *L, int id, const char *key)函数
里面的lua_setfield(L, -6 , key);这一行触发的,
水平比较差,没有思路,求大神指点。
-------------------环境
系统:mac osx 10.10
lua版本:5.1 5.14 5.3都试过
protobuff版本: 3.0.0-alpha-3
the buffer type is : cdata<void *>, when call decode method from luajit, I get an error message : protobuf.lua:502: userdata 5 is nil.
lua: ./protobuf.lua:545: register fail
stack traceback:
[C]: in function '_env_register'
./protobuf.lua:545: in function 'register'
test2.lua:6: in main chunk
[C]: ?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.