Giter VIP home page Giter VIP logo

protobuf's Introduction

Node v6.x.x/7.x.x Now Supported

Many thanks to Mário Freitas/imkira for his work. Thanks to work by Nikolai Vavilov/seishun, package uses `node-gyp``, a bundled protobuf lib and compiles on Windows. Currently maintained by Anton Antonov/syndbg.

Protobuf

This is a fork of http://code.google.com/p/protobuf-for-node/

Works/worked with NodeJS versions:

  • 0.8.x,
  • 0.10.x,
  • 4.x.x,
  • 5.x.x,
  • 6.x.x,
  • 7.x.x

To install:

# NPM version (may be slightly out-dated, but surely stable)
> npm install protobuf
# 0.8.x/0.10.x support version
> npm install https://github.com/chrisdew/protobuf.git#c4622aa2b954d43902a1444a5bf99957ac5d10c1
# 4.x.x/5.x.x support version
> npm install https://github.com/chrisdew/protobuf.git#399f22901dadecc8132aa86cc638c7a379bc61b2
# 6.x.x/7.x.x support version
> npm install https://github.com/chrisdew/protobuf.git#235983e4a991e2b25dfce26b4180a84edd2a925e

All the best,

Chris.

P.S. Breaking change in 0.8.6: uint64 and int64 are now read as Javascript Strings, rather than floating point numbers. They can still be set from Javascript Numbers (as well as from string).

P.P.S. Here's an example I did for #29 - most users won't need the complication of bytes fields.

buftest.proto

package com.chrisdew.buftest;

message BufTest {
  optional float num  = 1;
  optional bytes payload = 2;
}

buftest.js

var fs = require('fs');
var Schema = require('protobuf').Schema;

// "schema" contains all message types defined in buftest.proto|desc.
var schema = new Schema(fs.readFileSync('buftest.desc'));

// The "BufTest" message.
var BufTest = schema['com.chrisdew.buftest.BufTest'];

var ob = { num: 42 };
ob.payload = new Buffer("Hello World");

var proto = BufTest.serialize(ob);
console.log('proto.length:', proto.length);

var outOb = BufTest.parse(proto);
console.log('unserialised:', JSON.stringify(outOb));

var payload = new Buffer(outOb.payload);
console.log(payload);

Makefile: (second line begins with a TAB not spaces)

all:
    protoc --descriptor_set_out=buftest.desc --include_imports buftest.proto

output:

$ node buftest.js 
proto.length: 18
unserialised: {"num":42,"payload":{"0":72,"1":101,"2":108,"3":108,"4":111,"5":32,"6":87,"7":111,"8":114,"9":108,"10":100,"length":11}}
payload: <Buffer 48 65 6c 6c 6f 20 57 6f 72 6c 64>

Older instructions for use with the NodeJS 0.6.x series.

Prerequisites:

NodeJS v0.6.X npm

To install on Ubuntu and OSX:

The first steps are to build and install Google's protobuf library. Make sure you have the right version by running "protoc --version" after the install.

wget http://protobuf.googlecode.com/files/protobuf-2.4.1.tar.gz
tar -xzvf protobuf-2.4.1.tar.gz
cd protobuf-2.4.1/
./configure && make && sudo make install
cd

This installs the npm package.

npm install protobuf

For Ubuntu, update library paths.

sudo ldconfig

For OSX, you might need to add the path:

export DYLD_LIBRARY_PATH=/home/chris/node_modules/protobuf/build/Release:/usr/local/lib:$DYLD_LIBRARY_PATH

And test that it works... Run node, try

require('protobuf');

you should see:

{ Schema: [Function: Schema] }

As seen from the instructions above, this is my first attempt at packaging a slightly complex C++ module for NPM.

If you can help me simplify these instructions, please submit a patch.

Good luck,

Chris.

protobuf's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

protobuf's Issues

Cannot work with 64-bit ints

This library tries to represent 64-bit signed and unsigned integers as Numbers, which leads to precision problems. Instead, it should use two 32-bit integers, a string or (probably the best way) a buffer.

Node cannot find the protobuf module

Compile seems to work out except for a few warnings but node can't find the module when requiring.

rsa:pbtest bhelx$ npm install protobuf
npm http GET https://registry.npmjs.org/protobuf
npm http 304 https://registry.npmjs.org/protobuf

> [email protected] install /Users/bhelx/node_modules/protobuf
> PROTOBUF=/usr/local/lib node-waf configure clean build

Checking for program g++ or c++          : /usr/bin/g++ 
Checking for program cpp                 : /usr/bin/cpp 
Checking for program ar                  : /usr/bin/ar 
Checking for program ranlib              : /usr/bin/ranlib 
Checking for g++                         : ok  
Checking for node path                   : ok /usr/local/lib/node_modules/ 
Checking for node prefix                 : ok /usr/local 
'configure' finished successfully (0.044s)
'clean' finished successfully (0.005s)
Waf: Entering directory `/Users/bhelx/node_modules/protobuf/build'
[1/7] cxx: protobuf_for_node.cc -> build/Release/protobuf_for_node_1.o
[2/7] cxx: addon.cc -> build/Release/addon_2.o
[3/7] cxx: example/protoservice.pb.cc -> build/Release/example/protoservice.pb_3.o
[4/7] cxx: example/protoservice.cc -> build/Release/example/protoservice_3.o
[5/7] cxx_link: build/Release/protobuf_for_node_1.o -> build/Release/libprotobuf_for_node_lib.dylib
ld: warning: directory '/usr/local/lib/lib' following -L not found
[6/7] cxx_link: build/Release/example/protoservice.pb_3.o build/Release/example/protoservice_3.o -> build/Release/protoservice.node
[7/7] cxx_link: build/Release/addon_2.o -> build/Release/protobuf_for_node.node
ld: warning: directory '/usr/local/lib/lib' following -L not found
ld: warning: directory '/usr/local/lib/lib' following -L not found
Waf: Leaving directory `/Users/bhelx/node_modules/protobuf/build'
'build' finished successfully (4.046s)
[email protected] ../../../node_modules/protobuf
rsa:pbtest bhelx$ ls ../../../node_modules/protobuf/build/Release/
total 1736
drwxr-xr-x  8 bhelx  staff     272 Apr 22 14:58 .
drwxr-xr-x  6 bhelx  staff     204 Apr 22 14:58 ..
-rw-r--r--  1 bhelx  staff   45184 Apr 22 14:58 addon_2.o
drwxr-xr-x  4 bhelx  staff     136 Apr 22 14:58 example
-rwxr-xr-x  1 bhelx  staff   79848 Apr 22 14:58 libprotobuf_for_node_lib.dylib
-rwxr-xr-x  1 bhelx  staff    9064 Apr 22 14:58 protobuf_for_node.node
-rw-r--r--  1 bhelx  staff  664688 Apr 22 14:58 protobuf_for_node_1.o
-rwxr-xr-x  1 bhelx  staff   77312 Apr 22 14:58 protoservice.node

rsa:pbtest bhelx$ node
> require('protobuf')
Error: Cannot find module 'protobuf'
    at Function._resolveFilename (module.js:332:11)
    at Function._load (module.js:279:25)
    at Module.require (module.js:354:17)
    at require (module.js:370:17)
    at repl:1:2
    at REPLServer.eval (repl.js:80:21)
    at Interface.<anonymous> (repl.js:182:12)
    at Interface.emit (events.js:67:17)
    at Interface._onLine (readline.js:162:10)
    at Interface._line (readline.js:426:8)

Do you think it has something to do with those warnings?

on MacOS, node v4.4.2:: [Release/obj.target/protobuf_for_node/protobuf_for_node.o] Error 1

5 warnings generated.
  CXX(target) Release/obj.target/protobuf_lite/protobuf/src/google/protobuf/io/zero_copy_stream.o
  CXX(target) Release/obj.target/protobuf_lite/protobuf/src/google/protobuf/io/zero_copy_stream_impl_lite.o
../protobuf/src/google/protobuf/io/zero_copy_stream_impl_lite.cc:159:16: warning: comparison of integers of different signs: 'int' and 'size_type' (aka 'unsigned long') [-Wsign-compare]
  if (old_size < target_->capacity()) {
      ~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~
../protobuf/src/google/protobuf/io/zero_copy_stream_impl_lite.cc:179:3: warning: comparison of integers of different signs: 'int' and 'size_type' (aka 'unsigned long') [-Wsign-compare]
  GOOGLE_CHECK_LE(count, target_->size());
  ^               ~~~~~  ~~~~~~~~~~~~~~~
../protobuf/src/google/protobuf/stubs/common.h:712:48: note: expanded from macro 'GOOGLE_CHECK_LE'
#define GOOGLE_CHECK_LE(A, B) GOOGLE_CHECK((A) <= (B))
                                            ~  ^   ~
../protobuf/src/google/protobuf/stubs/common.h:708:26: note: expanded from macro 'GOOGLE_CHECK'
  GOOGLE_LOG_IF(FATAL, !(EXPRESSION)) << "CHECK failed: " #EXPRESSION ": "
                         ^~~~~~~~~~
../protobuf/src/google/protobuf/stubs/common.h:705:5: note: expanded from macro 'GOOGLE_LOG_IF'
  !(CONDITION) ? (void)0 : GOOGLE_LOG(LEVEL)
    ^~~~~~~~~
2 warnings generated.
  LIBTOOL-STATIC Release/protobuf_lite.a
  CXX(target) Release/obj.target/protobuf_for_node/protobuf_for_node.o
In file included from ../protobuf_for_node.cc:22:
../node_modules/nan/nan.h:337:54: error: too many arguments to function call, expected at most 2, have 4
    return v8::Signature::New(nan_isolate, receiver, argc, argv);
           ~~~~~~~~~~~~~~~~~~                        ^~~~~~~~~~
/Users/ty/.node-gyp/4.4.2/include/node/v8.h:4675:3: note: 'New' declared here
  static Local<Signature> New(
  ^
In file included from ../protobuf_for_node.cc:22:
../node_modules/nan/nan.h:409:19: error: no type named 'ExternalAsciiStringResource' in 'v8::String'; did you mean 'ExternalStringResource'?
      v8::String::ExternalAsciiStringResource *resource) {
      ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~
                  ExternalStringResource
/Users/ty/.node-gyp/4.4.2/include/node/v8.h:2210:19: note: 'ExternalStringResource' declared here
  class V8_EXPORT ExternalStringResource
                  ^
In file included from ../protobuf_for_node.cc:22:
../node_modules/nan/nan.h:408:36: error: redefinition of 'NanNew'
  NAN_INLINE v8::Local<v8::String> NanNew(
                                   ^
../node_modules/nan/nan.h:403:36: note: previous definition is here
  NAN_INLINE v8::Local<v8::String> NanNew(
                                   ^
../node_modules/nan/nan.h:427:3: error: redefinition of 'NanNew'
  NanNew<v8::StringObject, v8::Handle<v8::String> >(
  ^
../node_modules/nan/nan.h:420:3: note: previous definition is here
  NanNew<v8::StringObject, v8::Local<v8::String> >(
  ^
../node_modules/nan/nan.h:444:36: error: redefinition of 'NanNew'
  NAN_INLINE v8::Local<v8::RegExp> NanNew(
                                   ^
../node_modules/nan/nan.h:438:36: note: previous definition is here
  NAN_INLINE v8::Local<v8::RegExp> NanNew(
                                   ^
../node_modules/nan/nan.h:456:36: error: redefinition of 'NanNew'
  NAN_INLINE v8::Local<v8::RegExp> NanNew(
                                   ^
../node_modules/nan/nan.h:450:36: note: previous definition is here
  NAN_INLINE v8::Local<v8::RegExp> NanNew(
                                   ^
../node_modules/nan/nan.h:767:13: error: no member named 'smalloc' in namespace 'node'
    , node::smalloc::FreeCallback callback
      ~~~~~~^
../node_modules/nan/nan.h:777:12: error: no matching function for call to 'New'
    return node::Buffer::New(nan_isolate, data, size);
           ^~~~~~~~~~~~~~~~~
/Users/ty/.node-gyp/4.4.2/include/node/node_buffer.h:31:40: note: candidate function not viable: no known conversion from 'uint32_t' (aka 'unsigned int') to 'enum encoding' for 3rd argument
NODE_EXTERN v8::MaybeLocal<v8::Object> New(v8::Isolate* isolate,
                                       ^
/Users/ty/.node-gyp/4.4.2/include/node/node_buffer.h:43:40: note: candidate function not viable: 2nd argument ('const char *') would lose const qualifier
NODE_EXTERN v8::MaybeLocal<v8::Object> New(v8::Isolate* isolate,
                                       ^
/Users/ty/.node-gyp/4.4.2/include/node/node_buffer.h:28:40: note: candidate function not viable: requires 2 arguments, but 3 were provided
NODE_EXTERN v8::MaybeLocal<v8::Object> New(v8::Isolate* isolate, size_t length);
                                       ^
/Users/ty/.node-gyp/4.4.2/include/node/node_buffer.h:36:40: note: candidate function not viable: requires 5 arguments, but 3 were provided
NODE_EXTERN v8::MaybeLocal<v8::Object> New(v8::Isolate* isolate,
                                       ^
In file included from ../protobuf_for_node.cc:22:
../node_modules/nan/nan.h:781:12: error: no viable conversion from returned value of type 'v8::MaybeLocal<v8::Object>' to function return type 'v8::Local<v8::Object>'
    return node::Buffer::New(nan_isolate, size);
           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/ty/.node-gyp/4.4.2/include/node/v8.h:210:7: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'v8::MaybeLocal<v8::Object>' to 'const v8::Local<v8::Object> &' for 1st argument
class Local {
      ^
/Users/ty/.node-gyp/4.4.2/include/node/v8.h:210:7: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'v8::MaybeLocal<v8::Object>' to 'v8::Local<v8::Object> &&' for 1st argument
/Users/ty/.node-gyp/4.4.2/include/node/v8.h:214:13: note: candidate template ignored: could not match 'Local' against 'MaybeLocal'
  V8_INLINE Local(Local<S> that)
            ^
/Users/ty/.node-gyp/4.4.2/include/node/v8.h:326:13: note: candidate template ignored: could not match 'S *' against 'v8::MaybeLocal<v8::Object>'
  V8_INLINE Local(S* that)
            ^
In file included from ../protobuf_for_node.cc:22:
../node_modules/nan/nan.h:788:26: error: no member named 'Use' in namespace 'node::Buffer'
    return node::Buffer::Use(nan_isolate, data, size);
           ~~~~~~~~~~~~~~^
../node_modules/nan/nan.h:1728:12: error: no member named 'IsExternalAscii' in 'v8::String'; did you mean 'IsExternal'?
  if (str->IsExternalAscii()) {
           ^~~~~~~~~~~~~~~
           IsExternal
/Users/ty/.node-gyp/4.4.2/include/node/v8.h:2174:8: note: 'IsExternal' declared here
  bool IsExternal() const;
       ^
In file included from ../protobuf_for_node.cc:22:
../node_modules/nan/nan.h:1729:23: error: no type named 'ExternalAsciiStringResource' in 'v8::String'; did you mean 'ExternalStringResource'?
    const v8::String::ExternalAsciiStringResource* ext;
          ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~
                      ExternalStringResource
/Users/ty/.node-gyp/4.4.2/include/node/v8.h:2210:19: note: 'ExternalStringResource' declared here
  class V8_EXPORT ExternalStringResource
                  ^
In file included from ../protobuf_for_node.cc:22:
../node_modules/nan/nan.h:1730:16: error: no member named 'GetExternalAsciiStringResource' in 'v8::String'
    ext = str->GetExternalAsciiStringResource();
          ~~~  ^
../node_modules/nan/nan.h:1731:11: error: assigning to 'const char *' from incompatible type 'const uint16_t *' (aka 'const unsigned short *')
    *data = ext->data();
          ^ ~~~~~~~~~~~

Install errors

I've tried installing on a debian machine, and an ubuntu machine and they both output the same error and refuse to install

npm http GET https://registry.npmjs.org/protobuf
npm http 304 https://registry.npmjs.org/protobuf
npm WARN engine [email protected]: wanted: {"node":">= 0.8.0 < 0.9.0"} current: {"node":"v0.11.2-pre","npm":"1.2.20"})

[email protected] install /opt/websteam/node_modules/protobuf
node-gyp rebuild

gyp ERR! configure error
gyp ERR! stack Error: "pre" versions of node cannot be installed, use the --nodedir f
gyp ERR! stack at install (/usr/local/lib/node_modules/npm/node_modules/node-gyp/
gyp ERR! stack at Object.self.commands.(anonymous function) [as install] (/usr/lo
gyp ERR! stack at getNodeDir (/usr/local/lib/node_modules/npm/node_modules/node-g
gyp ERR! stack at /usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/confi
gyp ERR! stack at ChildProcess.exithandler (child_process.js:630:7)
gyp ERR! stack at ChildProcess.EventEmitter.emit (events.js:100:17)
gyp ERR! stack at maybeClose (child_process.js:730:16)
gyp ERR! stack at Socket. (child_process.js:943:11)
gyp ERR! stack at Socket.EventEmitter.emit (events.js:97:17)
gyp ERR! stack at Pipe.close (net.js:455:12)
gyp ERR! System Linux 2.6.32-042stab072.10
gyp ERR! command "node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/no
gyp ERR! cwd /opt/websteam/node_modules/protobuf
gyp ERR! node -v v0.11.2-pre
gyp ERR! node-gyp -v v0.9.5
gyp ERR! not ok
npm ERR! weird error 1
npm ERR! not ok code 0

Support for repeated string fields is not working

Not sure if the problem is in serialization or de-serialization logic but when I send a message with a repeated string field I get back a single string instead of an array. To be more exact, the repeated string field is set by a string array in the outgoing message but I got back a comma-separated string where each comma-separated part is an element of the original string array.

Here is the proto file:
message TT {
repeated string str = 1;
optional uint32 uu = 2;
}

Here is a simple test I did in node (v0.10.4):
$ node

var fs = require('fs');
undefined
var Schema = require('protobuf').Schema;
undefined
Schema
[Function: Schema]
var schema = new Schema(fs.readFileSync('t.desc'))
undefined
schema
{}
var tt = schema['TT']
undefined
var msg = {str: ['abc', '123'], uu: 23}
undefined
var buf = tt.serialize(msg)
undefined
buf
<SlowBuffer 0a 07 61 62 63 2c 31 32 33 10 17>
tt.parse(buf)
{ str: 'abc,123', uu: 23 }
var m = tt.parse(buf)
undefined
m.str
'abc,123'

[] serialize, then parse, but not get [], is {}, why?

test.proto:
message test{
repeated int32 history = 1;
}


my nodejs code:
var Schema = require('protobuf').Schema;
var schema = new Schema(fs.readFileSync(test.desc'));
var req_ob = {
hisatory: []
};
var Req = schema['test'];
var s = Req.serialize(req_ob);
console.log("request:" + JSON.stringify(req_ob));

var Res = schema['test'];
var r = Req.parse(s);

console.log("response:" + JSON.stringify(r));

console:
request:{"hisatory":[]}
response:{}

why response is {}, not {"hisatory":[]} ???

Does not support parameter name like "a_b_c"

if use parameter name like "a_b_c" , get the exception follow :
Can't parse message of type "***" because it is missing required fields: "a_b_c"

because object.serialize() not handle the fields: "a_b_c"
//common.proto
package org.commmon;
message BaseInfo
{
required string id = 1;
required int32 last_go_time = 8;
}
var fs = require('fs');
var Schema = require('protobuf').Schema;

var common = new Schema(fs.readFileSync("./protos/common.desc"));
var Info = fishingjoy3["org.commmon.BaseInfo"];
var pmsg = Info.serialize({
"id": 'id',
"last_go_time": 12452475
});
console.log(pmsg.length);
var jmsg=uInfo.parse(pmsg);
console.log(jmsg);

Are there any plans on adding node 0.10 to the engines?

Hey, while using protobuf and installing my dependencies via npm i have seen an engine mismatch between node 0.10 and the stated engine in the package.json

"engines": { "node": ">= 0.8.0 < 0.9.0" }

Is there any plan to increase it to simething like:

"engines": { "node": ">= 0.8.0 < 0.11.0" }

because its working fine there as well....

Thanks

Compiling Errors on Mac

Unfortunately I tried the new release, seishun's fork and chromium's protobuf gyp.
I get compiling errors for all of them, so I assume it is a system specific problem (although compiling chromium works).

Anyway this is the output of npm install protobuf.

Maybe someone can help, I can't figure out what the problem is
I am running MacOSX 10.7 or can tell me howto get more verbose output from gyp to atleast know how he configures the build.

Edit:
On ubuntu everything works fine. I deinstalled all ports from macports completely just in case, but no change...

Malformed descriptor

var schema = new Schema(fs.readFileSync('./buftest.proto'));
Error: Malformed descriptor
at Error (native)
at Object. (D:\mytest\game\test.js:76:14)
at Module._compile (module.js:460:26)
at Object.Module._extensions..js (module.js:478:10)
at Module.load (module.js:355:32)
at Function.Module._load (module.js:310:12)
at Function.Module.runMain (module.js:501:10)
at startup (node.js:129:16)
at node.js:814:3

buftest.proto

package com.game.buftest;

message BufTest{
optional float num =1;
optional bytes payload =2;
}

I don't know where it was,

extensions?

how are extensions handled? when parsing data with an extension, the parsed result is empty

Cannot serialize object when key has underline

Key hello_world in js object need to be changed to helloWorld. weird..

But what if I have another key named helloWorld ?

message {
  required string hello_world = 1;
  required string helloWorld = 2;
}

example in readme

could you please place a complete example in the readme? for example, how do i use the included protobuf lib to compile protobufs?

Build error on OS X (node v0.6.6)

Mac OS 10.7.2, NodeJs v0.6.6

when invoking
npm install protobuf
got following error:

Waf: Entering directory `/Users/dolphin278/Playground/rfc/library/node_modules/protobuf/build'
[1/7] cxx: protobuf_for_node.cc -> build/Release/protobuf_for_node_1.o
[2/7] cxx: addon.cc -> build/Release/addon_2.o
[3/7] cxx: example/protoservice.pb.cc -> build/Release/example/protoservice.pb_3.o
[4/7] cxx: example/protoservice.cc -> build/Release/example/protoservice_3.o
../protobuf_for_node.cc:28:17: error: eio.h: No such file or directory
../protobuf_for_node.cc: In static member function ‘static void protobuf_for_node::WrappedService::AsyncInvocation::Start(protobuf_for_node::WrappedService*, google::protobuf::MethodDescriptor*, google::protobuf::Message*, google::protobuf::Message*, protobuf_for_node::Schema::Type*, v8::Handle<v8::Function>)’:
../protobuf_for_node.cc:562: error: invalid conversion from ‘int (*)(eio_req*)’ to ‘void (*)(eio_req*)’
../protobuf_for_node.cc:562: error:   initializing argument 1 of ‘eio_req* eio_custom(void (*)(eio_req*), int, int (*)(eio_req*), void*)’
Waf: Leaving directory `/Users/dolphin278/Playground/rfc/library/node_modules/protobuf/build'
Build failed:  -> task failed (err #1): 
    {task: cxx protobuf_for_node.cc -> protobuf_for_node_1.o}

How to intercept parse error messages

Hi!
Upon parsing incorrect protobuf message the parse method throws indescriptive error message and true error message outputs to stdout. Is there any way to intercept this message in my program?

Example:
My code:

try {
    var parsed = schema[inMsgType].parse(data);
} catch(e) {
    log(e)
}

log outputs just a meaningless message:

Error: Malformed message

But stdout has a string

libprotobuf ERROR google/protobuf/message_lite.cc:123] Can't parse message of type "Message" because it is missing required fields: smth.id

In server where stdout is directed to /dev/null this message will be lost. I want to catch and pass this message to my logger. Does anyone know how to do this?

String 'Error: Malformed message' goes from here: protobuf_for_node.cc:268

success = message->ParseFromArray(...)
result = success ? type->ToJs(*message) : v8::ThrowException("Malformed message")

Code from google/protobuf/message_lite.cc:123:

GOOGLE_LOG(ERROR) << InitializationErrorMessage("parse", *message);

So i think there should be some way to catch GOOGLE_LOG messages. Has anyone solved this problem?

How to process bytes field

How do I obtain the data of a 'bytes' field (shown as msg.delegate) into a Buffer?

new Buffer( msg.delegate ); ==> TypeError: First argument needs to be a number, array or string.

Building on Windows (7)

When compiling protobuf on Windows 7, the builder can't find these includes: string, vector, map, algorithm, stack, ...

Example output:

"string": No such file or directory [C:\Users\Jan\Desktop\respotify\node_modules\spotify-web\node_modules\protobuf\build\binding.sln]

What am I doing wrong?

make: *** [Release/obj.target/protobuf_for_node/protobuf_for_node.o] Error

14 errors generated.
make: *** [Release/obj.target/protobuf_for_node/protobuf_for_node.o] Error 1
gyp ERR! build error
gyp ERR! stack Error: make failed with exit code: 2
gyp ERR! stack at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:276:23)
gyp ERR! stack at emitTwo (events.js:100:13)
gyp ERR! stack at ChildProcess.emit (events.js:185:7)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:204:12)
gyp ERR! System Darwin 15.4.0
gyp ERR! command "/usr/local/Cellar/node/5.6.0/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /data1/nodejs/Firehose/node_modules/protobuf
gyp ERR! node -v v5.8.0
gyp ERR! node-gyp -v v3.3.1
gyp ERR! not ok
/data1/nodejs/Firehose
├── [email protected]
├── [email protected]
└── [email protected]

npm WARN Failed to parse json
npm WARN No data, empty input at 1:1
npm WARN
npm WARN ^
npm WARN File: /data1/nodejs/Firehose/package.json
npm WARN Firehose No description
npm WARN Firehose No repository field.
npm WARN Firehose No README data
npm WARN Firehose No license field.
npm ERR! Darwin 15.4.0
npm ERR! argv "/usr/local/Cellar/node/5.6.0/bin/node" "/usr/local/bin/npm" "install" "protobuf"
npm ERR! node v5.8.0
npm ERR! npm v3.8.3
npm ERR! code ELIFECYCLE

npm ERR! [email protected] install: node-gyp rebuild
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script 'node-gyp rebuild'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the protobuf package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node-gyp rebuild
npm ERR! You can get information on how to open an issue for this project with:
npm ERR! npm bugs protobuf
npm ERR! Or if that isn't available, you can get their info via:
npm ERR! npm owner ls protobuf
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR! /data1/nodejs/Firehose/npm-debug.log

why?

When parsing, there is no way to get int value of an enum

Right now when I parse a message in node, the enum value is set to string. However, string is not an efficient representation of enum. It takes longer to compare, it takes more space in the database, etc.

Possible solutions:

  1. Provide a mapping from string to int.
  2. Set option to always return int for enums.
  3. Generate a simple module that contains constants for all the enums in the protobuf.

Solutions #2 + #3 are likely to generate the nicest and most efficient server-side code.

Fails to build on OSX

OS: 16.4.0 Darwin Kernel Version 16.4.0: Thu Dec 22 22:53:21 PST 2016; root:xnu3789.41.3~3/RELEASE_X86_64 x86_64
npm: 4.3.0
node: v7.6.0

Apart from a shitload of warning about for-looping over and comparing integers and unsigned long.. There is those pesky error-messages that stops the whole thing:

../protobuf/src/google/protobuf/message.cc:130:60: error: implicit instantiation of undefined template
'std::__1::basic_istream<char, std::__1::char_traits >'
return ParseFromZeroCopyStream(&zero_copy_input) && input->eof();
^
/Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/iosfwd:109:33: note: template is declared here
class _LIBCPP_TYPE_VIS_ONLY basic_istream;
^
../protobuf/src/google/protobuf/message.cc:135:67: error: implicit instantiation of undefined template
'std::__1::basic_istream<char, std::__1::char_traits >'
return ParsePartialFromZeroCopyStream(&zero_copy_input) && input->eof();
^
/Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/iosfwd:109:33: note: template is declared here
class _LIBCPP_TYPE_VIS_ONLY basic_istream;
^
../protobuf/src/google/protobuf/message.cc:175:16: error: implicit instantiation of undefined template
'std::__1::basic_ostream<char, std::__1::char_traits >'
return output->good();
^
/Library/Developer/CommandLineTools/usr/bin/../include/c++/v1/iosfwd:111:33: note: template is declared here
class _LIBCPP_TYPE_VIS_ONLY basic_ostream;

Error when attempting to build a 32 bit version

I am trying to use this module with node-webkit, but after some debugging im stuck at the following error: This is after running npm install --target=0.10.22 --arch=ia32

>> dyld: lazy symbol binding failed: Symbol not found: __ZN2v86String3NewEPKci
>>   Referenced from: /Users/james/projects/deildu-time/dist/DeilduTime.app/Contents/Resources/app.nw/node_modules/castv2-client/node_modules/castv2/node_modules/protobuf/build/Release/protobuf_for_node.node
>>   Expected in: dynamic lookup
>>
>> dyld: Symbol not found: __ZN2v86String3NewEPKci
>>   Referenced from: /Users/james/projects/deildu-
>> time/dist/DeilduTime.app/Contents/Resources/app.nw/node_modules/castv2-client/node_modules/castv2/node_modules/protobuf/build/Release/protobuf_for_node.node
>>   Expected in: dynamic lookup
>> Exited with code: 6.

Builds broken on macOS Mojave

Currently installing protobuf-node is broken on macOS Mojave:

Build logs:

$ npm install --save protobuf

> [email protected] install /Users/james/Projects/cloud-admin/node_modules/protobuf
> node-gyp rebuild

protobuf/protobuf_full_do_not_use.target.mk:186: warning: overriding commands for target `Release/protobuf_full_do_not_use.a'
protobuf/protobuf_full_do_not_use.host.mk:200: warning: ignoring old commands for target `Release/protobuf_full_do_not_use.a'
protobuf/protobuf_lite.target.mk:180: warning: overriding commands for target `Release/protobuf_lite.a'
protobuf/protobuf_lite.host.mk:194: warning: ignoring old commands for target `Release/protobuf_lite.a'
  CXX(host) Release/obj.host/protobuf_full_do_not_use/protobuf/src/google/protobuf/stubs/strutil.o
../protobuf/src/google/protobuf/stubs/strutil.cc:687:21: warning: unused function 'PlaceNum' [-Wunused-function]
static inline char* PlaceNum(char* p, int num, char prev_sep) {
                    ^
In file included from ../protobuf/src/google/protobuf/stubs/strutil.cc:33:
../protobuf/src/google/protobuf/stubs/strutil.h:307:25: warning: unused typedef 'sizeof_int64_is_not_sizeof_long_long' [-Wunused-local-typedef]
                        sizeof_int64_is_not_sizeof_long_long);
                        ^
../protobuf/src/google/protobuf/stubs/strutil.h:313:25: warning: unused typedef 'sizeof_uint64_is_not_sizeof_long_long' [-Wunused-local-typedef]
                        sizeof_uint64_is_not_sizeof_long_long);
                        ^
../protobuf/src/google/protobuf/stubs/strutil.cc:1013:39: warning: unused typedef 'DBL_DIG_is_too_big' [-Wunused-local-typedef]
  GOOGLE_COMPILE_ASSERT(DBL_DIG < 20, DBL_DIG_is_too_big);
                                      ^
../protobuf/src/google/protobuf/stubs/strutil.cc:1068:39: warning: unused typedef 'FLT_DIG_is_too_big' [-Wunused-local-typedef]
  GOOGLE_COMPILE_ASSERT(FLT_DIG < 10, FLT_DIG_is_too_big);
                                      ^
5 warnings generated.
  CXX(host) Release/obj.host/protobuf_full_do_not_use/protobuf/src/google/protobuf/stubs/substitute.o
../protobuf/src/google/protobuf/stubs/substitute.cc:129:3: warning: comparison of integers of different signs: 'long' and 'std::__1::basic_string<char,
      std::__1::char_traits<char>, std::__1::allocator<char> >::size_type' (aka 'unsigned long') [-Wsign-compare]
  GOOGLE_DCHECK_EQ(target - output->data(), output->size());
  ^                ~~~~~~~~~~~~~~~~~~~~~~~  ~~~~~~~~~~~~~~
../protobuf/src/google/protobuf/stubs/common.h:733:26: note: expanded from macro 'GOOGLE_DCHECK_EQ'
#define GOOGLE_DCHECK_EQ GOOGLE_CHECK_EQ
                         ^
../protobuf/src/google/protobuf/stubs/common.h:709:48: note: expanded from macro 'GOOGLE_CHECK_EQ'
#define GOOGLE_CHECK_EQ(A, B) GOOGLE_CHECK((A) == (B))
                                            ~  ^   ~
../protobuf/src/google/protobuf/stubs/common.h:708:26: note: expanded from macro 'GOOGLE_CHECK'
  GOOGLE_LOG_IF(FATAL, !(EXPRESSION)) << "CHECK failed: " #EXPRESSION ": "
                         ^~~~~~~~~~
../protobuf/src/google/protobuf/stubs/common.h:705:5: note: expanded from macro 'GOOGLE_LOG_IF'
  !(CONDITION) ? (void)0 : GOOGLE_LOG(LEVEL)
    ^~~~~~~~~
In file included from ../protobuf/src/google/protobuf/stubs/substitute.cc:33:
In file included from ../protobuf/src/google/protobuf/stubs/substitute.h:36:
../protobuf/src/google/protobuf/stubs/strutil.h:307:25: warning: unused typedef 'sizeof_int64_is_not_sizeof_long_long' [-Wunused-local-typedef]
                        sizeof_int64_is_not_sizeof_long_long);
                        ^
../protobuf/src/google/protobuf/stubs/strutil.h:313:25: warning: unused typedef 'sizeof_uint64_is_not_sizeof_long_long' [-Wunused-local-typedef]
                        sizeof_uint64_is_not_sizeof_long_long);
                        ^
3 warnings generated.
  CXX(host) Release/obj.host/protobuf_full_do_not_use/protobuf/src/google/protobuf/stubs/structurally_valid.o
  CXX(host) Release/obj.host/protobuf_full_do_not_use/protobuf/src/google/protobuf/descriptor.o
In file included from ../protobuf/src/google/protobuf/descriptor.cc:35:
In file included from ../protobuf/src/google/protobuf/stubs/hash.h:43:
/Library/Developer/CommandLineTools/usr/include/c++/v1/ext/hash_map:213:5: warning: Use of the header <ext/hash_map> is deprecated. Migrate to <unordered_map> [-W#warnings]
#   warning Use of the header <ext/hash_map> is deprecated.  Migrate to <unordered_map>
    ^
In file included from ../protobuf/src/google/protobuf/descriptor.cc:35:
In file included from ../protobuf/src/google/protobuf/stubs/hash.h:44:
/Library/Developer/CommandLineTools/usr/include/c++/v1/ext/hash_set:205:5: warning: Use of the header <ext/hash_set> is deprecated. Migrate to <unordered_set> [-W#warnings]
#   warning Use of the header <ext/hash_set> is deprecated.  Migrate to <unordered_set>
    ^
../protobuf/src/google/protobuf/descriptor.cc:134:21: warning: comparison of integers of different signs: 'int' and 'std::__1::basic_string<char, std::__1::char_traits<char>,
      std::__1::allocator<char> >::size_type' (aka 'unsigned long') [-Wsign-compare]
  for (int i = 0; i < input.size(); i++) {
                  ~ ^ ~~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor.cc:492:21: warning: comparison of integers of different signs: 'int' and 'std::__1::vector<void *, std::__1::allocator<void *>
      >::size_type' (aka 'unsigned long') [-Wsign-compare]
  for (int i = 0; i < allocations_.size(); i++) {
                  ~ ^ ~~~~~~~~~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor.cc:516:21: warning: comparison of integers of different signs: 'int' and 'std::__1::vector<const char *, std::__1::allocator<const
      char *> >::size_type' (aka 'unsigned long') [-Wsign-compare]
  for (int i = 0; i < symbols_after_checkpoint_.size(); i++) {
                  ~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor.cc:519:21: warning: comparison of integers of different signs: 'int' and 'std::__1::vector<const char *, std::__1::allocator<const
      char *> >::size_type' (aka 'unsigned long') [-Wsign-compare]
  for (int i = 0; i < files_after_checkpoint_.size(); i++) {
                  ~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor.cc:522:21: warning: comparison of integers of different signs: 'int' and 'std::__1::vector<std::__1::pair<const
      google::protobuf::Descriptor *, int>, std::__1::allocator<std::__1::pair<const google::protobuf::Descriptor *, int> > >::size_type' (aka 'unsigned long') [-Wsign-compare]
  for (int i = 0; i < extensions_after_checkpoint_.size(); i++) {
                  ~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor.cc:536:50: warning: comparison of integers of different signs: 'int' and 'std::__1::vector<void *, std::__1::allocator<void *>
      >::size_type' (aka 'unsigned long') [-Wsign-compare]
  for (int i = allocations_before_checkpoint_; i < allocations_.size(); i++) {
                                               ~ ^ ~~~~~~~~~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor.cc:989:25: warning: comparison of integers of different signs: 'int' and 'std::__1::vector<int, std::__1::allocator<int> >::size_type'
      (aka 'unsigned long') [-Wsign-compare]
      for (int i = 0; i < numbers.size(); ++i) {
                      ~ ^ ~~~~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor.cc:1498:21: warning: comparison of integers of different signs: 'int' and 'std::__1::vector<const google::protobuf::FieldDescriptor *,
      std::__1::allocator<const google::protobuf::FieldDescriptor *> >::size_type' (aka 'unsigned long') [-Wsign-compare]
  for (int i = 0; i < fields.size(); i++) {
                  ~ ^ ~~~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor.cc:1534:23: warning: comparison of integers of different signs: 'int' and 'std::__1::vector<std::__1::basic_string<char>,
      std::__1::allocator<std::__1::basic_string<char> > >::size_type' (aka 'unsigned long') [-Wsign-compare]
    for (int i = 0; i < all_options.size(); i++) {
                    ~ ^ ~~~~~~~~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor.cc:2361:20: warning: comparison of integers of different signs: 'int' and 'const typename basic_string<char, char_traits<char>,
      allocator<char> >::size_type' (aka 'const unsigned long') [-Wsign-compare]
  if (name_dot_pos == string::npos) {
      ~~~~~~~~~~~~ ^  ~~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor.cc:2607:23: warning: comparison of integers of different signs: 'int' and 'std::__1::basic_string<char, std::__1::char_traits<char>,
      std::__1::allocator<char> >::size_type' (aka 'unsigned long') [-Wsign-compare]
    for (int i = 0; i < name.size(); i++) {
                    ~ ^ ~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor.cc:2623:21: warning: comparison of integers of different signs: 'int' and 'std::__1::basic_string<char, std::__1::char_traits<char>,
      std::__1::allocator<char> >::size_type' (aka 'unsigned long') [-Wsign-compare]
  for (int i = 0; i < name.size(); i++) {
                  ~ ^ ~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor.cc:2728:21: warning: comparison of integers of different signs: 'int' and 'std::__1::vector<std::__1::basic_string<char>,
      std::__1::allocator<std::__1::basic_string<char> > >::size_type' (aka 'unsigned long') [-Wsign-compare]
  for (int i = 0; i < tables_->pending_files_.size(); i++) {
                  ~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor.cc:2731:16: warning: comparison of integers of different signs: 'int' and 'std::__1::vector<std::__1::basic_string<char>,
      std::__1::allocator<std::__1::basic_string<char> > >::size_type' (aka 'unsigned long') [-Wsign-compare]
      for (; i < tables_->pending_files_.size(); i++) {
             ~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor.cc:3650:17: warning: reference cannot be bound to dereferenced null pointer in well-defined C++ code; comparison may be assumed to
      always evaluate to true [-Wtautological-undefined-compare]
         &file->options() != NULL &&
          ~~~~~~^~~~~~~~~    ~~~~
../protobuf/src/google/protobuf/descriptor.h:1274:1: note: 'options' returns a reference
PROTOBUF_DEFINE_OPTIONS_ACCESSOR(FileDescriptor, FileOptions);
^
../protobuf/src/google/protobuf/descriptor.h:1185:29: note: expanded from macro 'PROTOBUF_DEFINE_OPTIONS_ACCESSOR'
  inline const TYPE& CLASS::options() const { return *options_; }
                            ^
../protobuf/src/google/protobuf/descriptor.cc:211:23: warning: unused variable 'bucket_size' [-Wunused-const-variable]
  static const size_t bucket_size = 4;
                      ^
../protobuf/src/google/protobuf/descriptor.cc:212:23: warning: unused variable 'min_buckets' [-Wunused-const-variable]
  static const size_t min_buckets = 8;
                      ^
In file included from ../protobuf/src/google/protobuf/descriptor.cc:54:
../protobuf/src/google/protobuf/stubs/strutil.h:307:25: warning: unused typedef 'sizeof_int64_is_not_sizeof_long_long' [-Wunused-local-typedef]
                        sizeof_int64_is_not_sizeof_long_long);
                        ^
../protobuf/src/google/protobuf/stubs/strutil.h:313:25: warning: unused typedef 'sizeof_uint64_is_not_sizeof_long_long' [-Wunused-local-typedef]
                        sizeof_uint64_is_not_sizeof_long_long);
                        ^
21 warnings generated.
  CXX(host) Release/obj.host/protobuf_full_do_not_use/protobuf/src/google/protobuf/descriptor.pb.o
  CXX(host) Release/obj.host/protobuf_full_do_not_use/protobuf/src/google/protobuf/descriptor_database.o
../protobuf/src/google/protobuf/descriptor_database.cc:303:21: warning: comparison of integers of different signs: 'int' and 'std::__1::vector<void *, std::__1::allocator<void
      *> >::size_type' (aka 'unsigned long') [-Wsign-compare]
  for (int i = 0; i < files_to_delete_.size(); i++) {
                  ~ ^ ~~~~~~~~~~~~~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor_database.cc:441:21: warning: comparison of integers of different signs: 'int' and 'std::__1::vector<const
      google::protobuf::FieldDescriptor *, std::__1::allocator<const google::protobuf::FieldDescriptor *> >::size_type' (aka 'unsigned long') [-Wsign-compare]
  for (int i = 0; i < extensions.size(); ++i) {
                  ~ ^ ~~~~~~~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor_database.cc:464:21: warning: comparison of integers of different signs: 'int' and
      'std::__1::vector<google::protobuf::DescriptorDatabase *, std::__1::allocator<google::protobuf::DescriptorDatabase *> >::size_type' (aka 'unsigned long') [-Wsign-compare]
  for (int i = 0; i < sources_.size(); i++) {
                  ~ ^ ~~~~~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor_database.cc:475:21: warning: comparison of integers of different signs: 'int' and
      'std::__1::vector<google::protobuf::DescriptorDatabase *, std::__1::allocator<google::protobuf::DescriptorDatabase *> >::size_type' (aka 'unsigned long') [-Wsign-compare]
  for (int i = 0; i < sources_.size(); i++) {
                  ~ ^ ~~~~~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor_database.cc:498:21: warning: comparison of integers of different signs: 'int' and
      'std::__1::vector<google::protobuf::DescriptorDatabase *, std::__1::allocator<google::protobuf::DescriptorDatabase *> >::size_type' (aka 'unsigned long') [-Wsign-compare]
  for (int i = 0; i < sources_.size(); i++) {
                  ~ ^ ~~~~~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor_database.cc:525:21: warning: comparison of integers of different signs: 'int' and
      'std::__1::vector<google::protobuf::DescriptorDatabase *, std::__1::allocator<google::protobuf::DescriptorDatabase *> >::size_type' (aka 'unsigned long') [-Wsign-compare]
  for (int i = 0; i < sources_.size(); i++) {
                  ~ ^ ~~~~~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor_database.cc:237:21: warning: comparison of integers of different signs: 'int' and 'std::__1::basic_string<char,
      std::__1::char_traits<char>, std::__1::allocator<char> >::size_type' (aka 'unsigned long') [-Wsign-compare]
  for (int i = 0; i < name.size(); i++) {
                  ~ ^ ~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor_database.cc:93:8: note: in instantiation of member function 'google::protobuf::SimpleDescriptorDatabase::DescriptorIndex<const
      google::protobuf::FileDescriptorProto *>::ValidateSymbolName' requested here
  if (!ValidateSymbolName(name)) {
       ^
../protobuf/src/google/protobuf/descriptor_database.cc:68:10: note: in instantiation of member function 'google::protobuf::SimpleDescriptorDatabase::DescriptorIndex<const
      google::protobuf::FileDescriptorProto *>::AddSymbol' requested here
    if (!AddSymbol(path + file.message_type(i).name(), value)) return false;
         ^
../protobuf/src/google/protobuf/descriptor_database.cc:264:17: note: in instantiation of member function 'google::protobuf::SimpleDescriptorDatabase::DescriptorIndex<const
      google::protobuf::FileDescriptorProto *>::AddFile' requested here
  return index_.AddFile(*file, file);
                ^
../protobuf/src/google/protobuf/descriptor_database.cc:237:21: warning: comparison of integers of different signs: 'int' and 'std::__1::basic_string<char,
      std::__1::char_traits<char>, std::__1::allocator<char> >::size_type' (aka 'unsigned long') [-Wsign-compare]
  for (int i = 0; i < name.size(); i++) {
                  ~ ^ ~~~~~~~~~~~
../protobuf/src/google/protobuf/descriptor_database.cc:93:8: note: in instantiation of member function
      'google::protobuf::SimpleDescriptorDatabase::DescriptorIndex<std::__1::pair<const void *, int> >::ValidateSymbolName' requested here
  if (!ValidateSymbolName(name)) {
       ^
../protobuf/src/google/protobuf/descriptor_database.cc:68:10: note: in instantiation of member function
      'google::protobuf::SimpleDescriptorDatabase::DescriptorIndex<std::__1::pair<const void *, int> >::AddSymbol' requested here
    if (!AddSymbol(path + file.message_type(i).name(), value)) return false;
         ^
../protobuf/src/google/protobuf/descriptor_database.cc:312:19: note: in instantiation of member function
      'google::protobuf::SimpleDescriptorDatabase::DescriptorIndex<std::__1::pair<const void *, int> >::AddFile' requested here
    return index_.AddFile(file, make_pair(encoded_file_descriptor, size));
                  ^
In file included from ../protobuf/src/google/protobuf/descriptor_database.cc:41:
../protobuf/src/google/protobuf/stubs/strutil.h:307:25: warning: unused typedef 'sizeof_int64_is_not_sizeof_long_long' [-Wunused-local-typedef]
                        sizeof_int64_is_not_sizeof_long_long);
                        ^
../protobuf/src/google/protobuf/stubs/strutil.h:313:25: warning: unused typedef 'sizeof_uint64_is_not_sizeof_long_long' [-Wunused-local-typedef]
                        sizeof_uint64_is_not_sizeof_long_long);
                        ^
10 warnings generated.
  CXX(host) Release/obj.host/protobuf_full_do_not_use/protobuf/src/google/protobuf/dynamic_message.o
In file included from ../protobuf/src/google/protobuf/dynamic_message.cc:66:
In file included from ../protobuf/src/google/protobuf/stubs/hash.h:43:
/Library/Developer/CommandLineTools/usr/include/c++/v1/ext/hash_map:213:5: warning: Use of the header <ext/hash_map> is deprecated. Migrate to <unordered_map> [-W#warnings]
#   warning Use of the header <ext/hash_map> is deprecated.  Migrate to <unordered_map>
    ^
In file included from ../protobuf/src/google/protobuf/dynamic_message.cc:66:
In file included from ../protobuf/src/google/protobuf/stubs/hash.h:44:
/Library/Developer/CommandLineTools/usr/include/c++/v1/ext/hash_set:205:5: warning: Use of the header <ext/hash_set> is deprecated. Migrate to <unordered_set> [-W#warnings]
#   warning Use of the header <ext/hash_set> is deprecated.  Migrate to <unordered_set>
    ^
2 warnings generated.
  CXX(host) Release/obj.host/protobuf_full_do_not_use/protobuf/src/google/protobuf/extension_set_heavy.o
../protobuf/src/google/protobuf/extension_set_heavy.cc:362:14: warning: comparison of two values with different enumeration types in switch statement ('FieldDescriptor::Type'
      and 'google::protobuf::internal::WireFormatLite::FieldType') [-Wenum-compare-switch]
        case WireFormatLite::TYPE_MESSAGE:
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
../protobuf/src/google/protobuf/extension_set_heavy.cc:361:14: warning: comparison of two values with different enumeration types in switch statement ('FieldDescriptor::Type'
      and 'google::protobuf::internal::WireFormatLite::FieldType') [-Wenum-compare-switch]
        case WireFormatLite::TYPE_GROUP:
             ^~~~~~~~~~~~~~~~~~~~~~~~~~
../protobuf/src/google/protobuf/extension_set_heavy.cc:360:14: warning: comparison of two values with different enumeration types in switch statement ('FieldDescriptor::Type'
      and 'google::protobuf::internal::WireFormatLite::FieldType') [-Wenum-compare-switch]
        case WireFormatLite::TYPE_BYTES:
             ^~~~~~~~~~~~~~~~~~~~~~~~~~
../protobuf/src/google/protobuf/extension_set_heavy.cc:359:14: warning: comparison of two values with different enumeration types in switch statement ('FieldDescriptor::Type'
      and 'google::protobuf::internal::WireFormatLite::FieldType') [-Wenum-compare-switch]
        case WireFormatLite::TYPE_STRING:
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~
4 warnings generated.
  CXX(host) Release/obj.host/protobuf_full_do_not_use/protobuf/src/google/protobuf/generated_message_reflection.o
  CXX(host) Release/obj.host/protobuf_full_do_not_use/protobuf/src/google/protobuf/message.o
In file included from ../protobuf/src/google/protobuf/message.cc:36:
In file included from ../protobuf/src/google/protobuf/stubs/hash.h:43:
/Library/Developer/CommandLineTools/usr/include/c++/v1/ext/hash_map:213:5: warning: Use of the header <ext/hash_map> is deprecated. Migrate to <unordered_map> [-W#warnings]
#   warning Use of the header <ext/hash_map> is deprecated.  Migrate to <unordered_map>
    ^
In file included from ../protobuf/src/google/protobuf/message.cc:36:
In file included from ../protobuf/src/google/protobuf/stubs/hash.h:44:
/Library/Developer/CommandLineTools/usr/include/c++/v1/ext/hash_set:205:5: warning: Use of the header <ext/hash_set> is deprecated. Migrate to <unordered_set> [-W#warnings]
#   warning Use of the header <ext/hash_set> is deprecated.  Migrate to <unordered_set>
    ^
../protobuf/src/google/protobuf/message.cc:130:60: error: implicit instantiation of undefined template 'std::__1::basic_istream<char, std::__1::char_traits<char> >'
  return ParseFromZeroCopyStream(&zero_copy_input) && input->eof();
                                                           ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/iosfwd:109:32: note: template is declared here
    class _LIBCPP_TEMPLATE_VIS basic_istream;
                               ^
../protobuf/src/google/protobuf/message.cc:135:67: error: implicit instantiation of undefined template 'std::__1::basic_istream<char, std::__1::char_traits<char> >'
  return ParsePartialFromZeroCopyStream(&zero_copy_input) && input->eof();
                                                                  ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/iosfwd:109:32: note: template is declared here
    class _LIBCPP_TEMPLATE_VIS basic_istream;
                               ^
../protobuf/src/google/protobuf/message.cc:175:16: error: implicit instantiation of undefined template 'std::__1::basic_ostream<char, std::__1::char_traits<char> >'
  return output->good();
               ^
/Library/Developer/CommandLineTools/usr/include/c++/v1/iosfwd:111:32: note: template is declared here
    class _LIBCPP_TEMPLATE_VIS basic_ostream;
                               ^
2 warnings and 3 errors generated.
make: *** [Release/obj.host/protobuf_full_do_not_use/protobuf/src/google/protobuf/message.o] Error 1
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:196:23)
gyp ERR! stack     at ChildProcess.emit (events.js:210:5)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:272:12)
gyp ERR! System Darwin 18.6.0
gyp ERR! command "/usr/local/Cellar/node/12.12.0/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/james/Projects/cloud-admin-node_modules/protobuf
gyp ERR! node -v v12.12.0
gyp ERR! node-gyp -v v5.0.3
gyp ERR! not ok
npm WARN enoent ENOENT: no such file or directory, open '/Users/james/Projects/cloud-admin/package.json'
npm WARN [email protected] requires a peer of [email protected] but none is installed. You must install peer dependencies yourself.
npm WARN cloud-admin No description
npm WARN cloud-admin No repository field.
npm WARN cloud-admin No README data
npm WARN cloud-admin No license field.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `node-gyp rebuild`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/james/.npm/_logs/2019-10-27T23_04_05_709Z-debug.log

NodeJS version:

$ node --version
v12.12.0

I see there are some related PRs to fix some warnings #42 and #56 but even after applying these patches to my fork there are still numerous compilation problems. Near as I can tell many of these are because Mojave uses clang by default? I've tried to brew tap / brew install appropriate versions of GCC and set CC and CXX env vars but have not manage to successfully build the protobuf-node module yet :/

Appreciate some help here :)

Required message fields are not handled properly

Everything works if the proto file is this:

package ptest;
message LoginReq {
optional string login_name = 1;
optional string password = 2;
}

But if the login_name field becomes required then deserialization fails and provides the following error:

libprotobuf ERROR google/protobuf/message_lite.cc:123] Can't parse message of type "ptest.LoginReq" because it is missing required fields: login_name

undefined:1
function(self) { var f = this; return function(arg) { return f.call(self,
^
Error: Malformed message
at Function.parse (unknown source)
at Object. (/home/ec2-user/dev/tests/msgs/test.js:14:24)
at Module._compile (module.js:449:26)
at Object.Module._extensions..js (module.js:467:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.runMain (module.js:492:10)
at process.startup.processNextTick.process._tickCallback (node.js:244:9)

Extensions

Extensions don't seem to work. For example, here is my base.proto:

message Request {
  enum RequestType {
    // Let us know you are a player
    AUTH  = 1;

    // These apply to `channels`
    JOIN  = 2;
    LEAVE = 3;

    // This is a passthrough type
    // Pocket will pass this to the server listeners
    // for decoding
    MESSAGE = 4;
  }

  message Auth {
    optional string key = 1;
    optional string verifier = 2;
  }

  message Channel {
    optional string name = 1;
  }

  required RequestType request_type = 1;

  optional Channel channel = 2;
  optional Auth auth = 4;

  extensions 100 to 199;
}

When I try to extend in extension.proto:

import "base.proto";

extend Request {
  required string name = 100;
}

Loading extension.proto and encoding anything always results in blank messages being encoded, causing the decode to fail since required fields are missing.

Can't build on node v0.8.0

I see this error when trying to build the module on node v0.8.0 (Mac OS X 10.7):

 Checking for program g++ or c++          : /usr/bin/g++ 
 Checking for program cpp                 : /usr/bin/cpp 
 Checking for program ar                  : /usr/bin/ar 
 Checking for program ranlib              : /usr/bin/ranlib 
 Checking for g++                         : ok  
 Checking for node path                   : not found 
 Checking for node prefix                 : ok /Users/muir/.nvm/v0.8.0 
 'configure' finished successfully (0.120s)
 'clean' finished successfully (0.011s)
 Waf: Entering directory `/Users/muir/projects/nearbuy/protobuf/build'
 [1/3] cxx: protobuf_for_node.cc -> build/Release/protobuf_for_node_1.o
 [2/3] cxx: addon.cc -> build/Release/addon_1.o
 ../protobuf_for_node.cc: In static member function ‘static void protobuf_for_node::WrappedService::AsyncInvocation::Init()’:
 ../protobuf_for_node.cc:549: error: invalid conversion from ‘void (*)(ev_async*, int)’ to ‘void (*)(ev_loop*, ev_async*, int)’
 ../protobuf_for_node.cc: In static member function ‘static void protobuf_for_node::WrappedService::AsyncInvocation::Done(protobuf_for_node::WrappedService::AsyncInvocation*)’:
 ../protobuf_for_node.cc:605: error: cannot convert ‘ev_async*’ to ‘ev_loop*’ for argument ‘1’ to ‘void ev_async_send(ev_loop*, ev_async*)’
 ../protobuf_for_node.cc: In static member function ‘static int protobuf_for_node::WrappedService::AsyncInvocation::Returned(eio_req*)’:
 ../protobuf_for_node.cc:615: error: cannot convert ‘ev_async*’ to ‘ev_loop*’ for argument ‘1’ to ‘void ev_async_start(ev_loop*, ev_async*)’
 ../protobuf_for_node.cc: In static member function ‘static void protobuf_for_node::WrappedService::AsyncInvocation::InvokeAsyncCallbacks(ev_async*, int)’:
 ../protobuf_for_node.cc:639: error: cannot convert ‘ev_async*’ to ‘ev_loop*’ for argument ‘1’ to ‘void ev_async_stop(ev_loop*, ev_async*)’
 Waf: Leaving directory `/Users/muir/projects/nearbuy/protobuf/build'
 Build failed:  -> task failed (err #1): 
    {task: cxx protobuf_for_node.cc -> protobuf_for_node_1.o}

The problem is that the new ev-emul.h undefines a bunch of important EV_MULTIPLICITY-related macros (in particular, EV_P_). I'm not sure exactly why it does this, but the solution is probably to switch protobuf to uv instead of ev.

Error occurred when I serialize a PB and turned it to base64 and decode it and then parse.

var str = encodeBase64(schema.serialize(pb));
var obj = schema.parse(new Buffer(decodeBase64(str));

And the error is:

undefined:1
function(self) {  var f = this;  return function(arg) {    return f.call(self,
                                                                    ^
Error: Malformed message
    at Function.parse (<anonymous>:1:70)
    at Function.module.exports.createFromPB (/Users/XadillaX/code/node/getui/lib/transparent.js:93:36)
    at Function.module.exports.createFromBase64 (/Users/XadillaX/code/node/getui/lib/transparent.js:106:17)
    at Object.<anonymous> (/Users/XadillaX/code/node/getui/test/template.js:15:25)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Function.Module.runMain (module.js:497:10)
    at startup (node.js:119:16)

How to read enum types and their values?

var cdmSchema = new Schema(fs.readFileSync('../proto/CommonDimensionProto.desc'));
var AppCategory = cdmSchema['com.lge.telepathy.proto.AppCategory'];

AppCategory is a enum class defined in CommonDimensionProto.proto. I can read the enum class? Please help!

Add support for node.js v0.11.13 / v0.12

The protobuf module does not build with node.js v0.11. I just tried it with v0.11.13 which apparently is fairly close to what we'll get for node v0.12

The problem is that there have been changes to node and v8's interfaces for native code. A good summary of what's changed is at https://github.com/rvagg/nan
(That site is actually a library which might be helpful in dealing with the issues).

Does anyone have plans to work on this?

Compile for io.js 1.0.4

Protobuf is not compiling for io.js latest (1.0.4), node-gyp fails with a lot of errors. Latest error is included here:

../protobuf_for_node.cc: In function ‘v8::Local<v8::Function> protobuf_for_node::SchemaConstructor()’:
../protobuf_for_node.cc:724:26: error: base operand of ‘->’ has non-pointer type ‘v8::Persistent<v8::FunctionTemplate>’
     return SchemaTemplate->GetFunction();
                          ^
../protobuf_for_node.cc:725:3: warning: control reaches end of non-void function [-Wreturn-type]
   }
   ^
make: *** [Release/obj.target/protobuf_for_node/protobuf_for_node.o] Error 1
make: Leaving directory `/home/ubuntu/adserver/node_modules/protobuf/build'

protobuf 0.8.6 is recompiled every time

Every time I run "npm install protobuf" it is recompiled. This does not happen with other packages, even those with native code. It makes our deployment process very slow. On Mac OS X, Ubuntu and Debian.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.