socyery / pb4php Goto Github PK

The _check_type() method doesn't seem to be handling namespace properly.

Example proto file:
message BlahClass {
    option string blah = 1;
}

message TestClass {
   message TestGroup {
       optional BlahClass blah =1;
   }
}

Result:
This throws the exception "Protofile type BlahClass is unknown!"

Expected:
BlahClass was defined as the top level namespace and should not have thrown
this error.

I replaced the __check_type() function with this code to fix this issue:
    private function _check_type($type, $array, $path)
    {
        if (isset($this->scalar_types[strtolower($type)]))
            return array(strtolower($type), '');


        $apath = explode(".", $path);
        while ( array_pop($apath ) ) {
            $namespace = join(".", $apath) . "." . $type;

            foreach ($this->m_types as $message)
            {
                if ($message['name'] == $namespace)
                {
                    return array($type, $namespace);
                }
            }
        }

        $namespace = $type;
        foreach ($this->m_types as $message)
        {
            if ($message['name'] == $namespace)
            {
                return array($type, $namespace);
            }
        }

        // @TODO TYPE CHECK
        throw new Exception('Protofile type ' . $type . ' unknown!');
    }

Class file: /message/reader/pb_input_reader.php
Class method: get_message_from()

public function get_message_from($from)
{
    return substr($this->string, $from, $this->pointer - $from);
}

The get_message_from() may return the "FALSE" value if the value of $from 
is greater than or equal to the length of "$this->string".

It is suggested that returing empty string instead of returning "FALSE" 
value. See the following example:

public function get_message_from($from)
{
    $str = substr($this->string, $from, $this->pointer - $from);
    if ($str === FALSE)
    { // empty string
        $str = '';
    }

    return $str;
}

In the parser/pb_parser.php class, parse() method, it was possible to
specifie a destination path for the generated classfile.

Now it puts the class in the current directory from the caller page.

It would be nice to be able to specify it again.

Here is a quick diff..

$ diff pb_parser_old.php pb_parser.php 
37c37
<     public function parse($protofile)

---
>     public function parse($protofile, $classpath='')
55c55
<         $this->_create_class_file( $this->created_php_file_name );

---
>         $this->_create_class_file( $classpath .
$this->created_php_file_name );

What steps will reproduce the problem?
1. my.proto
message my_test {
    required sint32 id = 1;
} 
2. generate php-class:
$test = new PBParser();
$test->parse('./my.proto'); 
3. php-code  
----
<?
require_once('../message/pb_message.php');
require_once('./pb_proto_my.php');
error_reporting(E_ALL);
ini_set('display_errors','On');
$A = new my_test();
$A->set_id(123);

$str = $A->SerializeToString();

$B = new my_test();
$B->parseFromString($str);
echo $B->id();
?>

What is the expected output? What do you see instead?
123

Notice: Object of class PBSignedInt could not be converted to int in 
/Users/max_m/Downloads/protocolbuf/message/type/pb_signed_int.php on line 19

Call Stack:
    0.0005     328212   1. {main}() /Users/max_m/Downloads/protocolbuf/example/test_my.php:0
    0.0052     541284   2. PBMessage->ParseFromString() 
/Users/max_m/Downloads/protocolbuf/example/test_my.php:13
    0.0052     541856   3. PBMessage->_ParseFromArray() 
/Users/max_m/Downloads/protocolbuf/message/pb_message.php:148
    0.0054     542952   4. PBSignedInt->ParseFromArray() 
/Users/max_m/Downloads/protocolbuf/message/pb_message.php:223

What version of the product are you using? On what operating system?
[18:37:48] max_m: ~/Downloads/protocolbuf/example> uname -v
Darwin Kernel Version 9.8.0: Wed Jul 15 16:55:01 PDT 2009; root:xnu-
1228.15.4~1/RELEASE_I386
[18:37:50] max_m: ~/Downloads/protocolbuf/example> php -v
PHP 5.3.0 (cli) (built: Jul  5 2009 23:21:56) 
Copyright (c) 1997-2009 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2009 Zend Technologies
    with Xdebug v2.0.5, Copyright (c) 2002-2008, by Derick Rethans

Please provide any additional information below.
I think PBSignedInt should be inherited from  PBInt (now it inherited from 
PBScalar)
That's solved my problem

When I use the scalar types of double, float, uint32, uint64, fixed32,
fixed64, sfixed32, or sfixed64 it throws the error "Protofile type XXXX
unknown!"

Expected:
I assume it should treat those as PBInt type (or be specifically coded if
needed)

FILE: parser/pb_parser.php
private function _strip_comments(&$string)
{
    $string = preg_replace('/\/\/.+/', '', $string);
    // now replace empty lines and whitespaces in front
    $string = preg_replace('/\\r?\\n\s*/', "\n", $string);
}

The first preg_replace() call in the _strip_comments() function is used to 
strip out comments from a proto message. However, there might be no 
characters after the precending '//'.

Example:
//
// COMMENT
//

So the regular expression should be written in the form of '/\/\/.*/' to 
match any characters (except newline) that has zero or more duplicates.

Added two helper methods to pb_message.php
- get_value(): Convert PBMessage objects to PHP nested objects
- set_value(): Convert PHP nested objects to PBMessage object

In addition, it is also required to make little 
modification to pb_parser.php. See the following generated code as example.

<?php
class PB_User extends PBMessage
{
    var $wired_type = PBMessage::WIRED_LENGTH_DELIMITED;
    public function __construct($reader = null)
    {
        parent::__construct($reader);
        $this->names[1] = 'name';  // <--- Newly Generated Code
        $this->fields[1] = 'PBString';
        $this->values[1] = '';
    }
    function name()
    {
        return $this->_get_value(1);
    }
    function set_name($value)
    {
        return $this->_set_value(1, $value);
    }
}

* Protocol Buffers Message Example:
message PB_User {
    required string name = 1;
}

$pb = new PB_User();
$user = $pb->get_value();
echo $user->name;

$user->name = 'Test';
$pb = new PB_User();
$pb->set_value($user);


class pb_message {

public function get_value()
{
    unset($value);

    foreach ($this->names as $index => $_name)
    {
        if (is_array($this->values[$index]))
        {
            $value->$_name = array();
            foreach ($this->values[$index] as $_value)
            {
                array_push($value->$_name, $_value->get_value());
            }
        }
        else
        {
            $value->$_name = ($this->values[$index]) ? $this->values
[$index]->get_value() : null;
        }
    }

    return $value;
}

public function set_value($value)
{
    foreach ($this->names as $index => $_name)
    {
        if (is_array($this->values[$index]) AND is_array($value->$_name))
        {
            $this->values[$index] = array();
            foreach ($value->$_name as $_value)
            {
                $class_object = new $this->fields[$index]();
                $class_object->set_value($_value);
                $this->values[$index][] = $class_object;
            }
        }
        else
        {
            if ( ! is_object($this->values[$index]))
            {
                $this->values[$index] = new $this->fields[$index]();
            }

            $this->values[$index]->set_value($value->$_name);
        }
    }
}

}

Hi,

The generatation of classes does not work if primitive type is repeated
instead of a message type.


message AddressBook {
  repeated string person = 1;
}

Code generated:

<?php
class AddressBook extends PBMessage
{
  var $wired_type = PBMessage::WIRED_STRING;
  public function __construct($reader=null)
  {
    parent::__construct($reader);
    $this->fields["1"] = "PBString";
    $this->values["1"] = array();
  }
  function person($offset)
  {
    return $this->_get_arr_value("1", $offset);
  }
  function add_person()
  {
    return $this->_add_arr_value("1");
  }
  function person_size()
  {
    return $this->_get_arr_size("1");
  }
}
?>


I can generate a person, but there is no way for me to set any data in it,
as no message object is returned.
Serializing that object will then fail.

I also suspect the same problem when parsing a file having the same kind of
definitions.

If I define two classes in the proto file like so:

message TestClass {
   optional BlahClass blah = 1;
}

message BlahClass {
   optional string text = 1;
}

It will throw the error "Protofile type BlahClass unknown!". I would expect
it to be able to tell BlahClass is defined later in the proto file.

Another example:
message TestClass {
   repeated TestClass test = 1;
}

This also throws the error "Protofile type TestClass unknown!"

What steps will reproduce the problem?
1. Viewing "test.php" on my web server...

What is the expected output? What do you see instead?
Expected Output: Whatever test.php is supposed to display. 
Actual Output: Fatal error: Undefined class constant
'PBMessage::WIRED_STRING' in /var/www/protocolbuf/example/test.php on line 15

What version of the product are you using? On what operating system?
Version 0.25 
OS: Debian 5.0.3 (lenny) 

Please provide any additional information below.
PHP5 installed over an Apache 2.2 HTTP server. 

What steps will reproduce the problem?
1. Create a proto file containing 'option' commands
2. Run the parser
3. Parser will crash

What is the expected output? What do you see instead?
Correct proto-file with the options just ignored

What version of the product are you using? On what operating system?
SVN commit 41 on Linux

Please provide any additional information below.
To fix, add the following to _strip_comments:
$string = preg_replace('/^option\s+.*/m', '', $string);

On parsing .proto file i am receiving this error: Fatal error: Uncaught 
exception 'Exception' with message 'Protofile type float unknown!'

The reproduce the issues here is the proto message:

message Geo {
  optional float lat         = 1;    
  optional float lon         = 2;    
  optional string country    = 3;    
  optional string city       = 4;    
  optional string zip        = 5;
  optional int32 type        = 6;
  optional string continent  = 7;       
  optional string state      = 8;      
  optional int32  dma        = 9;
}


The php code is:

require_once('parser/pb_parser.php');
$test = new PBParser();
$test->parse('test.proto');
var_dump('File parsing done!');


Is there any workaround?

P.S. If i change it to int32 it's working fine without exception

PBMessage::WIRED_STRING
is not define

Please add an explicit encoding conversion routine into the SerializeToString() 
method of PBString class to handle a unicode characters correctly. 
mb_convert_encoding($this->value, "UTF-8") worked for me but I'm not proficient 
in PHP, so there can be a better way.

File "pb_double.php":
-------------------------
<?php
/**
 * @author Dmitry Vorobyev (http://dmitry.vorobyev.name)
 */
class PBDouble extends PBScalar
{
    var $wired_type = PBMessage::WIRED_64BIT;

    public function ParseFromArray()
    {
        $this->value = '';

        // just extract the string
        $pointer = $this->reader->get_pointer();
        $this->reader->add_pointer(8);
        $this->value = unpack('d', $this->reader->get_message_from($pointer));
    }

    /**
     * Serializes type
     */
    public function SerializeToString($rec=-1)
    {
        $string = '';
        if ($rec > -1)
        {
            $string .= $this->base128->set_value($rec << 3 |
$this->wired_type);
        }

        $string .= pack("d", (double)$this->value); 

        return $string;
    }
}
-------------------------

Don't forget to include this file in pb_message.php:
-------------------------
require_once(dirname(__FILE__). '/' . 'type/pb_double.php');
-------------------------

And specify this class as class to work with this type in pb_parser.php:
-------------------------
var $scalar_types = array(..., 'double' => 'PBDouble', 'float', 'int32' =>
'PBInt', 'int64' => 'PBInt',
-------------------------

Attached you will find a patch for PBParser to support the "import" statement.

(See also "Importing Definitions" in
http://code.google.com/intl/de-DE/apis/protocolbuffers/docs/proto.html)

I also attached pb_parser.php as a whole.

The patch is against the version of pb_parser.php in protocolbuf_025.zip


All the best,

Patrick

Below are the code changes of the attached file:

1. The 'import' and 'package' specifiers is supported in 
parser/pb_parser.php
2. Added bytes scalar types to parser/pb_parser.php
3. Fixed a bug in the set_value() function in 
message/encoding/pb_base128.php
   - If the length of $newstring is larger than 32, the following lines 
will result in an integer overflow. Avoid to use bindec() in this case.
     $hexstring = dechex(bindec($newstring));
4. Fixed a bug in the SerializeToString() function in 
message/pb_message.php
   - Use strlen($stringinner) instead of mb_strlen($stringinner), or the 
length might be incorrect if the internal character encoding is UTF8 or 
others.
5. Replace the name of WIRED_STRING with WIRED_LENGTH_DELIMITED, which 
would be a proper name announced by Google.
6. Remove the 'var_dump()' code from message/type/pb_bool.php

[deleted issue]

The split() method is deprecated in php5, using pb4php generates several 
warning messages about it.

The PBMessage class has memory leaks issues in PHP 5. The solution is to 
add a __destruct() method, and manually call __destruct() and unset() on 
every used object. This code has been proven no memory leaks by the use of 
memory_get_usage() function.

The right way of dealing with PBMessage object.
=====================================
$pb = new PB_MessageObjectExample()
$pb->ParseFromString('....');
   :   :   :
   :   :   :
$pb->__destruct();
unset($pb);
=====================================

* PBMessage
=====================================
abstract class PBMessage
{
    /**
     * Fix Memory Leaks with Objects in PHP 5
     * http://paul-m-jones.com/?p=262
     */
    public function __destruct()
    {
        if (isset($this->reader))
        {
            unset($this->reader);
        }
        if (isset($this->value))
        {
            unset($this->value);
        }
        // base128
        if (isset($this->base128))
        {
           unset($this->base128);
        }
        // fields
        if (isset($this->fields))
        {
            foreach ($this->fields as $name => $value)
            {
                unset($this->$name);
            }
            unset($this->fields);
        }
        // values
        if (isset($this->values))
        {
            foreach ($this->values as $name => $value)
            {
                if (is_array($value))
                {
                    foreach ($value as $name2 => $value2)
                    {
                        if (is_object($value2) AND method_exists
($value2, '__destruct'))
                        {
                            $value2->__destruct();
                        }
                        unset($value2);
                    }
                    unset($value->$name2);
                }
                else
                {
                    if (is_object($value) AND method_exists
($value, '__destruct'))
                    {
                        $value->__destruct();
                    }
                    unset($value);
                }
                unset($this->values->$name);
            }
            unset($this->values);
        }
    }
}
=====================================

run test.php in zendstudio, and I change the "split()" to "explode()" in 
pb_parser.php for php 5.3

output: 

1. so many code like "reader = $reader; $this->value = $this; $this->base128 = 
new base128varint(PBMessage::MODUS); } /** * Get the wired_type and field_type 
* @param $number as decimal * @return array wired_type, field_type */ public 
function get_types($number) { $binstring = decbin($number); $types = array(); 

and so on 

2. output the Fatal error, like blew 
Fatal error: Class 'PBEnum' not found in 
D:\wamp\www\Omaha_poker\pb\example\pb_proto_test.php on line 3

What steps will reproduce the problem?
1. run an example (test_new.php or test.php)
2.
3.

What is the expected output? What do you see instead?
expected : var_dump of personn object, but I get this error : Undefined class 
constant 
'PBMessage::WIRED_STRING'

What version of the product are you using? On what operating system?
OS: MAC OS X
Version  025

Please provide any additional information below.

I am trying to parse the serialized string.

Serialized String:

4"I¸ÒÄʯ+è¾dªð¬£"ë/@2`Mozilla/5.0 (Windows; U; Windows NT 5.1; 
en-US; rv:1.8.1.2pre) Gecko/20070118 
Firefox/2.0.0.2preZhttp://some.gcn.site.combeà3lá8G³QõB?1??sy>r¿"  2
©³´¶¾Jùñ­#y%·x ªgfbk_It_czw8BMZ5WDHeyAGai8²GCèèøÂ<2753¸Ö=ÈNÑ
X$·