Giter VIP home page Giter VIP logo

protobuf-net's Introduction

protobuf-net logo protobuf-net

protobuf-net is a contract based serializer for .NET code, that happens to write data in the "protocol buffers" serialization format engineered by Google. The API, however, is very different to Google's, and follows typical .NET patterns (it is broadly comparable, in usage, to XmlSerializer, DataContractSerializer, etc). It should work for most .NET languages that write standard types and can use attributes.

Build status

Release Notes

v3 is here!

Change history and pending changes are here.


Supported Runtimes

  • .NET 6.0+ (.NET 5 etc will use .NET Standard 2.1)
  • .NET Standard 2.0, 2.1
  • .NET Framework 4.6.2+

Build tools

Build tools to help you use protobuf-net correctly are available via protobuf-net.BuildTools

Runtime Installation

All stable and some pre-release packages are available on NuGet. CI Builds are available via MyGet (feed URL: https://www.myget.org/F/protobuf-net/api/v3/index.json ).

You can use the following command in the Package Manager Console:

Install-Package protobuf-net
Package NuGet Stable NuGet Pre-release Downloads MyGet
protobuf-net protobuf-net protobuf-net protobuf-net protobuf-net MyGet

Basic usage

1 First Decorate your classes

[ProtoContract]
class Person {
    [ProtoMember(1)]
    public int Id {get;set;}
    [ProtoMember(2)]
    public string Name {get;set;}
    [ProtoMember(3)]
    public Address Address {get;set;}
}
[ProtoContract]
class Address {
    [ProtoMember(1)]
    public string Line1 {get;set;}
    [ProtoMember(2)]
    public string Line2 {get;set;}
}

Note that unlike XmlSerializer, the member-names are not encoded in the data - instead, you must pick an integer to identify each member. Additionally, to show intent it is necessary to show that we intend this type to be serialized (i.e. that it is a data contract).

2 Serialize your data

This writes a 32 byte file to "person.bin" :

var person = new Person {
    Id = 12345, Name = "Fred",
    Address = new Address {
        Line1 = "Flat 1",
        Line2 = "The Meadows"
    }
};
using (var file = File.Create("person.bin")) {
    Serializer.Serialize(file, person);
}

3 Deserialize your data

This reads the data back from "person.bin" :

Person newPerson;
using (var file = File.OpenRead("person.bin")) {
    newPerson = Serializer.Deserialize<Person>(file);
}

Notes

Notes for Identifiers

  • they must be positive integers (for best portability, they should be <= 536870911 and not in the range 19000-19999)
  • they must be unique within a single type but the same numbers can be re-used in sub-types if inheritance is enabled
  • the identifiers must not conflict with any inheritance identifiers (discussed later)
  • lower numbers take less space - don't start at 100,000,000
  • the identifier is important; you can change the member-name, or shift it between a property and a field, but changing the identifier changes the data

Advanced subjects

Inheritance

Inheritance must be explicitly declared, in a similar way that it must for XmlSerializer and DataContractSerializer. This is done via [ProtoInclude(...)] on each type with known sub-types:

[ProtoContract]
[ProtoInclude(7, typeof(SomeDerivedType))]
class SomeBaseType {...}

[ProtoContract]
class SomeDerivedType {...}

There is no special significance in the 7 above; it is an integer key, just like every [ProtoMember(...)]. It must be unique in terms of SomeBaseType (no other [ProtoInclude(...)] or [ProtoMember(...)] in SomeBaseType can use 7), but does not need to be unique globally.

.proto file

As an alternative to writing your classes and decorating them, You can generate your types from a .proto schema using protogen; the protogen tool is available as a zip from that location, or as a "global tool" (multi-platform).

Alternative to attributes

In v2+, everything that can be done with attributes can also be configured at runtime via RuntimeTypeModel. The Serializer.* methods are basically just shortcuts to RuntimeTypeModel.Default., so to manipulate the behaviour of Serializer., you must configure RuntimeTypeModel.Default.

Support

I try to be responsive to Stack Overflow questions in the protobuf-net tag, issues logged on GitHub, email, etc. I don't currently offer a paid support channel. If I've helped you, feel free to buy me a coffee or see the "Sponsor" link at the top of the GitHub page.

protobuf-net's People

Contributors

benaadams avatar bferullo avatar bjorkstromm avatar deaglegross avatar dxdjgl avatar floatinghotpot avatar hellokitty avatar iamcarbon avatar jnyrup avatar kittyfisto avatar ladeak avatar lbargaoanu avatar markpflug avatar matti-koopa avatar mgravell avatar mirhagk avatar nickcraver avatar nickdarvey avatar randyridge avatar rayokota avatar remibou avatar sandorfr avatar schulz3000 avatar seldary avatar sotn0r avatar stephenfung avatar szehetner avatar thecentury avatar xpaw avatar yaakov-h avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

protobuf-net's Issues

Nested / Jagged-Arrays with IgnoreListHandling throwing NotSupportedException

It seems like IgnoreListHandling is still being ignored when applied to child list items.

class Program
{
    static void Main(string[] args)
    {
        using (MemoryStream ms = new MemoryStream())
        {
            Serializer.Serialize(ms, new OuterList());
        }
    }

    [ProtoContract]
    public class OuterList
    {
        [ProtoMember(1)]
        public List<InnerList> InnerList = new List<InnerList>();
    }

    [ProtoContract(IgnoreListHandling = true)]
    public class InnerList : IEnumerable<int>
    {
        [ProtoMember(1)]
        public List<int> innerList = new List<int>();

        public IEnumerator<int> GetEnumerator()
        {
            return this.innerList.GetEnumerator();
        }

        System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
        {
            return this.GetEnumerator();
        }
    }
}

... throws a NotSupportedException, although it should theoretically serialize because of the IgnoreListHandling-flag.

Sliverlight RuntimeTypeModel.CompileInPlace realization

Target- add RuntimeTypeModel.CompileInPlace issue into Silverlight platform too.

Tell please, if we'll have some problems, if do next:

1- in protobuf-net_Silverlight project -add Build-ConditionalCompile Symbol [FEAT_COMPILER]
we will get two errors:
2- DateTime doesn't contain -ToBinary and FromBinary -->so we set to use DateTime as FromFileTime -the same as for FX11 Precompile Symbol
3- DynamicMethod separately for silverlight - only with 3 params

-so after such changes we able to CompileInPlace() even on silverlight. How you think is that possible?

KnownTypeName usage leads to type mismatches

When assemblies are being loaded into multiple load contexts, it is possible for typeof(T) != TypeModel.ResolveKnownType(typeof(T).AssemblyQualifiedName, null, typeof(T).Assembly). This breaks inheritance support because MetaType.ApplyDefaultBehaviour prefers KnownTypeName over KnownType, and then the base type is not assignable from the subtype, causing the subtype to serialize and deserialize as if it were a base type.

To avoid this issue, MetaType.ApplyDefaultBehaviour should prefer KnownType, and ProtoInclude should store KnownType and KnownTypeName using separate fields such that KnownTypeName always returns the name as before, but KnownType can return null if the type was not provided by the CLR when the assembly was loaded (ProtoIncludeAttribute(int, string) vs ProtoIncludeAttribute(int, Type)). I don't know why MetaType.ApplyDefaultBehaviour prefers KnownTypeName currently.

Additionally, it might be a good idea for TypeModel.ResolveKnownType to somehow use the assembly context in the first attempt to resolve, maybe by searching that assembly and its referenced assemblies, but this may slow things down and it may be acceptable to instead require KnownType instead of KnownTypeName when using ProtoInclude in a product where assemblies may be loaded into an app domain multiple times.

I encountered this problem because my employer has an application that dynamically downloads assemblies as necessary, but also loads bundled plugins. There is a plugin that contains a copy of an assembly that can also be downloaded on demand, and that assembly contains protobuf-net models. When the plugin is loaded, the assembly is loaded from the plugin, but then it also gets downloaded into the appdomain application base directory causing it to become the preferred copy for Type.GetType. When the plugin code tries to create a deserializer it uses its copy of the assembly, but then protobuf-net uses the other copy of the assembly for ProtoIncludes. When loading the plugin a second time, we get the copy of that assembly from the application base directory instead, and deserialization succeeds.

This seems very convoluted, but there are probably other ways for this to happen if somebody is using multiple assembly load contexts or otherwise causing multiple assemblies with the same fully qualified name to be loaded into the same appdomain.

cloning boolean bug?

Hello Marc,

I guess I've found an error in current version 2.0.0.668 while cloning/deserializing boolean values.
I've the following test class, create a new instance of it, change the values and clone the instance with the provided method "DeepClone".

//[ProtoContract(SkipConstructor = true)]
[ProtoContract(SkipConstructor = false)]
public class ProtobufData
{
    [ProtoMember(1)]
    public bool ValueBool { get; set; }
    [ProtoMember(2)]
    public int ValueInt { get; set; }

    public ProtobufData()
    {
        this.ValueBool = false;
        //this.ValueBool = true;
        this.ValueInt = int.MinValue;
    }
}

A special combination of [ProtoContract(SkipConstructor = true|false)], initialisation in constructor "this.ValueBool = true|false" and change of instance values "someInstance.ValueBool = true|false" result in an unexpected result (see tables below).

[ProtoContract
(SkipConstructor = true)]
[ProtoContract
(SkipConstructor = false)]
values after construction:
this.ValueBool = false;
this.ValueInt = int.MinValue;
"ValueBool": false,
"ValueInt": -2147483648

"ValueBool": false,
"ValueInt": -2147483648

values after change:
someInstance.ValueBool = true;
someInstance.ValueInt = 1;
"ValueBool": true,
"ValueInt": 1

"ValueBool": true,
"ValueInt": 1

values after cloning:

"ValueBool": true,
"ValueInt": 1
"ValueBool": true,
"ValueInt": 1
[ProtoContract
(SkipConstructor = true)]
[ProtoContract
(SkipConstructor = false)]
values after construction:
this.ValueBool = true;
this.ValueInt = int.MinValue;
"ValueBool": true,
"ValueInt": -2147483648

"ValueBool": true,
"ValueInt": -2147483648

values after change:
someInstance.ValueBool = false;
someInstance.ValueInt = 1;
"ValueBool": false,
"ValueInt": 1

"ValueBool": false,
"ValueInt": 1

values after cloning:


"ValueBool": false,
"ValueInt": 1
"ValueBool": true,
"ValueInt": 1

It's about the lower right cell in the second table. Is this behaviour/result correct? I would have expected "ValueBool: false" because that's the value that was "set/serialized" before. Of course, "SkipConstructor = true" would always result in the correct clone, but I would like to use "SkipConstructor = false" due to lists I want to initilize in constructor.

Thanks in advance.

Mathias

Serializing a list properly with a type model

So I have a protomember like this:
http://hastebin.com/farecukosa.xml

Whenever I have the List added like above, I cannot build my serializer as it errors, removing the protomember causes things to work properly.

However I am convinced the above error I am having is causing things to not serialize properly as I get this error:

ProtoException: Invalid field in source data: 0

Here is my serialization,deserialization & type model.

http://hastebin.com/etukatoxub.cs

Basically I have a circular buffer (which is a basic array) and I want that to hold a list of elements. This is what is giving me trouble, as I cannot seem to get this to serialize and deserialize correctly (serializtion works, but deserialization complains) If this is a massive oversight on my part, please let me know!

Thank you again for any and all help!

Add GeneratedCode attribute to classes generated by ProtoGen

Projects scanned with Code Analysis in Visual Studio may return messages for generated files. Adding the GeneratedCode attribute to the template files above class declarations should allow for suppression of results if specified by the user.

Install Package in C++ Native Project

I got the following error:
Install-Package : Could not install package 'protobuf-net 2.0.0.668'. You are trying to install this package into a project that targets 'Native,Version=v0.0', but the
package does not contain any assembly references or content files that are compatible with that framework. For more information, contact the package author.
At line:1 char:16

+ Install-Package <<<< protobuf-net
+ CategoryInfo : NotSpecified: (:) [Install-Package], InvalidOperationException
+ FullyQualifiedErrorId : NuGetCmdletUnhandledException,NuGet.PowerShell.Commands.InstallPackageCommand

Please make it available to C++ Native Project
If not, is there another way to import the package?

Trust Level and Security Exception

My hosting has set and not overridable by web.config

The error is:
"System.Security.SecurityException: That assembly does not allow partially trusted callers."

Is possibile to change assemby or simple recompile with
<Assembly: Security.AllowPartiallyTrustedCallers>

thanks

Adding attributes to the fields of RuntimeTypeModel types

I'm constructing non-default RuntimeTypeModel for third-party library classes. However, I require some of the fields to be marked as references, just as you can do that with [ProtoMember(N), AsReference=true]. How can I apply those attributes to the fields of the types defined in my non-default RuntimeTypeModel?

Extensible.GetValues returns only the last value instead of all values

I am using Extensible to implement a serialized type that can handle arbitrary data members. One of those members is a list of items. To set the values of this members I use AppendValue and to get it's values I use GetValues. Unfortunately I find that GetValues only returns the last value added. By inspecting the byte stream I see that the data is there.

Below you'll find a small sample reproducing the behavior. I would expect the output of this program to be:
foo
bar

Instead you get:
bar

I believe is caused the loop in ProtoBuf.ExtensibleUtil.GetExtendedValues (around line 76):

            while (model.TryDeserializeAuxiliaryType(reader, format, tag, type, ref value, true, false, false, false) && value != null)

Passing false as the "asListItem" parameter causes TryDeserializeAuxiliaryType to return only the last value.

class Program
{
static void Main(string[] args)
{
var x = new SampleExt();
x.AddData("foo");
x.AddData("bar");
foreach (var value in x.GetData())
{
Console.WriteLine(value);
}
}

class SampleExt : Extensible
{
    public string[] GetData()
    {
        return GetValues<string>(this, 1).ToArray();
    }

    public void AddData(string value)
    {
        AppendValue(this, 1, value);
    }
}

}

cannot convert `ProtoBuf.PrefixStyle' expression to type `ProtoBuf.PrefixStyle'

Using Unity3D 4.5.2f1, I have this error only in the Unity compiler, as the code shows no errors in Visual Studio.

I am baffled as to what is causing it and how to go about fixing it, here is the line that is tripping things up:

sanitySerializer.SerializeWithLengthPrefix(bs, newObjStateData, typeof(ObjectPastStateData), PrefixStyle.Base128, 1);

Any help in this matter would be great. Thank you!

Possible to get an example of serializing different objects into one file?

Hi! I am using Unity3D and just discovered the beauty of protobuf-net (was using JSON before) and it has solved a MAJOR problem I was having saving and loading tons of data. It is also much faster than json (0.8s to 0.2s) by a large margin and file sizes are in the fractions. (down from 50MB to 4MB)

The only problem is I need to be able to read and write very often and in order to balance the load of that, I need to do so over several frames to prevent hitching in the game from the saving and loading (this happens while the game is playing). I have solved this problem by serializing over serveral frames into one file in a file stream. Problem is when I go to deserialize it in this manner, I cannot get one of the different item types I need to serialize. I want to serialize to different types imagine, foo and bar. Foo I serialize about 70-100 times over into the stream, then lastly, i serialize just 1 bar. The bar contains all the data I need to understand the list of foo. I cannot understand how to do so without making two files on two different streams? Here is what I have currently:

http://hastebin.com/zewuqolago.coffee

This works for the ObjectPastStateData as it gets all the objects saved in this manner, however the SHWrapperData does not come through. I still don't quite understand the field order (last argument) though, I figured if I changed this number to be unique I could easily pull the data by that number (it being some sort of header) but that isn't the case. Right now I serialize ObjectPastStateData as 2 and SHWrapperData as 1.
Here is what I tried to do to achieve this (based on some examples I have seen around the net) but I still cannot get this to work as needed.

http://hastebin.com/tiragefufe.coffee

Any help or information on this or even just an example doing this would be HUGE.

OutOfMemoryException in Serializer.Deserialize

I'm getting a System.OutOfMemoryException when calling Serializer.Deserialize on a FileStream. This happens only on one specific file, and not any other files. Any ideas why that would be?

Code (sorry for the VB code - it wasn't my choice)

        Dim loadObj As Object = Nothing
        Using tempStream As FileStream = File.OpenRead(dataFilename)
            loadObj = Serializer.Deserialize(Of MeterReadingsByMeterId)(tempStream)    'fails on 724 every time
            If Not loadObj.GetType() Is GetType(MeterReadingsByMeterId) Then
                loadObj = Nothing
            End If
        End Using

Stack Trace:

Exception of type 'System.OutOfMemoryException' was thrown.
   at System.Collections.Generic.Dictionary`2.Resize()
   at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add)
   at System.Collections.Generic.Dictionary`2.System.Collections.Generic.ICollection<System.Collections.Generic.KeyValuePair<TKey,TValue>>.Add(KeyValuePair`2 keyValuePair)
   at proto_6(Object , ProtoReader )
   at ProtoBuf.Serializers.CompiledSerializer.ProtoBuf.Serializers.IProtoSerializer.Read(Object value, ProtoReader source)
   at ProtoBuf.Meta.RuntimeTypeModel.Deserialize(Int32 key, Object value, ProtoReader source)
   at ProtoBuf.ProtoReader.ReadTypedObject(Object value, Int32 key, ProtoReader reader, Type type)
   at ProtoBuf.ProtoReader.ReadObject(Object value, Int32 key, ProtoReader reader)
   at proto_4(Object , ProtoReader )
   at ProtoBuf.Serializers.CompiledSerializer.ProtoBuf.Serializers.IProtoSerializer.Read(Object value, ProtoReader source)
   at ProtoBuf.Meta.RuntimeTypeModel.Deserialize(Int32 key, Object value, ProtoReader source)
   at ProtoBuf.ProtoReader.ReadTypedObject(Object value, Int32 key, ProtoReader reader, Type type)
   at ProtoBuf.ProtoReader.ReadObject(Object value, Int32 key, ProtoReader reader)
   at proto_2(Object , ProtoReader )
   at ProtoBuf.Serializers.CompiledSerializer.ProtoBuf.Serializers.IProtoSerializer.Read(Object value, ProtoReader source)
   at ProtoBuf.Meta.RuntimeTypeModel.Deserialize(Int32 key, Object value, ProtoReader source)
   at ProtoBuf.Meta.TypeModel.DeserializeCore(ProtoReader reader, Type type, Object value, Boolean noAutoCreate)
   at ProtoBuf.Meta.TypeModel.Deserialize(Stream source, Object value, Type type, SerializationContext context)
   at ProtoBuf.Serializer.Deserialize[T](Stream source)

Using multiple Protobuf-net implementations (multiple protobuf dlls)

So I have two other assets in my unity3d project that also now use protobuf-net. I see that when you create a type model and a serializer, you use your own built version of the library. At first I thought it would be easy as swapping out the protobuf-net.dll file, but it isn't. It causes things from whichever asset that had that dll file to have things missing. Seeing how I have now 3 different protobuf-net.dll files (my own, and 2 others), how can I centralize into just depending off of one so that they all can work in unison? I hope my question makes sense

In other words how can I get them to all play nicely?

Thank you!

Change property type

Is it possible to change property type, for example from class to simple int, and then on deserialization to customize deserialize if deserializing as int fails, to try deserialize class which was used in place of int before, filling out int property based on the values that used to be in that class?

So basically I want to convert stored value to a different type on deserialize. How would I go around doing it?

Unexpected behaviour when deserialising Property if SetProperty method is defined

I am not sure if it's undocumented feature or genuine bug but in code such as:

    class Program
    {
        static void Main()
        {
            using (Stream stream = new MemoryStream())
            {
                Serializer.Serialize(stream, new TestData(){SomeProperty = "Test"});

                stream.Position = 0;
                var obj = Serializer.Deserialize<TestData>(stream);
            }
        }
    }

    [DataContract]
    public class TestData
    {
        [DataMember(Order = 1)]
        public string SomeProperty { get; set; }

        public void SetSomeProperty(string data)
        {
            throw new Exception();
        }
    }

it would call SetSomeProperty method on deserialisation.
It doesn't do it for fields only for properties.
If it's a feature then it would be nice to be able to disable it through RuntimeTypeModel at least

Thank you

Deserialization causes unrecoverable StackOverflowException

When deserializing class Foo with a member class Bar where Bar has two implementations ( BarA, BarB) and one of the implementations is assigned (as a default) by the constructor of Foo, a StackOverflowException is thrown.

Skipping Foo's constructor fixes the issue, but perhaps a more graceful exception is in order?

https://gist.github.com/mpb27/f67a7bfe7018c78b8e4d

    class Program
    {
        static void Main(string[] args)
        {
            // ... try to reproduce the stack overflow, by serializing and deserializing from memory stream ...

            // ... use a serializer without defaults, not necessary ...
            var serializer = TypeModel.Create();
            serializer.UseImplicitZeroDefaults = false;

            var fooWithBarA = new Foo() { SomeBar = new BarA() { Value = 321 } };
            var fooWithBarB = new Foo() { SomeBar = new BarB() { Value = 321.0 } };

            // ... serialize fooWithBarA ...
            var ms1 = new MemoryStream();
            serializer.Serialize(ms1, fooWithBarA);

            // ... serialize fooWithBarB ...
            var ms2 = new MemoryStream();
            serializer.Serialize(ms2, fooWithBarB);

            // ... attempt deserialize fooWithBarA ...
            ms1.Position = 0;
            var desFooWithBarA = (Foo) serializer.Deserialize(ms1, null, typeof (Foo));
            var barA = (BarA) desFooWithBarA.SomeBar;
            Console.WriteLine("A = {0}", barA.Value);

            // ... attempt deserialize fooWithBarB ...
            ms2.Position = 0;
            // ... stack overflows in here ...
            var desFooWithBarB = (Foo) serializer.Deserialize(ms2, null, typeof (Foo));            var barB = (BarB) desFooWithBarB.SomeBar;
            Console.WriteLine("B = {0}", barB.Value);

            Console.ReadLine();
        }
    }

    [ProtoContract]
    public class BarA : Bar
    {
        [ProtoMember(1)] public int Value;
    }

    [ProtoContract]
    public class BarB : Bar
    {
        [ProtoMember(1)] public double Value;
    }

    [ProtoContract]
    [ProtoInclude(10, typeof(BarA))]
    [ProtoInclude(11, typeof(BarB))]
    public class Bar
    {
        // ... not usually empty but this is just an example ...
    }

    [ProtoContract(SkipConstructor = false)] // ... true fixes by not assigning a default ...
    public class Foo
    {
        [ProtoMember(1)]
        public Bar SomeBar { get; set; }

        public Foo()
        {
            // ... a default value ...
            SomeBar = new BarA() { Value = 123 };
        }
    }
}

Configuring ProtoBehaviorExtension is not endpoint specific

Issue
When configuring a service endpoint with the ProtoBehaviorExtension, the ProtoEndpointBehavior is applied to all subsequent endpoints configured with the same contract regardless of address, binding and behavior configurations.

Simplifeid example:
The following configuration works as expected. I have 2 service enpoints, one with DataContractSerializer and one with XmlProtoSerializer:

...
    <behavior name="dcs">
        <dataContractSerializer maxItemsInObjectGraph="1000000" />
    </behavior>
    <behavior name="proto">
        <dataContractSerializer maxItemsInObjectGraph="1000000" />
        <protobuf/>
    </behavior>
...
    <endpoint address="http://service:port/Http"      behaviorConfiguration="dcs"   binding="basicHttpBinding" bindingConfiguration="HttpBinding" contract="ITestService" name="dcs"/>
    <endpoint address="http://service:port/HttpProto" behaviorConfiguration="proto" binding="basicHttpBinding" bindingConfiguration="HttpBinding" contract="ITestService" name="proto"/>
...

If the proto endpoint is configured first, then both endpoints have the ProtoEndpointBehavior - which is not desired.

Expected behavior
The ProtoEndpointBehavior is only applied to the endpoints that explicitly have it configured.

Workaround
Ensure that all endpoints that require the ProtoEndpointBehavior are registered after any other endpoints in the service configuration. The documentation is not explicit about this.

I am currently working around this with a custom EndpointBehavior that applies the the ProtoOperationBehvior to the ClientRuntime.Operations and DisptachRuntime.Operations instead of the ServiceEndpoint.Operations.

[default] in .proto file not recognized

I use .proto files to specify my protobuf messages. I use [default=], but I cannot see where the values are set. They are not in the autogenerated .cs files. I would like to set some default values when the message is created. I cannot use the default constructor, because it is inside the autogenerated .cs files.

message Ack
{
    required bool is_error = 1 [default=false];
    required string message = 2 [default="ok"];
    required string request_id = 3;
}

I already ask this in stackoverflow, see here

Marc confirmed this as a bug.

Null nested messages

Hi,

I think there is a bug in protobuf-net but I'm not sure--I can't find any specification for this from Protocol Buffers.

It's a nested object that is null; when protobuf-net serializes this, it serializes a 0x0. I think this is wrong, if the nested object is null, there shouldn't be anything there. This is at least as I suspect and feel would be most consistent with rest of Protocol Buffers.

I have posted in the Protocol Buffers Google group, please read: https://groups.google.com/forum/#!topic/protobuf/vgEbT6s5Kyo

Best regards,
Johannes

Support for "oneof"

Would be nice to see support for the new oneof keyword in protobuf-net, as it certainly tidies up the cases where you, umm, want one of many optional things.

ListDecorator.EmitReadAndAddItem doesn't support URI collections

ValueMember.TryGetCoreSerializer intentionally returns a StringSerializer for URI types, claiming it will be wrapped in a UriDecorator later. However, this never happens in the case of ArrayDecorator.

I'm happy to contribute a patch, but I'm having a hard time finding the right place to make the correction without understanding the codebase better.

Assembly error with protobuf-net v2.0.0.668 in a Windows 8 Store C# project

Hello,
I am having issues with protobuf-net, which is used as a dependency of the NuGet package "OsmSharp".

I am quite new to the C# ecosystem so it might be some obvious thing i do not know. What I do is use Visual Studio Express 2013 for Windows (the one to create Windows 8 Store apps) and create a new C# Project for Windows 8.
I pull "OsmSharp" from the NuGet package manager which pulls protobuf-net v2.0.0.668 and Zlib.Portable 1.9.2.
For some unknown reason, it doesn't work like that, I also have to install "Antlr3 Portable" so that Antlr works.

All the OsmSharp code works until it reach the deserialiaze part which tried to deserialize a protobuf file. The error is this one:

Could not load file or assembly 'protobuf-net, Version=2.0.0.668, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The assembly version has a public key token that does not match that of the request. (Exception from HRESULT: 0x80132001)

This prevents the program from going further. I looked up online before but It didn't really help.
This old issue looks similar but not the exact same error code:
https://code.google.com/p/protobuf-net/issues/detail?id=378&q=assembly
This one has the same error on another library but i wasn't able to understand the fix (using a PCL library?)
http://thesoftwarechef.blogspot.fr/2013/11/mvvmlight-sharing-code-between-wpf-and.html
This one has a comment from "Bin Xie" that explain that the issue comes from NuGet that does not allow changing the "Strong Name" property but I didn't manage to do what was explained:
http://blogs.msdn.com/b/windowsazurestorage/archive/2012/12/10/updated-known-issues-for-windows-azure-storage-client-library-2-0-for-net-and-windows-runtime.aspx?PageIndex=1

This is what the protobuf-net properties look like:
image

As I said before, I am not really an expert in that environnement but some help would be greatly appreciated !

Thanks,
Fabien

IReadOnlyList<> fails

When serializing a type with IReadOnlyList<> Protobuf-net throws some error about missing an Add function. It would be nice if IReadOnlyList<> worked as well as IList<>. Thanks.

Boxing garbage

It appears that at the lowest level this library is working with all values as objects, and that this will generate garbage do to boxing/unboxing at least once for each value written or read. This is a major concern for games, or other fps dependent applications for which unexpected GC passes are noticeable to the user.

Keeping Object References in Multiple Serializations

Hello. I have been trying to find a way to preserve object references between multiple serializations to streams. I tried serializing a list of objects to a file stream (with the reference features enabled) and it worked for that single list of objects in the stream. But, I need to keep object references across multiple streams or serializations. Is it possible?

Forgive me if I've overlooked some documentation on how to do this.

Performance vs protobuf-csharp-port

Hi! I'm trying to evaluate different implementations of protobuf and saw yours and decided to try. I really like it's nice and flexible and allows storing properties and utilizes attributes. Kudos to you for doing it so simple!

Unfortunately, when testing performance, I found it's almost twice as slow as https://code.google.com/p/protobuf-csharp-port/. Why is that? Is it because of using reflection vs. precompiled? But Microsoft Live Labs' OptimizedSerializer also uses runtime reflection and it's also faster than your implementation...

Any hint on how to make your implementation faster will be greatly appreciated!

Thank you!
Andrew

Out-of-order pseudo inheritance

As of build r668 it is still mandatory to serialize the pseudo-inheritance fields before any member of a given class. This is done by the protobuf-net serializer automatically, but not by other protobuf serializers like the JAVA one. Failing that protobuf-net will not be able to deserialize the object correctly.

According to the google documentation it should be agnostic to the order to the fields.

C# 6 automatic property initializer forces its value over the deserialized one

Hi,
Just playing around in a project that uses the latest protobuf-net nuget package (just checked) in the new Visual Studio 2015 Preview and noticed something a little 'funky' about deserialization of properties that are initialized using the new automatic property syntax.

E.g.

[ProtoContract]
public class Container 
{
    [ProtoMember(0)]
    public bool ThisIsAFlag { get; set; } = true;
}

Will result in that flag ALWAYS being set to true when the containing object is deserialized. I assume this is because of the new compiler generated code being executed post-deserialization unintentionally, but I couldnt suppress the behaviour using [ProtoContract(SkipConstructor = true)]so it must be happening elsewhere from that path.

Worth knowing about even if there is nothing you can do about it?

Let me know if you need more details.

AqlaSerializer: fork link

I'm not sure whether I should post it here (as this is not an advertisement place) but I would like to let the protobuf-net community know that

I've released a fork of the protobuf-net which is focused on 100% .NET features compatibility.

Did you know that protobuf-net has significant issues with reference tracking? I fixed a lot of them but unfortunately maked it wire-incompatible with the original project. I don't expect my changes to be pulled back so I started a seperate project.

If you don't like to see this as an "issue" here I understand that. If it's possible I would like a link to my fork to be added somewhere in the wiki.

protogen specifies invalid defaults for externally defined enums

I have a simple enum defined in a common proto file:

enum ServerType { Unknown = 0; Core = 1; Account = 2; Zone = 3; Test = 4; }
In another file I then reference and use this type:

import "common.proto";

package Core;

message MsgCore_LogMessage_Request {
    option (Common.message_id) = 22;

    optional string message = 1;
    optional LogLevel level = 2;
    optional Common.ServerType server_type = 3;
    optional uint32 server_index = 4;
}

This produces invalid code on the server_type attribute:

private Common.ServerType _server_type = Common.ServerType.;
    [global::ProtoBuf.ProtoMember(3, IsRequired = false, Name=@"server_type", DataFormat = global::ProtoBuf.DataFormat.TwosComplement)]
    [global::System.ComponentModel.DefaultValue(Common.ServerType.)]

Note the default value doesn't resolve.
Manually specifying a default value is a workaround for this.

EF entities not getting serialized ?

Hi,

I tried using the protobuf for my EF entities. It does serialize all public fields of my objects but none of the EF entity attributes or nodes. These entity nodes are not getting serialized r:EntityKey z:Id="i2".
Also the nested objects are not getting serialized either. I tried changing the Runtime Model both true and false and it does not affect the results. So how do you tell the serializer to include all EF entities ?

TupleSerializer has problems with System.Uri constructor arguments

It seems that the ProtoBuf.Serializers.TupleSerializer doesn't check for the System.Uri constructor parameters. It uses the ValueMember.TryGetCoreSerializer which returns a StringSerializer for that type, which seems to be OK as the note says the it should be wrapped later by the UriDecorator, but this doesn't happen. In the result, unwrapped StringSerializer throws various exceptions as it tries to treat System.Uri value as a System.String while i.e. obtaining number of chars, etc.

Unfortunately I'm unable to avoid falling into the AutoTuple behaviour with some custom class while creating the runtime models, which renders this issue as a major one.

Thanks

OutOfMemoryException in BufferPool

The BufferPool throws OutOfMemory exception when serializing large objects on x86 platforms. In our case one of the fields is a byta array containing 64 MB data. Possible improvements:

  • Remove buffering from ProtoWriter: dirently write to the stream, buffering should be performed by the underlying stream.
  • Do not only cache default-sized buffers. The real gain would be in caching big buffers allocated on the Large Object Heap. They are responsible for memory fragmentation and also more expensive to re-allocate.
  • Use weak references for cached buffers. They could be automatically collected in low-memory situations.
  • Re-use cached buffers in ResizeAndFlushLeft method.

Support PCL Profile111

The current version includes support for the Profile136 PCL version, and it includes support for WinRT. Unfortunately, these leave a gap in a core scenario:

Try to create a portable library which references protobuf-net, which can be included in both a Windows 8.1 project and a Windows Phone 8.1 project. It is currently not possible. This is very painful, as it means that protobufs cannot be used in any shared library consumed by a universal app (an app which contains both a windows 8.1 and a windows phone 8.1 flavor).

As it turns out, the current WinRT flavor of protobuf-net CAN be directly referenced by a Windows 8.1 project, and it CAN be directly referenced by a Windows Phone 8.1 project, but it CANNOT be referenced by any PCL library which can then be referenced by those two project types.

To solve this, please add another PCL flavor (Profile 111). From File->New Project in Visual Studio, choose the PCL class library template, and then select this combination of items:

.Net 4.5
Windows 8.1
Windows Phone 8.1

Unselect everything else.

Error upon serializing EntityObject EF 6

Hi,
I am using Protobuf-net with EF 6. When I try to serialize an entity object, which has at least one EntityReference property, I get the following exception:

"The EntityReference has already been initialized. InitializeRelatedReference should only be used to initialize a new EntityReference during deserialization of an entity object."

Server stack trace:
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.RuntimePropertyInfo.SetValue(Object obj, Object value, BindingFlags invokeAttr, Binder binder, Object[] index, CultureInfo culture)
at System.Reflection.RuntimePropertyInfo.SetValue(Object obj, Object value, Object[] index)
at ProtoBuf.Serializers.PropertyDecorator.Read(Object value, ProtoReader source)
at ProtoBuf.Serializers.TypeSerializer.Read(Object value, ProtoReader source)
at ProtoBuf.Meta.RuntimeTypeModel.Deserialize(Int32 key, Object value, ProtoReader source)
at ProtoBuf.ProtoReader.ReadTypedObject(Object value, Int32 key, ProtoReader reader, Type type)
at ProtoBuf.ProtoReader.ReadObject(Object value, Int32 key, ProtoReader reader)
at ProtoBuf.Serializers.SubItemSerializer.ProtoBuf.Serializers.IProtoSerializer.Read(Object value, ProtoReader source)
at ProtoBuf.Serializers.TagDecorator.Read(Object value, ProtoReader source)
at ProtoBuf.Serializers.TypeSerializer.Read(Object value, ProtoReader source)
at ProtoBuf.Meta.RuntimeTypeModel.Deserialize(Int32 key, Object value, ProtoReader source)
at ProtoBuf.ProtoReader.ReadTypedObject(Object value, Int32 key, ProtoReader reader, Type type)
at ProtoBuf.ProtoReader.ReadObject(Object value, Int32 key, ProtoReader reader)
at ProtoBuf.Serializers.SubItemSerializer.ProtoBuf.Serializers.IProtoSerializer.Read(Object value, ProtoReader source)
at ProtoBuf.Serializers.TagDecorator.Read(Object value, ProtoReader source)
at ProtoBuf.Serializers.PropertyDecorator.Read(Object value, ProtoReader source)
at ProtoBuf.Serializers.TypeSerializer.Read(Object value, ProtoReader source)
at ProtoBuf.Meta.RuntimeTypeModel.Deserialize(Int32 key, Object value, ProtoReader source)
at ProtoBuf.ServiceModel.XmlProtoSerializer.ReadObject(XmlDictionaryReader reader, Boolean verifyObjectName)
at System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.PartInfo.ReadObject(XmlDictionaryReader reader, XmlObjectSerializer serializer)
at System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.DeserializeParameterPart(XmlDictionaryReader reader, PartInfo part, Boolean isRequest)
at System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.DeserializeParameter(XmlDictionaryReader reader, PartInfo part, Boolean isRequest)
at System.ServiceModel.Dispatcher.DataContractSerializerOperationFormatter.DeserializeBody(XmlDictionaryReader reader, MessageVersion version, String action, MessageDescription messageDescription, Object[] parameters, Boolean isRequest)
at System.ServiceModel.Dispatcher.OperationFormatter.DeserializeBodyContents(Message message, Object[] parameters, Boolean isRequest)
at System.ServiceModel.Dispatcher.OperationFormatter.DeserializeReply(Message message, Object[] parameters)
at System.ServiceModel.Dispatcher.ProxyOperationRuntime.AfterReply(ProxyRpc& rpc)
at System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime operation, ProxyRpc& rpc)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at ...

With the EF 4 I had no problem. Any help would be appreciated.

Thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.