Issue Serielizing DataTable with column of Type DateTimeOffset about protobuf-net-data HOT 6 CLOSED

It would be good to support DateTimeOffset (because it is a SQL Server type) but unfortunately protobuf-net doesn't support this natively yet so we wouldn't have any way of serializing it.

It seems protobuf-net doesn't support it because it would break .NET 2.0 compatibility. But that was nearly 3 years ago, much fewer people care about .NET 2.0 now so maybe we will see that change soon.

from protobuf-net-data.

Thanks for taking the time to reply.

I've another unrelated question, is it possible with protobuf-net-data to write linq type queries against a serialized stream. Something like mystream.AsEnumerable().AsQueryable().Select(.....). Currently I store Datatables using protobuf-net-data (which in terms of speed and size has proved a brilliant solution). What I have to do though to execute Linq queries is deserialize the data back to a datatable then execute a Linq query. Since my datatables can be quite large resources can come under pressure with this approach.

from protobuf-net-data.

Yep, IDataReader implements IDataRecord which you can use as a sort-of iterator which you can then query:

public static class DataReaderExtensions
{
    public static IEnumerable<IDataReader> AsEnumerable(this IDataReader reader)
    {
        if (reader == null) throw new ArgumentNullException("reader");

        while (reader.Read())
            yield return reader;
    }
}

...

var lastNames = reader
    .AsEnumerable()
    .Where(r => r.GetString(0) == "Richard") // have to use index unfortunately
    .Select(r => r.GetString(1))
    .Distinct();

This will still enumerate the entire data reader, but it won't need to buffer the entire DataTable in memory at once (only 1 row at a time is required).

from protobuf-net-data.

Great, thanks. I'll try that out

from protobuf-net-data.

Sorry to trouble you again, do you mean by the above that I need to write extension methods for Select, Where etc.

Taking the code as you have doesn't work as IDataReader doesn't implement Where, I added an extension method similar to:

public static IEnumerable Where(this IDataReader reader,
Func<IDataReader, IDataReader> whereFunc)
{
while (reader.Read())
{
yield return whereFunc(reader);
}
}

Now I can call with something like:
stream.Seek(0, SeekOrigin.Begin);
using (var reader = DataSerializer.Deserialize(stream))
{
while (reader.Read())
{
var lastNames = reader.Where(r => r.GetString(1).Length > 0);
}
}

the trouble is that I'm now not sure how to implement Select. Am I making any sense?

from protobuf-net-data.

Sorry the snippet should return type IEnumerable, which you can call linq methods e.g Where().

from protobuf-net-data.