Update to Cassandra 1.1 about astyanax HOT 10 CLOSED

I'd also like to see this.

Currently most of the driver seems to work fine but I'm having a problem when loading a composite column name with Integer and Long components. The Integer and Long Serializers have problems now that the byte buffer returned is a variable number of bytes, I think due to a change in libthrift.

from astyanax.

+1

from astyanax.

any update on this?

from astyanax.

I will look into this later this week.

from astyanax.

calvinfo, can you provide more details about the problems you are seeing and perhaps some code to replicate the problem. So far I have not seeing any issues upgrading to libthrift 0.7

from astyanax.

My mistake, my problem is happening in Cassandra 1.0 as well, so probably unrelated! There seems to be a bug with how the buffers on composites are returned versus how the serializers expect the data to come back.

My column family is set up in the following way:

ColumnFamily: Test
   Key Validation Class: org.apache.cassandra.db.marshal.IntegerType
   Default column value validator: org.apache.cassandra.db.marshal.BytesType
   Columns sorted by: org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.LongType,org.apache.cassandra.db.marshal.IntegerType)

And here's some code to demonstrate the problem, when inserted to a new row.

Keyspace keyspace = connection.get();

ColumnFamily<Integer, Composite> cf = 
    new ColumnFamily<>("Test", IntegerSerializer.get(), CompositeSerializer.get());

try {

    int key = 1;

    keyspace.prepareColumnMutation(cf, key, new Composite(2012L, 2012))
            .putValue("test", null)
            .execute();

    OperationResult<ColumnList<Composite>> result = keyspace.prepareQuery(cf)
                                                            .getKey(key)
                                                            .execute();

    ColumnList<Composite> columns = result.getResult();
    Column<Composite> col = columns.getColumnByIndex(0);

    Composite name = col.getName();
    Long prefix = (Long) name.getComponent(0)
                             .getValue(LongSerializer.get()); // 2012L

    Integer suffix = (Integer) name.getComponent(1)
                                   .getValue(IntegerSerializer.get()); // null
}

The values seem to write properly to cassandra, as seen from the CLI. The problem seems to be happening at com.netflix.astyanax.serializers.IntegerSerializer.fromByteBuffer.

Depending upon the numeric size of the integer, a variable number of bytes seems to be returned. For example, any number less than 256 needs only one byte. This causes the check for 4 bytes remaining to fail unless the returned int actually uses all 4 bytes in its full representation.

from astyanax.

seeing the same problem as @calvinfo

from astyanax.

The problem here is that the IntegerType in cassandra is NOT a 4 byte integer. It is a variable length BigInteger. This has always been a source of confusion with cassandra. Interestingly the Composite class actually does serialize ints as BigInteger so you should be fine here.

My recommendation would first be to fix the column family definition to be,

ColumnFamily: Test
   Key Validation Class: org.apache.cassandra.db.marshal.Int32Type
   Default column value validator: org.apache.cassandra.db.marshal.BytesType
   Columns sorted by: org.apache.cassandra.db.marshal.CompositeType(org.apache.cassandra.db.marshal.LongType,org.apache.cassandra.db.marshal.IntegerType)

The only change here is to change the key validation class to Int32Type since the ints you are serializing for the key are actually 32 bit ints.

For reading the data all you need to do is change the code for reading the suffix to read a big integer.

    BigInteger suffix = (BigInteger) name.getComponent(1)
                                   .getValue(BigIntegerSerializer.get()); // 2012

You could also use the AnnotatedCompositeSerializer but make sure the suffix part is a BigInteger.

from astyanax.

Great, thanks for the help! I hadn't realized composites serialize as BigInteger rather than Integer.

from astyanax.

Version 1.0.4 now uses cassandra 1.1.0

from astyanax.