problem using SharedHashMapBuilder ... about hugecollections-old HOT 6 CLOSED

It is possible there have been changes in the format. Can you delete the
old files first? We need to get to a point where old formats are supported.

On 26 June 2014 20:53, Ben Cotton [email protected] wrote:

Hi,

I fetched/merged into my clone'd workspaces all updates from each of my
upstream views of the various OpenHFT parent repo modules.

I then re-ran my Test at
https://github.com/Cotton-Ben/HugeCollections/blob/master/collections/src/test/java/net/openhft/collections/fromdocs/com/jpmorgan/pingpong_latency/PingPongPlayerLeft.java

via

java org.junit.runner.JUnitCore net.openhft.collections.fromdocs.com.jpmorgan.pingpong_latency.PingPongPlayerLeft

Since this update, when the following line #18
#18 in my Test is
executed:

SharedHashMap<String, BondVOInterface> shmLeft = new SharedHashMapBuilder()
.generatedValueType(true)
.entrySize(320)
.create(
new File("/dev/shm/BondPortfolioSHM"),
String.class,
BondVOInterface.class
);

This test now fails with the following exception:

Time: 0.107
There was 1 failure:

1. bondExample(net.openhft.collections.fromdocs.com.jpmorgan.pingpong_latency.PingPongPlayerLeft)
   java.lang.IllegalArgumentException: invalid size: 343048307875968
   at net.openhft.lang.io.MappedStore.(MappedStore.java:54)
   at net.openhft.lang.io.MappedStore.(MappedStore.java:49)
   at net.openhft.collections.AbstractVanillaSharedHashMap.createMappedStoreAndSegments(VanillaSharedHashMap.java:148)
   at net.openhft.collections.VanillaSharedHashMap.(VanillaSharedHashMap.java:45)
   at net.openhft.collections.SharedHashMapBuilder.create(SharedHashMapBuilder.java:264)
   at net.openhft.collections.fromdocs.com.jpmorgan.pingpong_latency.PingPongPlayerLeft.bondExample(PingPongPlayerLeft.java:18)
   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   at java.lang.reflect.Method.invoke(Method.java:606)
   at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
   at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
   at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
   at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
   at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
   at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
   at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
   at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
   at org.junit.runners.Suite.runChild(Suite.java:127)
   at org.junit.runners.Suite.runChild(Suite.java:26)
   at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
   at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
   at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
   at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
   at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
   at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
   at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
   at org.junit.runner.JUnitCore.run(JUnitCore.java:138)
   at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
   at org.junit.runner.JUnitCore.runMain(JUnitCore.java:96)
   at org.junit.runner.JUnitCore.runMainAndExit(JUnitCore.java:47)
   at org.junit.runner.JUnitCore.main(JUnitCore.java:40)

FAILURES!!!
Tests run: 1, Failures: 1

Should I expect the way that line #18
#18 above is coded in my
Test to work as before without exception? Any recommendation for remedy?

—
Reply to this email directly or view it on GitHub
#36.

from hugecollections-old.

Not really sure what you mean by "Can you delete the
old files first?"

from hugecollections-old.

Ah! I got it

rm -fr /dev/shm/*

did the trick!! ;-)

Thanks Peter ...

from hugecollections-old.

Like I said, you should be able to read old file with newer versions of SHM.
We are in the process of documenting and solidifying the format so this
issue doesn't arise.

I am also working on ways to make data changes more robust with
self-describing data formats. I have some plans to support both zero-copy
in place data structures which also support data format changes.

On 26 June 2014 21:13, Ben Cotton [email protected] wrote:

Closed #36 #36.

—
Reply to this email directly or view it on GitHub
#36 (comment).

from hugecollections-old.

As always, great work!

I am also working on ways to make data changes more robust with
self-describing data formats

Oh man ... you've got our minds percolating now (also, as always).

from hugecollections-old.

You can have a format which allows random access, but also has self
describing data embedded (which doesn't change of course) so it can still
be read if the format changes or you don't have access to such information.

On 26 June 2014 21:24, Ben Cotton [email protected] wrote:

As always, great work!

I am also working on ways to make data changes more robust with
self-describing data formats

Oh man ... you've got our minds percolating now (also, as always).

—
Reply to this email directly or view it on GitHub
#36 (comment)
.

from hugecollections-old.