iamthiago / cassandra-phantom Goto Github PK
View Code? Open in Web Editor NEWCassandra + Phantom Example
Cassandra + Phantom Example
Hi, I have a use case to insert a list of entities like List[Song], any idea on how to do this efficiently?
Hi ,
First Thank you for this awesome example .
however running sbt test fails with the following exception :
15:10:38.807 [pool-1-thread-1] DEBUG c.d.driver.core.ControlConnection - [Control connection] error on localhost/0:0:0:0:0:0:0:1:9142 connection, trying next host
com.datastax.driver.core.exceptions.TransportException: [localhost/0:0:0:0:0:0:0:1] Cannot connect
at com.datastax.driver.core.Connection$1.operationComplete(Connection.java:157) ~[cassandra-driver-core-3.0.0.jar:na]
at com.datastax.driver.core.Connection$1.operationComplete(Connection.java:140) ~[cassandra-driver-core-3.0.0.jar:na]
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680) ~[netty-common-4.0.33.Final.jar:4.0.33.Final]
at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603) ~[netty-common-4.0.33.Final.jar:4.0.33.Final]
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563) ~[netty-common-4.0.33.Final.jar:4.0.33.Final]
at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:424) ~[netty-common-4.0.33.Final.jar:4.0.33.Final]
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:276) ~[netty-transport-4.0.33.Final.jar:4.0.33.Final]
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:292) ~[netty-transport-4.0.33.Final.jar:4.0.33.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528) ~[netty-transport-4.0.33.Final.jar:4.0.33.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468) ~[netty-transport-4.0.33.Final.jar:4.0.33.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382) ~[netty-transport-4.0.33.Final.jar:4.0.33.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354) ~[netty-transport-4.0.33.Final.jar:4.0.33.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112) ~[netty-common-4.0.33.Final.jar:4.0.33.Final]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102]
Caused by: java.net.ConnectException: Connection refused: localhost/0:0:0:0:0:0:0:1:9142
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.8.0_102]
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[na:1.8.0_102]
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224) ~[netty-transport-4.0.33.Final.jar:4.0.33.Final]
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:289) ~[netty-transport-4.0.33.Final.jar:4.0.33.Final]
... 6 common frames omitted
is there any fix / workaround ?
Got this error, when trying to stress the system with JMeter:
18:11:28.576UTC ERROR[cluster14095-worker-2] ExceptionCatchingRunnable - Unexpected error while executing task
io.netty.channel.ChannelException: Unable to create Channel from class class io.netty.channel.socket.nio.NioSocketChannel
at io.netty.bootstrap.AbstractBootstrap$BootstrapChannelFactory.newChannel(AbstractBootstrap.java:455) ~[spray.jar:1.0]
at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:306) ~[spray.jar:1.0]
at io.netty.bootstrap.Bootstrap.doConnect(Bootstrap.java:133) ~[spray.jar:1.0]
at io.netty.bootstrap.Bootstrap.connect(Bootstrap.java:115) ~[spray.jar:1.0]
at com.datastax.driver.core.Connection.initAsync(Connection.java:138) ~[spray.jar:1.0]
at com.datastax.driver.core.HostConnectionPool.initAsync(HostConnectionPool.java:123) ~[spray.jar:1.0]
at com.datastax.driver.core.SessionManager.forceRenewPool(SessionManager.java:256) ~[spray.jar:1.0]
at com.datastax.driver.core.Cluster$Manager.onUp(Cluster.java:1689) ~[spray.jar:1.0]
at com.datastax.driver.core.Cluster$Manager.access$1100(Cluster.java:1268) ~[spray.jar:1.0]
at com.datastax.driver.core.Cluster$Manager$NodeRefreshRequestDeliveryCallback$3.runMayThrow(Cluster.java:2659) ~[spray.jar:1.0]
at com.datastax.driver.core.ExceptionCatchingRunnable.run(ExceptionCatchingRunnable.java:32) ~[spray.jar:1.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_66]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_66]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_66]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_66]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66]
Caused by: io.netty.channel.ChannelException: Failed to open a socket.
at io.netty.channel.socket.nio.NioSocketChannel.newSocket(NioSocketChannel.java:62) ~[spray.jar:1.0]
at io.netty.channel.socket.nio.NioSocketChannel.<init>(NioSocketChannel.java:72) ~[spray.jar:1.0]
at sun.reflect.GeneratedConstructorAccessor17.newInstance(Unknown Source) ~[na:na]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_66]
at java.lang.reflect.Constructor.newInstance(Constructor.java:422) ~[na:1.8.0_66]
at java.lang.Class.newInstance(Class.java:442) ~[na:1.8.0_66]
at io.netty.bootstrap.AbstractBootstrap$BootstrapChannelFactory.newChannel(AbstractBootstrap.java:453) ~[spray.jar:1.0]
... 15 common frames omitted
Caused by: java.net.SocketException: Too many open files
at sun.nio.ch.Net.socket0(Native Method) ~[na:1.8.0_66]
at sun.nio.ch.Net.socket(Net.java:411) ~[na:1.8.0_66]
at sun.nio.ch.Net.socket(Net.java:404) ~[na:1.8.0_66]
at sun.nio.ch.SocketChannelImpl.<init>(SocketChannelImpl.java:105) ~[na:1.8.0_66]
at sun.nio.ch.SelectorProviderImpl.openSocketChannel(SelectorProviderImpl.java:60) ~[na:1.8.0_66]
at io.netty.channel.socket.nio.NioSocketChannel.newSocket(NioSocketChannel.java:60) ~[spray.jar:1.0]
... 21 common frames omitted
Hi @thiagoandrade6,
I wanted to thank you for your work in highlighting some of the features offered in phantom through better documentation and examples than perhaps we have made available.
There are however a few very small issues in the implementation, primarily to do with the fact that you are not meant to use models in the way you are highlighting them and you may advocate patterns that for various technical reasons we don't support or test for. There is a lot of reflection code in phantom and a lot of type magic that is very specifically designed to be bullet proof for a certain access pattern, and it can easily break especially since we don't consider it when building the library. I'm aware this is a bit of a chicken and egg problem but nonetheless.
Some of the observations:
Database
to centralise application code and tables and strongly discourages sharing fields in the model.Database
object like you are advocating.CassandraUtils class
you have included in the project is somewhat very misleading, since it's unintentionally introducing blocking behaviour to address specifying a consistency level, something phantom can already do with great ease through its existing Options
API.Perhaps our efforts would be best spent building a common resource that we can all collaborate on and review since it would provide much better insight for all users and it would follow a pattern we all agree is the "right one".
I will contact you later on this week separately via email to highlight a proposal on how we can achieve this.
Please add the details about run
Removing the old, unmaintained embedded Cassandra in favor of the official Cassandra image.
I have a question about the way data is being stored here
def saveOrUpdate(songs: Song): Future[ResultSet] = {
for {
_ <- SongsModel.store(songs).future()
byArtist <- SongsByArtistsModel.store(songs).future
} yield byArtist
}
If the songs model store is okay but songs by artists model fails, won't you end up in an inconsistent state? Wouldn't it be more appropriate to use a logged batch here because if the second persist fails, the rollback will occur to the first table. This way you won't end up in an inconsistent state.
non issue, sorry
This is an extension to this question:
This question has also been asked here.
In Thiagos example the two tables; 'songs' & 'songs_by_artist' both have the same rows but with different partitions (primary keys / clustering columns)
CREATE TABLE test.songs (
song_id timeuuid PRIMARY KEY,
album text,
artist text,
title text
);
CREATE TABLE test.songs_by_artist (
artist text,
song_id timeuuid,
album text,
title text,
PRIMARY KEY (artist, song_id)
) WITH CLUSTERING ORDER BY (song_id ASC);
This means inserting, updating and deleting across both tables within the SongsService works with the same base data / rows.
How would you for example have a table such as 'artist_songs_counts', with columns 'song_id' (K) and 'num_songs' (++) and ensure that 'SongsService' adds corresponding row to each table; 'songs' & 'songs_by_artist' & 'artist_songs_counts' (where there are different numbers of row but information should be linked, such as the artist info).
CREATE TABLE test.artist_songs_counts (
artist text PRIMARY KEY,
num_songs counter);
sbt:compile
produces
[info] Compiling 4 Scala sources to ...
[error] .../cassandra-phantom/src/main/scala/com/cassandra/phantom/modeling/service/SongsService.scala:41: could not find implicit value for parameter session: com.datastax.driver.core.Session
[error] Await.ready(database.autocreate().future(), 5.seconds)
[error] ^
[error] one error found
[error] (compile:compileIncremental) Compilation failed
code in question:
def createTables(): Unit = {
Await.ready(database.autocreate().future(), 5.seconds)
}
my workaround. Extracting values from database
to pass them back in implicitly looks a bit weird.
def createTables(): Unit = {
implicit val s = database.session
implicit val ks = database.space
Await.ready(database.autocreate().future(), 5.seconds)
}
Hi Thiago,
As one of the last pieces of feedback, you are using a lot of straight CQL examples on the readme, which might mislead people from one of phantom's more powerful capabilities, namely the ability to auto-generate tables and to pre-create the keyspace.
You don't actually need to manually create any of the CQL, although I would agree it's useful to have side by side examples to understand exactly how phantom maps to the adjacent CQL code. Let me know what you think.
Regards,
Flavian
there are lot of changes from 2.9.x , I would like to refactor to newer version
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.