frees-io / freestyle-cassandra Goto Github PK

Research and discussion about the CQL Interpolator feature

Based on #14 we should add support for inserting and updating case classes.

Planned features:

• Spike #48
• case classes annotation for defining the table
• insert/select operation on case classes that returns FS
• index support
• CREATE TABLE statement generator (and potentially an FS operation)

For #22 we need a model with all possible configuration values at statement level. For example:

• setConsistencyLevel
• setRetryPolicy
  ...

See Statement docs

Operations for fetching cluster information and health status

As part of this ticket we should create some coding issues

We need to add the following methods to LowLevelAPI to increase its safety:

Datastax driver code

public interface PreparedStatement {
  ...
    public BoundStatement bind();
  ...
}

public class BoundStatement  {
  ...
    public BoundStatement setBytesUnsafe(int i, ByteBuffer v) {
        return wrapper.setBytesUnsafe(i, v);
    }

    public BoundStatement setBytesUnsafe(String name, ByteBuffer v) {
        return wrapper.setBytesUnsafe(name, v);
    }
  ...
}

Freestyle-cassandra

trait LowLevelAPI {
  ...
  def bind: FS[BoundStatement]

  def setBytesUnsafe(index: Int, value: ByteBuffer): FS[BoundStatement]

  def setBytesUnsafe(name: String, value: ByteBuffer): FS[BoundStatement]
  ...
}

As part of this ticket, we should also:

• Redefine the MappedField to something like

  trait MappedField {
    def name: String
    def mapField[M[_]](implicit M: MonadError[M, ByteBuffer]): ByteBuffer
  }

• Modify the interpolator acordingly

This ticket depends on #71

Currently, in #80 we're making always a prepareStatement

We should add the option for executing the query without preparing the statement.

A possible solution could be to use SessionAPI.executeStatement and override the SimpleStatement class with:

import scala.collection.JavaConverters._
class ByteBufferSimpleStatement(query: String, values: List[ByteBuffer]) extends SimpleStatement(query, values.toArray) {
  override def getValues(protocolVersion: ProtocolVersion, codecRegistry: CodecRegistry): Array[ByteBuffer] = values.toArray
}

Implements a decoder for Cluster using case classy

Using the library cql-parser we should implement a local schema provider.

For a "compile time" version of the interpolator, the user needs to define his own interpolator in a different compile unit.

TBD

Research and discussion about the High Level API feature

• Spike #46
• FieldMapper (from case class to List[(String, ByteBuffer)]) - #38
• Insert statement from a case class - #124
• Populate a prepared statement with a case class instance - #125
• Method for inserting case class instances - #126
• Select statements from case classes - #127
• Fetch case classes instances - #128
• Unify all methods under the same class/trait

Research and discussion about the Schema Validator feature

The interpolator returns a M[(String, List[OutputValue])]. We should add an implicit def for converting this value into an M[ResultSet]

The first release should be performed under the name frees-cassandra.

Research and discussion about the Streaming feature

From #30

In the SessionAPI and potentially in the interpolator

• Spike #43
• Define the configuration values for statements #28
• High-Level API algebra #29
• Implement a handler based on LowLevelAPI

The schema information should be read from a cassandra connection

As part of #17 we need a mechanism for retrieving information about the schema (keyspace and tables).

We need a FieldMapper that maps a case class into a list with the field names and the values exported to ByteBuffer values.

In order to test the interpolator, we need a Cassandra instance running before the test compilation. In this way, one approach could be to start and stop the Cassandra instance from SBT.

Also, finish the test MetadataInterpolatorTest in macros-tests project.

See #57

In this ticket, we should take a decision about how the delete operation will be written with the Datastax driver and/or LowLevelAPI.

• Spike #42

It should be type safe and fail at compile time

Maybe we could use the propensive/contextual library

• Spike #45
• Basic Interpolator #60
• Parameter support #71
• Macro expansion #68
• Type checking #91
• as[Model] and asList[Model] operations #93

• Spike #44

See #57

In this ticket, we should take a decision about how the insert operation will be written with the Datastax driver and/or LowLevelAPI.

We should allow the creation of a MetadataSchemaProvider that receives a properties files with the cluster configuration.

This ticket depends on #57

We need to define how all these operations will be written with a CQL Interpolator and what features will be supported.

For example:
Select

cql"SELECT name FROM keyspace1.users WHERE id = $myId"

Aspects to consider:

• The interpolator should accept variables, but how are the values decoded. Currently, we have a ByteBufferCodec definition, but the instances should be available in the scope
• What is the value type returned by the interpolator? FreeS[F, ResultSet] for the selects?

See #57

In this ticket, we should take a decision about how the delete operation will be written with the Datastax driver and/or LowLevelAPI.

Currently, the schema provider and schema validator classes contains methods that return Either. We should abstract all of these methods and return M[_] instead, receiving an implicit MonadError in the methods.

As part of this story we should:

• Spike #47
• Define the keyspace and table models - Issue #24
• Add a mechanism (through implicits?) for retrieving schema information, providing different strategies (local schema sql, remote connection to the db, ...) - Issue #27
• Add a schema parser for keyspaces - PR #15
• Add a scheme parser for tables - Not needed (#49)
• Implement a local schema sql provider - Issue #49
• Implement a remote scheme provider - Issue #51

We need a statement generator able to create valid SQL insert and select statements from a Type

Research and discussion about the Auto Insert/Update for case classes feature

Implement a basic CQL interpolator based on propensive/contextual and with support for statements without arguments

As part of ticket #11 we should decide how to test the decoders, preferably using scalacheck

We want to give support for CRUD operations. All operations will be delegated to the LowLevelAPI. The LowLevelAPI allows the execution of all async operations of the Session object from the Datastax driver.

In this ticket, we should take a decision about how the select operation will be written with the Datastax driver and/or LowLevelAPI.

For example:

Select

# CQL
SELECT name FROM keyspace1.users WHERE id = <myId>

// Datastax code
val prepSt = session.prepare("SELECT name FROM keyspace1.users WHERE id = ?")
val boundStatement = prepSt.bind()
boundStatement.setBytesUnsafe("id", byteBufferCodec.serialize(myId))
val resultSet = session.execute(boundStatement)

def prepareStatement(implicit c: ByteBufferCodec[UUID]): FreeS[F, BoundStatement] =
  lowLevelAPI.prepare("SELECT name FROM keyspace1.users WHERE id = ?") map { st =>
    val boundStatement = st.bind()
    boundStatement.setBytesUnsafe("id", c.serialize(newUser.id))
    boundStatement
}

for {
  prepSt    <- prepareStatement
  resultSet <- lowLevelAPI.execute(prepSt)
} resultSet

See the freestyle-cassandra-example to learn about how to use LowLevelAPI

We should take care of what's the recommended way to make each operation in Cassandra

Research and discussion about the Object Mapper feature.

Basically, we need to describe what features we want and how they should like (in code)

Using the version 0.0.1 we should add an example using the interpolator in the frees-io/freestyle-cassandra-example repo.

As part of this ticket, we should take the decision of creating a new project inside the repo or just a new main App.

I've tested the interpolator with this code, that can be used to create the example:

package freestyle.cassandra.sample

import java.util.UUID

import cats.instances.future._
import com.datastax.driver.core.{ProtocolVersion, ResultSet, Session, TypeCodec}
import freestyle._
import freestyle.FreeS
import freestyle.async.implicits._
import freestyle.asyncMonix.implicits._
import freestyle.cassandra.api.{ClusterAPI, SessionAPI}
import freestyle.cassandra.codecs._
import freestyle.cassandra.query.interpolator._
import freestyle.cassandra.query.interpolator.RuntimeCQLInterpolator._
import freestyle.cassandra.sample.Implicits._
import freestyle.implicits._
import monix.cats._
import monix.eval.{Task => MonixTask}
import monix.execution.Scheduler

import scala.collection.JavaConverters._
import scala.concurrent.Await
import scala.concurrent.duration.Duration
import scala.util.control.NonFatal

object Test extends App {

  implicit val executionContext: Scheduler = Scheduler.Implicits.global

  def connect[F[_]](implicit clusterAPI: ClusterAPI[F]): FreeS[F, Session] =
    clusterAPI.connectKeyspace("demodb")

  def closeSession[F[_]](implicit sessionAPI: SessionAPI[F]): FreeS[F, Unit] = sessionAPI.close

  def close[F[_]](implicit clusterAPI: ClusterAPI[F]): FreeS[F, Unit] = clusterAPI.close

  implicit val session: Session =
    Await.result(connect[ClusterAPI.Op].interpret[MonixTask].runAsync, Duration.Inf)

  val uuid = java.util.UUID.fromString("43aab350-f393-45be-a10e-d2244e45818c")

  implicit val uuidCodec: ByteBufferCodec[UUID] =
    freestyle.cassandra.codecs.byteBufferCodec(TypeCodec.uuid(), ProtocolVersion.V5)

  val task: MonixTask[ResultSet] = cql"SELECT id, name FROM User WHERE id = $uuid".asResultSet[MonixTask]
  val resultSet: ResultSet       = Await.result(task.runAsync, Duration.Inf)
  try {
    resultSet.iterator().asScala.foreach { row =>
      println(s"User ${row.get[java.util.UUID](0, TypeCodec.uuid()).toString}")
    }
  } catch {
    case NonFatal(t) =>
      t.printStackTrace()
  }

  Await.result(closeSession[SessionAPI.Op].interpret[MonixTask].runAsync, Duration.Inf)

  Await.result(close[ClusterAPI.Op].interpret[MonixTask].runAsync, Duration.Inf)

}

https://github.com/scalameta/paradise/issues/59

Actual version: 2.1.0
Patch: 2.1.1
Minor: -
Major: -

Create an algebra with methods for inserting, updating and reading columns and tables and for modifying counts

Related to frees-io/sbt-freestyle#17

Research and discussion about the Query Builder feature

For #17 we need a model for the table

Base definition on Create table DDL