Giter VIP home page Giter VIP logo

loopback-connector-cassandra's People

Contributors

0candy avatar alexdm0 avatar altsang avatar candytangnb avatar dhmlau avatar hliu9 avatar kallenboone avatar nabdelgadir avatar rmg avatar setogit avatar shenghu avatar ssh24 avatar tatemz avatar virkt25 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

loopback-connector-cassandra's Issues

Unconfigured table <tablename>

Im working with loopback 4 and cassandra,

Model, controller, repository and datasource are created successfully, Also loopback 4 server started without any error.
At the time of post or get a method form model im getting error msg "unconfigured Table tablename". Table already created in database manually.

Unhandled error in GET /testmodels: 500 ResponseError: unconfigured table Testmodel at FrameReader.readError (D:\version2\lb4\explore\node_modules\cassandra-driver\lib\readers.js:326:15) at Parser.parseBody (D:\version2\lb4\explore\node_modules\cassandra-driver\lib\streams.js:194:66) at Parser._transform (D:\version2\lb4\explore\node_modules\cassandra-driver\lib\streams.js:137:10) at Parser.Transform._read (_stream_transform.js:190:10) at Parser.Transform._write (_stream_transform.js:178:12) at doWrite (_stream_writable.js:415:12) at writeOrBuffer (_stream_writable.js:399:5) at Parser.Writable.write (_stream_writable.js:299:11) at Protocol.ondata (_stream_readable.js:693:20) at Protocol.emit (events.js:198:13) at addChunk (_stream_readable.js:288:12) at readableAddChunk (_stream_readable.js:269:11) at Protocol.Readable.push (_stream_readable.js:224:10) at Protocol.Transform.push (_stream_transform.js:151:32) at Protocol.readItems (D:\version2\lb4\explore\node_modules\cassandra-driver\lib\streams.js:109:10) at Protocol._transform (D:\version2\lb4\explore\node_modules\cassandra-driver\lib\streams.js:32:10)

Error: unconfigured table

I use cassandra 3.10
When I create record with loopback API, server show error Error: unconfigured table UserEvent
Check it please !

Unable to use your connector ( model configuration issue, automigration gives NaN 'cassandra' )

Hi guys,

I am trying to use integrate loopback-connector-cassandra with no success.

First issue I stumbled upon is how models should be configured.
According to the documentation, they should be configured similar to this:

zipCodes = db.define('zipCodes', {
  state: String,
  zipCode: Number,
  });

But loopback configures them totally differently! The common structure is:

{
  "name": "AppUser",
  "base": "User",
  "idInjection": true,
  "options": {
    "validateUpsert": true
  },
  "properties": {
  },
  "validations": [],
  "relations": {},
  "acls": [],
  "methods": {}
}

How exactly they should be configured?

I also tried to create tables in Cassandra using this code ( I tried both methods autoupdate & automigrate ):

var server = require('./server');
var ds = server.dataSources.cassandra;
var models = []; // will be aplied to all models
ds.autoupdate(models, function(er) {
  if (er) throw er;
  console.log('Loopback tables [' - models - '] created in ', ds.adapter.name);
  ds.disconnect();
});

My datasource name is "cassandra". However, code just prints:

NaN 'cassandra'

Do you have any basic example of fully configured Loopback project with cassandra datastore? I believe it will help a lot.

Thank you in advance.

Best,

Model.updateAttributes doesn't consider composite primary keys

Description/Steps to reproduce

  1. Create a table w/ composite primary key, e.g.
    CASS_SORTABLE = db.define('CASS_SORTABLE', {
      patBool: {type: Boolean, id: 2},
      str: String,
      patStr: {type: String, id: true},
      num: Number,
      patNum: {type: Number, id: 1},
      yearMonth: {type: String, index: true},
      }, {
      cassandra: {
        clusteringKeys: ['str', 'num DESC'],
        },
      });
  1. Insert a record
    var ROW
    CASS_SORTABLE.create({
      str: cassTestString + '10',
      num: 10,
      patBool: true,
      patNum: 100,
      patStr: cassTestString + '100',
    }, function(err, m) {
      ROW = m;
    });
  1. Update attributes of the row,
    ROW_1.updateAttributes({ 
      str: ROW_1.str,
      num: ROW_1.num,
      patBool: ROW_1.patBool,
      patNum: ROW_1.patNum,
      patStr: ROW_1.patStr,
      yearMonth: '2018-06'}, function(err, rows) {
      });

The update failed w/ error

     Uncaught AssertionError: expected ResponseError {
  name: 'ResponseError',
  info: 'Represents an error message from the server',
  message: 'PRIMARY KEY part str found in SET part',
  code: 8704,
  query: 'UPDATE "CASS_SORTABLE" SET "str"=?,"num"=?,"yearMonth"=? WHERE "patStr"=?'
} to not exist

Link to reproduction sandbox

Expected result

The operation is expected to succeed.

Additional information

ROW.updateAttributes called these codes,
SQLConnector.prototype.updateAttributes = function(model, id, data, options, cb) , which is called by
DataAccessObject.prototype.updateAttributes.

In DataAccessObject.prototype.updateAttributes, it calls

              invokeConnectorMethod(connector, 'updateAttributes', Model,
                [getIdValue(Model, inst), Model._forDB(context.data)],
                options, updateAttributesCallback);

getIdValue(Model, inst) doesn't consider the composite primary key.

The ideal fix is in juggler to get an object from getIdValue(Model, inst) instead of a string value. But before that, a workaround can be in SQLConnector.prototype.updateAttributes, to put primary keys inwhere and remove them from data.

schema migration for add/drop column(s) does not work due to cql syntax

Description/Steps to reproduce

Cassandra loopback connector uses incorrect syntax for schema migration.

Issue 1 - adding a column
Error: no viable alternative at input 'NOT' (ALTER TABLE table1 ADD newCol [TEXT] NOT...)
Correct syntax: ALTER TABLE table1 ADD col1 TEXT;

Issue 2 - dropping multiple columns
Error: mismatched input ',' expecting EOF (ALTER TABLE table1 DROP col1[,]...)
Correct syntax: ALTER TABLE table1 DROP (col1, col2);

Link to reproduction sandbox

Expected result

Cassandra table is succussfully migrated with the correct syntax below:
ALTER TABLE table1 ADD col1 TEXT;
ALTER TABLE table1 DROP (col1, col2);

Additional information

https://github.com/strongloop/loopback-connector-cassandra/blob/master/lib/migration.js#L180
https://github.com/strongloop/loopback-connector-cassandra/blob/master/lib/migration.js#L270
https://docs.datastax.com/en/cql/3.3/cql/cql_reference/cqlAlterTable.html

Creating Arrays?

Suggestion

Use Cases

I wanna store an array of strings.

Examples

I tried to use this in Loopback 4, but all I get in Cassandra is a simple text field:

  @property.array(String, {
    default: [],
    jsonSchema: {
      uniqueItems: true,
      type: 'string',
      format: 'uri-reference',
      errorMessage: 'An array of file URLs is expected',
    },
  })
  attachments?: string[];

How to create an array?

Acceptance criteria

TBD - will be filled by the team.

Getting Error Code : 8192 (Represents an error message from the server)

Respected Sir,

I am using strongloop with data source CASSANDRA. The test connection is success. I can try to add one new row in the DB but getting error code 8192 as a response from server. See attached screenshot for your reference.

Can you please help?. Thanks in advance for your help.

loopback-connector-cassandra-error

Thanks,
Bhavin Bhalodia.

Fix CI failures

Travis CI is failing due to Connection fails: NoHostAvailableError: All host(s) tried for query failed. First host tried, 127.0.0.1:9042: Error: connect ECONNREFUSED 127.0.0.1:9042. See innerErrors. It will be retried for the next request.

Jenkins has more random failures but one the consistent ones are:

  • Node 12:
    • Many Error: Timeout of 10000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves. errors (sometimes happens on other Node versions, but most often Node 12)
    • 1) cassandra custom tests discoverPrimaryKeys:
      Uncaught AssertionError: expected Array [ 'num', 'patBool', 'patNum', 'patStr', 'str' ] to equal Array [ 'patNum', 'patBool', 'patStr', 'str', 'num' ] (at '0', A has 'num' and B has 'patNum')
      + expected - actual

       [
      -  "num"
      +  "patNum"
         "patBool"
      -  "patNum"
         "patStr"
         "str"
      +  "num"
       ]

Acceptance Criteria

  • Fix the connection issue on Travis CI
  • Fix the discoverPrimaryKeys failure (could sort the array to fix)
  • Try increasing the timeout to see if the tests pass, otherwise figure out why these tests are failing for Node 12 and fix it

user updateAttribute throws exception while updating password

I am trying update user password and I am using updateAttribute function.

I am seeing an interesting issue. It does update the field but it also sets the error. I am see this error
It tries to delete accesstoken but in accesstoken key is id. If I change the schema to change primary_key to user_id then other functions will start throwing errors;

message: 'Some partition key parts are missing: id',
info: 'Represents an error message from the server',
code: 8704,
coordinator: 'local:9042',
query: 'DELETE FROM "AccessToken" WHERE "userId" IN (?)' }

  user.updateAttribute('password', password, function(error, response) {
               if (error) {
                 return callback(error);
              }
              console.log(response);

              return callback(null, {userId: response.id});
          });

migration failed drop multiple columns case

Description/Steps to reproduce

migration tables that have multiple drop columns do not work.

Steps to reproduce

  1. create table model with 5 columns
  2. apply migration
  3. change table to have 2 columns
  4. apply alter model.
  5. check cassandra to confirm migration status

Link to reproduction sandbox

https://github.com/strongloop/loopback-connector-cassandra/blob/eb176bb71475276031e298d005aa05b4810f5a9a/lib/migration.js#L274

`

  if (actualFieldNotPresentInModel(actualField, model)) {

    sql.push('DROP ' + self.escapeName(actualField.column));

  }

`

it generates sql like

ALTER TABLE "abc" DROP "column3", DROP "column4", DROP "column5"
and that will fail with
line 1:42 mismatched input ',' expecting EOF (ALTER TABLE "abc" DROP "column3"[,]...),
while from below cql, it should be DROP column | ( column_list )

https://docs.datastax.com/en/dse/6.0/cql/cql/cql_reference/cql_commands/cqlAlterTable.html

Expected result

DROP ("column3", "column4", "column5")

Additional information

https://docs.datastax.com/en/dse/6.0/cql/cql/cql_reference/cql_commands/cqlAlterTable.html
https://github.com/strongloop/loopback-connector-cassandra/blob/eb176bb71475276031e298d005aa05b4810f5a9a/lib/migration.js#L274

Datasource not available after installing loopback-connector-cassandra

We are wondering about the status of the project because the datasource is not available even after installing loopback-connector-cassandra with npm install <local-directory> (so it's not related to issue #2).

Also the link for the official documentation at http://docs.strongloop.com/display/LB/Cassandra+connector (from README.md) is dead and searching for "cassandra" in the strongloop documentation yields nothing.

Could you clarify the status of the project please ?

pagination support

design question: do we expose the TOKEN to the C* connector API, or can we completely hide in under the hood and support pagination?

Run shared tests from both v3 and v4 of juggler

At the moment, our connectors are installing juggler 3.x to run the test suite. When we make a change to juggler@3, cis-jenkins detects downstream dependency and triggers CI build of connectors with the modified juggler version. When we make a change to juggler's master, no downstream builds are triggered.

In the past, we were maintaining multiple branches to test connectors against different juggler versions. E.g. loopback-2.x containing code from "master" but installing juggler 2.x for testing. This is rather impractical, because we don't have bandwidth to update those other branches with changes made to master.

Let's rework the connector test setup to execute tests from both juggler versions 3.x and 4.x, as we are already doing in other connector repositories.

The reference pull request to draw inspiration from: loopbackio/loopback-connector-mongodb#519. See also loopbackio/loopback-connector-postgresql#376.

switch keyspace

v1.0.0 assumes all connector calles aredone in one keyspace.
Note: USE query in lib/migration.js/createTable

Unable to save array entry for list<text> column

I am using Cassandra datasource in loopback 3, and trying to store values in list datatype column of a table(keyspace) of Cassandra datasource. But while creating an entry from loopback explorer, we are getting TypeError: Not a valid list value, expected Array obtained '["Morning","Afternoon","Evening"]'

Here is my loopback model:

{
  "name": "Time",
  "base": "PersistedModel",
  "idInjection": true,
  "options": {
    "validateUpsert": true
  },
  "properties": {
    "options": {
      "type": "array",
      "format": "list"
    }
  },
  "validations": [],
  "acls": [],
  "methods": {}
}

Tried debugging loopback-cassandra-datasource and saw options converted to string in params.

Data passed from loopback explorer:

{
  "options": ["Morning","Afternoon","Evening"]
}

Data logged from beforeRemote hook of the model:

{ options:
   [ 'Morning',
     'Afternoon',
     'Evening' ]
}

package.json

"dependencies": {
  "loopback": "3.26.0",
  "loopback-boot": "2.28.0",
  "loopback-component-explorer": "6.4.0"
  "loopback-connector-cassandra": "1.7.1"
}

The typeof options is object. I was debugging loopback cassandra connector and found typeof options as string in encoder.js of cassandra-driver node module. We have migrated from MongoDB to Cassandra and didn't face any issues for MongoDB driver. To what type or which format do I need to set for options property? Please let me know if any additional information required. Thank you in advance.

Milliseconds are missing when save and retrieve date type from cassandra.

Description/Steps to reproduce

Create a model which uses Date type. Try to save with milliseconds. Check inside Cassandra how the value was saved. It loses milliseconds.
Save manually a date in Cassandra using milliseconds. Retrieve it from loopback 4, you will see milliseconds are set to 0.

The problem is inside cassandra.js
When you save the date it uses the function:

function dateToMysql(date) {
  return date.getUTCFullYear() + '-' +
    fillZeros(date.getUTCMonth() + 1) + '-' +
    fillZeros(date.getUTCDate()) + ' ' +
    fillZeros(date.getUTCHours()) + ':' +
    fillZeros(date.getUTCMinutes()) + ':' +
    fillZeros(date.getUTCSeconds());

  function fillZeros(v) {
    return parseInt(v) < 10 ? '0' + v : v;
  }
}

As you can see millisecons are not being taking into account.

This could be a possible solution:

function dateToMysql(date:Date) {
  function fillZeros(v, n = 2) {
    return ('0'.repeat(n-1) + v).slice(-n);
  }
  return date.getUTCFullYear() + '-' +
    fillZeros(date.getUTCMonth() + 1) + '-' +
    fillZeros(date.getUTCDate()) + ' ' +
    fillZeros(date.getUTCHours()) + ':' +
    fillZeros(date.getUTCMinutes()) + ':' +
    fillZeros(date.getUTCSeconds()) + '.' +
    fillZeros(date.getUTCMilliseconds(),3); 
}

When you retrieve a Date it uses function:

Cassandra.prototype.fromColumnValue

case 'Date':

        // Cassandra allows, unless NO_ZERO_DATE is set, dummy date/time entries
        // new Date() will return Invalid Date for those, so we need to handle
        // those separate.
        if (val == '0000-00-00 00:00:00') {
          val = null;
        } else {
          val = new Date(val.toString().replace(/GMT.*$/, 'GMT'));
        }

val.toString() get rid of millisecons, val is actually a Date, you can return it without any conversion. Return val without any conversion solves the issue.

Connecting to cloud based cassandra nodes requires missing field

To connect to certain cloud providers they require "localDataCenter" this is missing in both cassandra.js and requires an update to cassandra-connector in package.json.

I added to following and it now allows me to connect.

"cassandra-driver": "3.5.0"" needs to be updated to ^4.0.0
reference "https://docs.datastax.com/en/developer/nodejs-driver/4.6/upgrade-guide/"

and 'function generateOptions(settings)' needs the following line added.
"clientOptions.localDataCenter = settings.localDataCenter;"

Project Status

Whitch is the status of the project?. I tried install the package from npm and anything was download, and not exist any commit from many time.

I have and cassandra prjoect and would like to use loopback with this connector ...

Support cassandra collections

Description

  • Define model to use collections
  • Insert/update column of collection type
  • Extended operation on collection type, e.g. updating an individual element of a Map
  • Nested collections

Link to reproduction sandbox

Expected result

Additional information

unable to create save basic user in cassandra

I am unable to save basic user for authentication loopback 3.
form.js:128:32)',
message: 'IN predicates on non-primary-key columns (property) is not yet supported',
info: 'Represents an error message from the server',
code: 8704,
coordinator: 'localhost:9042',
query: 'SELECT "model","property","accessType","permission","principalType","principalId","id" FROM "ACL" WHERE "model"=? AND "property" IN (?,?) AND "accessType" IN (?,?) ALLOW FILTERING' }
Unhandled error for request POST /RMUser: Error: IN predicates on non-primary-key columns (property) is not yet supported.

Model config below.

"User": {
"dataSource": "db",
"public": false
},
"AccessToken": {
"dataSource": "db",
"public": false
},
"ACL": {
"dataSource": "cassandra",
"public": false
},
"RoleMapping": {
"dataSource": "cassandra",
"public": false
},
"Role": {
"dataSource": "cassandra",
"public": false
},
"RMUser": {
"dataSource": "cassandra",
"public": true,
"options": {
"emailVerificationRequired": true
}
}

I used automigrate to create tables. It created table like this. Which is incorrect for the query below

CREATE TABLE nfeeds."ACL" (
id uuid PRIMARY KEY,
"accessType" text,
model text,
permission text,
"principalId" text,
"principalType" text,
property text
) WITH bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND comment = ''
AND compaction = {'class': 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
AND dclocal_read_repair_chance = 0.1
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99PERCENTILE';

NPM empty

npm i loopback-connector-cassandra

Results in an empty folder in node_modules/. Can you publish up a new version?

geoPoint support

create user defined type and support GeoPoint type as many of other DB connectors do.

CI tests Failed with [email protected]

Description/Steps to reproduce

Imported tests from loopback-datasource-juggler failed. They passed with [email protected]. But failed w/ [email protected].

Failed tests are in manipulation.tests.js it.only('should have updated password hashed with replaceAttributes'. The error occurred in dao.js, see following code snippet,

        function replaceCallback(err, data) {
          if (err) return cb(err);
          context.data = data;  // This is new code. Data is undefined as update in cassandra returns nothing. 
          if (typeof connector.generateContextData === 'function') {
            context = connector.generateContextData(context, data);
          }
          var ctx = {
            Model: Model,
            hookState: hookState,
            data: context.data,
            isNewInstance: false,
            options: options,
          };
          Model.notifyObserversOf('loaded', ctx, function(err) {
            if (err) return cb(err);

            if (ctx.data[pkName] !== id && !Model._warned.cannotOverwritePKInLoadedHook) { // Here error is thrown!!!
              Model._warned.cannotOverwritePKInLoadedHook = true;

Link to reproduction sandbox

Expected result

Additional information

keyspace parameters

In lib/migration.js/createTable, it tries to create keyspace if not exists with predetermined parameters such as:

      ' WITH REPLICATION = { \'class\' : \'SimpleStrategy\',' +
      ' \'replication_factor\' : \'3\' }';

make it client definable

automigration create database name 'undefined', create tables within 'undefined' database.

datasource.json

"cassandraConn": {
"host": "192.168.4.21",
"port": 9042,
"database": "emsv2_testing",
"password": "",
"name": "cassandraConn",
"user": "",
"connectTimeout": 30000,
"readTimeout": 30000,
"connector": "cassandra"
}

automigration create database name 'undefined', create tables within 'undefined' database. when posting data showing error,

{
"error": {
"statusCode": 500,
"name": "NoHostAvailableError",
"message": "Connecting after shutdown is not supported",
"info": "Represents an error when a query cannot be performed because no host is available or could be reached by the driver.",
"innerErrors": null,
"stack": "NoHostAvailableError: Connecting after shutdown is not supported\n at Client._connectCb (/home/rd/enms/node_modules/cassandra-driver/lib/client.js:397:21)\n at Client.promiseWrapper (/home/rd/enms/node_modules/cassandra-driver/lib/utils.js:410:13)\n at Client.connect (/home/rd/enms/node_modules/cassandra-driver/lib/client.js:384:31)\n at Array.utils.series.next (/home/rd/enms/node_modules/cassandra-driver/lib/client.js:854:18)\n at next (/home/rd/enms/node_modules/cassandra-driver/lib/utils.js:714:17)\n at Object.series (/home/rd/enms/node_modules/cassandra-driver/lib/utils.js:698:3)\n at Client._innerExecute (/home/rd/enms/node_modules/cassandra-driver/lib/client.js:853:9)\n at Client.handler (/home/rd/enms/node_modules/cassandra-driver/lib/client.js:487:10)\n at Client.promiseWrapper (/home/rd/enms/node_modules/cassandra-driver/lib/utils.js:410:13)\n at Client.execute (/home/rd/enms/node_modules/cassandra-driver/lib/client.js:485:31)\n at Cassandra.executeSQL (/home/rd/enms/node_modules/loopback-connector-cassandra/lib/cassandra.js:238:15)\n at /home/rd/enms/node_modules/loopback-connector/lib/sql.js:651:10\n at /home/rd/enms/node_modules/loopback-datasource-juggler/lib/observer.js:259:9\n at doNotify (/home/rd/enms/node_modules/loopback-datasource-juggler/lib/observer.js:155:49)\n at Cassandra.ObserverMixin._notifyBaseObservers (/home/rd/enms/node_modules/loopback-datasource-juggler/lib/observer.js:178:5)\n at Cassandra.ObserverMixin.notifyObserversOf (/home/rd/enms/node_modules/loopback-datasource-juggler/lib/observer.js:153:8)"
}
}
my cassandra database is alive..

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.