Giter VIP home page Giter VIP logo

mongolike's Introduction

Mongolike

Mongolike is an experimental MongoDB clone being built on top of PLV8 and Postgres.

Implemented (so far)

  • create_collection()
  • drop_collection()
  • save()
  • find()
  • runCommand() (Map/Reduce)
  • ensureIndex()
  • removeIndex()
  • getIndexes()

Installing

Install PLV8

Visit http://code.google.com/p/plv8js/wiki/PLV8 and follow the build instructions.

Install Mongolike

The Easy Way

The easy way to install is to use node.js.

$ npm install -g mongolike
$ mongolike-install -d yourdb

The Slightly Less Easy Way

$ for file in sql/*.sql; do psql yourdb < $file; done

Running Tests

Mongolike includes a test suite and a test runner.

$ test/test_runner.js -d yourdb

Additional tests can be added to test/tests.sql.

Using

All commands must be prefixed by SELECT, and are modified slightly to work in the Postgres environment.

create_collection(collection)

Create a collection.

Example:

SELECT create_collection('test');

drop_collection(collection)

Drop a collection.

Example:

SELECT drop_collection('test');

save(collection, object)

Save an object into a collection.

Example:

SELECT save('test', '{ "foo": "bar" }');

find(collection /*, terms, limit, skip */)

Find an object, with optional terms, limit, and skip.

Example:

SELECT find('test', '{ "type": { "$in": [ "food", "snacks" ] } }');

runCommand(command)

Run a command on the Database. Currently only mapReduce is supported.

NOTE The JSON object cannot have carriage returns, the example below does for readability.

Example:

SELECT runCommand('{
  "map": "function MapCode() {
    emit(this.Country, {
      \"data\": [
        {
          \"city\": this.City, 
          \"lat\":  this.Latitude, 
          \"lon\":  this.Longitude
        }
      ]
    });
  }",
  "reduce": "function ReduceCode(key, values) {
    var reduced = {
      \"data\": [ ]
    };
    for (var i in values) {
      var inter = values[i];
      for (var j in inter.data) {
        reduced.data.push(inter.data[j]);
      }
    }
    return reduced;
  }",
  "mapreduce": "cities",
  "finalize": "function Finalize(key, reduced) {
    if (reduced.data.length == 1) {
      return {
        \"message\" : \"This Country contains only 1 City\"
      };
    }

    var min_dist = 999999999999;
    var city1 = { \"name\": \"\" };
    var city2 = { \"name\": \"\" };
    var c1;
    var c2;
    var d;

    for (var i in reduced.data) {
      for (var j in reduced.data) {
        if (i >= j) continue;
        c1 = reduced.data[i];
        c2 = reduced.data[j];
        d = Math.sqrt((c1.lat-c2.lat)*(c1.lat-c2.lat)+(c1.lon-c2.lon)*(c1.lon-c2.lon));

        if (d < min_dist && d > 0) {
          min_dist = d;
          city1 = c1;
          city2 = c2;
        }
      }
    }
    return {
      \"city1\": city1.city,
      \"city2\": city2.city,
      \"dist\": min_dist
    };
  }" }');

ensureIndex(collection, terms /*, type */)

Creates a new index on a collection.

Example:

SELECT ensureIndex('test', '{ "foo", "bar" }', '{ "unique": true }');

removeIndex(collection, name)

Removes an index from a collection by name.

Example:

SELECT removeIndex('test', 'idx_col_woo_foo');

removeIndex(collection, terms)

Removes an index from a collection by terms.

NOTE in order to remove an index with terms you MUST cast the query due to how Postgres handles JSON.

Example:

SELECT removeIndex('test', '{ "foo", "bar" }'::json);

getIndexes(collection)

Retrieves all indexes for a given collection.

Example:

SELECT getIndexes('test');

Importing the Data

I have included a modest amount of data for testing and benchmarking, both for Postgres and for MongoDB (1,706,873 rows).

Importing into Postgres:

$ psql yourdb < data/cities.sql

This will create the collection and save() all of the data.

Importing into MongoDB

$ mongoimport --collection cities --type csv --headerline --file data/cities.csv --db yourdb

Follow along at http://legitimatesounding.com/blog/

mongolike's People

Contributors

jerrysievert avatar e1ven avatar ryleysill93 avatar alex avatar msakrejda avatar

Stargazers

Grey avatar Charles avatar Paul G avatar Sandalots avatar Alexey Palazhchenko avatar  avatar Ruben Diaz avatar Leonid Shirmanov avatar Toan Tran avatar Stathis Sideris avatar Christian Grabowski avatar Bruno Lavoie avatar  avatar Igor Leahu avatar Keegan McCallum avatar Alexander avatar  avatar Shawn Deleurme avatar  avatar Val Packett avatar  avatar Thuc Nguyen Canh avatar Steven Lee avatar  avatar RYeah Sh avatar Alex Steffes avatar Ruben avatar Asad Dhamani avatar Mikhail Kuzmin avatar Kevin Ridgway avatar Angus H. avatar Qasim Zaidi avatar Michał Czyż avatar Alan Grosskurth avatar  avatar  avatar Dammian Miller avatar Sunguk Lee avatar brawnski armstrong avatar Raphaël Huchet avatar Gwen Glaser avatar Fabien Bourgeois avatar Ken Keiter avatar Masroor Naved avatar KS Chan avatar Jits avatar Rob Morgan avatar Gabriel Dumitrescu avatar Dick Tang avatar Kensuke Nagae avatar Nana GB avatar  avatar Mark Chapman avatar Rafal Piekarski avatar Nejuf Rowan avatar renzo avatar David avatar Ray Vanderborght avatar Alesei N avatar Michael Andrews avatar  avatar RK Aranas avatar Tad avatar James Walker avatar  avatar Zubair Quraishi avatar Alexander Behrens avatar Stipe Kotarac avatar Joonho Choi avatar Quentin Pré avatar Eugene Duvenage avatar Martin Heidegger avatar Howard Mei avatar Scott Ames-Messinger avatar Corey Kuwanoe avatar Sangcheol Park avatar  avatar J Wynia avatar  avatar John Lee avatar Luís Portela Afonso avatar Tom Rodenberg avatar Chris Sandulow avatar Mara Morton avatar Leslie avatar Riley Guerin avatar Marcello Prattico avatar Christoph Grabo avatar Patrick Shampine avatar Drew Miller avatar Sri Dodda avatar José Dulanto avatar Roman Neuhauser avatar Theron Boerner avatar Gopal Patel avatar  avatar Nikola Kotur avatar Armand Zerilli avatar Fedor Nikolaev avatar Owais Lone avatar

Watchers

Dan Shaw avatar Henry Huang avatar evandrix avatar Dax Reyes avatar  avatar Bob Henkel avatar Stacy Jeptha avatar  avatar James Cloos avatar q3]~c~g7RV7%V=H[4n avatar dai_yamashita avatar renzo avatar  avatar Chatchai Daecha avatar  avatar  avatar Vamsi avatar  avatar

mongolike's Issues

move tests out of tests.sql and into their own module [test/test_runner.js]

filename : test/test_runner.js
line # : 38
tag : todo
md5 : d07207f6c68c08e5df34b85bdbc50288

// [todo] - move tests out of tests.sql and into their own module
function loadTests (client) {
  client.query(tests, function (err) {
    if (err) {
      console.error('could not load tests', err.toString());

      client.end();
      return;
    }

    runTests(client);
  });
}
function runTests (client) {
  client.query("SELECT mongolike_tests()", function (err, results) {
    client.end();

add insert

add the functionality for insert to work. essentially save but with upsert explicit.

add versioning

add versioning to the collection schema.

if exists:

  • check version
  • run migrations from current version through most recent

if not:

  • run full creation scripts

Trying to test this locally

Thanks for writing this, it's a very cool idea! ;)

I'm trying to test locally, and ran into some problems-

I was getting a "DETAIL: undefined() LINE 0: [object Object]" error when trying to save any javascript, including the .sql file you included.

It looks like this is because it's treating it as an object, not a JS string.
I was able to get past this, by replacing

var obj = JSON.parse(data);

with

var obj = JSON.parse(JSON.stringify(data));

After making this change I was able to import as expected.

I am still having issues querying, however.

hostname=# SELECT find('cities', '{ "Country": "zw" }');
ERROR: type "undefined" does not exist
hostname=#

ERROR: type "undefined" does not exist

I tried a few variations of typeof() instead of === or !==, but I wasn't able to get past this.
Does this work on your machine as it is?
Any idea what I might be doing wrong?

I'm on OSX 10.8.2, using psql 9.2.1, v8 3.9.24, plv8 from git.

Any advise would be appreciated.

Why javaScriptStyle then python_style?

This looks awesome! Quick question: I notice some methods (getIndexes) are camel case per JavaScript itself, others (create_collection) use underscores - is there a particular rule here?

Sort Support

I tried a few different methods, but I think writing support for sorting is above my level.

It looks like postgres does allow you to sort using a function, but writing it requires more JS than I can really do.

If you ever had time, being able to specify a sort key and a direction would be very useful.

test [sql/103-find4.sql]

filename : sql/103-find4.sql
line # : 1
tag : todo
md5 : 81d4d9a7c4e910db09b480a268d7eb18

--- [todo] - test
CREATE OR REPLACE FUNCTION find (collection varchar, terms json, lim int, skip int) RETURNS
SETOF json AS $$
  var table = 'col_' + collection;
  var sql = "SELECT data FROM " + table;

  var where_clause = plv8.find_function("where_clause");
  var where = where_clause(terms);
  where = JSON.parse(where);

  sql += " " + where.sql;
  if (lim > -1 ) {
    sql += "LIMIT " + lim;
  }

  if (skip > 0) {
    sql += "OFFSET " + skip;

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.