Giter VIP home page Giter VIP logo

mora's Introduction

Mora - Mongo Rest API

REST server for accessing MongoDB documents and meta data

Documents

When querying on collections those parameters are available:

query  - use mongo shell syntax, e.g. {"size":42}
limit  - maximum number of documents in the result
skip   - offset in the result set
fields - comma separated list of (path-dotted) field names
sort   - comma separated list of (path-dotted) field names
extended_json - set to "true" to return responses in [MongoDB Extended JSON](http://docs.mongodb.org/manual/reference/mongodb-extended-json/) format
Examples
Listing aliases
$ curl 'http://127.0.0.1:8181/docs/' \
>   -D - \
>   -H 'Accept: application/json'
HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 22 Apr 2014 07:06:30 GMT
Content-Length: 61

{
  "success": true,
  "data": [
   "test",
   "local"
  ]
}
Listing databases
$ curl 'http://127.0.0.1:8181/docs/local/' \
>   -D - \
>   -H 'Accept: application/json'
HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 22 Apr 2014 07:07:11 GMT
Content-Length: 61

{
  "success": true,
  "data": [
   "local",
   "use1"
  ]
}
Listing collections
$ curl 'http://127.0.0.1:8181/docs/local/local' \
>   -D - \
>   -H 'Accept: application/json'
HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 22 Apr 2014 07:24:10 GMT
Content-Length: 98

{
  "success": true,
  "data": [
   "new-collection",
   "startup_log",
   "system.indexes"
  ]
}
Inserting document
$ curl 'http://127.0.0.1:8181/docs/local/local/new-collection/document-id' \
-D - \
-X POST \
-H 'Content-Type: application/json' \
-H 'Accept: application/json' \
--data '{"title": "Some title", "content": "document content"}'

HTTP/1.1 201 Created
Content-Location: /docs/local/local/new-collection/document-id
Content-Type: application/json
Date: Tue, 22 Apr 2014 07:23:33 GMT
Content-Length: 116

{
  "success": true,
  "data": {
   "created": true,
   "url": "/docs/local/local/new-collection/document-id"
  }
}
Finding document
$ curl 'http://127.0.0.1:8181/docs/local/local/new-collection/document-id' \
>   -D - \
>   -H 'Accept: application/json'
HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 22 Apr 2014 07:32:33 GMT
Content-Length: 123

{
  "success": true,
  "data": {
   "_id": "document-id",
   "content": "document content",
   "title": "Some title"
  }
}
Finding documents
$ curl 'http://127.0.0.1:8181/docs/local/local/new-collection?limit=1&skip=1' \
>    -D - \
>    -H 'Accept: application/json'
HTTP/1.1 200 OK
Content-Type: application/json
Date: Wed, 23 Apr 2014 23:18:39 GMT
Content-Length: 387

{
  "success": true,
  "prev_url": "/docs/local/local/new-collection?limit=1\u0026skip=0",
  "next_url": "/docs/local/local/new-collection?limit=1\u0026skip=2",
  "data": [
   {
    "_id": "535849cfb734f91cdc000002",
    "content": "document content",
    "title": "Some title"
   }
  ]
}
Updating document
$ curl 'http://127.0.0.1:8181/docs/local/database/new-collection/document-id' \
>  -D - \
>  -X PUT \
>  -H 'Content-Type: application/json' \
>  -H 'Accept: application/json' \
>  --data '{"title": "New title"}'
HTTP/1.1 200 OK
Content-Location: /docs/local/database/new-collection/document-id
Content-Type: application/json
Date: Tue, 22 Apr 2014 06:37:02 GMT
Content-Length: 133

{
  "success": true,
  "data": {
   "created": false,
   "url": "/docs/local/database/new-collection/document-id"
  }
}
Updating documents
$ curl 'http://127.0.0.1:8181/docs/local/local/new-collection' \
>   -D - \
>   -X PUT \
>   -H 'Content-Type: application/json' \
>   -H 'Accept: application/json' \
>   --data '{"$set": {"title": "New title"}}'
HTTP/1.1 200 OK
Content-Type: application/json
Date: Wed, 23 Apr 2014 23:33:11 GMT
Content-Length: 22

{
  "success": true
}
Removing document
$ curl 'http://127.0.0.1:8181/docs/local/local/new-collection/document-id'  \
>   -D - \
>   -X DELETE \
>   -H 'Accept: application/json'
HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 22 Apr 2014 07:42:47 GMT
Content-Length: 22

{
  "success": true
}
Removing collection
$ curl 'http://127.0.0.1:8181/docs/local/local/new-collection'  \
>   -D - \
>   -X DELETE \
>   -H 'Accept: application/json'
HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 22 Apr 2014 07:43:24 GMT
Content-Length: 22

{
  "success": true
}
Statistics
Database statistics
$ curl http://127.0.0.1:8181/stats/local/local -D -
HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 22 Apr 2014 08:17:46 GMT
Content-Length: 341

{
  "success": true,
  "data": {
   "avgObjSize": 595.6,
   "collections": 3,
   "dataFileVersion": {
    "major": 4,
    "minor": 5
   },
   "dataSize": 5956,
   "db": "local",
   "fileSize": 67108864,
   "indexSize": 0,
   "indexes": 0,
   "nsSizeMB": 16,
   "numExtents": 3,
   "objects": 10,
   "ok": 1,
   "storageSize": 10502144
  }
}
Collection statistics
$ curl http://127.0.0.1:8181/stats/local/local/startup_log -D -
HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 22 Apr 2014 08:18:16 GMT
Content-Length: 389

{
  "success": true,
  "data": {
   "avgObjSize": 728,
   "capped": true,
   "count": 8,
   "indexSizes": {},
   "lastExtentSize": 10485760,
   "max": 9223372036854775807,
   "nindexes": 0,
   "ns": "local.startup_log",
   "numExtents": 1,
   "ok": 1,
   "paddingFactor": 1,
   "size": 5824,
   "storageSize": 10485760,
   "systemFlags": 0,
   "totalIndexSize": 0,
   "userFlags": 0
  }
}

Install from source

go get -u github.com/emicklei/mora

Create a release

sh release.sh

Configuration

Mora uses a simple properties file to specify host,port,aliases and other options

# listener info is required
http.server.host=localhost
http.server.port=8181

# enable cross site requests
http.server.cors=true

# for swagger support (optional)
swagger.path=/apidocs/
swagger.file.path=./swagger-ui/dist

# mongo instances are listed here; specify an alias for each
mongod.{alias}.host=localhost
mongod.{alias}.port=27017
# initial and operational timeout in seconds
mongod.{alias}.timeout=5
# optional authentication
mongod.{alias}.username=
mongod.{alias}.password=
mongod.{alias}.database=
# read preference mode
# supported options (case-insensitive): https://godoc.org/gopkg.in/mgo.v2#Mode
mongod.{alias}.mode=primary
# alternatively, a mongodb connection string uri can be used instead
# supported options: https://godoc.org/gopkg.in/mgo.v2#Dial
mongod.{alias}.uri=mongodb://myuser:mypass@localhost:40001,otherhost:40001/mydb

# enable /stats/ endpoint
mora.statistics.enable=true

Run

$ mora -config mora.properties

Swagger

Swagger UI is displaying automatically generated API documentation and playground.

Swagger UI

© 2013, http://ernestmicklei.com. MIT License

mora's People

Contributors

crackcomm avatar emicklei avatar gui avatar hp03 avatar mindreframer avatar nvnoskov avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mora's Issues

XML serialization error

When I'm creating a request to mora using Google Chrome I'm getting error:

xml: unsupported type: bson.M

Request:

GET /docs/lab/appg_test_env1/clients/5334dce9b734f91194000001 HTTP/1.1
Host: 127.0.0.1:4000
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8

Can't authenticate with Mora

I am trying to bring up an API for my mongoDB instance.

I currently have a user with read only rules that I can connect with so my mora.properties file looks like this:

#listener info is required
http.server.host=0.0.0.0
http.server.port=9123

mongo.db.uri=mongodb://user:pass@localhost:27017/mydb

I have also tried the following

#listener info is required
http.server.host=0.0.0.0
http.server.port=9123

#Auth 
mongod.mydb.databse=mydb
mongod.mydb.username=user
mongod.mydb.password=pass

mondod.mydb.host=localhost
mondod.mydb.port=27017

But when I try to do the following

$ curl http://localhost:9123/docs/mydb 
 {
   "success": false,
   "error": {
    "code": 401,
    "name": "unauthorized"
   }
  }

And the Mora instance logs:

2015/06/20 05:32:00 [mora] connecting to [db=mongodb://user:pass@localhost:27014/mydb] with timeout [0 seconds]
2015/06/20 05:32:00 [mora][error] unauthorized

Needless to say - I have checked my user authentication info several times - it is correct.
The thing is - I have tried the exact same URI for mongoDB in another app (node.js mongodb-rest api) and it works, but for some reason Mora fails to authenticate.

Any ideas how to go around that?

Wondering about hooks

Been playing around with this and realized it almost meets the needs I have except for I want to be able to do some GPG application-level field encryption on the server side before any PUT or POST would make it actually into the DB. Of course this would be over HTTPs as well and hadn't read of support for that as well.

But my main question is really related to being able to do 99% CRUD with mora and then 1% application specific data transformations, log triggers and the like with some sort of hook. I suppose I could just customize mora to my needs, but was wondering if any thought had already been given into providing such hooks into the pipeline.

Mora reusing

Hey!
Today I was writing a small HTTP API using MongoDB as a storage and I just thought, what if I could reuse mora? Tell me what do You think about it and check code which is my first attempt, I have decoupled and a little bit documented the code. It makes mora a little bit more reusable but we can go further.

Does not handle documents with UUID's as Ids

If your MongoDB documents contain a certain type of UUID for an _id, Mora returns the ID as an object like this:

"_id": {
"Kind": 3,
"Data": "ABnJswg/6EKEoibo1A6/yQ=="
}

It does not appear possible to interact with the document level of the API (GET,PUT,DELETE) using this format with Swagger.

It looks like Mora supports some form of Hex object id. Not sure why this type shows up like this or what "Kind" refers to. These were documents which had a .NET GUID as an ID and were added to MongoDB using the official C-Sharp driver.

In MongoClient the id looks like: BinData(3,"ABnJswg/6EKEoibo1A6/yQ==")

In Robomongo the id looks like: LUUID("c1fad936-f455-d543-be73-07970d6eac1e")
or with .NET id encoding like: NUUID("36d9fac1-55f4-43d5-be73-07970d6eac1e")

I know this is just a demo project and I am rather new to GoLang but, if you have some general advice about how to solve this, I would be willing to try a fix and submit a pull request.

OPTIONS response prior to performing PUT does not contain Access-Control-Allow-Origin or Access-Control-Expose-Headers headers. Rejected by Chrome.

I was attempting to perform a PUT operation from Angular and kept getting rejected by CORS in Chrome. I checked the GET request I did prior to the PUT and noticed it went through fine (though no OPTIONS request was done for the GET). Upon attempting a PUT, the OPTIONS request goes through and returns a 200 as you'd expect with the Allow response header, but not the Access-Control-Allow-Origin or Access-Control-Expose-Headers headers. This caused Chrome to deny the request.

I've done some digging in mora and go-restful and it looks like the CrossOriginResourceSharing Filter method isn't being called on the OPTIONS request, though I'm not sure if that's by design and the OPTIONSFilter container func should be handling this instead. The CrossOriginResourceSharing filter method does get called successfully for the GET. I haven't checked to see if it gets called for DELETE or POST yet.

Also, for completeness sake, I haven't touched the cors setting in mora.properties. It was left in its default state of true.

Thanks for the great project!

len(data)=0 when I set skip=1 and limit=1

curl 'http://****?skip=1&limit=1'

response :
{
"success": true,
"data": []
}

change limit =2 , the resoponse is ok .

looks like : limit=1, skip >= 1 , empty
limit = 1, skip = 0 , ok
limit > 1, ok

I find len(iter.docData)=0 and iter.err = "not found"

Error on update when new document contains string "_id".

There is an error which occurs when we want to update a document using putDocument.

Example request:

Request URL: http://127.0.0.1:8080/docs/local/use1/services/531c584fb734f914a8000001
Request Method: PUT
Status Code: 500 Internal Server Error

Request payload:

{
  "_id": "531c584fb734f914a8000001",
  "domain": "",
  "icon": "",
  "name": "test",
  "state": "Ok",
  "type": "Big Platform"
}

Response:

cannot change _id of a document old:{ _id: ObjectId('531c584fb734f914a8000001'), domain: "", icon: "", name: "test", state: "Ok", type: "Computing Service" } new:{ _id: "531c584fb734f914a8000001", domain: "", icon: "", name: "test", state: "Ok", type: "Big Platform" }

As u can see _id value is a string. Mora doesn't treat it like a string from _id path parameter (it should transform it into a ObjectIdHex) that's why error occurs.

Every new post request create a new TCP connection.

Is it by design or how can I reuse the TCP connection.

Looking at netstat this appears to be resulting in a new connection for every post resulting in a large number of concurrent connections being open.

What is the correct way to reuse connections in this case?

Mora Authentication

Hey, I'd like to have some authentication options for mora API usage.
I think it should be some out-of mongo, distributable authentication service, maybe Keystone (api docs) which is common choice in public and private clouds today as it's part of popular and strong OpenStack.
I would like to hear other options and opinions about them, and maybe some voices against my word because Keystone was just a shot.

Is there any plan to add api input validation.

I think its a good idea to validate the documents before we insert into the database, Using some kind of configuration includes a schema definition for every resource managed by the API.

bind: cannot assign requested address

Each time I used another ip for http.server.host setting throws a binding exeception, here is the full log:

2016/10/24 18:24:55 [mora][info] loading configuration from [mora.properties]
[restful] 2016/10/24 18:24:55 log.go:30: [restful/swagger] listing is available at http://192.168.56.103:8181/apidocs.json
[restful] 2016/10/24 18:24:55 log.go:30: [restful/swagger] http://192.168.56.103:8181/apidocs/ is mapped to folder ./swagger-ui/dist
2016/10/24 18:24:55 [mora][info] ready to serve on http://192.168.56.103:8181
2016/10/24 18:24:55 listen tcp 192.168.56.103:8181: bind: cannot assign requested address

it works with localhost or empty string, whatever else fails.

Builds failing due to missing goproperties repo

I'm trying to build mora in a new environment, but it's failing because it looks like the emicklei/goproperties repository has been deleted from github: https://github.com/emicklei/goproperties Was this intentional? I think I was building successfully last month when I thinks this repo still existed.

$ go get
# cd .; git clone https://github.com/emicklei/goproperties /tmp/gocode/src/github.com/emicklei/goproperties
Cloning into '/tmp/gocode/src/github.com/emicklei/goproperties'...
ERROR: Repository not found.
fatal: Could not read from remote repository.

I found the original upstream repo at dmotylev/goproperties, so I'm not sure if mora could be switched over to use that directly, or if there was relevant changes in your forked repo that are needed.

Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.