bigcompany / resource Goto Github PK
View Code? Open in Web Editor NEWresource engine for javascript
License: MIT License
resource engine for javascript
License: MIT License
The JugglingDB and jugglingdb-nano modules should be removed.
The extra layer added to resource persistence is cumbersome and not very good.
It would be much better to remove JugglingDB support and only focus on supporting CouchDB for now.
so it has schemas, hooks, etc. #20 solves all of my use cases for wanting resource to be a resource, so that's enough for me, but eventually we could resource-ify all the methods.
when resource connects to a couch with a specified database, if that database is not found in the database, resource should create that database and then perform the desired request.
If I run this code:
var resource = require('resource');
var schema = {
properties: {
name: {
description: 'name property',
type : 'string',
required : true,
minLength: 1
}
}
};
var room = resource.define('room', { schema: schema });
room.persist('memory');
console.log('\ncreate resource');
console.log('===============');
console.log('\n> create a invalid room');
room.create({}, function(err, res) {
console.log('error must be not null: ', err);
console.log('constraint must be true: ' + room.schema.properties.name.required);
console.log('\n> create a invalid room');
room.create({name: ''}, function(err, res) {
console.log('error must be not null: ', err);
console.log('constraint must be true: ' + room.schema.properties.name.required);
console.log('\n> create a valid room');
room.create({name: 'my room'}, function(err, res) {
console.log('error must be null: ', err);
console.log('constraint must be true: ' + room.schema.properties.name.required);
console.log('\n> create a invalid room');
room.create({}, function(err, res) {
console.log('error must be not null: ', err);
console.log('constraint must be true: ' + room.schema.properties.name.required);
});
});
});
});
I get the following output:
create resource
===============
> create a invalid room
error must be not null: [Error: Validation error: [
{
"property": "name",
"constraint": "required",
"expected": true,
"actual": false,
"message": "Required value is missing"
}
]]
constraint must be true: true
> create a invalid room
error must be not null: [Error: Validation error: [
{
"property": "name",
"constraint": "required",
"expected": true,
"actual": false,
"value": "",
"message": "Required value is missing"
}
]]
constraint must be true: true
> create a valid room
error must be null: null
constraint must be true: false
> create a invalid room
error must be not null: null
constraint must be true: false
Seems like the parsing of the schema in mschema module does modify the schema itself. Take a look at the index.js inside the mschema module at line 98.
So after a valid room instance is created the schema changes of the room resource and all the following calls to create a invalid resource do not validate the model properly. Am I missing something?
I used the resource v 0.5.2
The persistence code should be kept in a separate resource.
Ideally, every different type of datasource would be a separate resource ( couch
, redis
, etc ) acting as an adapter to the persistence resource.
This would allow us to remove JugglingDB vendor from resource core, and put it into the persistence resource instead.
When an instance of a resource is returned from the datasource layer ( JugglingDB ) any resource methods for that resource should be mapped onto the resource instance scoped within the unique key of the current resource instance.
Wouldn't it be nice if we could chain .use
calls, all that would require would be to change this line to return this
, is that an option? If not how about passing arrays into .use like resource.use(['http', 'socket']);
this way resource
stores all the resources in the same format as other resources (such as auth
, socket
, persistence
, github
, template
, etc) can store subresources. this way we can implement docs
, package
, etc to iterate through subresources.
the proposed name of a subresource is parent-child
as in auth-github
. (or does auth::github
make more sense?) the proposed way to access a subresource is parent.child
as in auth.github
, which should be reflected when generating documentation.
this is something that needs to be finalized, implemented, and tested, so here's what i am thinking so far.
install (async):
npm install
to appDir + '/node_modules/'npm install
resource package from npm registryuse (async):
load (sync):
To reproduce, create a big project like the following:
josh@onix:/tmp/big-test$ ls -R
.:
node_modules test.js view
./node_modules:
big resource resources
./view:
josh@onix:/tmp/big-test$ cat test.js
var big = require('big');
var view = big.use('view');
var _view = view.create({ path: process.cwd() + '/view' });
_view.load();
josh@onix:/tmp/big-test$
When you run this, big tries to lazily install cheerio as a dependency for the view resource on view.create
, but because the method is synchronous _view
ends up being undefined and the call to load
fails:
josh@onix:/tmp/big-test$ node test.js
info: installing view to /tmp/big-test/resources/view
warn: view resource is missing a required dependency: cheerio
warn: spawning npm to install missing dependencies
exec: npm install [email protected]
warn: deffering execution of `view.create` since dependencies are missing
/tmp/big-test/test.js:7
_view.load();
^
TypeError: Cannot call method 'load' of undefined
at Object.<anonymous> (/tmp/big-test/test.js:7:7)
at Module._compile (module.js:449:26)
at Object.Module._extensions..js (module.js:467:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.runMain (module.js:492:10)
at process.startup.processNextTick.process._tickCallback (node.js:244:9)
If you install cheerio before running this, it works just fine because the dependency is already installed and view.create
is able to execute properly.
Now that every resource is also a package, resource.use
should require the package instead of being hard-coded to index.js
There is an issue with default options not being passed into resource.all
. Could just be version issue.
Only show one deferment message per resource method. Currently, the same message can spam the screen. Even though the output is technically correct, it's not a good experience for the user
inconsistency led to changes here: master...bigspace.
imo, consistency will make everything easier, including #6.
thoughts?
Basically, if you modify any documents using futon, you're going to have a bad time.
Here's tests:
tl;dr: updateOrCreate a creature, modify the creature using mikeal/request, get the creature, modify it with the persistence engine, and update it.
Here's the output from that test (tl;dr: modifications to life do not register after get, and returned object after get does not conform to update's schema):
# modify persisted documents outside of jugglingdb
ok 68 successfully created korben
ok 69 korben has life 10
ok 70 successfully requested couchdb document
ok 71 successfully modified couchd document
ok 72 successfully got korben
not ok 73 korben has life 100
---
file: /home/josh/dev/bigcompany/resource/index.js
line: 733
column: 9
stack:
- getCaller (/home/josh/dev/bigcompany/resource/node_modules/tap/lib/tap-assert.js:418:17)
- assert (/home/josh/dev/bigcompany/resource/node_modules/tap/lib/tap-assert.js:21:16)
- Function.equal (/home/josh/dev/bigcompany/resource/node_modules/tap/lib/tap-assert.js:162:10)
- Test._testAssert [as equal] (/home/josh/dev/bigcompany/resource/node_modules/tap/lib/tap-test.js:86:16)
- /home/josh/dev/bigcompany/resource/test/persistence.js:291:13
- /home/josh/dev/bigcompany/resource/index.js:685:29
- afterHooks (/home/josh/dev/bigcompany/resource/index.js:733:9)
- callbackWrap (/home/josh/dev/bigcompany/resource/index.js:681:18)
- _args (/home/josh/dev/bigcompany/resource/index.js:651:33)
- exports.enable.r.method.description (/home/josh/dev/bigcompany/resource/lib/persistence.js:140:7)
found: 10
wanted: 100
...
not ok 74 successfully updated korben
---
type: Error
message: |
Invalid arguments for method `creature.update`. [
{
"attribute": "type",
"property": "metadata",
"expected": "object",
"actual": "object",
"message": "must be of object type"
},
{
"attribute": "type",
"property": "items",
"expected": "array",
"actual": "object",
"message": "must be of array type"
}
]
code: ~
errno: ~
file: /home/josh/dev/bigcompany/resource/index.js
line: 535
column: 29
stack:
- {
- "attribute": "type",
- "property": "metadata",
- "expected": "object",
- "actual": "object",
- "message": "must be of object type"
- },
- {
- "attribute": "type",
- "property": "items",
- "expected": "array",
- "actual": "object",
- "message": "must be of array type"
- }
- ]
- execute (/home/josh/dev/bigcompany/resource/index.js:535:29)
- /home/josh/dev/bigcompany/resource/index.js:409:16
- beforeHooks (/home/josh/dev/bigcompany/resource/index.js:471:16)
- /home/josh/dev/bigcompany/resource/index.js:400:14
- beforeAllHooks (/home/josh/dev/bigcompany/resource/index.js:440:16)
- EventEmitter.fn [as update] (/home/josh/dev/bigcompany/resource/index.js:391:12)
- /home/josh/dev/bigcompany/resource/test/persistence.js:295:20
- /home/josh/dev/bigcompany/resource/index.js:685:29
- afterHooks (/home/josh/dev/bigcompany/resource/index.js:733:9)
- callbackWrap (/home/josh/dev/bigcompany/resource/index.js:681:18)
...
not ok 75 korben has life 50
---
file: /home/josh/dev/bigcompany/resource/index.js
line: 547
column: 20
stack:
- getCaller (/home/josh/dev/bigcompany/resource/node_modules/tap/lib/tap-assert.js:418:17)
- assert (/home/josh/dev/bigcompany/resource/node_modules/tap/lib/tap-assert.js:21:16)
- Function.equal (/home/josh/dev/bigcompany/resource/node_modules/tap/lib/tap-assert.js:162:10)
- Test._testAssert [as equal] (/home/josh/dev/bigcompany/resource/node_modules/tap/lib/tap-test.js:86:16)
- /home/josh/dev/bigcompany/resource/test/persistence.js:299:15
- execute (/home/josh/dev/bigcompany/resource/index.js:547:20)
- /home/josh/dev/bigcompany/resource/index.js:409:16
- beforeHooks (/home/josh/dev/bigcompany/resource/index.js:471:16)
- /home/josh/dev/bigcompany/resource/index.js:400:14
- beforeAllHooks (/home/josh/dev/bigcompany/resource/index.js:440:16)
found: ~
wanted: 50
type:
found: undefined
wanted: undefined
...
I suspect that once the schema issues are resolved, that couchdb will fail to update due to an update conflict because jugglingdb appears to be caching.
It seems that the property id is always added to the resource schema. In some cases this is superfluous.
Remove the id property as a default. Only add id to the schema if the resource is persisted.
Low priority.
When a resource is .use() in an application, it will always check the resources
package for the existence of that resource. If that resource is found in resources
, a copy of it will be made into the current applications directory + /resources.
This is really useful for developers building applications with resources, but it can be slightly confusing to the work-flow of developers who are building new resources.
The issue arises in that sometimes you don't want a copy of the resource made, like when you are inside the require.resolve('resources') and are making change to a resource / building a new resource.
A possible solution would be setting this as an option in on the resource singleton itself, or detecting if we are inside require.resolve('resources') and not using the cache.
a resource should be able to contain a resource instance within any of its properties. this is currently possible with using ids as a placeholder for child instances, but then it takes multiple requests to create or get the entire root instance.
We should move as much code out of core into separate resources as possible.
The core resource API should be as minimalistic as possible, where all additional functionality is a result of composition via resource.use
resource should have a method resource.start
that intelligently aggregates all resource.start methods for all included resources.
considerations:
.start
methods.start
gets calledThe issue is inside this block of code:
https://github.com/bigcompany/resource/blob/master/lib/load.js#L29
The code for a "find" eventually finds its way to this code chunk:
CradleAdapter.prototype.models = function(model, filter, callback, func) {
this.client.all(
{include_docs: true},
errorHandler(callback, function(res, cb) {
var docs = res.map(function(doc) {
return idealize(doc);
});
var filtered = filtering(docs, model, filter, this._models)
func ? func(filtered, cb) : cb(filtered);
}.bind(this))
);
};
This is wrong. Instead of pulling down ALL docs then filtering, the filtering function should be implicitly pushed to the couchdb and then used to run these queries on the database instead of locally.
Users should be aware of the various options that can get passed into resource.persist
, such as host
, port
, username
, password
, etc... based on the type of datasource being used.
When using a resource check to see if the resource has a resource.init
method. If this method is available on the resource then execute it immediately.
The Resource.init
method will be used to indicate the resource requires some asynchronous operation before it can be Resource.start
.
In Resource.start
it's expected that something will happen ( like starting a service ).
In Resource.init
it's expected that nothing outside of initializing the resource will happen.
info: installing to /home/kane/projects/bighack/bigspace/resources
fs.js:684
return binding.stat(pathModule._makeLong(path));
^
Error: ELOOP, too many symbolic links encountered '/home/kane/projects/bighack/resources/node_modules/resources/node_modules/resources/node_modules/resources/node_modules/resources/node_modules/resources/node_modules/resources/node_modules/resources/node_modules/resources/node_modules/resources/node_modules/resources/node_modules/resources/node_modules/resources/node_modules/resources/node_modules/resources/node_modules/resources/node_modules/resources/node_modules/resources/node_modules/resources/node_modules/resources/node_modules/resources/node_modules/.bin/express'
at fs.statSync (fs.js:684:18)
at /home/kane/projects/bighack/resource/lib/helper.js:6:16
at Array.forEach (native)
at helper.copyDir (/home/kane/projects/bighack/resource/lib/helper.js:5:7)
at /home/kane/projects/bighack/resource/lib/helper.js:12:7
at Array.forEach (native)
at helper.copyDir (/home/kane/projects/bighack/resource/lib/helper.js:5:7)
at /home/kane/projects/bighack/resource/lib/helper.js:12:7
at Array.forEach (native)
at helper.copyDir (/home/kane/projects/bighack/resource/lib/helper.js:5:7)
resource.use
and resource.load
should both be resource methods added to resource with resource.method
it would be beneficial to have a schema for the return values of a resource method because then we could get free documentation and validation, among probably other things.
There seems to be some issues with resource.all
while using the fs
datasource and a .DS_Store
file is encountered.
This will cause an error in the resource.all
call looking for a .DS_Store.json
file which doesn't exist.
This issue may also apply to other files that may errantly exist in the /db/
folders.
Time to clean up the code base and make sure it follows the style guidelines defined here:
Right now we if we can't find a document, we call back with an error. However, the error doesn't have too much information in it and user has to match against the message error to find out what happened.
Should we do that? In an ORM-ish/database access layer, not finding the document seems like a pretty normal occurrence, could we just return null
as the document?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.