Just a simple static site
- cd myblog
- zola build
- cp public/blog/choosing-backend-database/index.html ../choosing-backend-database/index.html
- cp public/blog/optimizing-postgres-reads/index.html ../optimizing-postgres-reads/index.html
API creation middleware with an admin interface
License: MIT License
I'm triying to install but I get:
Error: Argument passed in must be a single String of 12 bytes or a string of 24 hex characters
The db abstraction layer should be unified more. Currently different types handle adding a duplicate key differently.
Using the new admin interface i setup a database with the users storage set to 'postgres'. And the set up succeeds but it seems the default username and password ([email protected] and 123) are what is carried into the database, and not the values i entered in the setup page
Hello,
I understand that the JSON file support offers a nice "on file database" solution, but I would like to propose offering SQLite support as well. JSON Files have a locking issue, and are limited by the file system. SQLite 3 offers most of the power of PostgreSql with the advantages of being lighter weight in every way. Along with offering a single easy to backup file as opposed to a complex database system that requires dedicated backup tools. SQLite is also a popular developer alternative to PostgresSql, as it allows the developer to reduce load on their machine while developing.
SQLite recently added support for json fields which should allow for a fork of "mongo-query-to-postgres-jsonb" that turns it into "mongo-query-to-sqlite-json1".
Hi,
in the Admin interface how do i connected collections by relationship?
for example i have POST and TAG collections, and in a post item i want to have a field/property named tags which is type array of ids that holds the ids from the TAG collection...?
thanks.
Hi,
Was creating new collections from the /admin
panel and ran into the following problems:
number
and setting a min and max, the the request fails with a validation error but the collection creation succeeds_id
property is not populated, should be by default right? When using postgres, the id column is also empty. This seems to work fine for users thoughown
permissions do not get auto added to the Admin role, should they?true
and not 1
as with the default collections, this seems to be updated when the server is restartedTokens are currently valid forever. It would be good to add an expiration date to them and make this a configurable setting so admins can decide what works best for their app.
The 'Download All' button does not work in all locations its listed. In the console the error reads Uncaught TypeError: t.downloadCSV is not a function
I understand how to add an expressa.addListener to each document, but I would like to know if there is a way to add a listener to expressa that happens right before the final response is sent.
I'd like to add a copyright message to the end of my entire result before it's sent to the client.
Can't quite figure it out. :/
It seems they are more flaky when running on windows. Not sure yet how this can be solved. I think adding cypress to the CircleCI runner would help too to know if they're flaky in general.
I was thinking it might be wise to limit the below call in allowFirstUserAsAdmin
to top 1 result. Since it happens for every post query on the users collection it might be called frequently, theoretically getting worse as a site gets more users. Also as more users get added the json payload grows, and we just need a count.
const userCount = (await api.db.users.find()).length
I am not a mongo query expert but from some googling it seems this may be achievable with { $limit : 1 }
'/:collection` however, does allow listeners.
I guess there's a notify missing
not sure, but maybe this allows users with 'edit own: true'-permissions to overwrite non-owned collection-items by sending a PUT request with a modified {.....,meta:{owner:'hackeruid'}}
In the case of updating items, we might want expressa to retrieve the collection-item first from the db, so we'll know for sure we're dealing with a legit owner-property.
Hi,
So i installed a server and created an admin user. Then, logged in as that admin user in the /admin page, i created a new user with the predefined Authenticated
role. I then logged out of the super user, and back in as the new 'Authenticated' user, and was still able to do everything i did as an admin user, creating collections, adding users etc. Is this correct? I was thinking because the Authenticated role only has these limited { "users: view own": true, "users: edit own": true, "users: delete own": true }
permissions, that i shouldn't be able to?
Hi,
I really like this project it is very nice.
I tried few API interfaces but they were very limited, this looks like a good start.
In order for me to save time to create API REST endpoints when i do mocks i use "json-server" package - https://github.com/typicode/json-server
its super quick, you have a JSON file structure, that from within that i auto generate the routes also with nested sub routes with relationship and rules.
i think that adding this as an "import feature" could auto create collections schema and routes will be great! any thoughts?
I just noticed this on chrome v66:
the issue: /user/me endpoint get cached (which prevents account-switching).
Interestingly enough, this only happens on a server (which is behind SSL, not sure if that has something to do with it). Localhost seems to work as expected.
Im now trying this index.js patch:
router.get('/user/me', function(req, res, next) {
req.params.collection = 'users';
req.params.id = req.uid;
+ res.setHeader("Cache-Control", "no-cache, no-store, must-revalidate");
+ res.setHeader("Pragma", "no-cache");
+ res.setHeader("Expires", 0);
getById(req, res, next);
//res.redirect('/users/' + req.uid)
});
I will keep you posted
I tried updating some isolated properties of an object.
However, looking at the code it seems unfinished no?
the 'changed'-variable seems unused in router.post('/:collection/:id/update', > (req, res, next) {
in index.js
#node v6.9.1, npm 4.0.2, Windows 10 64 Pro
starting
node app.js
Example app listening on port 3000!
error reading settings
Error: could not load data
at C:\expressa\myapp\node_modules\jfs\lib\Store.js:209:15
at ReadFileContext.callback (C:\expressa\myapp\node_modules\jfs\lib\Store.js:94:16)
at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:365:13)
validators loaded.
navigation to http://localhost:3000/admin
my err handler
TypeError: Cannot read property 'get' of undefined
at getById (C:\expressa\myapp\node_modules\expressa\index.js:249:27)
at C:\expressa\myapp\node_modules\expressa\index.js:106:2
at Layer.handle [as handle_request] (C:\expressa\myapp\node_modules\express\lib\router\layer.js:95:5)
at next (C:\expressa\myapp\node_modules\express\lib\router\route.js:131:13)
at Route.dispatch (C:\expressa\myapp\node_modules\express\lib\router\route.js:112:3)
at Layer.handle [as handle_request] (C:\expressa\myapp\node_modules\express\lib\router\layer.js:95:5)
at C:\expressa\myapp\node_modules\express\lib\router\index.js:277:22
at Function.process_params (C:\expressa\myapp\node_modules\express\lib\router\index.js:330:12)
at next (C:\expressa\myapp\node_modules\express\lib\router\index.js:271:10)
at addRolePermissionsMiddleware (C:\expressa\myapp\node_modules\expressa\role_permissions.js:8:11)
selecting filebased gives
updated settings collection storage
updated collection collection storage
updated role collection storage
error adding collection permissions
Error: could not load data
at C:\expressa\myapp\node_modules\jfs\lib\Store.js:209:15
at ReadFileContext.callback (C:\expressa\myapp\node_modules\jfs\lib\Store.js:94:16)
at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:365:13)
updated users collection storage
error adding collection permissions
Error: could not load data
at C:\expressa\myapp\node_modules\jfs\lib\Store.js:209:15
at ReadFileContext.callback (C:\expressa\myapp\node_modules\jfs\lib\Store.js:94:16)
at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:365:13)
updated post collection storage
adding collection permissions for settings
updated settings collection storage
adding collection permissions for collection
updated collection collection storage
adding collection permissions for role
updated role collection storage
adding collection permissions for users
updated users collection storage
adding collection permissions for post
updated post collection storage
when selecting mongodb
Example app listening on port 3000!
error reading settings
Error: could not load data
at C:\expressa\myapp\node_modules\jfs\lib\Store.js:209:15
at ReadFileContext.callback (C:\expressa\myapp\node_modules\jfs\lib\Store.js:94:16)
at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:365:13)
validators loaded.
my err handler
TypeError: Cannot read property 'get' of undefined
at getById (C:\expressa\myapp\node_modules\expressa\index.js:249:27)
at C:\expressa\myapp\node_modules\expressa\index.js:106:2
at Layer.handle [as handle_request] (C:\expressa\myapp\node_modules\express\lib\router\layer.js:95:5)
at next (C:\expressa\myapp\node_modules\express\lib\router\route.js:131:13)
at Route.dispatch (C:\expressa\myapp\node_modules\express\lib\router\route.js:112:3)
at Layer.handle [as handle_request] (C:\expressa\myapp\node_modules\express\lib\router\layer.js:95:5)
at C:\expressa\myapp\node_modules\express\lib\router\index.js:277:22
at Function.process_params (C:\expressa\myapp\node_modules\express\lib\router\index.js:330:12)
at next (C:\expressa\myapp\node_modules\express\lib\router\index.js:271:10)
at addRolePermissionsMiddleware (C:\expressa\myapp\node_modules\expressa\role_permissions.js:8:11)
updated settings collection storage
updated collection collection storage
updated role collection storage
error adding collection permissions
TypeError: Parameter "url" must be a string, not undefined
at Url.parse (url.js:87:11)
at Object.urlParse [as parse] (url.js:81:5)
at module.exports (C:\expressa\myapp\node_modules\mongodb\lib\url_parser.js:17:23)
at connect (C:\expressa\myapp\node_modules\mongodb\lib\mongo_client.js:290:16)
at Function.MongoClient.connect (C:\expressa\myapp\node_modules\mongodb\lib\mongo_client.js:114:3)
at C:\expressa\myapp\node_modules\expressa\db\mongo.js:51:17
at Object.get (C:\expressa\myapp\node_modules\expressa\db\mongo.js:50:11)
at addCollectionPermissions (C:\expressa\myapp\node_modules\expressa\collection_permissions.js:51:23)
at C:\expressa\myapp\node_modules\expressa\index.js:118:11
at Array.map (native)
updated users collection storage
error adding collection permissions
TypeError: Parameter "url" must be a string, not undefined
at Url.parse (url.js:87:11)
at Object.urlParse [as parse] (url.js:81:5)
at module.exports (C:\expressa\myapp\node_modules\mongodb\lib\url_parser.js:17:23)
at connect (C:\expressa\myapp\node_modules\mongodb\lib\mongo_client.js:290:16)
at Function.MongoClient.connect (C:\expressa\myapp\node_modules\mongodb\lib\mongo_client.js:114:3)
at C:\expressa\myapp\node_modules\expressa\db\mongo.js:51:17
at Object.get (C:\expressa\myapp\node_modules\expressa\db\mongo.js:50:11)
at addCollectionPermissions (C:\expressa\myapp\node_modules\expressa\collection_permissions.js:51:23)
at C:\expressa\myapp\node_modules\expressa\index.js:118:11
at Array.map (native)
updated post collection storage
C:\expressa\myapp\node_modules\mongodb\lib\mongo_client.js:237
throw err
^
Error: Argument passed in must be a single String of 12 bytes or a string of 24 hex characters
at new ObjectID (C:\expressa\myapp\node_modules\bson\lib\bson\objectid.js:34:11)
at C:\expressa\myapp\node_modules\expressa\db\mongo.js:52:40
at connectCallback (C:\expressa\myapp\node_modules\mongodb\lib\mongo_client.js:315:5)
at C:\expressa\myapp\node_modules\mongodb\lib\mongo_client.js:234:11
at _combinedTickCallback (internal/process/next_tick.js:67:7)
at process._tickCallback (internal/process/next_tick.js:98:9)
...
Not totally sure about it, but i noticed a missing password-property.
Im suspecting that adding some roles (thru expressa-admin) caused this.
I'll do some more research.
I pulled the latest version, but got this:
$ node --version
4.0.0
$ node app.js
/mnt/data/home/sqz/projects/expressa/node_modules/promise-sequential/index.js:9
let count = 0;
^^^
SyntaxError: Block-scoped declarations (let, const, function, class) not yet supported outside strict mode
at exports.runInThisContext (vm.js:53:16)
at Module._compile (module.js:413:25)
at Object.Module._extensions..js (module.js:452:10)
at Module.load (module.js:355:32)
at Function.Module._load (module.js:310:12)
at Module.require (module.js:365:17)
at require (module.js:384:17)
at Object.<anonymous> (/mnt/data/home/sqz/projects/expressa/index.js:8:18)
at Module._compile (module.js:434:26)
at Object.Module._extensions..js (module.js:452:10)
Lets try strict:
$ node --use-strict app.js
/mnt/data/home/sqz/projects/expressa/node_modules/bcrypt-nodejs/bCrypt.js:519
password = password + (minor >= 'a' ? "\000" : "");
^^
SyntaxError: Octal literals are not allowed in strict mode.
at exports.runInThisContext (vm.js:53:16)
at Module._compile (module.js:413:25)
at Object.Module._extensions..js (module.js:452:10)
at Module.load (module.js:355:32)
at Function.Module._load (module.js:310:12)
at Module.require (module.js:365:17)
at require (module.js:384:17)
at Object.<anonymous> (/mnt/data/home/sqz/projects/expressa/auth/index.js:2:14)
at Module._compile (module.js:434:26)
at Object.Module._extensions..js (module.js:452:10)
I tried looking for a newer bcrypt, but it's already at the latest version.
I think the promise-sequential requires strict mode, but bcrypt needs to run outside strict mode.
Not sure how to solve this, but i think we should find a solution as many services still run < node v4
Hi, i am actually pretty new to backend world.
I just want to figure out what fits my need. So i am curious, is it the same as keystonejs or different?. I am looking for API-first / headless cms actually. Is expressa good choice for my case?
Thanks.
While i don't think it will necessarilly be quick, i do think it would be relatively straightforward to generate a javascript client script containing all the default calls to all the collections.
Something similar to what is exposed by expressa in node ie: api.db.users.get
but for the endpoints specified in the the readme. The idea being the client script does the more laborious url construction and the consumer on the client just does something like api.users.get( ... )
or await api['email_signups].update( ... )
etc.
Exposing all the collections is probably not a great idea, so a whitelisting approach could be used by setting a property on the collection schema 'for client'. Client script would then be generated when the database is created and when collections are added/removed.
Just an idea!
Quite low in terms of severity: Once you install a server, you are unable to log out and log back in again. You have to restart the server in order to be shown the log in screen at /admin
idea: would be neato to be able to setup collections by 馃憤
I need a setTimeout for adding some per-collection functions (expressa.db.collection.bless = function(){}
):
app.listen( port, function(){
setTimeout( function(){
expressa.db.mycollection.bless = function(doc){
doc.disable = function(){
doc.state = "disabled"
}.bind(doc)
}
}, 500 ) // otherwise only expressa.db.settings is available (collections didn't settle down yet)
})
I think something like this is what we want:
expressa.db.on('ready',function(){
expressa.db.mycollection.bless = function(doc){
doc.disable = function(){
doc.state = "disabled"
}.bind(doc)
}
}
app.listen( port )
For those who want to execute custom queries etc on the database, it would be better if the request was made from the existing Pool already created within expressa. The alternative is requiring pg in your project and creating another pool/connection which to me is inefficient and bypasses the contract between your server and expressa. I think all comms should happen via expressa.
Small side note -> i noticed you create a pool per collection, which could result in quite a few pools depending on application, and the documentation recommends limited number, even just 1: https://node-postgres.com/features/pooling
Super small issue. Breadcrumb in the admin ui does not work
can you reproduce:
/api/users
in the browserWe need that the any kind of sensitive information should not be visible to anonymous, authenticated user-role. Only admin role should have access to that.
can you assist in resolving this issue?
Error: could not load data
at /var/www/app/node_modules/jfs/lib/Store.js:209:15
at ReadFileContext.callback (/var/www/app/node_modules/jfs/lib/Store.js:94:16)
at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:365:13)
j
after installing via npm and adding necessary code changes, when I go to http://localhost:3000/admin/ i am redirected to install page but when I try to save changes, i get error.
there is no data/settings/development.json file being created.
Any help would be appreciated.
Im in the situation where i would like to use data/settings/development.json on my buildserver (without production configvalues).
Any idea how to initialize the development-settings?
Most nodejs applications distinguish between production/development using the NODE_ENV environment flag:
if( process.env.NODE_ENV == "production" ){
}else{ // by default nodejs sets it to "development"
}
expressa could do something similar, by creating both development.json
and production.json
in data/settings/.
upon first run...and load one based on the NODE_DEV value.
index.js
28 router.settings = {}
29
- 30 db.settings.get('production')
+ 30 db.settings.get( process.env.NODE_ENV )
31 .then(function(data) {
32 router.settings = data;
NOTE: if this sounds ok let me know, and we'll prepare a PR
Hi,
from your readme I would have expected to have a login. But there are no credentials mentioned. So I fail attempting to get through the admin UI. Any help appreciated!
I want my expressa api to sit on the root of the server, everything seems to work except the /admin page is no longer accessible:
For example, running this:
var express = require('express');
var app = express();
var expressa = require('expressa');
app.use('/', expressa.api()); // optionally pass in settings
app.use('/admin', expressa.admin({ apiurl: '/' }));
app.listen(3000, function () {
console.log('Example app listening on port 3000!');
});
then navigating to http://localhost:3000/admin/
results in the following error:
{"error":"unknown collection"}
I installed expressa following readme and got an error after install:
error adding collection permissions
Error: could not load data
at /rbd/pnpm-volume/db2e9f30-6bfe-4850-b75e-e887ea2d0f78/node_modules/.registry.npmjs.org/jfs/0.2.6/node_modules/jfs/lib/Store.js:209:15
at ReadFileContext.callback (/rbd/pnpm-volume/db2e9f30-6bfe-4850-b75e-e887ea2d0f78/node_modules/.registry.npmjs.org/jfs/0.2.6/node_modules/jfs/lib/Store.js:94:16)
at FSReqWrap.readFileAfterOpen [as oncomplete] (fs.js:420:13)
adding collection permissions for settings
adding collection permissions for collection
adding collection permissions for role
adding collection permissions for users
adding collection permissions for post
I think we need a way to add custom endpoints that also have expressa features like req.user and req.hasPermission so that these custom endpoints can use the current permission model.
In the era of node >= 10 is it necessary to use BlueBird?
Since native promises have come a long way, I don't see any of Bluebirds special features being used so could now be a good time to reconsider?
My other concern is expressa overwrites the global.Promise giving no way for the consumer to choose native promises themselves.
it seems /users/me seems to bypass the listeners (which could hide/delete sensitive properties).
Expressa currently has a mixed policy when it comes to responding (results/errors in json or plain text).
I would opt for json-only responses, by creating an error function in index.js
like this:
+ router.error = function(res,msg,code,info){
+ res.status(204).send({"error":msg, "code": code || 1,"info": info || ""}).end()
+ }
router.use(>(req, res, next) {
路 req.settings = router.settings;
+ res.set('Content-Type','applicatioin/json'')
+ res.error = router.error
路 next()
})
Now anywhere in the expressa code, or listeners, one could call res.error("some error")
and the rest is taken care of.
Browser client-execution wouldn't crash on JSON parse-errors (in case of plaintext) or 404-statuses, they can recover appropriately by checking for the error
-key in every response (and present a custom, more human errormessage based on the code
-field)
WDYT?
Any idea how could append them into the permission-mechanism and append them to the 'Permission'-table in expressa-admin?
// this will output all custom express endpoints
app._router.stack.map(>(endpoint){
console.log( endpoint.regexp.toString() )
})
It would make expressa the first express middleware (I've seen) to apply role/permissions on programmatically added endpoints.
The problem with all express applications is a lack of centrally securing all endpoints.
So cases like this:
app.use('/foo/bar',function(req,res,next){ .... })
app.get('/flop',function(req,res,next){ .. })
Totally bypasses authentication/roles.
Ofcourse this can be solved like this:
app.get('/flop',myAuthMiddleware, function(req,res,next){ .. })
However, then it still bypasses all finer grained permissions.
Since expressa already has finegrained controlled of permissions, it would be neato to add the non-expressa endpoints in there as well.
WDYT?
roles, collections and settings are all required to be stored in files. All of these are accessed very frequently, so they'll need to be cached in memory. I'm not sure yet how often the cached version should be updated, maybe only when expressa starts up.
I'm planning on adding a basic password reset feature. Here's my plan
Hi,
I am following the instructions on the README to get a simple server and database set up. I have the following file server.js
`
var express = require('express');
var app = express();
var expressa = require('expressa');
app.use('/api', expressa.api()); // optionally pass in settings
app.use('/admin', expressa.admin({ apiurl: '/api/' }));
app.listen(3000, function () {
console.log('Example app listening on port 3000!');
});`
I have created an empty database 'test' in pgAdmin
I run my server, and navigate to localhost:3000/admin
But when i click i get the following error(s):
I am on a laptop using windows 10
Thanks
I have a user who could not edit their own document in another collection despite having the correct editown permission. I noticed the documents owners were coming in as undefined
and i believe it is tracked down to the following:
In collections.js
line 127
const owner = doc.owner && doc.meta.owner
Maybe supposed to be
const owner = doc.meta && doc.meta.owner
?
This seems to have started since the 0.5.5 update. Basically if you have multiple listeners such as
var express = require('express');
var app = express();
var expressa = require('expressa');
var api = expressa.api();
app.use('/admin', expressa.admin({apiurl: '/api'}));
app.use('/api', api); // optionally pass in settings
api.addListener('ready', function ready() {
app.listen(3000, function () {
console.log('Example app listening on port 3000!');
});
})
api.addCollectionListener('get', 'users', async function usersGet(req, collection, data) {
})
api.addCollectionListener('post', 'users', async function usersPost(req, collection, doc) {
})
api.addLateCollectionListener('post', 'users', async function usersPostLate(req, collection, doc) {
})
api.addCollectionListener('put', 'users', async function usersPut(req, collection, doc) {
})
And then try fresh install via the /admin panel (using users storage type as 'postgresql', you will notice the following error:
From what i can tell this is a race condition, the install is somehow losing its usual order of operations, and in this case, expressa is trying to retrieve the 'Admin' role document before it has been created
Still trying to make something 馃樃
I don't get why it throw that since addListener is present in listerner.js.
Here's the beginning of server.js:
var express = require('express');
var app = express();
var expressa = require('expressa');
app.use(express.static('public'));
app.use('/api', expressa.api()); // optionally pass in settings
app.use('/admin', expressa.admin({ apiurl: '/api/', showCriticalFeaturesOnProduction: true }));
app.get('/', function(request, response) {
response.sendFile(__dirname + '/views/index.html');
});
expressa.addListener('get', -101, function (req, collection, doc) {
...
I tried following https://github.com/thomas4019/expressa/blob/master/doc/relationships.md but didn't get expected results.
Is this property in meta? Or declared as object in root, then additional properties in root?
Do we need to change additionalProperties to true?
Thanks!
Hi,
In the documentation it says we are able to execute queries on a collection via the api:
/:collection/?query={..}
Where the query has to be in the form of a mongo-query. If my databse is jsonb postgres can i execute jsonb sql queries?
Hi,
I had to migrate file-collections to postgres using expressa-cli, and found out all my tests failed.
It turns out all the database adapters return an id after a create() db-call, except postgres returns an object.
I introduced an environment-flag to not cause harm for existing postgres expressa-users for now.
However, I do believe we want to make this default behaviour, so we could, at somepoint bump the versionnumber to indicate a breaking change for postgres users (v0.4.x etc).
A declarative, efficient, and flexible JavaScript library for building user interfaces.
馃枛 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 馃搳馃搱馃帀
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google 鉂わ笍 Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.