jamesplease / api-pls Goto Github PK

Atm a user could inject arbitrary SQL into a resource file, and it will execute. No good!

There should be a system for generating a resource. Generation of a resource creates two documents:

1. a resource definition file. this has validation, for instance.
2. an initial up/down migration

Eventually, there will also be resource editing. Editing a resource will possibly update the resource definition, and always generate a new migration file.

Right now, postgres errors will be reported as generic server errors. I can do better than that. For instance, hitting an endpoint for a resource that doesn't exist in the DB returns the generic error, yet the logged err is:

16:33:01.294Z  WARN JSON-API-Builder: There was an unknown server error. (err.code=42P01)
    error: relation "transaction" does not exist
        at Connection.parseE (/Users/jmeas/webdev/json-api-builder/node_modules/pg/lib/connection.js:539:11)
        at Connection.parseMessage (/Users/jmeas/webdev/json-api-builder/node_modules/pg/lib/connection.js:366:17)
        at TLSSocket.<anonymous> (/Users/jmeas/webdev/json-api-builder/node_modules/pg/lib/connection.js:105:22)
        at emitOne (events.js:96:13)
        at TLSSocket.emit (events.js:188:7)
        at readableAddChunk (_stream_readable.js:176:18)
        at TLSSocket.Readable.push (_stream_readable.js:134:10)
        at TLSWrap.onread (net.js:551:20)

I can use err.code to map these to more human-readable things.

List of postgres errors: https://www.postgresql.org/docs/8.2/static/errcodes-appendix.html

from the endpoints example repo:

{
  "authors": "/v1/authors?include={books,books.chapters,photos}&filter[{id,name,alive,dead,date_of_birth,date_of_death,born_before,born_after}]",
  "books": "/v1/books?include={chapters,firstChapter,series,author,stores,photos}&filter[{author_id,series_id,date_published,published_before,published_after,title}]",
  "chapters": "/v1/chapters?include={book}&filter[{book_id,title,ordering}]",
  "photos": "/v1/photos?include={imageable}",
  "series": "/v1/series?include={books,photos}&filter[{title}]",
  "stores": "/v1/stores?include={books,books.author}"
}

Endpoints does this

The migrations need to be stored in a database, so that someone doesn't need to carry them around on their computer or on a server.

Follow up of #4

Because the migrate script is hardcoded to look for careen in the immediate node_modules, rather than the top-level node_modules!

This will make the example a better representation of the final API

JSON Schema supports this.

Atm the migrate command always logs success, even if careen explodes.

I should validate the resource model a little more thoroughly. I can use JSON Schema for this, most likely.

Also, validating the types could be useful. this list of types may help with that.

Right now, I have attributes, meta, and relationships. I need to add links!

Read has been set up. I need the other three.

• Respect silent mode
• Incorporate verbose mode
• Pipe logs to bunyan for better formatting in dev mode. This may require using exec instead of require

Doesn't help with #16

What's the authorization and authentication story here?

Are all the deps being used?

This is an incomplete checklist of how incomplete my compliance with JSON API is.

• attributes (CRUD)
• meta attributes (CRUD)
• relations
  • many-to-one
  • many-to-many
  • one-to-one
  • relationship endpoints /v1/:resource/relationships/:related
  • related endpoints /v1/:resource/:related
• standard errors
• sparse fieldsets
• sorting
• pagination
• filtering
• links

These don't need to be two scripts

Careen is great for migrations, but it requires them to be on the disk. I could:

1. temporarily write them out to the disk for development purposes
2. wait out to see the resolution of programble/careen#5

Eventually, the migration strings, and their unique IDs, will be stored in the DB that manages the other DB.

This would bootstrap a basic resource file for you.

Is dotenv for the DB url a good idea? Bad idea? What are other alternatives?

1. where do resource models go?
2. what do resource models look like?

This seems silly, but I may not need Lerna anymore. It was really helpful to get me to think about separating out the example from the tool itself, but now it just seems like the utils are arbitrarily split out from the CLI and the server.

I should either move it all back together, or maybe separate out the CLI from the server.

Consider the methods in the helpers namespace as a better way of generating inserts and updates, especially when one needs multi-row updates and multi-row inserts.

See also:

• http://stackoverflow.com/questions/37300997/multi-row-insert-with-pg-promise
• http://stackoverflow.com/questions/39119922/postgresql-multi-row-updates-in-node-js

I'm thinking this lib could be pulled into a project repo, like Moolah. Then you'd write resources / store migrations there to configure the app that the project uses.

Also, instead of having two databases, there can probably just be one DB that stores info about itself. That would keep things simpler.

I suspect that under-the-hood, things will change quite a bit as I add more features. What I can do to prevent regressions is to write unit tests that just run migrations, then hit endpoints. That way, I can be sure that I'm not introducing bugs into existing features, like editing regular attrs or meta.

ty @ericvaladas

pls start should start a dashboard for managing resources. it would be a basic crud app

Careen is exploding when the app reboots. The error is:

Unhandled rejection error: relation "category" already exists

It's like it's not keeping track of which migrations are being run, or something, because this shouldn't be an issue. I'll have to hop into the DB to see what's being stored.

This may be related to #16

It could be useful if this tool could also spin up a mock API that's JS-only. This could be useful for:

1. experimenting with changes before affecting the real db
2. testing
3. offline development without needing to set up a local DB

Eventually, I'll need to generate migrations from changes to the resource model. Resource models can be represented by JSON, so this is a two step process:

1. Have both old and new resource model
2. Diff them
3. Interpret diff, creating migration file from it
4. Eventually, applying the migration

Some tools that may help with this:

https://github.com/chbrown/rfc6902

For some reason, the files that I'm deleting with del are still getting picked up by careen...even after they're gone. I'm not sure what could be causing that...

Careen intermittently errors on startup with the following error:

Unhandled rejection TypeError: Cannot read property '0' of undefined
    at matchesToMigration (/Users/jmeas/webdev/json-api-builder/node_modules/careen/lib/files.js:81:54)
    at /Users/jmeas/webdev/json-api-builder/node_modules/ramda/dist/ramda.js:3865:27
    at /Users/jmeas/webdev/json-api-builder/node_modules/ramda/dist/ramda.js:920:27
    at XWrap.f (/Users/jmeas/webdev/json-api-builder/node_modules/ramda/dist/ramda.js:5350:24)
    at XWrap.@@transducer/step (/Users/jmeas/webdev/json-api-builder/node_modules/ramda/dist/ramda.js:755:25)

There are two ways to do this:

1. Hardcoding the routes in. I can do this because I will know all of the possible relationships from the resource definitions. The downside to this approach is that the route config will become a bit less tidy.

2. Star routes, then managing finding the routes from within the middleware. This can still use the definitions, but it would keep the route config file more orderly.

There are two types of routes I need to support:

/books/1/relationships/author

/books/1/author

@tbranyen mentioned the common situation of a User resource that has a password attribute. That attribute is "hidden"; although it exists, it's never returned from a GET and can't be updated with PATCH.

Out of the box, there should be a hidden flag for attributes that provides this exact behavior.

More generally, there should be a system for advanced attributes, including:

1. hidden attributes (described above)
2. encrypted attributes (such as passwords)

there may be other use cases.

You should be able to specify any number of DBs to connect to. Three common ones would be:

dev, staging, prod

but you could also have one-offs for feature branches, or whatever.

JSON API specifies that CUD are optional. Default should have them enabled, but they can be turned off.

ref

to start tracking updates

They no longer work – not too sure why!

I use Postgres.app for local development and it does not have SSL enabled out of the box. It looks like the database config appends ?ssl=true for us, but it would be nice if we could have control over that parameter. Maybe we could include the ssl parameter in our DATABASE_URL variable so that we have control over whether SSL is used in our environments.

• What if the resources directory isn't found?
• What if there are zero resources?
• What if there is a resource parsing error?

All of these things should be logged out.