Integrate DataFire on Express with '/api' already reserved about datafire HOT 13 CLOSED

Hi, thanks for reaching out!

DataFire does support Express directly (and uses Express under the hood). Have you looked at the cookbook?

I think what you want to do is edit this file in nodetomic and add something like this:

let datafire = require('datafire');
let project = datafire.Project.fromDirectory(__dirname); // Dir containing DataFire.yml
let projectServer = new datafire.ProjectServer(project);

projectServer.getRouter().then(router => {
  app.use('/df', router);
});

You should also set openapi.basePath in DataFire.yml:

openapi:
  host: api.example.com
  schemes:
    - https
  basePath: /df

paths:
  /gifs:
    get:
      action: reddit_rss/subreddit
      input:
        subreddit: gifs

That example should serve r/gifs at https://api.example.com/df/gifs. You should also be able to mount on the same /api path used by nodetomic.

Hope I understood your question correctly...if the above doesn't work let me know.

Good luck!

from datafire.

Well, I pretty much have what you set up. This express server I am using is set up very 'elegantly' so If this is an edge case to you don't bother, I'll keep hammering away until I figure something out so I can use DataFire...on the other hand if you are inclined to curiosity and maybe finding a more bulletproof integration for express DF here is a summary of what I have:

src
-config
--config.js
-lib
--datafire
---DataFire.yaml (or should this be .yml, you have it as both in different parts of your doc)
---goodbye.js
--express
---client.js
---index.js

datafire/DataFire.yaml

openapi:
  host: localhost:3434
  schemes:
    - http
  basePath: /v1

paths:
  /goodbye:
    get:
      action: ./goodbye.js

datafire/goodbye.js

let datafire = require('datafire');

module.exports = new datafire.Action({
  handler: input => "Goodbye, y'all come back now ya hear",
})

https://github.com/kevoj/nodetomic-api-swagger/blob/develop/src/lib/express/client.js

express/client.js

export default (app) => {

  // Paths 404 from url
  app.get(config.path.disabled, (req, res) => {
    res.status(404).sendFile(`${config.base}/views/404.html`);
  });

  // Point static path to client by default
  let client = config.client;
  let file = 'index';



  // If not exits client, when set internal default
  if (!fs.existsSync(config.client)) {
    client = `${config.base}/views/default`;
    file = config.mode;
    if (config['socket.io'].example) {
      app.use('/socket', express.static(`${client}/socket.html`));
      app.use('/token', express.static(`${client}/token.html`));
    }
  }

  app.use(express.static(client));
  app.use(favicon(path.join(client, 'favicon.ico')));

  // Folder client
  app.get('/*', (req, res) => {
    res.sendFile(`${client}/${file}.html`);
  });

  let datafire = require('datafire');
  let project = datafire.Project.fromDirectory('../datafire'); // Dir containing DataFire.yml
  let projectServer = new datafire.ProjectServer(project);
  projectServer.getRouter().then(router => {
    console.log(chalk.greenBright(`Datafire-> serving /v1`));
    app.use(':url(v1)/*', router);

   //also tried     app.use('/v1', router);
   //also tried    app.use(':url(v1)/*', router);
   //also tried    app.use(':url(v1)', router);
  });

  // Examples
  // app.use('/bower_components', express.static(`${config.root}/bower_components`));
  // app.get('/:url(admin)/*', (req, res) => {
  //     res.sendFile(`${config.client2}/index.html`);
  // });

}

https://github.com/kevoj/nodetomic-api-swagger/blob/develop/src/lib/express/index.js

express\index.js

export function index(app) {

  return new Promise((resolve, reject) => {

    app.use(bodyParser.json({ limit: '5mb' }));
    app.use(bodyParser.urlencoded({ extended: false }));
    app.use(fileUpload());
    app.use(cookieParser());
    app.use(methodOverride());
    app.use(compression());
    app.use(helmet());
    app.use(cors({ origin: true, credentials: true }));

    if ("twitter" in config.oAuth && config.oAuth.twitter.enabled)
      app.use(session({ secret: config.secret, resave: false, saveUninitialized: false }));

    app.use(passport.initialize());
    app.use(passport.session());

    // Morgan
    if (config.log)
      app.use(morgan('dev'));

    resolve();

  })

};

https://github.com/kevoj/nodetomic-api-swagger/blob/develop/src/config/index.js

config/index.js
...
  // globals
  mode: process.env.NODE_ENV || 'development', // mode
  name: APP_NAME, // name
  node: parseInt(process.env.NODE_APP_INSTANCE) || 0, // node instance
  root: path.normalize(`${__dirname}/../..`), // root
  base: path.normalize(`${__dirname}/..`), // base
  client: `${path.normalize(`${__dirname}/../..`)}${CLIENT}`, // client
  log: true // logs

My console.log statement in the client.js where I register DataFire fires on startup so I assume it is registered.

Morgan reports...

GET /v1/goodbye 304 6.392 ms - -
GET /v1/style.css 304 1.090 ms - -
GET /v1/logo.svg 304 0.652 ms - -

Nothing else returned in console like my action requests "Goodye..."

Finally, to ensure datafire is running I can datafire run ./goodbye.js in the /datafire directory and I receive the expected "Goodbye..." action in terminal.

from datafire.

Ahh I think changing to DataFire.yml (rather than .yaml) should do the trick.

Sorry for the confusion there. I found one instance of .yaml in the docs, but if you spot it anywhere else I'll change it.

We should also throw an error if the file isn't there, will push a fix shortly.

from datafire.

Well glad I could help there... 😢 but I tried all variants with .yml and .yaml. Like I said, if this is an edge case then so be it, I'll figure it out. It's probably something with the way this stack is setup redirecting away from my intended path for DataFire. I hope to get a break through...really have big plans to use the plug and play nature of your framework without rewriting all my endpoints as we discussed in email a few days ago...

from datafire.

Another related question on en Express integration...Am I still required to WRITE_GOLDEN=true npm test if I change the openApi spec? I did it once and it ran the local server script for npm test so don't know if it had any effect on the openapi yaml

from datafire.

OK a couple ideas:

1. Project.fromDirectory() should be relative to cwd, not the current file. Can you try Project.fromDirectory(__dirname + '/../../../datafire')?

2. You may need to register the DataFire router before the other routes, which means moving everything inside of project.getRouter().then().

With those two changes I was able to mount the router on /df.

export default (app) => {
  let datafire = require('datafire');
  let project = datafire.Project.fromDirectory(__dirname + '/../../../datafire'); // Dir containing DataFire.yml
  let projectServer = new datafire.ProjectServer(project);
  projectServer.getRouter().then(router => {
    app.use('/df', router);

    // Paths 404 from url
    app.get(config.path.disabled, (req, res) => {
      res.status(404).sendFile(`${config.base}/views/404.html`);
    });

    // Point static path to client by default
    let client = config.client;
    let file = 'index';

    // If not exits client, when set internal default
    if (!fs.existsSync(config.client)) {
      client = `${config.base}/views/default`;
      file = config.mode;
      if (config['socket.io'].example) {
        app.use('/socket', express.static(`${client}/socket.html`));
        app.use('/token', express.static(`${client}/token.html`));
      }
    }

    app.use(express.static(client));
    app.use(favicon(path.join(client, 'favicon.ico')));

    // Folder client
    app.get('/*', (req, res) => {
      res.sendFile(`${client}/${file}.html`);
    });
 });
}

from datafire.

Also, the WRITE_GOLDEN=true is only needed if you're making changes to the DataFire repo, not if you're making changes in your project. It just updates a test file in the DataFire repo.

from datafire.

Sadly no luck...exact result (or lack thereof) as before.

I think there is some path redirects in the code I need to figure out. In the config/index.js he is abstracting away the api path among others

  path: {
    disabled: '/:url(api|auth|assets|lib)/*' // paths 404
  },

Hey I'm feeling guilty you're spending your time on this, not my intention. I was hoping it was just a boneheaded mistake on my part you'd be able to catch right away. You've got to be up to your eyeballs with your new public release so I won't waste any more of your time sorting my problem. I really appreciate all you've done so far and will figure this out because I want to use your products..! I'll report back if I make progress (hopefully with a subscription 😉 )

from datafire.

Glad to hear it :) FWIW, here's my working example

Changing the path to :url(v1)/* seems to break it though.

Good luck, and I'm always happy to help...this kind of stuff is great for exercising the framework and making sure it plays nicely with other tooling.

from datafire.

Thanks for all Bobby,

I'm still pounding away. Here's a silly question, do I have to use the DataFire.yaml? This stack uses yaml for its API descriptor also, calling functions in js files.

example

/api/hello:
  x-swagger-router-controller: hello
# list
  get:
    operationId: list
    tags:
      - Hello
    summary: Get list Hello's
    description: Returns all hello
    responses:
      200:

Is this DataFire.yaml hard coded or is it ok that I have the project server service /v1 path? I'm going to give it a try by changing the path and assigning a name to the function in the goodbye.js

• will report back if successful.

Thank you again!

from datafire.

You don't need a DataFire.yml, you can also create a Project programatically. The contents of DataFire.yml just get passed to the Project constructor:

let datafire = require('datafire');
let proj = new datafire.Project({
  directory: __dirname, // the directory that would normally contain DataFire.yml
  paths: {/* ... */},
  tasks: {/* ... */},
})

You could also never create a Project, and just let nodetomic handle all the routing. You can then run your DataFire actions manually in the nodetomic routes. E.g. the example you gave above might have something like:

./hello.js

let datafire = require('datafire');
let action = new datafire.Action({
  handler: input => "Hello!",
})

module.exports.list = function(req, res) {
  action.run().then(result => res.json(result));
}

from datafire.

You could also never create a Project, and just let nodetomic handle all the routing

THIS!!! is exactly where I am traveling this morning and your timely hello.js snippet just unlocked the door and allowed a datafire action.

SUCCESS!

Now I can start moving forward again this time with datafire sauce. Going to try a few more advanced actions and integrations now and see where it leads.

You are very much appreciated here! Suggest you put this in your cookbook (using DataFire with API routes or without a Project Server or something like that) as it opens up your platform for existing API projects

from datafire.

Great! Just added to the cookbook

Let me know if I can help with anything else!

from datafire.