Recently, a change was committed (cf2ac9d) to stop monkey-patching ServerRequest and ServerResponse. The result is an API that looks like:

function(env, next) {
  env.response.statusCode = 200;
  env.response.headers = { 'Content-Type': 'text/plain' };
  env.responseBody = 'Aloha, Gaia!';

  next(env);
}

That API mismatch is unpretty. The goal should be to create a change so that the API does NOT monkey-patch, but instead wraps ServerResponse (in this case). The end result should look something like:

function(env, next) {
  env.response.statusCode = 200;
  env.response.headers = { 'Content-Type': 'text/plain' };
  env.response.body = 'Aloha, Gaia!';

  next(env);
}

Ah, that's better. BONUS: Add a shortcut that allows something like:

env.response.set([200, { 'Content-Type': 'text/plain' }, 'Aloha, Gaia!']);

But better than that (hopefully).

Somewhat on the fence with this...

It would be easier for developers to work with the response body after it has been fully retrieved. This happens at the expense of asynchronous piping from the target through the proxy to the client.

Will likely need this for the request and the response body.

The HAR spec[1] has picked up some traction. Plus, I like it. We should investigate using it for Argo's trace output.

There are some existing tools that can help with visualizing this. In particular, I like the detailed info around send, wait, and receive times regarding HTTP requests.

[1] http://www.softwareishard.com/blog/har-12-spec/

Hi @kevinswiber,

I would like to use Argo to proxy calls to my API when a specific url is matched.

For example, proxy: /api/v2/datasets -> example.com/api/v2/datasets

Its clear how to do this generically for all urls, but how do I trigger proxy only when url contains /api/v2?

I've been tracking this library since your talk in Toronto. You mentioned that Apigee is working on something for Node.js. Any updates on that?

var http = require('http');
var argo = require('argo');

var proxy = argo().target('http://weather.yahooapis.com');

http.createServer(function(req, res) {
  req.pipe(proxy).pipe(res);
}).listen(1337);

That might not give us what we want. Maybe something like...

var http = require('http');
var argo = require('argo');
var streamer = require('argo-streamer');

var proxy = argo().target('http://weather.yahooapis.com');

http.createServer(function(req, res) {
  var stream = streamer(proxy, req, res);
  req.pipe(stream).pipe(res);
}).listen(1337);

Better error handling and process longevity.

File names gone wild.

./linkedList.js -> linked_list.js
./example/oauthOptions.js -> ./example/oauth_options.js

Support this:

argo()
  .use(function(handle) {
    handle('error', function(env, next) {
      response.statusCode = 500;
      next(env);
    });
  })

1. Catch an error.
2. env.argo.pipeline.siphon('error', env, next);

Maybe make it so you can siphon between Argos?

Right now, the pipeline is rebuilt on each request. Stop that.

This is needed for tracing to determine which trace lines are correlated with the same request and to record the time sequence. Async communication can create a partially ordered queue. These proposed changes will help to resolve ordering by the client.

Example:

{
  "requestId": "9cff0456-84b3-401a-89f6-bc9eae559369",
  "sequenceNumber": 1
}

Right now, runner.js clusters on max number of CPUs. This should be configurable and default to no clustering.

All new code, including this, should be submitted with tests. The goal is 100% coverage.

Argo hangs when using the proxy. This appears to be a problem introduced by the switch to Streams 2 support.

Thanks for the heads up, @timanglade! Next time, don't hesitate to create an issue. For important fixes like this, @mdobson and I tend to act fast.

Export Argo as a module...

module.exports = require('argo')().target('http://weather.yahooapis.com');

Run it with argod.

$ argod --port=1337 --hostname=127.0.0.1 --backlog=128 ./server.js

1. What's going in?
2. What's coming out?
3. How long did each step take?

First off, thanks so much for the rad product!

Totally my fault here, but I'm going a little batty trying to sort this out, and would totally appreciate any pointers in the right direction :)

https://github.com/patcon/polis-api-proxy/blob/master/path_parameterization.js#L33

It's pretty messy, but I've intercepted POST requests, caught the body, and merged my changes in, but I can't seem to sort out the best way to "pass this along"..! :)

Break this beast apart. It has grown too big.

Protect the backend!

Prevent the backend server from getting slammed with a 1,000,000 requests while waiting for the cache to fill.

There seem to be quite a lot of broken links, primarily to example configs. Here are some I came across quickly:

• At https://argoproj.github.io/docs/argo/getting-started.html, link to Argo Workflows by Example is broken - perhaps it should link to https://argoproj.github.io/docs/argo/examples/readme.html
• At https://argoproj.github.io/docs/argo/workflow-executors.html, link to the configmap is broken
  • The same broken link also appears at https://argoproj.github.io/docs/argo/links.html, https://argoproj.github.io/docs/argo/offloading-large-workflows.html and https://argoproj.github.io/docs/argo/resource-duration.html
• At https://argoproj.github.io/docs/argo/fields.html#workflow, open the "Examples" rollup and these all appear to be broken - first example is archive-location.yaml
• At https://argoproj.github.io/docs/argo/resource-duration.html, link to ../util/resource/duration.go is broken
• At https://argoproj.github.io/docs/argo/rest-api.html, link to golang code example is broken

I will stop there :-)

Just the sexy bits on the README. Move more complex features into the Wiki.

This could be neat.

1. Sniff Accept-Encoding.
2. Modify (request|response).getBody so it auto-inflates when accessed. (proxies matter)
3. Re-deflate prior to serving.

Take inspiration from old Tabasco work (https://github.com/kevinswiber/Tabasco). Check out Sinatra's routing (https://github.com/sinatra/sinatra). Investigate routing in Express, as this might already be a copy of Sinatra (https://github.com/visionmedia/express).

@wzhanw

See title.

Argo currently has undefined behavior when setting up a pipeline with .route before .use.

Example:

var assert = require('assert');
var argo = require('argo');

argo()
  .route('/hello', function(handle) {
    handle('request', function(env, next) {
      assert(env.test, 'success'); // FAILS
      next(env);
    });
  })
  .use(function(handle) {
    handle('request', function(env, next) {
      env.test = 'success';
      next(env);
    });
  })
  .listen(3000);

This will currently produce an AssertionError.

The behavior should be defined so that pipelines are executed in this order:

1. Middleware Request Pipeline
2. Route Request Pipeline
3. Proxy Target
4. Route Response Pipeline
5. Middleware Response Pipeline

The HTTP spec requires the HTTP 405 Method Not Allowed response to have an "Allow" header listing the methods.

I noticed this because Apigee Edge complains about it.

In perf testing I found it's quite easy to get argo to die if the target can't be reached or if a new request can't connect, do the DNS lookup, or whatever. That makes some benchmarks run for a very short time. argo should at least have an 'error' handler on the HTTP request and return an error to the client, and eventually have a whole error pipeline.

We're using parsed.hostname but it should be parsed.host for the Host header.

The server at localhost:1337 will get a request with the Host header set to only localhost not localhost:1337.

argo()
  .target('http://localhost:1337')
  .listen(3000)

Steps to repro

1. Use Nodejs 0.11 with this code.

var argo = require('argo');

argo()
  .use(function(handle) {
    handle("response", function(env, next) {
      env.response.statusCode = 200;
      env.response.headers = { 'content-type':'text/plain'};
      env.response.body = 'Hello';
      next(env);
    });
  })
  .listen(3000);

Stack trace

_http_incoming.js:187
        if (util.isUndefined(dest[field])) dest[field] = value;
                                 ^
TypeError: Cannot read property 'host' of undefined
    at IncomingMessage._addHeaderLine (_http_incoming.js:187:34)
    at IncomingMessage.incoming._addHeaderLine (/Users/ApigeeCorporation/node/paas-router/node_modules/argo/argo.js:36:22)
    at IncomingMessage._addHeaderLines (_http_incoming.js:131:12)
    at HTTPParser.parserOnHeadersComplete (_http_common.js:92:19)
    at Socket.socketOnData (_http_server.js:358:22)
    at Socket.EventEmitter.emit (events.js:98:17)
    at readableAddChunk (_stream_readable.js:156:16)
    at Socket.Readable.push (_stream_readable.js:123:10)
    at TCP.onread (net.js:509:20)

I've confirmed that this code works with node 0.10

We've got tests that are broken in Argo. Mostly around routing.

There's a lot of duplication in platform.js when it comes to figuring out the route handlers. Perhaps caching the handlers would make sense.

Example:

var assert = require('assert');
var argo = require('argo');

argo()
  .get('^/hello$', function(handle) {
    handle('request', function(env, next) {
      next(env);
    });
  })
  .get('^/world$', function(handle) {
    handle('response', function(env, next) {
      env.count = env.count || 0;
      env.count++;

      assert.equal(env.count, 1);
      next(env);
    });
  })
  .listen(3000);

A call to /world results in AssertionError: 2 == 1.

I thought there was an issue for this, but apparently not. I'll add one in retrospect, despite the lack of value implied. :)

We need a way to nest complex proxy configs. "When going to /api, use CORS middleware."

Using .map would be fitting.

Example:

var argo = require('../');
var cors = require('./cors');

argo()
  .use(cors)
  .get('/outer', function(addHandler) {
    addHandler('request', function(env, next) {
      env.response.body = 'Outer scope!';
      next(env);
    });
  })
  .map('/web', function(proxy) {
    proxy 
      .use(function(addHandler) {
        addHandler('response', function(env, next) {
          env.response.headers['X-Stuff'] = 'Yep';
          next(env);
        });
      })
      .get('/greeting', function(addHandler) {
        addHandler('request', function(env, next) {
          env.response.body = 'Hello, world!';
          next(env);
        }); 
      });
  })
  .listen(process.env.PORT || 3000);

See commit c25a25d.

Run some benches, yo.

When nesting proxies, such as...

argo()
  .map('/locations', function(proxy) {
    proxy
      .target('http://4sq/locations');
  })
  .target('http://foodcity/products')
  .listen(1337)

...the Frame of current execution doesn't include the responsibility of target execution. Therefore, hitting http://localhost:3000/locations will execute the target request twice.

Fix, yo.

This is important for targeting backend servers.

Cache it on the proxy, yo.

This code sample appends the mapped endpoint to the target URL.

0.0.0.0:3000/mtraining proxies to api.usergrid.com/mtraining/sandbox/mtrainings

var argo = require('argo'),
    router = require('argo-url-router');

var server = argo();

server.map('/mtraining', function(server) {
  server.target('http://api.usergrid.com/mtraining/sandbox');
});

server.map('/mdobson', function(server) {
  server.target('http://api.usergrid.com/mdobson/sandbox');
});

server.listen(3000);

Psst. Hey, Kevin, stop being lazy.

Adding a placeholder for testing.

I've been working to get test coverage up. Metrics are calculated using jscoverage. This is only one metric for quality, and it's pretty weak (IMHO), but people pay attention to it.

The end goal is to have super thorough testing (without changing the structure of code to only accommodate better testing).

We're at 85% at this very moment on my local branch (for Argo core, not the OAuth package).

I should get to 100% today. This is pretty important.

Not sure what happened, but this no longer works:

var argo = require('argo');
var router = require('argo-url-router');

argo()
  .use(router)
  .get('/hello', function(handle) {
    handle('request', function(env, next) {
      env.response.body = 'World.';
      env.target.skip = true;
      next(env);
    });
  })
  .target('https://api.usergrid.com')
  .listen(3000);

Expected output of hitting http://localhost:3000/hello is "World." Instead, the request gets proxied to the target.

Bit throttle super deluxe.