stackvana / hook.io Goto Github PK

It's pretty common for webhooks to receive a JSON payload, and it would be cool if hook.io supported application/json as a content type.

A simpler/smaller change that would immediately improve the experience is adding collect-stream to the module list so I can at least skip the boilerplate for concatenating a stream into a big old string before calling JSON.parse.

It would be nice to have some visualizations of the growth of the project.

We are already collecting all the data necessary to create these visualizations / statistics.

see: user.ctime, hook.ran Hook.all

The current script for installing modules is not ideal.

It should be able to check if the current version of the module is already installed before attempting to install the same module again ( overwriting the old one and taking time )

see: https://github.com/bigcompany/hook.io/blob/master/modules/install.js

The hooks are currently self-contained files with integrated schema. Maybe a next step could be to integrate micro tests to automatically run whenever a hook gets updated.

Of course we can already test the hooks easily with the web-interface, but it is not quite the same as running some assertions from code.

Minimum requirements: async, multiple cases, labels, and maybe setup/teardown.

A nice way could be to let module['exports'].test be a function that returns the micro suite, like a one-level deep Vows or Expresso.

Could also be fun to do it with Tap(e): every hook could get a dedicated endpoint /test that publishes a TAP stream other hooks can consume.

Cool to see this project come back - I remember on the original project I added a bunch of wiki pages. Any chance that these can come back?

https://github.com/digitalocean has stepped up and sponsored hook.io for a year of hosting.

We should add their logo to the homepage and add them to a small page of all our sponsors.

Hi! I need to do some mucking about with AWS, could you add xml2js and aws-sdk?

mime

The cron process should now safely ignore invalid crons, but since they are treated as plain strings with no validation, it's possible to enter invalid cron's into the system.

A validator should be added to Hook.cron so that invalid cron patterns are not allowed to be saved.

Thanks to @wyzwhp for discovering this issue.

It would be nice to have a way of storing and associating key/pair settings within my account. These settings would then be available as env variables and accessible via process.env within hooks I have ownership of.

Use case: Lets say I want to write a hook that talks to an api. I do not want to pass the api key as a query param or post data to the hook. I would instead grab it from the env var.

An example - https://gist.github.com/desmondmorris/aa4628375ba9b38adffb

When installing hook.io there should be a bash script for configuring the server environment.

The first item to add to this script is setting EXPORT variables for the hook.io log file paths. This way, can don't have to hard-code the log paths into our start scripts.

appcached

The entire hook.io project should be as accessible as possible for any hardware, software, language, culture, location, or physical or mental ability.

The guidelines for web accessibility can be found here:

http://www.w3.org/standards/webdesign/accessibility
http://en.wikipedia.org/wiki/Web_Content_Accessibility_Guidelines

This will require a complete review of the current site. The current HTML should be close to what is needed, but much additional work will still need to be done.

This is probably a bit crazy but what if we added some docker magic?

My thought was by adding npm module docker-stream

docker containers run as microservices through pipes FTW

Thanks for fixing #6, would it be possible to accept/parse custom media types (e.g. application/vnd.github.v3+json)?

Am getting the same http response for all hooks (when run in browser or curl)

error happened in your connection

~ $ curl --verbose http://hook.io/Marak/echo
* About to connect() to hook.io port 80 (#0)
*   Trying 72.2.115.57...
* connected
* Connected to hook.io (72.2.115.57) port 80 (#0)
> GET /Marak/echo HTTP/1.1
> User-Agent: curl/7.26.0
> Host: hook.io
> Accept: */*
>
* additional stuff not fine transfer.c:1037: 0 0
* HTTP 1.1 or later with persistent connection, pipelining supported
< HTTP/1.1 200 OK
< x-powered-by: Express
< date: Wed, 03 Dec 2014 14:18:53 GMT
< connection: close
< transfer-encoding: chunked
< set-cookie: xxx; Path=/; HttpOnly
<
* Closing connection #0
error happened in your connection

Currently, all validation errors ( such as bad input on a form submit ) are sent back to the client as JSON.

This is okay, since the data is being passed back to the user. It would be better if there was a nice template shown to the user.

The production site is currently down due to non-payment of server bills.

I've been paying the cost out of pocket for months and can't afford it anymore.

We need to secure some funding to keep the project running until billing #36 is implemented and the platform can pay for it's own hosting.

Custom domain routing for hooks is almost complete.

The stubs for custom domains should be completed and deployed.

There are two small issues ( so far ) with using the stripe.js checkout widget.

1. When using the slider on the /billing page. If a value is selected, then the stripe overlay is loaded by clicking on the pay button, then the value appears in the overlayed stripe form. If this overlay is then closed, and a new value is selected on the slider, then the pay button is clicked again...the value of the overlay is not updated. Unsure of the solution. The value inside the stripe iframe needs to be updated...or the entire element needs to be removed and added everytime it's closed / loaded.
2. If data-email is provided to the stripe checkout.js widget, then the email field will be pre-filled. This is good, but the input box becomes un-editable, which is not good. I'd like to see the email field pre-filled, but still be editable. Possible solution might involve contacting Stripe / making a change to the checkout.js file itself / or just finding a configuration variable we can set.

Currently, there is no easy way to remove a hook from the database.

There should be a button added on the hooks.html page for logged in users to delete the hook from hook.io

When processing crons ( https://github.com/bigcompany/hook.io/blob/master/lib/resources/cron/index.js#L10 ) we currently create N concurrent outgoing HTTP requests where N is the amount of Hooks that need to be run at that specific cron interval.

It would be better if the cron script had a concurrency limit of X, where only X concurrent outgoing connections will be established. The async library should have a good API for this.

I'm doing some experimentation with hook.io. I assumed I would have access to something like the requests api, but all I see is a method called hook.open in the documentation but that appears to open a stream, when I just want to fire off an HTTP request. I'm just trying to proxy some data from one API to another after manipulating it a little. Am I missing something obvious?

Hi,

when i go back in the browser and press run again (in debug theme) than it does not allways use the latest version. Sometimes it seems to run a old one, presenting me an old error.

There should be a better way to view referral data and affiliate payouts.

Referrals are multi-level ( a tree ), so the visualization should reflect this.

There is a minor bug where the schema for the Hook will not display correctly after it's created / forked.

A solution should be in production within a day.

After reviewing the logs for the past 24 hours, it appears it's still possible that certain run-time user errors in Hooks can cause the worker process to crash.

The worker shouldn't crash, but much more importantly, all error messages must be sent back to the client.

The solution might involve putting an uncaughtexception handler on the worker, but in theory, try / catch should work. More investigation is required.

A solution should land in production within a few days.

See http://hook.io/grncdr/echo for example.

Something appears wrong with Crons. The hook run count is not updating.

Investigation is required

Crons are possibly not working as intended.

Solution should be in production within a day.

Getting error "error happened in your connection" on all hooks.

The code for running a hook with a generic input / output buffer in hook.runUntrustedHook should be a separate module / project.

The application of themes and presenters should be transform streams applied to the output of this module.

I've started this refactor locally, but am now marking it as low-priority since we don't immediately need it. Probably will be done within a month.

If anyone needs this / wants to collaborate on this let me know.

Hey @Marak I found an error testing an old hook I was using to test this module. I believe the module crash when the gist is no longer available. Maybe it would we good to avoid this exception and show a pretty message? Let me know and i'll try to merge a PR with a fix.

//output
/root/hook.io/temp/vrunoa/560bb84d38e24078b66a.js:1
(function (exports, require, module, __filename, __dirname) { <!DOCTYPE html>
                                                              ^
SyntaxError: Unexpected token <
    at vm.js:73:16
    at Module._compile (module.js:443:25)
    at Object..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.<anonymous> (module.js:310:12)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)
    at EventEmitter.attemptToRequireUntrustedHook (/root/hook.io/lib/resources/hook/attemptToRequireUntrustedHook.js:18:21)
    at /root/hook.io/lib/resources/hook/runHook.js:22:11
    at /root/hook.io/lib/resources/hook/fetchHookSourceCodeFromGithub.js:43:13

http://hook.io/vrunoa/echo
https://gist.github.com/vrunoa/560bb84d38e24078b66a

All the examples at http://hook.io/examples are returning "error happened in your connection"

It looks like in the database the resource property user.name is not normalized to lowercase. This has probably also leaked into the owner property on other resources.

A quick migration script and a few lines of code should fix this.

The issue is that since a lot of our incoming requests are coming over HTTP, case-sensitivity can be a problem. Better to normalize all resources keys to be lower-case.

The platform has been up and running three months with no major issues.

I've been paying for the server hosting out of pocket, and am at the point where I need to add the ability for paid accounts ( to help pay the hosting bill ).

The billing system will start out as purely optional. I'm going to start with only crypto-currency payments ( so no credit cards ) using our Safewallet platform

Key Goals

• Integrate payment options using Safewallet ( bitcoin / litecoin / dogecoin / peercoin )
• Add Graphical Interface for managing Billing options

Initial testing shows we might now be occasionally experiencing socket resets from workers.

Possible regression introduced by #16

The problem was most likely introduced by switching to request from hyperquest in this file: https://github.com/bigcompany/hook.io/blob/master/lib/resources/hook/runRemote.js

Solution should be in production within a few days.

Hooks should be able to execute based on a cron pattern.

see: http://hook.io/cron

I've gotten this somewhat working locally, but it requires more testing.

This should be ready and in production within a few days.

First: Awesome service that I've been wanting for a while!

I would like to use this to transform Web Hook data from one service into a format that another service expects.

Example:

Service A WebHook -> hook.io -> Service B Rest Request

This use case absolutely requires HTTPS support. Is HTTPS support coming any time soon?

Cheers,
Lyle

When I browse the available npm packages I miss a line-splitter transform-stream. With the focus on (unix) steams and pipes one of these would very handy to be able to do line-based text input.

I'm thinking https://www.npmjs.org/package/byline could be a good candidate, but I defer to your experience.

The JSON example you gave in another issue does not work:

curl -H "Content-Type: application/json" -d '{"username":"xyz","password":"xyz"}' http://hook.io/Marak/echo

Returns:

Error communicating with worker 10000

Error: socket hang up
    at createHangUpError (_http_client.js:210:15)
    at Socket.socketOnEnd (_http_client.js:288:23)
    at Socket.emit (events.js:129:20)
    at _stream_readable.js:898:16

Currently, it's possible for users to submit bad gist links which could cause the worker to crash.

This is not acceptable behavior. Bad gist links should be caught in validation, and the worker should not have any uncaught error conditions.

Fix should be easy. Estimated to be in production within a few days.

There is a minor bug with the body parser causing submitted form variables to not appear in Hook.params.

Resolution should be in production within a day.

After switching to the multi-server architecture we are now experiencing an issue where the passport session on the server is being destroyed after the server pipes a request to a worker.

The worker itself is stateless and does not require the session info.

The issue is that after the request is sent to the worker, the server itself is losing track of the session. Maybe it's being overwritten by the worker response.

The consequence is that the user will have to login to the website every time they run a hook through the web gui. It's actually not that bad since github holds a cookie and hook.io will redirect back to the requested page. Should be somewhat seamless.

Issue is currently non-critical.

Use https://github.com/icholy/ttygif to create gif screencasts for all curl examples!

Seems like a regression in error handling has caused user errors in hook source code to not be sent to the client.

An overhaul of the server / worker architecture has been completed locally. This should be deployed within a few days and fix the issue of errors not being returned to the user.

Instead of having a specific url for registering new user referrals, the referral should be automatically set based on the owner of the first hook visted.

For example: A new user visits http://hook.io/marak/echo and then signs up. That user referral is then credited to marak

Users should be able to always manage / administrate a hook by visiting the hook in the browser with an admin url variable.

• Create new theme admin
• Use admin theme when ?admin is provided to the hook

Currently by default, all of a users hooks will be listed publicly on their homepage ( hooks.html ).

In certain cases users might not want their hooks to be public / enumerable.

• Add new property status to hook resource
• Add ability in UI to toggle hook status

Reported by @scoin