Giter VIP home page Giter VIP logo

apex / up Goto Github PK

View Code? Open in Web Editor NEW
8.8K 8.8K 368.0 23.29 MB

Deploy infinitely scalable serverless apps, apis, and sites in seconds to AWS.

Home Page: https://up.docs.apex.sh

License: MIT License

Makefile 0.30% Go 96.75% JavaScript 0.92% Shell 2.03%
api api-gateway aws aws-infrastructure aws-lambda deploy-tool deployment golang graphql heroku http lambda microservice microservices nodejs serverless static up

up's Introduction

No longer maintained

This software is no longer being maintainted and should not be chosen for new projects. See this issue for more information

Apex Serverless Architecture

Apex lets you build, deploy, and manage AWS Lambda functions with ease. With Apex you can use languages that are not natively supported by AWS Lambda through the use of a Node.js shim injected into the build. A variety of workflow related tooling is provided for testing functions, rolling back deploys, viewing metrics, tailing logs, hooking into the build system and more.

This project is designed for event-driven pipelines as it does not abstract away FaaS (functions as a service). If you are building web applications, APIs, or sites, consider using Apex Up, which provides a more out-of-the-box experience for these use-cases.

Installation

On macOS, Linux, or OpenBSD run the following:

curl https://raw.githubusercontent.com/apex/apex/master/install.sh | sh

Note that you may need to run the sudo version below, or alternatively chown /usr/local:

curl https://raw.githubusercontent.com/apex/apex/master/install.sh | sudo sh

On Windows download binary.

After downloading, rename binary file 'apex.exe', then add to PATH.

If already installed, upgrade with:

apex upgrade

Runtimes

Currently supports:

  • Node.js
  • Golang
  • Python
  • Ruby
  • Java
  • Rust
  • Clojure

Example projects for all supported runtimes can be found in _examples directory.

Features

  • Supports languages Lambda does not natively support via shim
  • Binary install (install apex quickly for continuous deployment in CI etc)
  • Hook support for running commands (transpile code, lint, dependency management, etc)
  • Batteries included but optional (opt-in to higher level abstractions)
  • Environment variable population via command-line, file, or inline config
  • Idempotent deployments (checksums skip already-deployed code)
  • Multiple environments via project.ENV.json and function.ENV.json files
  • Configuration inheritance and overrides
  • Command-line function invocation with JSON streams
  • Command & function name autocompletion
  • Function name globbing (ex: apex deploy api_*)
  • Transparently generates a zip for your deploy
  • Project bootstrapping with optional Terraform support
  • Function metrics and cost analysis
  • Ignore deploying files with .apexignore
  • Function rollback support
  • Tail function logs
  • Concurrency for quick deploys
  • Dry-run to preview changes
  • VPC support
  • Multiple region support
  • Lambda@Edge support

Sponsors

Does your company use Apex? Help keep the project bug-free and feature rich by sponsoring the project.

Backers

Love our work and community? Become a backer.

Example

Apex projects are made up of a project.json configuration file, and zero or more Lambda functions defined in the "functions" directory. Here's an example file structure:

project.json
functions
├── bar
│   ├── function.json
│   └── index.js
└── foo
    ├── function.json
    └── index.js

The project.json file defines project level configuration that applies to all functions, and defines dependencies. For this simple example the following will do:

{
  "name": "example",
  "description": "Example project"
}

Each function uses a function.json configuration file to define function-specific properties such as the runtime, amount of memory allocated, and timeout. This file is completely optional, as you can specify defaults in your project.json file. For example:

{
  "name": "bar",
  "description": "Node.js example function",
  "runtime": "nodejs4.3",
  "memory": 128,
  "timeout": 5,
  "role": "arn:aws:iam::293503197324:role/lambda"
}

Now the directory structure for your project would be:

project.json
functions
├── bar
│   └── index.js
└── foo
    └── index.js

Finally the source for the functions themselves look like this in Node.js:

console.log('start bar')
exports.handle = function(e, ctx) {
  ctx.succeed({ hello: e.name })
}

Apex operates at the project level, but many commands allow you to specify specific functions. For example you may deploy the entire project with a single command:

$ apex deploy

Or whitelist functions to deploy:

$ apex deploy foo bar

Invoke it!

$ echo '{ "name": "Tobi" }' | apex invoke bar
{ "hello": "Tobi" }

See the Documentation for more information.

Links


Build Status Slack Status GoDoc OpenCollective OpenCollective

up's People

Contributors

bnoguchi avatar chemidy avatar corysimmons avatar davidmc24 avatar desm avatar harlow avatar jaimeiniesta avatar joshdickson avatar kruppel avatar louisrli avatar markelog avatar mhemmings avatar naartjie avatar philippgille avatar sanderhahn avatar sbussard avatar shaicoleman avatar skabbes avatar stephenmathieson avatar tj avatar tlight avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

up's Issues

Status pages

Wouldn't be hard to have a little self-hosted CLI driven status page, especially if you don't need all the random stuff statuspage.io provides.

Would you use this if it:




Do you care about integrations?




Do you have a status page right now?




Pricing model for Pro

I haven't decided on this yet, likely tiered, I'd be happy if small companies with little funding don't have to pay much, startups and established companies can afford more. Perhaps by company size? Maybe just by feature-set, we'll see, happy to hear suggestions!

Recording the time required to create Up, that might help gauge how much time it would save someone as well haha, most teams should not be wasting time developing tooling, engineers are pricey!

Chrome extension for GH logs / metrics / CI

Big maybe but it might be kind of cool to inject "Logs" and "Metrics" tabs into Github for your apps. Tying your GH org together with Up could be a nice way to leverage existing org data.

Downside of course is that you need to install the extension, but it's one less tab to leave open all day.

Dynamic SVG dashboard for repo readmes, displaying status, metrics, etc

Building https://github.com/tj/gh-polls today got me thinking, maybe it would be cool (kind of gimmicky) to have a single image you place in a project Readme, and it shows you an overview of metrics such as API latency, uptime, errors, CI status and others.

Basically an alternative to Geckoboard without paying for that, and your team automatically gets access to it since it's just in a readme or Wiki!

Sooner

  • optional animations
  • params for title, metric, stat, etc
  • timeseries
  • timeseries error overlay similar to Heroku
  • timeseries deployment annotations
  • timeseries with comparison to previous period
  • histograms
  • metrics (simple request / error counts etc)
  • alarm status
  • uptime status
  • billing related stuff?
  • heatmaps for latency etc
  • stacked bars might be nicer for requests/4xx/5xx errors
  • maybe some Geckoboard style widgets for displaying MRR etc with percentage changes

Later

  • abstract out the datasource so it doesn't have to be CloudWatch
    • Is there an open-source data fetching layer?
  • themes
  • date range switching

Upgrade command to update up(1)

Just like apex(1). It would be cool to check remotely maybe once per day too, and show the changelog

  • move apex's logic to a distinct package
  • add to up and test

Faster gitignore style pattern matching

Current pkg uses regexps, not a big deal for most projects but node_modules typically has an easy 10k files. On my machine the current regexp based stuff is ~22k ops/s, not super quick, likely easy to notice on large projects.

Licensing

Figure out what makes sense, complete license noob.

  • OSS license
  • Pro license
  • Make sure deps are permissive
  • CLA? Don't bother?

Resource management (opinions please!)

Curious what peoples thoughts are regarding managing things like DNS records, alarms, etc.

One option is individual CLI commands, this would be a little easier to implement, but awkward since the team can't review/audit them, commit them to source control, and requires listing a bunch of resources to see the state of the world.

$ up dns ls
$ up dns add apex.sh blog.apex.sh A 52.0.16.118,52.1.119.170,....
$ up dns add apex.sh ping.apex.sh CNAME ping.netlify.com

The more "correct" approach these days would be more like Terraform infra-as-code style, where the config in your repo is the source of truth, though it may be less familiar/intuitive to some people.

For DNS looking something like the following (for apex.sh zone):

{
  "domain": "apex.sh",
  "dns": {
    "blog.apex.sh": {
      "type": "A",
      "value": [
        "52.0.16.118",
        "52.1.119.170"
      ]
    }
  },
  "ping.apex.sh": {
    "type": "CNAME",
    "value": "ping.netlify.com"
  }
}

Changes would be previewed via up plan which shows what will be updated/created/deleted etc and applied with up apply or similar. Note this is not a replacement for Terraform or CloudFormation, just need to provide enough for people to get apps bootstrapped.

Pros of CRUD style:

  • Familiarity due to matching Heroku and friends
  • ?

Pros of Terraform style:

  • Single source of truth / overview of app just from reading app.json
  • Preview changes before applying them
  • Easily reviewable via GitHub
  • ?

Any opinions?




What needs implementing for now:

  • IAM role(s)
  • DNS (explicit)
  • DNS (implicit)
  • API Gateway
  • ACM
  • parameterize tmpl
  • outputs
  • create stack
  • update stack
  • delete stack
  • plan
  • apply
  • event streaming
  • CLI integration
  • styling / ux

Scheduled requests

Big maybe, this starts to enter into apex(1) / Serverless territory, not sure I really want to go there, but it would be nice to not use verbose Terraform stuff just to set up scheduled tasks.

Add more features to the list

Some more that are either somewhat implied or not big items but should still be listed, maybe in a smaller below the main set. Maybe move this all to apex.sh/up

  • revise the list in general
  • Zero-downtime deployments
  • Server restarts upon crash
  • Pay only for what you use
  • Ignore files
  • Declarative TF style config planning / diffing following best practices
  • Private endpoints
  • Immutability of deploys / environment variables
  • Error pages (404.html, 4xx.html, error.html)
  • Implicit http -> https redirect
  • local dev server
  • list of middleware (CORS etc)
  • google analytics / segment injection etc
  • _headers support matching Netlify
  • _redirects support matching Netlify
  • path-based script/css injection rules
  • over a dozen regions
  • AWS itself is a feature (vs some sketchy startup)

Load testing command

Basically latency.apex.sh as a command, but a distributed version of hey(1) so you can still use all the flags and test from N regions. This should be a cool way to highlight the benefits of #27.

Might make sense to just package this up as its own up application, instead of a command. Also then serves as an example of using up.

Desktop app

App and web app will utilize the same code base. The desktop app will be targeted more towards people like myself, while the web app will be optionally self-hosted in your own network for private access by team members.

The app will features pretty much everything the CLI will, ranging from stats, alarms, continuous integration, and so on.

What would you prefer?



Replicated Dynamo example

With at least a few regions to span along the equator. I haven't tried Fauna, but if it's easy to set up that might be cool.

Optimize reverse proxy

Do some profiling and get the latency as low as possible, possible replace Go's reverse proxy with https://github.com/valyala/fasthttp, test unix domain sockets in Lambda as well, check heap and see what can be pooled. It's also making some assumptions about having concurrency of 1 at the moment, since that's the case with Lambda.

Make sure it's not decompressing gzip and recompressing either, not sure what the proxy does but I know net/http's client does this transparently, test with some larger responses.

  • profile cpu
  • profile mem
  • test larger responses
  • implement any quick wins
  • either remove Accept-Encoding or ignore when already compressed

Continuous Integration examples for Circle, Travis, etc

So it's easy for people to get rolling with the existing CI services out there. What do you use?






If Up provided a "serverless" (via AWS CodeBuild) self-hosted CI out of the box, and it was cost effective, complete with CLI and dashboard, would you use it?



Timelines?

So by when can we start using this? Seems really exciting. I don't like aws's console and UI its just too slow and I prefer automated stuff. Maybe this can replace my current server deployment and devops hell.

Rust support

Looks like cross compiling is not super easy right now, might have to do docker builds. Any suggestions from Rust users?

Install via npm

Not a big deal since it'll be a single copy/paste command anyway, but the install script could be published to npm to make that a little smoother for Node people.

https://github.com/Automattic/cloudup-cli is using the "up" package currently, but it's basically a dead project.

Ruby support

Out of the box. This one seems like it's pretty tricky to get running in Lambda.

Comparisons

Some kind of table showing some of the pros/cons of Up vs other solutions so people know what to expect, for ex Lambda doesn't let you deploy using custom Dockerfiles, not really a problem in most real-world apps that I've written though.

Ignore file support

There will of course be .upignore support, but I'd love some feedback on default behaviour:






  • add filter support
  • add .gitignore if present
  • add .npmignore if present

Upload progress indicator

Might be nice to add, could possible do s3 multi-part as well to speed things up for larger cases. Depends on your connection of course.

Serverless Metrics solution

I have a ~60% complete "serverless" metrics solution, very low cost to get started, and scales indefinitely built on AWS tech. May or may not make this a hosted product or an optional part of Up (or neither).

Easily an order of magnitude more cost-effective than any other solution I've seen, even Prometheus etc running on EBS, and easier to operate since you don't have to manage any nodes, set up HA, replication etc.

So far my thing uses the same protocol as Influx, latency is still good however price, volume, simplicity of operation and query flexibility are given priority. The query engine has more in common with Elasticsearch, making it easy to return nested results etc, SQL is more appealing for simple stuff but pretty awkward when it comes to nesting.

Another possibility is a CloudWatch compatibility layer so it's a vastly more cost effective drop-in replacement.

Curious what you use already:











What's most important for your use-cases?




Graphql API

API idea from @gjohnson to provide pro users for easier centralized management in an org for less IAM mayhem. Tempted to add some IAM management stuff to Up as well to make that a bit easier as an admin.

Pricing table

For now list out the AWS cost so people don't have to go hunt down the cost of logging, API Gateway, Lambda, and so on. Later it would be great to have part of the CLI or app estimate this stuff for you.

Twitter

kinda made the mistake of having accounts per product, but oh well, basically will act as a changelog, but then again my personal twitter is basically this as well, might not be worth it

Gitignore parity support

Currently ** is not supported. The pkg I'm using isn't very well documented, probably best to just fork or write a new one.

Default error pages

Maybe, it would be nice to at least check Accept and output some nice HTML instead of boring text.

  • default template
  • tests for it
  • tests for Accept
  • add color option
  • add support link option

Ruby support

Out of the box. This one seems a little involved from what I've seen, definitely need Docker, and to figure out how to get the gems working etc.

CLI polish

Make sure up stack and friends all have nice output and user-friendly errors, remapping of names (no one wants to see AWS::Lambda::Permission blah blah), etc.

FAQ

How it compares to other solutions, pros and cons, pricing of AWS stuff, APIGW/Lambda limitations etc.

Improve zip performance

It's pretty decent now, 15k files / 60MB with filtering in ~350ms but needs profiling to see if we can improve that some more.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.