Lambda Comments
This project which implements a minimal blog commenting service.
Blog posts:
- Introducing lambda-comments (you can try leaving a comment there)
- A day on the Hacker News home page: lambda-comments
Hacker News thread: https://news.ycombinator.com/item?id=11644042
It is completely "serverless", designed to use the following Amazon services:
The Lambda functions are written in ES6, with async/await, transpiled using Babel, and bundled using Webpack.
The AWS resources are provisioned using the CloudFormation service.
Additionally, we use Apex to simplify the uploading of the Lambda functions (without the shim).
Costs
It should cost very little to run. The following resources are provisioned:
- DynamoDB - only provisioned for 1 read capacity unit, 1 write capacity unit (which limits it to 1 job per second). This is the most expensive resource, approximately $0.65 a month.
- S3 - storage for comments and private data, plus requests and data transfer
- CloudWatch Logs
- Lambda functions - only pay for invocations, first million requests per month are free (hopefully your blog isn't that popular)
- API gateway - only pay for API calls
Deployment Instructions
Prerequisites
- You will need an AWS Account
- You will need OS X, Linux, *BSD or another Unix-based OS (scripts will need some modifications for Windows)
- Install the AWS CLI and ensure credentials are setup under ~/.aws/credentials (Instructions)
- Install Node.js (tested with v4.2.6 and v5.7.0)
git clone https://github.com/jimpick/lambda-comments.git(https)
or
git clone [email protected]:jimpick/lambda-comments.git(git)cd lambda-commentsnpm run install-all(runsnpm installin top directory, and then sets up sub-packages underpackages)- Install Apex
Configuration
Copy .env.SAMPLE to .env and customize it.
cp .env.SAMPLE .env
The default .env.SAMPLE contains:
# The URL of your blog/website that will be hosting the comments
# Used to generate CORS headers and also for Akismet
BLOG=https://example.com/blog/
# A name for your CloudFormation stack
# Also prefixed to the API Gateway REST API name
CLOUDFORMATION=myBlogComments
# The AWS region to provision the resources in
REGION=us-west-2
# The name for the API Gateway stage
STAGE=prod
# The Akismet.com API key (optional, but recommended)
# Akismet is a service for combatting blog spam from Automattic (WordPress)
#AKISMET=0123456789ab
# A Slack webhook to send notifications to (optional)
#SLACK=https://hooks.slack.com/services/XXXXXXXXX/YYYYYYYYY/ZZZZZZZZZZZZZZZZZZZZZZZZ
We use dotenv so it is also possible to configure the project by setting environment variables.
Parameters
BLOG: The full base url of the blog/website
CLOUDFORMATION: The name of the CloudFormation stack
REGION: The AWS region
STAGE: The API Gateway stage to create
AKISMET: (Optional, but recommended) API key from akismet.com for spam filtering
SLACK: (Optional) Slack webhook - configure this if you want a notification in a Slack channel each time a comment is posted
Installation
For now, follow the step-by-step instructions below. In the future, we will develop a streamlined installation procedure.
Use CloudFormation to create the AWS resources
npm run create-cloudformation
The command returns immediately, but it will take a while to complete (typically 3-4 minutes). It's a good idea to watch the CloudFormation task in the AWS Web Console to ensure that it completes without errors.
Note: When working with the CloudFormation recipe, you can also use
npm run update-cloudformation and npm run delete-cloudformation
Save the references to the provisioned CloudFormation resources
npm run save-cloudformation
This will create a file in deploy/state/cloudFormation.json
Temporary fix - update Lambda functions to use Node.js 4.3
CloudFormation has issues setting the runtime to 'nodejs4.3', see:
https://forums.aws.amazon.com/thread.jspa?threadID=229072
Until CloudFormation is updated, here's an extra step to update the Lambda functions to use Node.js 4.3 using a custom script.
npm run flip-lambdas-to-4.3
Generate an API key
npm run gen-api-key
This will create a file in deploy/state/apiKey.json containing an
apiKey variable that will be baked into the client to sign requests.
The purpose of the API key is to try to minimize spam to the API, but as the API key is distributed publicly as part of the javascript, it's only there to stop non-sophisticated spammers who don't care enough to extract the API key and sign their requests.
Setup the Apex build directory
npm run setup-apex
This generates build/apex/project.json
Compile the Lambda scripts using babel
npm run compile-lambda
This will use webpack and babel to compile the source code in src/server/lambdaFunctions into build/apex/functions
The webpack configuration is in deploy/apex/webpack.config.es6.js
Deploy the lambda functions
npm run deploy-lambda
This will run apex deploy in the build/apex directory to upload the
compiled lambda functions.
Alternatively, if you want to execute the compile and deploy steps in one
command, you can run: npm run deploy-backend
Build the frontend javascript
npm run build-frontend
This builds the code in the packages/frontend directory.
Upload the frontend javascript script to S3
npm run upload-script
This will copy lambda-comments.js to the S3 bucket.
Alternatively, if you want to execute the compile and deploy steps in one
command, you can run: npm run deploy-frontend.
If you want to deploy the backend and frontend all in one step, you can
use: npm run deploy
Run the test suite
npm run test
This will run both the local tests, and remote test which test the deployed API and lambda functions.
The local tests can be run as npm run test-local, and the remote tests can
be run as npm run test-remote.
Currently the test suite expects some data to pre-exist in the S3 bucket. Until the tests are properly mocked, they will fail unless the data is created.
View logs
You can tail the CloudWatch logs:
npm run logs
This just executes apex logs -f in build/apex
Embed the front-end JavaScript client
First, get the URL for the script:
npm run get-client-js-url
This will return a URL you will use below. eg.
//s3-us-west-2.amazonaws.com/myblogcomments-websites3-1ttpk69ph7gr7/lambda-comments.js
In the target web page (perhaps a blog generated by a static site generator such as Jekyll or Hugo), add the following HTML to insert the comments from the development server into the page:
<div id="lambda-comments"></div>
<script src="*** url from above ***"></script>
Providing that the webpage is located at the web address matching the 'BLOG' setting in the .env configuration file, the comments form should appear on the page. If not, check the developer tools console in the web browser to see if there are any errors (typically due to CORS).
Front-end development
The code for the "front-end" javascript that displays the comments and the
comment form embedded in a web page lives in the packages/frontend
directory.
To run a development server, change into the packages/frontend directory,
copy .env.SAMPLE to .env, and run the development server.
cd packages/frontend
cp .env.SAMPLE .env
npm start
The development server is based on react-project
with a heavily modified webpack configuration in webpack.config.js.
In the target web page (perhaps a blog generated by a static site generator such as Jekyll or Hugo), add the following HTML to insert the comments from the development server into the page:
<div id="lambda-comments"></div>
<script src="http://localhost:8081/lambda-comments.js"></script>
To Do List
- Limit length of comments and metadata
- Simplified installation
- Check that permalink and blog match
- Override for path location
- Fetch source page to confirm script is installed on first post
- Test on various browsers, polyfills
- CORS override
- Rearrange code: put lambda scripts under packages directory
- Admin: auth
- Admin: moderation
- Admin: submit ham/spam to akismet
- Admin: Turn comments on/off
- Support for editing blog posts for a limited time
- Detect DDoS style attacks and automatically throttle API Gateway to prevent unlimited charges
- Mocks for AWS/API calls
- Integration test
- Selenium tests
- Coverage
- Emoji support
- Handle DynamoDB ProvisionedThroughputExceededException
- Investigate Swagger
- Generate API docs
- Webpack 2 tree-shaking support
- Optimize download size
- Plugins for server-side rendering on common static site generators
- Optimized bundle for ES6-capable platforms
- Library for bundling with existing client-side javascript builds
- Investigate deep integration with React.js static site generators
- Gatsby
- Phenomic
- Instructions for static-ish hosting platforms
- GitHub Pages
- Surge
- Netlify
- Aerobatic
- Firebase
- Zeit
Interesting links
Lots of related projects, many of which I haven't investigated yet.
Lambda / Serverless Frameworks
- http://apex.run/ (Go, Terraform - we use Apex, but just for convenience to upload the functions)
- https://github.com/serverless/serverless (CloudFormation)
- https://github.com/motdotla/node-lambda
- Azure Cloud Functions vs. AWS Lambda
Lambda Libraries
Awesome Lists
- https://github.com/donnemartin/awesome-aws#lambda
- https://github.com/donnemartin/awesome-aws#api-gateway
Serverless Comment Systems
- http://kevinold.com/2016/02/01/serverless-graphql.html
- https://github.com/serverless/serverless-graphql-blog
- https://github.com/ummels/jekyll-aws-comments
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent