Background

Use case is that we want to get Next.js configured for the server-side as follows:

• Ideally mimic something like assignDefaults in https://unpkg.com/browse/[email protected]/dist/server/config.js wherein we pass a config object from reading next.config.js and get it all fixed up per that function.
• Unfortuately, we can't do this because (1) assignDefaults is not exported, and (2) we don't want to require anything that has Next's webpack stuff involved (for loadConfigs webpack hooks).

For our current code, we have a pure object (with object values but no arrays), which we can get away with a naive Object.assign() on, but this lacks:

1. Handling arrays, functions, etc.
2. Validation, checking, etc.

Like the real assignDefaults and loadConfig do in Next.js proper.

Task

• Find some way to get all the Next.js configuration goodness and parity.

Ideas

• Find a third party library
• Build and monkey patch dist/server/config.js to a new file that exports assignDefaults
• At build time inflate the server object using assignDefaults and write it out to disk that gets included in the zip bundle.

• Upgrade serverles-jetpack to sls 3.
• Update deps in this demo.

• Have a target: "serverless" build.
• Bridge routing with next-aws-lambda or something similar to target one base Lambda
• Assess package size
  • Check if there are duplicates across page bundles and see if we can tweak Next.js webpack config to split common chunks out.
• Verify correctness.

• Follow the custom server Next.js guide with serverless-http
• Review packaging with Jetpack to make sure we don't bring in all the dev stuff we don't need.
• Verify it actually works.

Architecture

Notes

Use the lambda-at-edge library to break up next into two lambdas:

• default-lambda that contains all the pages
• api-lambda that contains just the api routes

Pre-rendered and other static assets are extracted from the .next build and are uploaded to S3.

Cloudfront cache behaviours are then configured to send the requests along to either these lambdas or to fetch static assets that were uploaded to S3 (no lambda hit necessary).

Pros

• AWS deployment is nearly identical to local server (with the proviso that cloudfront forwards on expected cookies or other original request headers e.g. referrer)
• Fast to deploy
• Global deployment on lambda@edge that promises lowest latency to end users
• Cloudfront cache plays well with webpack "chunks" generated by next build

Cons

• The deployment as configured in terraform is complex (900+ lines of terraform)
• Requires deeper knowledge of next internals (page routing, dynamic page routing, static generation, api routes, webpack chunks etc.). These details could easily change when upgrading next
• lambda@edge puts cloudwatch logs in the AZ closest to the user that made the request. So additional work might be needed to aggregate logs
• Cold starts seem very slow (5-10 seconds), but I've not looked into this much yet

The TF IAM permissions module isn't needed for this demo. Let's remove all the terraform.

• Remove policies in formidops
• Destroy the TF stack
• Destroy the AWS bootstrap
• Remove the TF and AWS code
• Remove script commands in package.json
• Update README

• Remove existing posts
• New posts
  • Ways to deploy Next.js to Lambda
  • Target server vs serverless

• Move application source to src to clarify files. Leave server as-is
• Add a new logo/profile
• Search on other TODOs in app and either fix or ticket.
• Memoize retrieving the post data and post content? (Doesn't seem to be a big deal)
• We're not doing any routing in express, with the Next.js router doing it all. Try out removing express and using http built-in module instead. (Fix or ticket) (Not doing because we use express.static() in root handler.

• Programmatic Solution?: Find a programmatic way if possible to see what's not used in server output. Maybe even compare against serverless output or Next.js internals in how it decides deps for serverless mode?
• Tune: Augment and tune all the jetpack / allowMissing / dynamic.resolutions config.
• Verify no log misses.
• Add to trace-deps https://github.com/FormidableLabs/trace-deps/blob/main/docs/configuration.md

Presumably lots of various changes to update to most current Next.js 12.

Research & Issues

Next has one dist file import a non-existent file in the package

https://unpkg.com/browse/[email protected]/dist/compiled/webpack/bundle5.js at line 141344 has in relevant part:

"use strict";
module.exports = require("next/dist/build/webpack/plugins/terser-webpack-plugin");
 
/***/ }),

However, https://unpkg.com/browse/[email protected]/dist/build/webpack/plugins/terser-webpack-plugin is just a directory with a sub src directory in it.

We're seeing 5+ second initial load time for code.

• Figure out where it is
• Consider adding timings as a part of the app
• Fix

It looks like a monkey-patch for node internals for webpack to decide between 4 and 5 around this code: https://www.runpkg.com/[email protected]/dist/next-server/server/config.js#397 (after prettifying)

await (0, _configUtils.loadWebpackHook)(phase, dir);

produces this error because of missing files, where awkwardly it looks like we have to include webpack even though we don't want to use it at runtime:

"Error: Cannot find module 'next/dist/compiled/webpack/webpack-lib'",
        "Require stack:",
        "- /var/task/node_modules/next/dist/build/webpack/require-hook.js",
        "- /var/task/node_modules/next/dist/next-server/server/config-utils.js",
        "- /var/task/node_modules/next/dist/next-server/server/config.js",
        "- /var/task/node_modules/next/dist/next-server/server/next-server.js",
        "- /var/task/node_modules/next/dist/server/next.js",
        "- /var/task/server/index.js",
        "- /var/runtime/UserFunction.js",
        "- /var/runtime/index.js",
        "    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:815:15)",
        "    at Function.resolve (internal/modules/cjs/helpers.js:80:19)",
        "    at /var/task/node_modules/next/dist/build/webpack/require-hook.js:4:1520",
        "    at Array.map (<anonymous>)",
        "    at Object.<anonymous> (/var/task/node_modules/next/dist/build/webpack/require-hook.js:4:1474)",
        "    at Module._compile (internal/modules/cjs/loader.js:999:30)",
        "    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)",
        "    at Module.load (internal/modules/cjs/loader.js:863:32)",
        "    at Function.Module._load (internal/modules/cjs/loader.js:708:14)",
        "    at Module.require (internal/modules/cjs/loader.js:887:19)"

We could just trace and package those files, but it would be nice to not bloat the package.

We've bandaided this with a tracing configuration like this: afc7d2e#diff-8cf16232f3b7094ea431234555d38ad71e6e62764c3e29a5841e25172aea5049R127-R130

Task:

• Figure out if we can avoid this...