Giter VIP home page Giter VIP logo

heroku-buildpack-cl2's Introduction

Heroku Buildpack for Common Lisp

My rework of the Common Lisp buildpack for Heroku by Mike Travers.

Differences from the Mike's buildpack:

  • Doesn't install patched portableaserve and wuwei. You are free to use any webserver. If you need a version not available in quicklisp, you can fetch it as a git submodule of your application.
  • Doesn't force your application to be saved into a lisp image at Heroku. We recommend the build output to consist of full application and libraries sources together with prebuild .fasl files to speedup loading. You can build a lisp image, but sometimes this will lead to problems. For example, many libraries keep static files (e.g css, javascript) in their repositories and access the files using asdf:system-relative-pathname. If lisp image is saved and copied to another location (as it happens with build results at Heroku), those libraries will not find their static files.

Usage

To feel comfortable, read about Heroku Procfile and Buildpack. You will then understand, that Heroku allows you to use any unix command to start your application. And buildpack provides a separate compile step, where you can prepare things for that command: fetch the lisp implementation binary, download libraries with quicklisp and build .fasl files. Again, using full power of unix.

The compile script of this buildpack installs SBCL available as sbcl/sbcl-1.0.54-x86-64-linux/run-sbcl.sh, and then invokes a heroku-compile.lisp script in your application root directory.

In your heroku-compile.lisp you typically want to prebuild .fasl files of your application by first (asdf:disable-output-translations) and then loading your ASDF system.

If you need Quicklisp, call cl-user::require-quicklisp, it installs quicklisp in the directory quicklisp/.

Your application is started according to what you specfy in your Procfile. Here you can invoke the SBCL and run the lisp code you need. In the Procfile command you also want to disable ASDF output translations.

Disabling ASDF output tranlastions is necessary becuse Heroku performs the compile step on one machine/directory, and then copies the result into another machine/directory. With default output translations ASDF caches .fasl files according to the full path of their source files. When the sources are moved to another location, ASDF can not match them to the cached .fasls. In result full recompilation will hapen at start time of your application.

With ASDF output translations disabled the .fasl files are placed near the sources, and when copied together, ASDF still matches them.

See the example application.

Notes

  • Heroku does not have a persistent file system. Applications should use S3 for storage; ZS3 is a useful CL library for doing that.
  • You can login into the server where your application is deployed and inspect it with heroku run bash command. See [one off dynos] (https://devcenter.heroku.com/articles/one-off-dynos).

Todos

  • require-quicklisp function should accept desirable quicklisp dist version and quicklisp client version, because when deploying application to server we want predictable environement.

Credits

heroku-buildpack-cl2's People

Contributors

avodonosov avatar ddollar avatar mtravers avatar oguzbilgic avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

heroku-buildpack-cl2's Issues

Suggestion of improvement, and incremental compilation

Hi Anton,

It's fun to notice you have enhanced the heroku buildpack for CL.
I have been seduced by your idea of not producing a self contained executable, so I backported some of the code in this buildpack to the one I use for my application (based on CCL).

One thing I wanted to suggest as an improvement is to loosen the coupling with the "disable output translations" setting in ASDF. Following the ASDF manual I added this to the compile script, before the last phase (actual app compilation):

mkdir -p $BUILD_DIR/.config/common-lisp/asdf-output-translations.conf.d/
echo ":disable-cache" > $BUILD_DIR/.config/common-lisp/asdf-output-translations.conf.d/99-disable-cache.conf
export ASDF_OUTPUT_TRANSLATIONS=/:

With this, you don't need to specify in the app code that ASDF should disable output translations, therefore the process command line is more simple (no need to prepend ASDF_OUTPUT_TRANSLATIONS=/:) and heroku-compile.lisp does not need (asdf:disable-output-translations).

The benefit of that is that if you later find another location for storing the fasl files (for example in cache directory), heroku-compile.lisp and the Procfile do not need to be changed.

One thing I wished I could improve is to avoid recompiling the entire application every time a push is made, which would involve using the cache directory for storage between each deployments. But I am not sure right now how the produced fasls can be copied back to the build directory, so they are available to the deployed app. It seems ASDF stores the entire root path in its cache directory structure, and I don't know how to avoid that.

If you have any idea to share, I'll be happy to hear about it.

Camille

Free Procfile from ASDF output translations knowledge

As suggested in the issue #2, we can isolate the application writer from specifying ASDF_OUTPUT_TRANSLATIONS=/: by placing ASDF configuration into a config file.

We only need to ensure that ASDF will be able to locate the config after deployment.
Does Heroku guarantee that the directory referenced by $BUILD_DIR at compile step will be pointed by $HOME at run step?

quicklisp/asdf.lisp is recompiled at startup

We disable ASDF output translations, so libraries are not recompiled.

But quicklisp/asdf.lisp file is compiled by quicklisp/setup.lisp, and ASDF output translations are not used.

Quicklisp places asdf.fasl into a subdirectory of quicklisp/cache. The subdirectory name is calculated as a hash of these values:

   (lisp-implementation-type)
   (lisp-implementation-version)
   (machine-type)
   (machine-version)

I think (machine-version) is different between the Heroku build machine and the machine where application runs.

When quicklisp/setup.lisp tries to find asdf.fasl on new machine and doesn't find it, it recompilates asdf.lisp. It takes seconds. In total startup upon the first request may take 6-10 seconds.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.