Drainpipe is a composer package which provides build tool and testing helpers for a Drupal site, including:
- Site and database updates
- Artifact packaging for deployment to a hosting provider
- Automated testing setup
- Integration with DDEV
- CI integration
- Installation
- Database Updates
- .env support
- SASS Compilation
- JavaScript Compilation
- Testing
- Hosting Provider Integration
- GitHub Actions Integration
- GitLab CI Integration
- Tugboat Integration
composer config extra.drupal-scaffold.gitignore true
composer config --json extra.drupal-scaffold.allowed-packages "[\"lullabot/drainpipe\", \"lullabot/drainpipe-dev\"]"
composer require lullabot/drainpipe
composer require lullabot/drainpipe-dev --dev
and if using DDEV, restart to enable the added features:
ddev restart
This will scaffold out various files, most importantly a Taskfile.yml
in the
root of your repository. Task is a task runner / build tool that aims to be
simpler and easier to use than, for example, GNU Make. Since it's written in Go,
Task is just a single binary and has no other dependencies. It's also
cross-platform with everything running through the same shell interpreter.
You can see what tasks are available after installation by running
./vendor/bin/task --list
or ddev task --list
if you're running DDEV. To get
more information on a specific task e.g. what parameters it takes, you can run
task [task name] --summary
.
Your Taskfile.yml
can be validated with JSON Schema:
curl -O https://taskfile.dev/schema.json
npx ajv-cli validate -s schema.json -d scaffold/Taskfile.yml
See .github/workflows/validate-taskfile.yml for an example of this in use.
๐ก If your docroot is not the standard `web/` path, you must create a symlink to it
ln -s web/ docroot
The drupal:update
command follows the same procedure as the
'drush deploy' command, with the
exception that it runs the configuration import twice as in some cases the
import can fail due to memory exhaustion before completion.
drush updatedb --no-cache-clear
drush cache:rebuild
drush config:import || true
drush config:import
drush cache:rebuild
drush deploy:hook
Drainpipe will add .env
file support for managing environment variables.
This is only used for locals - other environments such as CI and production should use their native environment variable mechanisms.
This consists of:
- Creation of a
.env
and.env.defaults
file - Default
Taskfile.yml
contains dotenv support note: real environment variables will override these - Drupal integration via
vlucas/phpdotenv
To enable this, add the following to yourcomposer.json
:You will need to restart DDEV if you make any changes to"autoload-dev": { "files": [ "vendor/lullabot/drainpipe/scaffold/env/dotenv.php" ] },
.env
or.env.defaults
This compiles CSS assets using Sass. It also supports the following:
- Globbing
// Base @use "sass/base/**/*";
- Modern Normalizer
- Autoprefixer configured through a
.browserslistrc
file in the project root
- Add @lullabot/drainpipe-sass to your project
yarn add @lullabot/drainpipe-sass
ornpm install @lullabot/drainpipe-sass
- Edit
Taskfile.yml
and addDRAINPIPE_SASS
in thevars
sectionvars: DRAINPIPE_SASS: | web/themes/custom/mytheme/style.scss:web/themes/custom/mytheme/style.css web/themes/custom/myothertheme/style.scss:web/themes/custom/myothertheme/style.css
- Run
task sass:compile
to check it works as expected - Run
task sass:watch
to check file watching works as expected - Add the task to a task that compiles all your assets e.g.
assets: desc: Builds assets such as CSS & JS cmds: - yarn install --immutable --immutable-cache --check-cache - task: sass:compile - task: javascript:compile assets:watch: desc: Builds assets such as CSS & JS, and watches them for changes deps: [sass:watch, javascript:watch]
JavaScript bundling support is via esbuild.
- Add @lullabot/drainpipe-javascript to your project
yarn add @lullabot/drainpipe-javascript
ornpm install @lullabot/drainpipe-javascript
- Edit
Taskfile.yml
and addDRAINPIPE_JAVASCRIPT
in thevars section
Source and target need to have the same basedir (web or docroot) due to being unable to provide separate entryNames. See evanw/esbuild#224DRAINPIPE_JAVASCRIPT: | web/themes/custom/mytheme/script.js:web/themes/custom/mytheme/script.min.js web/themes/custom/myotherthemee/script.js:web/themes/custom/myothertheme/script.min.js
- Run
task javascript:compile
to check it works as expected - Run
task javascript:watch
to check file watching works as expected - Add the task to a task that compiles all your assets e.g.
assets: desc: Builds assets such as CSS & JS cmds: - yarn install --immutable --immutable-cache --check-cache - task: sass:compile - task: javascript:compile assets:watch: desc: Builds assets such as CSS & JS, and watches them for changes deps: [sass:watch, javascript:watch]
This is provided by the separate drainpipe-dev package (so the development/testing dependencies aren't installed in production builds).
All the below static code analysis tests can be run with task test:static
Test Type | Task Command | Description |
---|---|---|
Security | task test:security | Runs security checks for composer packages against the FriendsOfPHP Security Advisory Database and Drupal core and contributed modules against Drupal's Security Advisories. |
Lint | task test:lint | - YAML lint on .yml files in the web directory- Twig lint on files in web/modules , web/profiles , and web/themes - composer validate These cannot currently be customised. See #9. |
PHPStan | task test:phpstan | Runs PHPStan with mglaman/phpstan-drupal onweb/modules/custom , web/themes/custom , and web/sites . |
PHPUnit | task test:phpunit:static | Runs Unit tests in web/modules/custom/**/tests/src/Unit and test/phpunit/**/Unit |
PHPCS | task test:phpcs | Runs PHPCS with Drupal coding standards provided by [Coder module](https://www.drupal.org/project/coder |
phpcs.xml
can be altered using Drupal's
composer scaffold.
- Edit
phpcs.xml
in the root of your project, e.g. to add an exclude pattern:<!-- Custom excludes --> <exclude-pattern>web/sites/sites.php</exclude-pattern>
- Create a patch file
diff -urN vendor/lullabot/drainpipe-dev/scaffold/phpcs.xml phpcs.xml > patches/custom/phpcs.xml.patch
- Add the patch to
composer.json
The pre hook is needed otherwise the composer scaffold attempts to re-patch a file it no longer has control over when running"scripts": { "pre-drupal-scaffold-cmd": [ "if [ -f \"phpcs.xml\" ]; then rm phpcs.xml; fi" ], "post-drupal-scaffold-cmd": [ "if [ -f \"phpcs.xml\" ]; then patch phpcs.xml < patches/custom/phpcs.xml.patch; fi" ] },
composer install --no-dev
- Delete the
vendor
directory andphpcs.xml
and then runcomposer install
to verify everything works as expected
Functional tests require some mechanism of creating a functing Drupal site to
test against. All the below tests can be run with task test:functional
task test:phpunit:functional
Runs PHPUnit tests in:
web/modules/custom/**/tests/src/Kernel
test/phpunit/**/Kernel
web/modules/custom/**/tests/src/Functional
test/phpunit/**/Functional
web/modules/custom/**/tests/src/FunctionalJavaScript
test/phpunit/**/FunctionalJavaScript
You will need to make sure you have a working Drupal site before you're able to run these.
Support for Drupal Test Traits
is included, set this in your Taskfile.yml
vars:
vars:
DRUPAL_TEST_TRAITS: true
This will additionally look for tests in:
web/modules/custom/**/tests/src/ExistingSite
test/phpunit/**/ExistingSite
web/modules/custom/**/tests/src/ExistingSiteJavascript
test/phpunit/**/ExistingSiteJavascript
beware: DTT tests will run against the main working Drupal site rather than installing a new instance in isolation
task test:nightwatch
Runs functional browser tests with Nightwatch.
Run test:nightwatch:setup
to help you setup your project to run Nightwatch
tests by installing the necessary node packages and DDEV configurations.
If you are using DDEV, Drainpipe will have created a
.ddev/docker-compose.selenium.yaml
file that provides standalone Firefox and
Chrome as containers, as well as an example test in test/nightwatch/example.nightwatch.js
.
To run the above test you will need to have a working Drupal installation in the
Firefox and Chrome containers. You can run the test:nightwatch:siteinstall
helper task to run the Drupal site installer for both sites with your existing
configuration.
After you've verified this test works, you can ignore it in your composer.json
:
"extra": {
"drupal-scaffold": {
"file-mapping": {
"[project-root]/test/nightwatch/example.nightwatch.js": {
"mode": "skip"
}
}
}
Nightwatch tests must have the suffix .nightwatch.js
to be recognised by
the test runner.
Whilst tests are running, you can view them in realtime through your browser.
https://:7900 for Chrome https://:7901 for Firefox
The password for all environments is secret
.
task test:autofix
attempts to autofix any issues discovered by tests.
Currently, this is just fixing PHPCS errors with PHPCBF.
Generic helpers for deployments can be found in tasks/snapshot.yml
,
tasks/deploy.yml
, and tasks/drupal.yml
task deploy:git |
Pushes a directory to a git remote |
task drupal:composer:development |
Install composer dependencies |
task drupal:composer:production |
Install composer dependencies without devDependencies |
task drupal:export-db |
Exports a database fetched with a *:fetch-db command |
task drupal:import-db |
Imports a database fetched with a *:fetch-db command |
task drupal:install |
Runs the site installer |
task drupal:maintenance:off |
Turn off Maintenance Mode |
task drupal:maintenance:on |
Turn on Maintenance Mode |
task drupal:update |
Run Drupal update tasks after deploying new code |
task snapshot:archive |
Creates a snapshot of the current working directory and exports as an archive |
task snapshot:directory |
Creates a snapshot of the current working directory |
Databases are by default fetched to /var/www/html/files/db/db.sql.gz
, this can
be overridden with a variable in Task:
`task drupal:import-db DB_DIR="/var/www/htdocs"`
When creating a snapshot of the current working directly files can be excluded
using a .drainpipeignore
file in the root of the repository that uses the same
format as .gitignore
, e.g.
# Files that won't be deployed to Pantheon
/.ddev
/.github
/.yarn
/files/db
/tests
/.env
/.env.defaults
/README.md
/Taskfile.yml
*.sql
*.sql.gz
This folder can then be deployed to a remote service either as an archive, or
pushed to a git remote with task deploy:git
.
Pantheon specific tasks are contained in tasks/pantheon.yml
.
Add the following to your Taskfile.yml
's includes
section to use them:
includes:
pantheon: ./vendor/lullabot/drainpipe/tasks/pantheon.yml
task pantheon:fetch-db |
Fetches a database from Pantheon. Set PANTHEON_SITE_ID in Taskfile vars |
See below for CI specific integrations for hosting providers.
Add the following to composer.json
for generic GitHub Actions that will be
copied to .github/actions/drainpipe
in your project:
"extra": {
"drainpipe": {
"github": []
}
}
They are composite actions which can be used in any of your workflows e.g.
- uses: ./.github/actions/drainpipe/set-env
- name: Install and Start DDEV
uses: ./.github/actions/drainpipe/ddev
with:
git-name: Drainpipe Bot
git-email: [email protected]
ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
ssh-known-hosts: ${{ secrets.SSH_KNOWN_HOSTS }}
Update Pull Request descriptions with a markdown table of any changes detected
in composer.lock
using composer-lock-diff.
"extra": {
"drainpipe": {
"github": ["ComposerLockDiff"]
}
}
To enable deployment of Pantheon Review Apps:
- Add the following composer.json
"extra": { "drainpipe": { "github": ["PantheonReviewApps"] } }
- Run
composer install
to install the workflow to.github/workflows
- Add the following secrets to your repository:
PANTHEON_TERMINUS_TOKEN
See https://pantheon.io/docs/terminus/install#machine-tokenPANTHEON_SITE_NAME
The canonical site nameSSH_PRIVATE_KEY
A private key of a user which can push to PantheonSSH_KNOWN_HOSTS
The result of runningssh-keyscan -H codeserver.dev.$PANTHEON_SITE_ID.drush.in
TERMINUS_PLUGINS
(optional) Comma-separated list of Terminus plugins to be availablePANTHEON_REVIEW_USERNAME
(optional) A username for HTTP basic auth localPANTHEON_REVIEW_PASSWORD
(optional) The password to lock the site with
Add the following to composer.json
for GitLab helpers:
"extra": {
"drainpipe": {
"gitlab": []
}
}
This will import scaffold/gitlab/Common.gitlab-ci.yml
,
which provides helpers that can be used in GitLab CI with includes and
references.
Updates Merge Request descriptions with a markdown table of any changes detected
in composer.lock
using composer-lock-diff.
Requires GITLAB_ACCESS_TOKEN
variable to be set, which is an access token with
api
scope.
"extra": {
"drainpipe": {
"gitlab": ["ComposerLockDiff"]
}
}
"extra": {
"drainpipe": {
"gitlab": ["Pantheon", "Pantheon Review Apps"]
}
}
- Add the following the composer.json to enable deployment of Pantheon Review Apps
"extra": { "drainpipe": { "github": ["PantheonReviewApps"] } }
- Run
composer install
- Add your Pantheon
site-name
to the last job in the new workflow file at.github/workflows/PantheonReviewApps.yml
- Add the following secrets to your repository:
PANTHEON_TERMINUS_TOKEN
See https://pantheon.io/docs/terminus/install#machine-tokenSSH_PRIVATE_KEY
A private key of a user which can push to PantheonSSH_KNOWN_HOSTS
The result of runningssh-keyscan -H codeserver.dev.$PANTHEON_SITE_ID.drush.in
TERMINUS_PLUGINS
Comma-separated list of Terminus plugins to be available (optional)
This will setup Merge Request deployment to Pantheon Multidev environments. See [scaffold/gitlab/gitlab-ci.example.yml] for an example. You can also just include which will give you helpers that you can include and reference for tasks such as setting up Terminus. See scaffold/gitlab/Pantheon.gitlab-ci.yml.
Add the following to composer.json
to add Tugboat configuration:
{
"extra": {
"drainpipe": {
"tugboat": {}
}
}
}
The following will be autodetected based on your .ddev/config.yml
:
- Web server (nginx or apache)
- PHP version
- Database type and version
- nodejs version
- Redis (Obtained with
ddev get ddev/ddev-redis
)
Additionally, Pantheon Terminus can be added:
{
"extra": {
"drainpipe": {
"tugboat": {
"terminus": true
}
}
}
}
It is assumed the following tasks exist:
sync
build
update
The build
, sync
, and update
tasks can be overridden with sync:tugboat
,
build:tugboat
, and update:tugboat
tasks if required (you will need to re-run
composer install
to regenerate the Tugboat scripts if you are adding this
task to your Taskfile.yml
for the first time).
sync:
desc: "Fetches a database from Pantheon and imports it"
cmds:
- task: pantheon:fetch-db
- task: drupal:import-db
sync:tugboat:
desc: "Fetches a database from Pantheon and imports it in Tugboat"
cmds:
- task: pantheon:fetch-db
vars:
DB_DIR: /var/lib/tugboat/files/db
- task: drupal:import-db
vars:
DB_DIR: /var/lib/tugboat/files/db
๐ก
composer install
should be re-run if any changes are made to the DDEV
configuration.
You can hook into the init
step of images by adding them to your
Taskfile.yml
, e.g.
tugboat:php:init:
cmds:
- apt-get install -y libldap2-dev
- docker-php-ext-install ldap