Performance tests divided into suites.

• Any Pillow version
• ImageMagick dev package for Wand
• OpenCV for Python, for example through opencv-python binary package

$ cd testsuite
$ pip install -r ./requirements.txt
$ ./run.py test_suite [test_suite ...]

test_suite is one of available test suites:

load, convert, composition, rotate_right, scale,
blur, wand_load, wand_convert, wand_composition,
wand_rotate_right, wand_scale, wand_blur, cv2_load,
cv2_rotate_right, cv2_scale, cv2_blur

The list can grow, please refer to:

$ ./run.py --help

$ ./run.py scale --progress
$ ./run.py scale --mode RGBA
$ ./run.py scale --runs 50
$ ./run.py scale --size 512x512

For some suites the chosen mode (scale, blur) does matter, while others (convert, composition) work regardless of current mode.

Automatic test launches test suites against relevant Pillow versions including SSE4 and AVX2. Currently it only works on Linux Ubuntu.

You have two ways to run the test: Vagrant or host (tested on Ubuntu 14.04 and 16.04).

Please, ensure that you are using Vagrant version 1.8.7 or later.

$ cd auto
$ vagrant up

It will automatically start a full test which can take over 10 minutes. Run the test again:

$ vagrant provision --provision-with run

First you need to install the prerequisites:

$ cd auto
$ sudo ./install_ubuntu.sh

Script options:

$ cd auto
$ ./run.sh                # Run all tests
$ ./run.sh no             # Skip AVX2 tests
$ ./run.sh - --json       # Pass extra arguments to test suite
$ ./run.sh no --json      # Pass extra arguments and skip AVX2 tests
$ CC="ccache cc" ./run.sh # Speed up subsequent builds using CC cache

We are concerned in tests on a AMD CPU with AVX2 (Zen microarchitecture) and on an Intel Skylake CPU.

1. Before running any tests, please ensure that you have exited all browsers on the host machine, closed all audio players and stopped any backup systems. The full test lasts up to 15 minutes.

2. Run automatic test with --json option. Please avoid custom --size, --mode and --runs settings.

   $ ./run.sh - --json

3. Collect the following information about your system: name, OS version, CPU. For example:

   • Name: MacBook Pro (Retina, 13-inch, Late 2013)
   • OS: Ubuntu 14.04.4 LTS 64bit
   • CPU: Intel® Core™ i5-4258U CPU @ 2.40GHz

4. Create an issue with the collected information and test output.

The docs can be found on the public page with benchmark results and details.

$ cd docs
$ npm install
$ npm run build

$ python -m SimpleHTTPServer &
$ npm run watch

--watch-poll will help if you have any problems with auto reload.