pyblish / pyblish-ci Goto Github PK

At the moment, two or more tests will cause concurrent runs to be triggered, but concurrency is unbounded and could easily exhaust memory and CPU. Limit this, and put things into a queue.

Motivation

What hosts are available for free?

As we're operating without a license, its possible that this isnt allowed. moreover, its likely at all needed hosts doesn't support being run without a license. additionally, some hosts won't run on linux.

Counterpoint, essential software does work, does exist and should work fine.

• maya
• nuke
• houdini
• clarisse?
• 3ds?
• softimage?
• hiero?

Currently, root is the current user, and the current working directory is /pyblish.

Make a common user, pyblish, and use his home directory as the working directory for each job.

Motivation

When processing exceeds 5-10 concurrent jobs, how can we keep things from getting slow?

We could use a droplet per worker, each worker capable of running around 5-10 concurrent jobs, each worker announcing itself to a central dispatcher.

Downside, we lose docker caching. processing will go quick, as images are only downloaded once, but how can we keep certain jobs from running on workers with the requested image available?

A job could first define what it needs. the central dispatcher could then evaluate, given the currently registered workers, which images are available where. it could then send a job to a worker with the necessary images available.

In cases of a single worker accumulating too many jobs, a new worker would have to be assigned and would have to re-download the images.

This should result in a natural propagation of images most heavily used across the number of available workers. e.g. if 90% of tests require maya2015sp5, then the first worker will quickly become overburdened which will in turn make other workers have to also download maya2015sp5, increasing its availability across the swarm.

The default python is currently being overridden to provide for a uniform experience across multiple images and multiple environments.

However, altering the default python also breaks the OS, as it depends on it's existence; most predominately, yum - CentOS's package manager - no longer works.

Solution

A single script is provided to run in any number of images, even images with a completely different set of software installed. The problem is that we can't provide direct access to mayapy under a Maya image, as the Nuke images won't have it. For the same script to run on every image, it's important that they all refer to the same distribution of Python.

To solve this, a new executable is introduced - hpython for Host Python, leaving python unspoiled and still accessible.

Future

Try making each common executable prefixed with h, such as hpip.

Currently, HTML and CSS is written in Python. Separate this. For good measure.

It's possible a later image exists when initiating a new build. For every build, pull latest image first.

Goal

Enable users re-running a job. Useful in case of an error occurring that is unrelated to the test, and instead related to an external problem such as a bad connection or temporarily missing resource.

Implementation

A docker kill should handle stop whereas re-running is a matter of putting the job back into the original queue, meaning we'll need to keep hold of completed jobs too, possibly serialise and store them on disk/in database.

Who is our audience? Large facilities are the most appropriate, but they will likely already have this infrastructure and won't need a cloud based option, unless we offer something more than simply a host.

Commercial houses is a good fit, but they typically won't bother with tests.

It seems medium sized facilities are most suited, like the ones using Pyblish currently.

Luckily, there are many medium sized facilities and they have money to spend. estimating between 50-100 studios are a direct match, 5-10 may eventually consider using the service. This is a very small userbase, and it likely wont get any bigger.

At 5 studios, using 5 hosts each, each host costing around £20/month, that's £100/month of service costs per studio, + £100/month for using pyblish ci..

How about they pay whatever their licenses cost, along with the amount of licenses, and I'll charge a fee per worker, which is a direct connection with droplets, £10/worker/month for Digital ocean, and say £50/worker/month in direct income = £60/worker/month/studio

We could potentially look at a service for vfx in particular, offering licenses on a per-host basis like before, but not limited to Pyblish

Goal

To provide for introspection into which environment a test is being run.

Motivation

When running a test capable of supporting multiple images, the test must somehow distinguish between what software is available so as to provide for appropriate setup/teardown procedures.

In the case of Pyblish, a Maya image must be set-up with a Pyblish for Maya integration, whereas a Nuke image is set-up with a Pyblish for Nuke integration.

Furthermore, it is currently the case that a dedicated binary is provided to bridge access to the python executable of each image (see #8). With some way to distinguish the system at run-time, this executable could instead be dynamically found.

# E.g.
if [ ${SYSTEM_IMAGE} == "mottosso/maya" ]; then
  mayapy run_testsuite.py

Implementation

An environment variable SYSTEM_IMAGE is provided in each job.

There is no home page at the moment, build one with an overview of what's available and going on.

Goal

100% uptime and 1 gigabit internet connection.

This has to do with when we can safely delete the temporary directory after Docker image having been run. It seems to keep things locked up for a period after closing (needs confirmation, could also be a simple permission issue).

Maya throws a segmentation fault upon test completion, unless sys._exit(0) is run during cleanup. However, running sys._exit(0) severely cripples the amount of output coming from the nose tests.

The test process cannot return anything other than 0, as it will imply a failed test. But we can't cut off the test output, as it is the only way in which to inspect the results of a test.

A possible solution is to look at the last outputted line, and override the return code in case output ends with anything containing the words "segmentation fault".

Goal

Expand the use of continuous integration for CG.

Motivation

Pyblish CI has commercial potential. It could either be a global offer, or a Pyblish specific offer. Pyblish means less crowd, but better specificity and support.

If we go commercial, we will need licenses. Luckily, most licenses are offered on a monthly basis, meaning we can scale dynamically as need arises. The cost of licenses could then go hand in hand with the cost of using the service, meaning zero overhead and infinite scale.

Repository is pulled in, and assigned as a volume to the Docker image.

This director is then used as the current working director,y meaning it changes. The next build will then suffer from it's change.

Copy this directory into a new temporary directory, that is wiped after each build.

Use image as opposed to environment as key to determine which Docker image to run tests within.

Implementation

Proposed format.

image:
- mottosso/maya2015sp5
- mottosso/maya2016sp1
- mottosso/nuke8
- mottosso/nuke9
script:
- git clone https://github.com/pyblish/pyblish-x.git
- export PYTHONPATH=$(pwd)/pyblish-x/pythonpath
- python run_testsuite.py