Genpei (源平) is a standard implementation conforming to the Global Alliance for Genomics and Health (GA4GH) Workflow Execution Service (WES) API specification. A simple and highly scalable REST API Server using Flask and cwltool that follows the philosophy of Microservice. It supports the execution and management of Workflow written in Common Workflow Language (CWL).
Genpei supports Python 3.6 or newer.
$ pip3 install genpei
$ genpeiWe also expect to launch using Docker.
Because of the compatibility of cwltool and Docker-in-Docker (DinD), you have to mount docker.sock, /tmp, etc.
Please check the documentation in DockerHub - cwltool for more information.
# Launch
$ docker-compose up -d
# Launch confirmation
$ docker-compose logsAs API specifications, please check GitHub - GA4GH WES and SwaggerUI - GA4GH WES.
As the simplest example of a REST API Request, here is the result of a GET /service-info.
GET /service-info
{
"auth_instructions_url": "https://github.com/suecharo/genpei",
"contact_info_url": "https://github.com/suecharo/genpei",
"default_workflow_engine_parameters": [],
"supported_filesystem_protocols": [
"http",
"https",
"file"
],
"supported_wes_versions": [
"1.0.0"
],
"system_state_counts": {},
"tags": {
"wes_name": "genpei"
},
"workflow_engine_versions": {
"cwltool": "3.0.20200324120055"
},
"workflow_type_versions": {
"CWL": {
"workflow_type_version": [
"v1.0",
"v1.1",
"v1.1.0-dev1",
"v1.2.0-dev1",
"v1.2.0-dev2"
]
}
}
}The host and port used by the application can be changed by specifying the startup arguments (--host and --port). And environment variables corresponding to these arguments are GENPEI_HOST and GENPEI_PORT.
$ genpei --help
usage: genpei [-h] [--host] [-p] [--debug] [-r] [--service-info]
An implementation of GA4GH Workflow Execution Service Standard as a microservice
optional arguments:
-h, --help show this help message and exit
--host Host address of Flask. (default: 127.0.0.1)
-p , --port Port of Flask. (default: 8080)
--debug Enable debug mode of Flask.
-r , --run-dir Specify the run dir. (default: ./run)
--service-info Specify `service-info.json`. The workflow_engine_versions, workflow_type_versions
and system_state_counts are overwritten in the application.
$ genpei --host 0.0.0.0 --port 5000Genpei manages the submitted workflows, workflow parameters, output files, etc. on the file system. The location of run dir can be overridden by the startup argument --run-dir or the environment variable GENPEI_RUN_DIR.
The run dir structure is as follows. Initialization and deletion of each run can be done by physical deletion with rm.
$ tree run
.
├── 11
│ └── 11a23a68-a914-427a-80cd-9ad6f7cfd256
│ ├── cmd.txt
│ ├── end_time.txt
│ ├── exe
│ │ └── workflow_params.json
│ ├── exit_code.txt
│ ├── outputs
│ │ ├── ERR034597_1.small_fastqc.html
│ │ ├── ERR034597_1.small.fq.trimmed.1P.fq
│ │ ├── ERR034597_1.small.fq.trimmed.1U.fq
│ │ ├── ERR034597_1.small.fq.trimmed.2P.fq
│ │ ├── ERR034597_1.small.fq.trimmed.2U.fq
│ │ └── ERR034597_2.small_fastqc.html
│ ├── run.pid
│ ├── run_request.json
│ ├── start_time.txt
│ ├── state.txt
│ ├── stderr.log
│ └── stdout.log
├── 14
│ └── ...
├── 2d
│ └── ...
└── 6b
└── ...The execution of POST /runs is very complex. Examples using Python's requests are provided by GitHub - genpei/tests/post_runs_examples. Please use this as a reference
The development environment starts with the following.
$ docker-compose -f docker-compose.dev.yml up -d --build
$ docker-compose -f docker-compose.dev.yml exec app bashWe use flake8, isort, and mypy as the linter.
$ bash ./tests/lint_and_style_check/flake8.sh
$ bash ./tests/lint_and_style_check/isort.sh
$ bash ./tests/lint_and_style_check/mypy.shWe use pytest as a Test Tool.
$ pytest .Apache-2.0. See the LICENSE.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent