Python SDK for working with https://github.com/conductor-oss/conductor.
Conductor is the leading open-source orchestration platform allowing developers to build highly scalable distributed applications.
Check out the official documentation for Conductor.
Show support for the Conductor OSS. Please help spread the awareness by starring Conductor repo.
- Install Conductor Python SDK
- Hello World Application Using Conductor
- Running Workflows on Conductor Standalone (Installed Locally)
- Running Workflows on Orkes Conductor
- Learn More about Conductor Python SDK
Before installing Conductor Python SDK, it is a good practice to set up a dedicated virtual environment as follows:
virtualenv conductor
source conductor/bin/activateThe SDK requires Python 3.9+. To install the SDK, use the following command:
python3 -m pip install conductor-pythonIn this section, we will create a simple "Hello World" application that executes a "greetings" workflow managed by Conductor.
Create greetings_workflow.py with the following:
from conductor.client.workflow.conductor_workflow import ConductorWorkflow
from conductor.client.workflow.executor.workflow_executor import WorkflowExecutor
from greetings_worker import greet
def greetings_workflow(workflow_executor: WorkflowExecutor) -> ConductorWorkflow:
name = 'greetings'
workflow = ConductorWorkflow(name=name, executor=workflow_executor)
workflow.version = 1
workflow >> greet(task_ref_name='greet_ref', name=workflow.input('name'))
return workflow
Create greetings_workflow.json with the following:
{
"name": "greetings",
"description": "Sample greetings workflow",
"version": 1,
"tasks": [
{
"name": "greet",
"taskReferenceName": "greet_ref",
"type": "SIMPLE",
"inputParameters": {
"name": "${workflow.input.name}"
}
}
],
"timeoutPolicy": "TIME_OUT_WF",
"timeoutSeconds": 60
}Workflows must be registered to the Conductor server. Use the API to register the greetings workflow from the JSON file above:
curl -X POST -H "Content-Type:application/json" \
http://localhost:8080/api/metadata/workflow -d @greetings_workflow.jsonNote
To use the Conductor API, the Conductor server must be up and running (see Running over Conductor standalone (installed locally)).
Using Python, a worker represents a function with the worker_task decorator. Create greetings_worker.py file as illustrated below:
Note
A single workflow can have task workers written in different languages and deployed anywhere, making your workflow polyglot and distributed!
from conductor.client.worker.worker_task import worker_task
@worker_task(task_definition_name='greet')
def greet(name: str) -> str:
return f'Hello {name}'Now, we are ready to write our main application, which will execute our workflow.
Let's add helloworld.py with a main method:
from conductor.client.automator.task_handler import TaskHandler
from conductor.client.configuration.configuration import Configuration
from conductor.client.workflow.conductor_workflow import ConductorWorkflow
from conductor.client.workflow.executor.workflow_executor import WorkflowExecutor
from greetings_workflow import greetings_workflow
def register_workflow(workflow_executor: WorkflowExecutor) -> ConductorWorkflow:
workflow = greetings_workflow(workflow_executor=workflow_executor)
workflow.register(True)
return workflow
def main():
# The app is connected to http://localhost:8080/api by default
api_config = Configuration()
workflow_executor = WorkflowExecutor(configuration=api_config)
# Registering the workflow (Required only when the app is executed the first time)
workflow = register_workflow(workflow_executor)
# Starting the worker polling mechanism
task_handler = TaskHandler(configuration=api_config)
task_handler.start_processes()
workflow_run = workflow_executor.execute(name=workflow.name, version=workflow.version,
workflow_input={'name': 'Orkes'})
print(f'\nworkflow result: {workflow_run.output["result"]}\n')
print(f'see the workflow execution here: {api_config.ui_host}/execution/{workflow_run.workflow_id}\n')
task_handler.stop_processes()
if __name__ == '__main__':
main()Set the following environment variable to point the SDK to the Conductor Server API endpoint:
export CONDUCTOR_SERVER_URL=http://localhost:8080/apiTo start the Conductor server in a standalone mode from a Docker image, type the command below:
docker run --init -p 8080:8080 -p 5000:5000 conductoross/conductor-standalone:3.15.0To ensure the server has started successfully, open Conductor UI on http://localhost:5000.
To run the application, type the following command:
python helloworld.py
Now, the workflow is executed, and its execution status can be viewed from Conductor UI (http://localhost:5000).
Navigate to the Executions tab to view the workflow execution.
For running the workflow in Orkes Conductor,
- Update the Conductor server URL to your cluster name.
export CONDUCTOR_SERVER_URL=https://[cluster-name].orkesconductor.io/api- If you want to run the workflow on the Orkes Conductor Playground, set the Conductor Server variable as follows:
export CONDUCTOR_SERVER_URL=https://play.orkes.io/api- Orkes Conductor requires authentication. Obtain the key and secret from the Conductor server and set the following environment variables.
export CONDUCTOR_AUTH_KEY=your_key
export CONDUCTOR_AUTH_SECRET=your_key_secretRun the application and view the execution status from Conductor's UI Console.
Note
That's it - you just created and executed your first distributed Python app!
There are three main ways you can use Conductor when building durable, resilient, distributed applications.
- Write service workers that implement business logic to accomplish a specific goal - such as initiating payment transfer, getting user information from the database, etc.
- Create Conductor workflows that implement application state - A typical workflow implements the saga pattern.
- Use Conductor SDK and APIs to manage workflows from your application.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent