Giter VIP home page Giter VIP logo

procrastinate's Introduction

Procrastinate: PostgreSQL-based Task Queue for Python

Deployed to PyPI

Deployed to PyPI

GitHub Repository

Documentation

Coverage

MIT License

Contributor Covenant

Procrastinate is looking for additional maintainers!

Procrastinate is an open-source Python 3.8+ distributed task processing library, leveraging PostgreSQL to store task definitions, manage locks and dispatch tasks. It can be used within both sync and async code and integrated to Django.

In other words, from your main code, you call specific functions (tasks) in a special way and instead of being run on the spot, they're scheduled to be run elsewhere, now or in the future.

Here's an example (if you want to run the code yourself, head to Quickstart):

# mycode.py
import procrastinate

# Make an app in your code
app = procrastinate.App(connector=procrastinate.AiopgConnector())

# Then define tasks
@app.task(queue="sums")
def sum(a, b):
    with open("myfile", "w") as f:
        f.write(str(a + b))

with app.open():
    # Launch a job
    sum.defer(a=3, b=5)

    # Somewhere in your program, run a worker (actually, it's often a
    # different program than the one deferring jobs for execution)
    app.run_worker(queues=["sums"])

The worker will run the job, which will create a text file named myfile with the result of the sum 3 + 5 (that's 8).

Similarly, from the command line:

export PROCRASTINATE_APP="mycode.app"

# Launch a job
procrastinate defer mycode.sum '{"a": 3, "b": 5}'

# Run a worker
procrastinate worker -q sums

Lastly, you can use Procrastinate asynchronously too:

import asyncio

import procrastinate

# Make an app in your code
app = procrastinate.App(connector=procrastinate.AiopgConnector())

# Define tasks using coroutine functions
@app.task(queue="sums")
async def sum(a, b):
    await asyncio.sleep(a + b)

async with app.open_async():
    # Launch a job
    await sum.defer_async(a=3, b=5)

    # Somewhere in your program, run a worker (actually, it's often a
    # different program than the one deferring jobs for execution)
    await app.run_worker_async(queues=["sums"])

There are quite a few interesting features that Procrastinate adds to the mix. You can head to the Quickstart section for a general tour or to the How-To sections for specific features. The Discussion section should hopefully answer your questions. Otherwise, feel free to open an issue.

The project is still quite early-stage and will probably evolve.

Note to my future self: add a quick note here on why this project is named "Procrastinate".

Where to go from here

The complete docs is probably the best place to learn about the project.

If you encounter a bug, or want to get in touch, you're always welcome to open a ticket.

procrastinate's People

Contributors

abe-winter avatar adibsaad avatar agateblue avatar aleksandr-shtaub avatar benjamb avatar bracketjohn avatar charlesaracil-ulti avatar corbott avatar daindwarf avatar dependabot[bot] avatar eliotberriot avatar ewjoachim avatar ignaciocabeza avatar k4nar avatar mecampbellsoup avatar mininao avatar paulzakin avatar pmav99 avatar pmourlanne avatar pre-commit-ci[bot] avatar renovate-bot avatar renovate[bot] avatar sbillion avatar sophie-ulti avatar t-eckert avatar thomasperrot avatar tomdottom avatar turicas avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.