Giter VIP home page Giter VIP logo

jobs's Introduction

Jobs

Robust Fiber-Driven Task-Based Parallelism Library

Features

  • Fast scalable performance
  • Lightweight dependencies with minimal switching overhead
  • Managed dependency memory
  • Unlimited dependencies per job, allowing for complex graphs
  • Fiber-aware mutexes that allow mid-execution interruption
  • High level algorithms to abstract individual job creation and management

Motivation

With the modern trend of increasing CPU cores, a highly scalable and low overhead automated work distribution system is a critical component of the backend for many real-time applications. While many of these already exist, few benefit from the power of user mode scheduling, which allows for minimal cost switching between jobs and full control over the task scheduling. This project was primarily created as a testbed to learn how to effectively operate with fibers to create a performance-focused scheduler.

Performance

Coming soon, Tracy profiler fiber support is a work-in-progress

Building

Jobs uses premake as a project file generator.

Windows

To create your solution files, run the included premake5.exe executable with the target project type as the first commandline argument, eg. vs2019 for Visual Studio 2019 project files. Open the generated solution/project files in the specified IDE and build the default project.

Linux

To create the makefiles, run the included premake5 executable with the target action being gmake2. Then, run make to compile the project.

Options

When running the build script, the behavior can be modified via the options below.

--logging

Used to retrieve internal logs for debugging purposes, this will have signficant performance impacts.

--profiling

Creates additional projects for compiling Tracy and the included examples, enables the emission of Tracy zones for internal profiling. You must include a build of Tracy (including it's dependencies) in the project root directory, with it's premake build scripts.

Samples

Full examples of using this library can be found in the Examples/ directory.

jobs's People

Contributors

adepke avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar

jobs's Issues

Mutex unlocked outside of task doesn't update waiters

If a mutex is unlocked outside of a task and all the workers have started to sleep, the unlock will not wake the workers, causing an infinite sleep until new work is enqueue. A condition variable may be required to solve this, used in the fallthrough case where a worker is about to sleep.

Move third party build scripts outside of the project directory, fix names

The build scripts for third party dependencies should be moved outside of their respective directories, and instead placed in the base ThirdParty/ directory. This allows less intrusion on the third party library, keeping it in a vanilla state. In addition, rename the third party build scripts to have the names of the library they build. See the xenia project for a demonstration of this practice.

Cross-thread fibers cause profiler halt

If a job is suspended on one worker thread, then resumed on another it will disconnect the Tracy profiler with a multiply-ended zone dialogue error. This is due to a lack of support for usermode threading in the profiler, see wolfpld/tracy#106 for tracking support.

Remove deprecated POSIX fiber functions

Unfortunately, the POSIX functions getcontext, makecontext, and swapcontext are deprecated and should not be used. These functions are currently used in the Linux implementation, which needs to be changed. Instead, assembly must be used for the Linux implementation.

Resumed fiber retains old transfer payload

The first time a fiber is scheduled, the transfer data is written based on the previous fiber's jump. If we leave this fiber and resume it later, the transfer payload won't be updated, and still refer to the original fiber. This causes a serious issue where we fix up the wrong fiber when repairing the previous fiber during a schedule. To fix this, we might need to modify the assembly of Boost.Context to not use a transfer payload, and to not invalidate a fiber after jumping to it.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.