Comments (1)
Here are some thoughts:
- Use Fenzo's tiered queues to define priority tiers for service versus batch tasks. We actually created two tiers - one for "critical" tasks that need to be launched right away (and most service style tasks fit into this tier), and one for "flex" tasks that have flexible needs for how quickly they need to be launched. Fenzo will assign resources to tasks in critical tier before considering assignments for tasks in the lower tier, flex. Note that tiers in Fenzo are numbered 0 to N-1 for N tiers. For us, critical is tier 0 and flex is tier 1.
- This does not, however, prevent the case of the cluster being saturated with lower tier tasks. In which case, a new task in the higher tier will have to wait until resources are made available due to completion of some tasks (e.g., batch tasks eventually complete).
- In the future, we will be introducing preemptions to ensure that the higher tier tasks can get resources immediately by terminating some lower tier tasks.
- Currently, we take the approach of guaranteeing resources for each tier using different set of agents. So, say, a set of agents are "earmarked" for tier 0 and a different set of agents are earmarked for tier 1. We do this by setting a Constraint that ensures tasks of a certain tier go to its preferred agents. This also allows us to ensure there is sufficient capacity for each tier, separately.
- We create separate sets of agents using the autoscale groups in Fenzo. See, AutoScaleByAttribute settings in the TaksScheduler's Builder class.
I spoke about capacity guarantees recently at QCon San Francisco. The slides are available here. The video should be available later from QCon.
from fenzo.
Related Issues (20)
- contributor agreement. HOT 8
- NamedResourceSetRequest HOT 2
- Resource ranges and "greedy" scheduling HOT 4
- Support v1 Protos (or provide utility to convert) HOT 5
- Support dynamic fitness threshold HOT 3
- Allow for custom shortfall evaluators HOT 4
- Execution failed for task ':fenzo-core:compileJava'.> 无效的源发行版: 1.8 HOT 2
- Update Mesos dependency
- BinPacking with weights HOT 2
- Incorrect handling of reserved resources HOT 2
- Unsafe concurrent access to unknown lease collection in AssignableVMs class
- Pluggable ENI fitness evaluator HOT 1
- After reading Fenzo, I don't understand how to get Framework information on Mesos?
- After reading Fenzo, I don't understand how to get Framework information on Mesos?
- DisableVM does not remember hosts HOT 2
- .travis.yml: The 'sudo' tag is now deprecated in Travis CI
- Dynamic ResAllocs
- CVE-2018-7489 on jackson-databind 2.4.5 HOT 1
- Unused variable
- Unreleased Resource: Streams HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fenzo.