Comments (6)
What kind of checking do you have in mind? Workflow should be a vector, containing vectors with two keyword elements?
Do you want to check for things like cycles?
from onyx.
I think, first, we can only validate the DAG form of the workflow - since the map form is now unpacked to a vector-of-vectors anyhow. Let's start with this:
- Workflow is a vector
- Workflow contains elements only of vectors
- Of those inner vectors, they are all size 2
- Of those inner vectors, the two elements are both keywords
- Both of those keywords refer to entries in the catalog
- "input" nodes in the DAG refer to entries in the catalog with
:onyx/type
:input
- There are no cycles
I'd be super happy with any subset of these.
from onyx.
See PR.
Checks not addressed:
Cycle checking.
Does not check whether tasks with no incoming nodes are type input (better checked when actually generating the DAG to check for cycles).
Checks additionally addressed:
- Input nodes should not have incoming edges.
- Output nodes should not have outgoing edges.
I assume that's correct?
from onyx.
That looks great to me. I'll merge these changes.
Want to make separate issues for cycling checks and no incoming nodes?
from onyx.
Merged and deployed under 0.4.1-SNAPSHOT
. Other changes pending.
from onyx.
Remaining issues can be tracked here #44
from onyx.
Related Issues (20)
- output task with a window does not record the window checkpoint size correctly
- Is Dire still used in Onyx platform? HOT 7
- Upgrade Apache Curator framework to 4.0.x - SSL Support HOT 6
- Flow conditions validation shows bad error message when tasks are not connected. HOT 3
- Peer group manager where communicator fails to start is recoverable HOT 2
- Validation error for min-max-n-peers for flux policy is not printed correctly.
- Input plugin's poll! continues to be invoked after completed? HOT 2
- Move task-lifecycle backoff-until-task-start! into state machine.
- http://www.onyxplatform.org/ links to https://github.com/onyx-platform/onyx/releases/tag/0.13.x which does not exist
- Resume point AssertionError: Assert failed: (= slot-migration :direct) for {:mode :initialize}
- Output from job-snapshot-coordinates does not match input schema for build-resume-point
- Onyx patch versions should not require new tenancy-id HOT 2
- Clojure 1.10.0-beta4 isn't happy with Onyx HOT 1
- Co-located task scheduler does not respect capacity contrains
- Output plugin :after-task-stop lifecycle fn doesn't receive :onyx.core/scheduler-event HOT 1
- Onyx hangs when provide ":onyx.messaging.aeron/media-driver-dir" setting in peer-config HOT 1
- Project maintanence going forward HOT 28
- Aeron Reliability HOT 4
- Feature request: Do not try to recover output checkpoints for plugins that don't use it HOT 3
- IndexOutOfBoundsException from aeron HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from onyx.