dad1617's People
Forkers
nuno-silvadad1617's Issues
Wait command
• Wait x_ms
. This command instructs the pupper master to sleep for x milliseconds
before reading and executing the following command in the script file.
Part of #1
Random Routing Policy
Random: tuples are output to a random replica of the downstream operator.
part of #7
Custom Stream Operator
part of #20
This is a hard one. Checkout the examples here:
https://fenix.tecnico.ulisboa.pt/downloadFile/282093452035684/OperatorTransferNew.zip
Operator commands
Tuple proccessing semantics
• At most once: Tuples may or be not processed, i.e., some tuples may be lost in
presence of operator failures. E.g., all the tuples sent towards an operator downstream
may be lost if this is unavailable.
• At least once: Tuples are guaranteed to be processed at least once, i.e., no tuple in
input is missed. Yet, it cannot be excluded that an operator processes twice a tuple,
e.g., once before crashing and a second time upon recovering.
• Exactly once: Tuples are guaranteed to be processed exactly once. This also ensures
exactly once semantics not only for the update of the internal state of the operators
involved in the processing, but also for the execution of any non-idempotent action
(e.g., interaction with external devices like DBMSs or storage systems) issued by the
operators during the processing of a tuple.
Part of #3
Uniq Stream Operator
part of #20
OP Stream Input
Stream Input that gets tuples from the registered OperatorService
part of #26
Start command
• Start operator_id
. Tells the operator id operator to start processing tuples.
Part of #1
Stream Operators
Uniq, Count, etc
part of #2
PuppetMaster
Implementation of the puppetmaster
Filter Stream Operator
part of #20
Tuple Routing (Routing Policy)
• Primary: tuples are output to the primary replica of the operator it is connected to
downstream.
• Random: tuples are output to a random replica of the downstream operator.
• Hashing(field id): tuples are forwarded to a replica of the downstream operator according
to the value of a hashing function computed on the tuple’s field with identifier
field id.
Operators
Operator implementation
Report
Operator Stream Engine
Takes care of getting input, processing it and outputting it.
part of #2
Dup Stream Operator
part of #20
Replication
submission requirements
- an example of a configuration file
- a text file with a short explanation of how to start the project
- a list of any missing features.
missing stuff
Unimplemented methods:
-
PuppetMaster
- PuppetMaster.Command.Log
- PuppetMaster.Command.SetSemantics
-
OperatorService
- OperatorService.forceInterval
- OperatorService.forceFreeze
- OperatorService.forceUnfreeze
- OperatorService.forceCrash
- OperatorService.getStatus
Interval command
• Interval operator_id x_ms
. The operator should sleep x milliseconds between two
consecutive events
Part of #1
Crash command
• Crash processname
. This command is used to force a process to crash.
Part of #1
File Stream Input
Stream Input that reads tuples from a given File.
part of #26
Hashing Routing Policy
Hashing(field id): tuples are forwarded to a replica of the downstream operator according
to the value of a hashing function computed on the tuple’s field with identifier
field id.
part of #7
File parser
Make a parser for the PuppetMaster to run commands
Refers to #1
Primary Routing Policy
• Primary: tuples are output to the primary replica of the operator it is connected to
downstream.
Refers to #7
Unfreeze command
• Unfreeze processname
. This command is used to put a process back to normal operation.
Pending messages that were received while the process was frozen, should be
processed when this command is received.
Part of #1
Status command
• Status
: This command makes all nodes in the system print their current status. The
status command should present brief information about the state of the system (who
is present, which nodes are presumed failed, etc...). Status information can be printed
on each nodes’ console and does not need to be centralised at the PuppetMaster.
Part of #1
Count Stream Operator
part of #20
Operator Stream Input
Takes care of obtaining input for the Stream Engine to consume.
part of #2
Operator subscriptions
Part of #1
Freeze command
• Freeze processname
. This command is used to simulate a delay in the process. After
receiving a freeze, the process continues receiving messages but stops processing them
until the PuppetMaster “unfreezes” it.
Part of #1
Parse config file
Parse the config file in operators.
Refers to #2
one PCS per IP
There should be one PCS per IP, not a single global PCS.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.