ngfk / vu-distributed-systems Goto Github PK

Every data structure defined in the system design (section 4.2) should be a POJO class.

From system design:

1. A user (U1) sends a job to one of the schedulers. This scheduler (S1) is chosen by our software on the user machine.
   a. S1 checks if the job was not already in the active jobs
   b. S1 adds the job to its local copy of active jobs
2. S1 communicates the job to all other schedulers
   a. Schedulers that receive the message add it to their local copies of active jobs
3. All other schedulers confirm receival of the previous message (step 2) to S1
4. S1 then confirms receival of the job to U1

After response from RM:

1. S1 returns the result to U1
2. U1 confirms it received the result to S1
   a. S1 removes the job from its local copy of active jobs
3. S1 communicates completion of the job to all other schedulers
   a. Schedulers that receive the message remove it from their local copies of active jobs
4. all other schedulers confirm receival of the previous message (step 13) to S1

Store data using data structures from #3.

User

• schedulers
• jobs

Scheduler

• schedulers
• users
• jobs

Arthur want to try Heroku as provider.

Would be nice if we can make it watch our master branch and let it serve both the java and website part.

From system design:

1. RM1 confirms receival of the job to S1
2. RM1 forwards the job to one of the worker nodes in its cluster (W1)
   a. W1 saves the job and corresponding data locally
3. W1 confirms receival of the job to RM1
4. Upon completion of the job, W1 returns the result to RM1
   a. RM1 saves the result locally
5. RM1 returns the result to S1
   a. S1 saves the result locally
   b. RM1 starts a timer and removes the result after one minute

Store data using data structures from #3.

Resource manager

• schedulers
• workers
• jobs

Worker

• job

Need some way to communicate between java grid and the browser interface. Best way is probably to just use a web socket connection.

know when a node dies

in the schedulers and resourcemanagers

Change the grid from a GUI application to a console "server" application. This server simply has to wait for web socket connections from the interface.

Eventually it will create the grid based on messages from the interface.

Try to figure out how to run the grid and the site simultaneously to each other on heroku

Random events are too unreliable. We need to find a way to trigger random events more reliably.

Do you guys mind if we restructure the java packages?

I was thinking about the structure below, makes it easier to know the purpose of a class just by looking at it's location.

/**
 * Model classes that form the grid & the communication between them.
 */
package distributed.systems.grid.model;
// ISocketCommunicator.java
// Job.java
// Message.java
// ResourceManager.java
// Scheduler.java
// Socket.java
// User.java
// Worker.java

/**
 * Data classes that are used within the model classes `grid.model`.
 */
package distributed.systems.grid.data;
// ActiveJob.java

/**
 * Classes used to setup the simulation.
 */
package distributed.systems.grid.simulation;
// GridSetup.java
// GridClusterSetup.java
// Simulation.java

/**
 * Classes used to communicate with the gui front-end.
 */
package distributed.systems.grid.gui;
// GuiMessage.java
// GuiMessageInit.java
// GuiMessageInitSizes.java
// GuiMessageQueue.java
// GuiMessageSetup.java
// GuiMessageState.java
// GuiMessageStop.java
// GuiMessageToggle.java
// NodeState.java
// NodeType.java
// WebSocketHandler.java

/**
 * Contains classes that start the system (have a main).
 */
package distributed.systems.grid;
// StartDebug.java
// WebSocketServer.java

The classes in grid.data are the classes from #3.

Someone will have to make an interface that looks like the system structure with a dynamic amount of schedulers/clusters/workers.

Should be easy using flexbox, if we skip the connection lines.

<div style="display:flex;">
    <div class="user"></div>
    <div style="display:flex;">
        <div class="scheduler"></div>
        <div class="scheduler"></div>
        ...
    </div>
    <div style="display:flex;">
        <div class="cluster" style="display:flex;">
            <div class="resource-manager"></div>
            <div>
                <div class="worker"></div>
                <div class="worker"></div>
                <div class="worker"></div>
                ...
            </div>
        </div>
        ...
    </div>
</div>

Right now every GridNode instance should be runnable and have a private Thread thread property. Maybe add this to the base class, let the base class implement Runnable and create an abstract run method that has to be implemented by every GridNode. The base class can then implement the start and stop method the same way User currently has.

When this is done the Simulation can keep a list of GridNodes and start/stop all of these node when they have to be started/stopped.