Option 1: write logs in per-node basis and use jq for querying
Can you run jq over multiple files?

Option 2: send logs to a log aggregator (Loki?). Can dashboard for data transmission

(1) the sink should be threaded (to send non-blocking requests with data to multiple targets)

(2) use a socket per a combination (match stream, target port)

Max sockets: match streams * target ports

The current implementation has limitations and is assumed to be in the same package (or jar) as both the server and the client

A separate server to accept control messages from the coordinator.

Functionality:

• both incoming and outcoming communication (receiving instructions and reporting available to commit)
• report healthy on the request for a switch (how to monitor for overload on the flink engine?)
• trigger buffering events of the partitioning input from t_0 until t_1 (separate data stream, same sink but triggered on demand?)
• flush the buffer to a new target

Implementation

• Send control message just like any other event. Create a super class from which AtomicEvent would inherit that would have a field with a type of a message (control or regular)
• filter the inputEventStream to branch it into the regular event stream and control stream based on the message type
• Apply Windowing based on t_0 and t_1?
• Define custom trigger?
• Reuse the SocketSink
• Define a custom time trigger for the event stream on t_1
• Override clear() method to clear state (all partial matches) on the trigger from (6) from the event stream on t_1

To see how the server side handles it with threading

On a match, all contributing events are forwarded.

Idea: regulate the number of attempts using retries and a reconnect loop on failure