harrystech / hyppo-worker Goto Github PK

The CommanderActor currently does not do a very good job isolating different failure domains within the execution process. Input file downloads cause task level failures instead of requeues, task actors die, etc.

The result listener should be wired up to read from the dead letters queue to detect expired work.

When jobs fail, they should definitely do more than just log it. We should fix the underlying worker protocol to be more explicit about the failure semantics we choose to enforce and capture as much as is possible about the failure context when reporting it (executor logs, etc).

Throttled resources should be implemented so that workers can detect that a task is runnable based on previous executions.