Workload manager

The workload manager's task is to enact a workload on an overlay. More specifically, the workload manager

• accepts a workload description (i.e. a set of tasks and relationships between those tasks);
• translates the tasks into actionable DataUnit and ComputeUnit descriptions;
• schedules those Units on a given Overlay (i.e. on pilots);
• dispatches and monitors (and potentially manages) the execution of the scheduled units;
• provides inspection on the workload.

The workload manager is stateful, in the sense that it can collate information about subsequently submitted workloads. On any of the individual actions (Translation, Scheduling, Dispatching), the Workload Manager can inspect the managed workload and the target overlay (Note that the overlay may not yet be defined during Translation, and may not yet be dispatched (i.e. running) during Scheduling -- early versus late binding). The actions are implemented as plugins -- different implementations are thus available.

The workload manager interface directly mirrors the above functionality:

class workload_manager (object) :
    def __init__ (self, translator = 'default', 
                        scheduler  = 'default',
                        dispatcher = 'default')                 : pass
    def translate_workload (self, workload_id, overlay_id=None) : pass
    def schedule_workload  (self, workload_id, overlay_id, ...) : pass (*)
    def dispatch_workload  (self, workload_id, overlay_id)      : pass
    def inspect_workload   (self, workload_id)                  : pass

The construction parameters define the set of plugins to be used for the individual actions.

(*) The schedule_workload method optionally accepts an additional binding parameter, which can be set to troy.EARLY or troy.LATE. If this parameter is set, it will trigger a test which ensures that the given Overlay is in the respective state, i.e. is not yet scheduled or dispatched in the case of early binding and is scheduled or dispatched in the case of late binding. Partially dispatched overlays will not be usable in either case -- for those, the binding parameter must be left unspecified (i.e. None).