This is the architecture we use to wrap the Wayne scheduler. We will use the same architecture to wrap around the next chosen scheduling tool. The effort is to provide forward compatibility, and to save as much work as possible. The architecture tries to decouple aspects of the scheduling that we will want to change in the future.
Scheduling phases
The scheduling process is divided into stages. To each stage will correspond an interface (a pure abstract class).
In the staging part, the dispatcher creates the environment for the job.
That is, create the file list, the environment variables and copies the file,
depending on the way the job will be executed.
In the execution part, the underlying scheduler (LSF, Condor...) is called.
In the de-staging part, the necessary job cleanup or copies are done. In some
cases this part might not be necessary.
The proposed interface is simply:
public interface Dispatcher {
void dispatch(JobRequest job);
void retrieveOutput(JobRequest job);
}
The details of JobRequest class, and other classes that might be necessary, is still under development.
Gabriele Carcassi - page was last modified