Dispatchers

The scheduler comes with 3 basic dispatchers and a CompositeDispatcher designed to be able to combine the different one in case one needs to submit to multiple clusters/sites.

LSFDispatcher

The LSFDispatcher is the one that has been most used, since LSF is the batch system put in production in STAR. What the LSF dispatcher essentially does is creating a script for each job, preparing the bsub command, with the correct LSF resource string, and executing it. Some of these functionalities are delegated to other classes.

The CSHApplication class encapsulates the creation of the csh scripts and filelists. It also handles the different filelists syntaxes. This delegation allows to recycle the same code across different dispatchers, and also to allow different applications to create different types of scripts. The interface between application and dispatcher, though, is still not well defined.

The LSFResourceStrategy, instead, encapsulate the code to create the resource option for bsub. In STAR, we use LSF static resources to balance the load across different file server, so each file it's checked and, if on a NFS server, the correct resource is added/incremented.

CondorGLSFDispatcher

The CondorGLSFDispatcher is a GRID dispatcher based on CondorG. To allow the distributed disk framework (see PassivePolicy description) and the use of LSFResources, we had to modify the Globus LSF job manager to be able to use some extra LSF option which wouldn't be available otherwise. So, in order, to use all the functionalities of this dispatcher as it is, some modification on the Globus installation are necessary.

This dispatcher is basically exactly the same than the LSFDispatcher, except that the jobs is not submitted directly to LSF, but is submitted to LSF via Condor-G/Globus.

CondorGDispatcher

This dispatcher is going to dispatch directly to Condor via condor_submit. It will create a class-ad, a script and a filelist for each job.

That fact that it derives from LSFDispatcher is basically a hack: they have many things in common, so they should have a common AbstractDispatcher ancestor.

CompositeDispatcher

The CompositeDispatcher is meant to combine different dispatcher in case of multiple sites/clusters. The idea is that for each cluster one would define a specific dispatcher, with the correct parameters, and instruct the CompositeDispatcher on how to redirect jobs. The DispatcherChooser class represent the policy that makes that choice.


Gabriele Carcassi - page was last modified