star-submit Command Line Options

What are Command Line Options ?

Command line options change the behavior of the star-submit command. To just submit your job with all the defaults one merely has to type:

star-submit request.xml

where request.xml holds the description of your jobs.

More details (for the experts):

The star-submit command is a /bin/csh script  that executes the scheduler's java jar. Java Jars are a collection of java classes. Some command line options are interpreted by the csh while others are interpreted inside java. For example using the "-r" (resubmit) option causes the csh script to call a different class for the Java Jar then the default one used for submitting the job. This scheme limits the scheduler to one executable command when different java classes may be being used. 

What Options are Available ?

Options can be broken down into two groups. The first group of  options used the submit a new jobs. These accept as there last argument the users xml file, which  hold the descript of there job. The second group of commands modify running jobs. This group accepts the [JOBID].session.xml as there last argument. Option for these two groups can not be mixed.

What are the Options  for Submitting New jobs ?

Option Examples Description
-debug [option] star-submit -debug debug myJob.xml Print all debugging info to the screen
star-submit -debug info myJob.xml Print all debugging/logging text to the screen that is above the level of info
star-submit -debug warn myJob.xml Print all debugging/logging text to the screen that is above the level of warning
star-submit -debug error myJob.xml Print all debugging/logging text to the screen that is above the level of error
star-submit -debug fatal myJob.xml Print all debugging/logging text to the screen that is above the level of fatal
-p [policyName] star-submit -p bnl_condor_rcf  myJob.xml Prepares all jobs and assigns them to the rcas cluster.
-u [option] star-submit -u grid myJob.xml Submit the jobs back through the grid gatekeeper if I am at the local site where I intend to have my jobs run.  This option is used for testing, otherwise the scheduler would always try to submit the job locally and there would be no way of testing grid functionality from your own site.

Note: This option requires that you have a valid grid certificate (or credentials) initialized for your account.   

star-submit -u nqc myJob.xml Do not do a queue check. The scheduler will do a very basic / limited check to verify that the queue is valid. By selecting this option that test will be bypassed.

Note: This is done a the users own risk and technical support will not be offered for jobs using the "-u nqc" option.

star-submit -u ie myJob.xml

This option ignores some errors. Of most use is the fact that it ignores files not found on the file list and error where the file distribution can not meet the userís parameters.

Note: This is done a the users own risk and technical support will not be offered for jobs using the "-u ie" option.

Notes: The logging levels from order of priority from lowest to highest are: debug --> info --> warn --> error --> fatal. 

The "default" way to submit a new job is: 

star-submit myjob.xml

This uses the default policy for the site. The policy is the algorithm that makes your jobs and  determines what batch system(s) and  queue(s) or pool(s) your jobs are placed into. There is a default policy for each site the scheduler runs at. However there may be other policies you can make use of, see the policies page on this site for a full list of available policies. To select a different policy to submit your job with, use the "-p [policyName]" option like in this example below:

star-submit -p bnl_condor_cas myJob.xml

The user should note that not all policies are always appropriate for all users. Some policies may hold queues that require special rights. For example there exist policies that are used for data reconstruction, that are intended to be used only by the reconstruction coordinator. Attempting to submit to queues where you do not have rights will result in your jobs being rejected  (not submitted at all)  or the job me be held idle in the queue indefinitely (until the job is killed). This my make it sound like using policies other then the default policy is a bad thing. However experimentation is encouraged because using different policies will give you access to nodes otherwise not available to your jobs, more nodes means more throughput of jobs faster.  At BNL there is for example across experiment policy which allows the use of other RHIC experiment nodes, but with sone limitations. Determining which policies are right for you can be determined by a combination of reading the documentation on the different  policies and experimentation (trial and error).

Additional Information for grid users:

The policy table also holds policy information about policies at other sites running the scheduler. The scheduler will forward and submit these jobs much in the same way as of the job where submitted locally. And all output described in your xml job description will be copied  back to your site just as it is done locally.

Note: Using policies on sites other then your own site requires that you have a valid grid certificate (or credentials) initialized for you account. 

What are the Options  for modifying jobs I have already submitted?

Option Examples Description Notes
-r [JobSelection] star-submit -r 1,2,3,5 DDD9CFB586F4139E8D14C6.session.xml resubmit the jobs 1,2,3 and 5. If you resubmit jobs that have not died or have not been killed you will have two instances of the same job running. This may be alright if you have constructed  your job is such a way as to overwrite the existing  output. Otherwise it could corrupt your output. It is also wasteful of cpu time.
star-submit -r all DDD9CFB586F4139E8D14C6.session.xml resubmit all of my jobs
star-submit -r 5-7 DDD9CFB586F4139E8D14C6.session.xml resubmit jobs 5 thought 7
-k [JobSelection] star-submit -k 1,2,3,5 DDD9CFB586F4139E8D14C6.session.xml kill the jobs 1,2,3 and 5.  
star-submit -k all DDD9CFB586F4139E8D14C6.session.xml kill all of my jobs
star-submit -k 5-7 DDD9CFB586F4139E8D14C6.session.xml kill jobs 5 thought 7
-kr [JobSelection] star-submit -kr 1,3,5 DDD9CFB586F4139E8D14C6.session.xml kill and resubmit jobs 1,3, and 5  
star-submit -kr all DD9CFB586F4139E8D14C6.session.xml kill and resubmit all of my jobs
star-submit -kr 5-8 DDD9CFB586F4139E8D14C6.session.xml kill and resubmit  jobs 5,6,7,8
-s [JobSelection] star-submit -s 5 DD9CFB586F4139E8D14C6.session.xml get the status of job 5 This command is somewhat slower then the bjobs, qstat, condor_q commands it calls in the background. This is normal because they run inside a special shell.
star-submit -s all DD9CFB586F4139E8D14C6.session.xml get the status of all of my jobs
star-submit -s 5-8 DDD9CFB586F4139E8D14C6.session.xml get the status of jobs 5,6,7 and 8
-n [JobSelection] star-submit -n all DD9CFB586F4139E8D14C6.session.xml submits all jobs if  they have no job ID (jobs that have been killed or failed to dispatch) Note: sometimes when dispatching, the submission will time out if the batch system is busy. If the job has already been submitted but communication to the batch system is hanging the scheduler may falsely report the job as having not been submitted using this option my result in two of the same job being submitted.
star-submit -n 3 DD9CFB586F4139E8D14C6.session.xml if job 3 failed to submit or was killed resubmit it, else don't do anything
star-submit -n 1-100 DD9CFB586F4139E8D14C6.session.xml submits jobs 1-100 if  they have no job ID (jobs that have been killed or failed to dispatch)
-f [JobSelection] star-submit -f all DDD9CFB586F4139E8D14C6.session.xml resubmit all jobs that failed to dispatch Note: sometimes when dispatching, the submission will time out if the batch system is busy. If the job has already been submitted but communication to the batch system is hanging the scheduler may falsely report the job as having not been submitted using this option my result in two of the same job being submitted.
star-submit -f 7 DDD9CFB586F4139E8D14C6.session.xml resubmit job 7 if it  failed to dispatch, else do nothing
star-submit -f 5-8 DDD9CFB586F4139E8D14C6.session.xml resubmit jobs 5,6,7,8  if they failed dispatch

This syntax is used after submission for modifying already submitted jobs is below:

star-submit -[OPTION] [JobSelection] [SessionFile.xml]

All of the options are detailed in the file above. The session.xml will need to be passed to the scheduler in order to take advantage of these options. The session.xml file is generated by the scheduler (SUMS) in the same directory from which your job has been submitted. If you kill the scheduler before submitting the last job, the file will not be written. All of the other files needed for submitting the job can be deleted such as .list, .csh, .report, .condor, and .condorG, these will all be regenerated. For this reason, any changes made to any of these files will be over written when the file is submitted. The session.xml file is human readable but can be very long. The session.xml file holds all of the objects generated by the scheduler right after it has submitted all of your jobs. Simple changes can be made to the jobs by editing this file, however it is not encouraged or recommended. The session.xml files are also bound to the version of the scheduler you use. So it is important to use the same version of the scheduler that was used to generate the session.xml file in the first place when running any of these post submission commands.

The job selection range is also required. The range can be a comma delimitated list. An example of such a list is 1,5,7 this means only apply this option to jobs sched674C9FE5F9B298C9232688D49F5002DA_1 and sched674C9FE5F9B298C9232688D49F5002DA_5 and sched674C9FE5F9B298C9232688D49F5002DA_7. Alternatively a range of jobs can be used. For example 5-24 would apply to all jobs between 5 and 24. This includes job 5 and job 24. These options may not be mixed. For example 2-6,20,45 would NOT be valid. If you want  the option to apply to all jobs you have submitted the key word "all" can be used.

Is there a help option?

Yes, type star-submit -h


Levente Hajdu - page was last modified