gov.bnl.star.offline.scheduler.lsf
Class LSFResourceStrategy

java.lang.Object
  extended bygov.bnl.star.offline.scheduler.lsf.LSFResourceStrategy

public class LSFResourceStrategy
extends java.lang.Object

Encapsulate the resource usage parameter (-R) for an LSF farm. Essentially, its a placeholder for the prepareResourceUsageSwitch() which will calculate the appriate string. The administrator should be able to create different subclasses for each farm as needed.

This class provides a scheleton for a typical resource usage for files residing on different NFS servers. It was designed to fit STAR needs at two sites. In case it doesn't fit the needs, the class can be subclassed to define new behaviour.

The resource switch is prepared in this way: Takes the list of input files and iterates over them.
Checks if each file needs a resource (resourceRequired()).
To determine that, checks if the file is on NFS, and looks for the typical data vault prefix (getDataVaultPrefix). For example, "/star/dataxx/..."; it extract the vaultNumber and it substitute it in the resourceSyntax
Increases the count of the resource (increaseResourceUsage)
Prepares the switch (prepareSwitch)

The resource value for each resource will the be given by the following formula:
min (base+nCount*fileFactor, max)

The subclass need to implement the abstract methods, which defines things that always change between different farms.

TODO Further work should be done to be able to configure the ResourceStrategy depending on the farm on which the job will be dispatched. This should be included in the GRID dispatchers.

Version:
July 24, 2003
Author:
Gabriele Carcassi, Jerome Lauret

Field Summary
protected  int base
          Base value for the resource value formula.
protected  int fileFactor
          FileFactor value for the resource value formula.
protected  int max
          Max parameter for the standard increaseResourceUsage implementation.
 
Constructor Summary
LSFResourceStrategy()
           
 
Method Summary
protected  void addResource(java.lang.String resName, int base)
          Addes a new resource, and initializes the value to base.
protected  void calculateLSFResources(Job job)
          Filles the resource table with all the resources.
protected  java.lang.String definedNames()
          A list of names, comma separated, of all the resources used.
protected  boolean determineResourceNotUsed()
          Determines whether any resources at all have been assigned.
 int getBase()
          Returns the base value for the resource value formula.
 java.lang.String getDataVaultPrefix()
          Returns the prefix with which a file on NFS identifies on which file server or data vault is residing.
 int getFileFactor()
          Returns the fileFactor value for the resource value formula.
 int getMax()
          Returns the max value for the resource value formula.
 java.lang.String getResourceNameSyntax()
          Returns the syntax to be used to convert a data vault number to the LSF resource.
 java.lang.String getSwitchSyntax()
          Returns the syntax that will be used to prepare the final switch
protected  int increaseResource(java.lang.String resName, int base, int amount)
          Increases the value associated to the given resource by a given amount.
protected  int increaseResourceUptoLimit(java.lang.String resName, int base, int amount, int limit)
          Increases the value associated to the given resource by a given amount.
protected  void increaseResourceUsage(java.lang.String resource)
          Increase the resource usage as part of the standard implementation.
protected  void initResources()
          Initializes the resource table.
 boolean isRemoveCharactersFromVaultNumber()
          If true removes all the characters from the vault number.
 boolean isTrimVaultNumberDecimal()
          Getter for property trimVaultNumberDecimal.
 boolean isTrimVaultNumberLeadingZeros()
          Returns true if the leading zeros of a vault number will be removed when creating the corresponding LSF resource.
protected  java.lang.String nameEqualValueColumnSeparated()
          Returns a string already formatted containing names and values column separated (i.e.
protected  java.lang.String nameEqualValueCommaSeparated()
          Returns a string already formatted containing names and values comma separated (i.e.
protected  java.lang.String prepareResourceName(java.lang.String vaultNumber)
          Given the vault id, returns the LSF resource name to be used in the -R parameter.
 java.lang.String prepareResourceUsageSwitch(Job job)
          Returns the -R parameter to be used to dispatch the given job.
protected  int prepareResourceValue(java.lang.String resName)
          Returns the value currently associated to the resource.
protected  java.lang.String prepareSwitch()
          Prepares the -R parameters to give to LSF at the end of the resource distribution.
protected  java.lang.String resourceRequired(PhysicalFile file)
          Determines which resource the job might need to access the file.
 void setBase(int base)
          Changes the base value for the resource value formula.
 void setDataVaultPrefix(java.lang.String dataVaultPrefix)
          Changes the prefix of a NFS file.
 void setFileFactor(int fileFactor)
          Changes the fileFactor value for the resource value formula.
 void setMax(int max)
          Changes the max value for the resource value formula.
 void setRemoveCharactersFromVaultNumber(boolean removeCharactersFromVaultNumber)
          Set to true removes all the characters from the vault number
 void setResourceNameSyntax(java.lang.String resourceNameSyntax)
          Changes the syntax to be used to convert a data vault number to the LSF resource.
 void setSwitchSyntax(java.lang.String switchSyntax)
          Changes the syntax that will be used to prepare the final switch.
 void setTrimVaultNumberDecimal(boolean trimVaultNumberDecimal)
          Setter for property trimVaultNumberDecimal.
 void setTrimVaultNumberLeadingZeros(boolean trimVaultNumberLeadingZeros)
          Set to true if the leading zeros of a vault number will be removed when creating the corresponding LSF resource.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

base

protected int base
Base value for the resource value formula.


fileFactor

protected int fileFactor
FileFactor value for the resource value formula.


max

protected int max
Max parameter for the standard increaseResourceUsage implementation.

Constructor Detail

LSFResourceStrategy

public LSFResourceStrategy()
Method Detail

getBase

public int getBase()
Returns the base value for the resource value formula.


setBase

public void setBase(int base)
Changes the base value for the resource value formula.


getFileFactor

public int getFileFactor()
Returns the fileFactor value for the resource value formula.


setFileFactor

public void setFileFactor(int fileFactor)
Changes the fileFactor value for the resource value formula.


getMax

public int getMax()
Returns the max value for the resource value formula.


setMax

public void setMax(int max)
Changes the max value for the resource value formula.


getDataVaultPrefix

public java.lang.String getDataVaultPrefix()
Returns the prefix with which a file on NFS identifies on which file server or data vault is residing. For example, "/star/data07/muDST/..." represents a file on data vault "07" (that is the data vault number or id), therefore the prefix is "/star/data".

Returns:
The prefix that identifies data vaults on NFS.

setDataVaultPrefix

public void setDataVaultPrefix(java.lang.String dataVaultPrefix)
Changes the prefix of a NFS file. The resource identifies the data vault number by looking at a prefix in the file name. For example "/star/data07/minbias/..." would have "/star/data" as a prefix.


getResourceNameSyntax

public java.lang.String getResourceNameSyntax()
Returns the syntax to be used to convert a data vault number to the LSF resource.


setResourceNameSyntax

public void setResourceNameSyntax(java.lang.String resourceNameSyntax)
Changes the syntax to be used to convert a data vault number to the LSF resource. It can be any string, and it should contain the $vaultNumber variable, which will be substituted with the particular vault number. For example, to have something like "dv7io", you should have "dv$vaultNumberio".


isTrimVaultNumberLeadingZeros

public boolean isTrimVaultNumberLeadingZeros()
Returns true if the leading zeros of a vault number will be removed when creating the corresponding LSF resource.


setTrimVaultNumberLeadingZeros

public void setTrimVaultNumberLeadingZeros(boolean trimVaultNumberLeadingZeros)
Set to true if the leading zeros of a vault number will be removed when creating the corresponding LSF resource. Basically, 07 will be changed to 7.


prepareResourceName

protected java.lang.String prepareResourceName(java.lang.String vaultNumber)
Given the vault id, returns the LSF resource name to be used in the -R parameter.

Parameters:
vaultNumber - the vault id, as identified in getDataVaultPrefix
Returns:
the LSF resource name for the vault

getSwitchSyntax

public java.lang.String getSwitchSyntax()
Returns the syntax that will be used to prepare the final switch


setSwitchSyntax

public void setSwitchSyntax(java.lang.String switchSyntax)
Changes the syntax that will be used to prepare the final switch. You can use the $definedNames, $nameEqualValueCommaSeparated or $nameEqualValueColumnSeparated variables in your switch. For example, the following switch: "\"select[$definedNames] rusage[$nameEqualValueCommaSeparated]\"" will produce something like: "select[defined(01)] rusage[01=50]"


prepareSwitch

protected java.lang.String prepareSwitch()
Prepares the -R parameters to give to LSF at the end of the resource distribution. It will use definedNames(), nameEqualValueCommaSeparated() or nameEqualValueColumnSeparated() to prepare it more easily.

Returns:
the -R parameter for the current job

increaseResourceUsage

protected void increaseResourceUsage(java.lang.String resource)
Increase the resource usage as part of the standard implementation. The standard formula starts counting from base, adds fileFactor for each usage until max is reached. In pseudo-code: if (nRes == 0) nRes = base; nRes += nRes; if (nRes > max) nRes = max;

One can modify the three protected parameters (base, fileFactor, max) to tune the usage.

Parameters:
resource - the resource for which to increase the counting.

addResource

protected void addResource(java.lang.String resName,
                           int base)
Addes a new resource, and initializes the value to base.

Parameters:
resName - the name of the resource
base - the initial value for the resource

increaseResource

protected int increaseResource(java.lang.String resName,
                               int base,
                               int amount)
Increases the value associated to the given resource by a given amount. If the resource doesn't exists, first it will be created and initialized.

Parameters:
resName - the name of the resource
base - the initial value if the resource has to be created
amount - the amount to increase the resource by
Returns:
the new value associated to the resource

increaseResourceUptoLimit

protected int increaseResourceUptoLimit(java.lang.String resName,
                                        int base,
                                        int amount,
                                        int limit)
Increases the value associated to the given resource by a given amount. If the resource doesn't exists, first it will be created and initialized. If the increased amount is over the limit, the value will be set to the limit.

Parameters:
limit - upper limit to which the resource value will be set
resName - the name of the resource
base - the initial value if the resource has to be created
amount - the amount to increase the resource by
Returns:
the new value associated to the resource

prepareResourceValue

protected int prepareResourceValue(java.lang.String resName)
Returns the value currently associated to the resource.

Parameters:
resName - resource name
Returns:
resource value

prepareResourceUsageSwitch

public java.lang.String prepareResourceUsageSwitch(Job job)
Returns the -R parameter to be used to dispatch the given job.

Parameters:
job - the job to be dispatched
Returns:
the -R parameter to use

initResources

protected void initResources()
Initializes the resource table.


nameEqualValueCommaSeparated

protected java.lang.String nameEqualValueCommaSeparated()
Returns a string already formatted containing names and values comma separated (i.e. "res1=34,res2=12").

Returns:
resource name/value pairs comma separated (i.e. "res1=34,res2=12")

nameEqualValueColumnSeparated

protected java.lang.String nameEqualValueColumnSeparated()
Returns a string already formatted containing names and values column separated (i.e. "res1=34:res2=12").

Returns:
resource name/value pairs column separated (i.e. "res1=34:res2=12")

definedNames

protected java.lang.String definedNames()
A list of names, comma separated, of all the resources used.

Returns:
(i.e. "res1,res2")

calculateLSFResources

protected void calculateLSFResources(Job job)
Filles the resource table with all the resources. It basically iterates over the input files and calls increaseResourceUsage for each file that map to a resource.

Parameters:
job - the job

resourceRequired

protected java.lang.String resourceRequired(PhysicalFile file)
Determines which resource the job might need to access the file.

Parameters:
file - input file the job will be using
Returns:
the resource needed

determineResourceNotUsed

protected boolean determineResourceNotUsed()
Determines whether any resources at all have been assigned.

Returns:
true if any resource is used.

isTrimVaultNumberDecimal

public boolean isTrimVaultNumberDecimal()
Getter for property trimVaultNumberDecimal.

Returns:
Value of property trimVaultNumberDecimal.

setTrimVaultNumberDecimal

public void setTrimVaultNumberDecimal(boolean trimVaultNumberDecimal)
Setter for property trimVaultNumberDecimal.

Parameters:
trimVaultNumberDecimal - New value of property trimVaultNumberDecimal.

isRemoveCharactersFromVaultNumber

public boolean isRemoveCharactersFromVaultNumber()
If true removes all the characters from the vault number.

Returns:
Value of property removeCharactersFromVaultNumber.

setRemoveCharactersFromVaultNumber

public void setRemoveCharactersFromVaultNumber(boolean removeCharactersFromVaultNumber)
Set to true removes all the characters from the vault number

Parameters:
removeCharactersFromVaultNumber - New value of property removeCharactersFromVaultNumber.


Copyright © 2002-2004 STAR collaboration - Brookhaven National Laboratory. All Rights Reserved.