Date:
August 5th - 6th 2004
Attendeee:
David Alexander, Dave Stampf, Levente Hajdu, Jerome Lauret, Valeri
Fine
General
discussion
In this
meeting, Levente first gave a presentation as per the current status
of the SUMS project. The main points related to the u_JDL from the
presentation follows
• Input,
output and, error streams
- Files (output files
/ files to be copied back from temp)
- Files catalog
queries
• Job name
• Resource
Requirements
Memory (min / max)
- Storage space (hard
disk)
CPU run time
- Scheduler adds more
before dispatching (StarData)
• Execution commands
• Type
- Simulated or real
job
What
the U-JDL does NOT define
Clusters
- Queues: this is
defined in SUMS in a configuration file. The main point is that some
queues or requirements may be static to a queue (i.e. hard limits)
and the RMS knows bout that so there was no need to date to
propagate this information via discovery service.
- Dispatching system
(LSF, CONDOR, CONDOR-G). Each queue has a queue type. Each type will
be associated to a specific dispatcher.
- Only one job / UJDL
- Advanced time
reservation
The
major point was that for the resource broker to make decisions,
either the user have to specify rates or ranges for the different
computing elements (memory, CPUtime, maybe clock time, IO,/SE etc
...). For example, specifying the time is an indication to the
resource broker to chose the optimal queue / slot to submit to (used
in SUMS).
The
question was raised on how much of the Glue
CE (click on the
hyperlink for a copy of the document) elements has to be implemented.
We also spoke of the DataSets issues. Since this area is more
developed in the AJDL, we thought to re-use some of its components.
Documentation can be found here.
We also
reviewed the available schemas as per the proposed proposal. David
had prepared a JDL
summary document.
They all have different flavors but are really barely different apart
from the DataGrid JDL allowing some limited depth job relation. We
believed that if we could demonstrate that a schema transformation
(XSLT) could be written for going from an existing to the AHJDL, the
proof would be done that there are equivalent.
Dave
Stampf suggested that instead of defining a schema, why not having
access methods delivering a bunch of properties ?? It was unclear how
this would fit within the tree structure approach needed for a truly
flexible JDL.
We had
some discussions on the WSDL. One issue which came to mind is, let's
suppose that an attribute is missing in an implementation. What do we
do ?? Use the validate() method and check if the client knows about
it then ?? This drove the layer for method calls i.e. validate() is
needed (then one can take action but at least, a warning can be
brought back to the user from the UI stand point). The question if
the request will be honored or not would be a service side decision
(can use default values, ignore it, etc ...).
We
diverged on the definition of tasks versus application lingo. Would
task or ExecutableName/Arguments can cover everything ? Preparation
of a “task” may be as complex as opening a port,
installing something etc ... This is the heart of choosing the task
concept in the initial STAR/JLab document. There, the idea is as
follow:
<rootTask>
<macro>myMacro.C(“arg1”, 10.0)</macro>
</rootTask>
while a a java application (main class, library jars) would present
itself like this
<javaTask>
<mainclass class="MyJob" />
<classpath>
<jar url="file:/~/jars/myjar.jar" />
<jar url="http://www.star.bnl.gov/~carcassi/myotherjar.jar" />
</classpath>
</javaTask>
This is described in the RDL document
which may be found on the PPDG Web site as PPDG-39.
Application and task are separate because the idea was that a task
can be shared by different applications.
WSDL
thoughts
Possible
methods would be “int submit(AbstractJobRequest job)” and
“int estimate(int jobReference)” that return job
reference integers; as well as “boolean execute(int
jobReference).” Tracking could be added later in Phase II with
a method such as “AbstractJobTracking getStatus(int
jobReference)”. Here the “AbstractJobRequest”
object will have a one-to-one correspondence to the AJHDL in that all
the information contained in an instance AJHDL file can be
represented by an instance of the AbstractJobRequest object.
//
AbstractJobHandler.java
public
interface AbstractJobHandler {
int submit(AbstractJobRequest
job);
int estimate(int jobReference);
boolean execute(int
jobReference);
AbstractJobTracking getStatus(int jobReference);
}
Glossary
Terminology
|
Definition
|
CompositeJob
|
|
Job
|
A Super-Job
|
|
|
Create()
|
Prepares a CompositeJob
|
Validate()
|
(Dave thinking it may be part of Create())
|
Estimate()
|
Estimates the length of the job
|
Submit()
|
Submits the job request
|
getStatus()
|
%tage failure, completion
|
getSubJobStatus()
|
|
Stop(),
Pause(), Cancel(), Resume(), Restart(), Requeue(),
Suspend(), Kill()
|
Selector.
|
Destroy()
|
|
Post
meeting
Levente
wrote a document with ideas he had. This was not discussed and left
for the second meeting. The document is available here.