JDL – Comparisons

1) Introduction

1.1 XML Short-hand

The job submission languages included here are summarized by simply taking the most recent documented representation (typically found directly in XML) and writing it out here in a quasi-xml way. An example would be:

<JobTitle>My Example Job</JobTitle>

</Header>

<ExtendDescription>Extends the SomeDescription.</ExtendDescription>

</ JobDescription >

In this document, I use a short-hand notation to write this out in a way that shows the heirarchy and the differences between elements and attributes of elements, as well as which elements are extended from which base elements. Thus, this example XML would appear as:

JobDescription

Header

JobTitle

SomeDescription { descriptionAttribute=[xs:string] }

SomeDescription/ExtendedDescription

2) Summary of Languages

2.1 Job Submission Description Language (JSDL)

Latest Info: Spec 0.4.4 draft July 2004

Names: Ali Anjomshoaa, Fred Brisard, An Ly, Stephen McGough, Darren Pulsipher, Andreas Savva (ed)

General Description: Global Grid Forum specification from JSDL-Working Group of the Scheduling and Resource Management Area.

job

JobIdentification

JobName

JobAnnotation

ExecutionUserID

User { name=[xs:string], group=[xs:string] }

JobProjectName

JobCategory

Extend { name=[xs:string] }

Resource

CPUDescription

CPUCount

CPUSpeed

PhysicalMemory

ProcessTopology

Environment

EnvironmentVariable { name=[xs:string] }

SoftwareRequirements

OperatingSystem

OperatingSystemVersion

OperatingSystemDescription

Limits

ProcessVirtualMemoryLimit

VirtualMemoryLimit

DataSegmentSizeLimit

CoreDumpSizeLimit

CPUTimeLimit

WallTimeLimit

Queue

Application

ExecutableDescription

ExecutableName

Argument

StdIn

StdOut

StdErr

WorkingDirectory

Log

Host { name=[xs:string], description=[xs:string] }

HostName

ExecutionUserID

Resource

ProcessTopology

Environment

SoftwareRequirements

HostGroup { name=[xs:string], description=[xs:string] }

HostName

ExecutionUserID

Resource

CPUDescription

CPUCount

CPUSpeed

PhysicalMemory

Network

NetworkDescription

NetworkCount

NetworkBandwidth

FileSystem { name=[xs:string], size=[xs:string] }

SwapSpace

TemporarySpace

ExclusiveExecution

ProcessTopology

Environment

SoftwareRequirements

DataAttributes

File

FileName

FileSystemName

Source

CreationFlag

Target

2.2 Analysis Job Description Language (AJDL)

Latest Info: ADJL: Analysis Job Description Language version0.3 March 2004

Names: David Adams

General Description: Proposal for a new language for general submission of requests for jobs by users, but specially designed for the ATLAS experimental data.

Job

Application { name=[xs:string], version=[xs:string] }

Task { id=[xs:string] }

TextFiles

TextFile { name=[xs:string] }

NamedLogicalFiles

NamedLogicalFile { name=[xs:string] }

LogicalFile { catalog_type=[xs:string],

catalog_name=[xs:string], file_id=[xs:string] }

BaseDataset { identity=[xs:string], mutability=[locked,appending],

location=[virtual,logical,physical,staged,mixed],

composite=[xs:boolean] }

Contentlist

Content { type=[xs:string], key=[xs:string] }

BaseDataset/EventDataset

BaseDataset

EventDatasetData

EventIdList { count=[xs=int] }

EventId { id=[xs:string] }

BaseDataset/LogicalFileDataset

BaseDataset

LogicalFileDatasetData { id=[xs:string] }

FileCatalog { type=[xs:string], name=[xs:string] }

BaseDataset/CompositeDataset

BaseDataset

CompositeDatasetData

DatasetIdlist

DatasetId { id=[xs:string] }

JobConfiguration

2.3 User Request Description Language (URDL)

Latest Info: User JDL- Draft version0.4 September 2003

Names: Gabriele Carcassi, Machael Haddox-Schatz, Andy Kowalski, Jerome Lauret

General Description: Proposal for a new language for general submission of requests for jobs by users, but specially designed for the STAR and JLab experiments.

Request

Application { name=[xs:string], ver=[xs:string] }

Task { type=[xs:string] }

Task/rootTask

macro { file=[xs:string], arguements=[xs:string] }

stdin { file=[xs:string] }

stdout { file=[xs:string] }

stderr { file=[xs:string] }

Task/javaTask

mainclass { class=[xs:string] }

classpath

pathelement { location=[xs:string], path=[xs:string] }

Task/scriptTask

script { file=[xs:string], arguements=[xs:string] }

stdin { file=[xs:string] }

stdout { file=[xs:string] }

stderr { file=[xs:string] }

Dataset { name=[xs:string] }

Dataset/catalogDataset

catalog

query

Dataset/logicalDataset

catalog

file

Dataset/physicalDataset

file

Extra

2.4 Job Description Language (JDL)

Latest Info: DataGrid JDL Attributes (DataGrid-01-TEN-0142-0_2) October 2003

Names: Fabrizio Pacini

General Description: This note provides the description of JDL attributes supported by the EDG WMS release-2 software.

Type = “Job”;

JobType = “[Normal,Interactive,Checkpointable,MPICH,Partitionable]”;

Executable = “string”;

Arguments = “string”;

StdInput = “string”;

StdOutput = “string”;

StdError = “string”;

InputSandbox = {“string”, “string”, ...};

OutputSandbox = {“string”, “string”, ...};

Environment = {“SOME_VARIABLE=value”, “SOME_VARIABLE=value”, ...};

InputData = {“string”, “string”, ...};

DataAccessProtocol = {“string”, “string”, ...};

OutputSE = “string”;

OutputData = {[OutputFile="string"; LogicalFileName=”string";],

[OutputFile="string"; LogicalFileName=”string";], ...}

VirtualOrganisation = “string”;

RetryCount = integer;

MyProxyServer = “string”;

HLRLocation = “string”;

NodeNumber = integer;

JobSteps = integer;

CurrentStep = integer;

ListnerPort = integer;

Requirements = other.someStringVar=="string" && other.someIntVar>2 &&

Member("string",other.someStringVar);

Rank = other.someNumericVar;

FuzzyRank = [true,false];

XML for the DataGrid JDL is simply the ClassAd XML (see http://www.cs.wisc.edu/condor/classad/). The ClassAd XML representation is the following...

classads

a { n=[xsd:string] }

b { v=[t,f] }

er { a=[xsd:string] }

un { a=[xsd:string] }

classads = root container. list of records, lists, literals, or values

c = classad record that contains a named attribute

a = named attribute. can be a record, list, literal, or value

l = list of records, attributes, literals, or values

e = expression, xsd:string

s = string literal, xsd:string

i = integer literal, xsd:int

r = real literal, xsd:double

b = binary literal, “t” or “f”

er = error value with annotation

un = undefined value with annotation

at = absolute time literal, xsd:dateTime

rt = relative time literal, xsd:duration

EXAMPLE:

<c>

<a n="VirtualOrganisation"><s>biome</s></a>

<a n="Arguments"><s>./script000.csh</s></a>

<a n="StdOutput"><s>outfile</s></a>

<a n="StdError"><s>errfile</s></a>

<s>outfile</s>

<s>errfile</s>

<s>Brain_radioth000.root</s>

</l></a>

<s>lfn:BrainTotal</s>

</l></a>

<s>gridftp</s>

</l></a>

<a n="JobType"><s>normal</s></a>

<s>/home/fpacini/NEWUI/bin/script000.csh</s>

<s>/home/fpacini/JOBS/jobsRAL/macro/macro000.mac</s>

<s>/home/fpacini/JOBS/jobsRAL/status/status000.rndm</s>

<s>/home/fpacini/JOBS/jobsRAL/required/prerunGate.mac</s>

<s>/home/fpacini/JOBS/jobsRAL/required/rangeInterfile2.dat</s>

<s>/home/fpacini/JOBS/jobsRAL/required/CJP_BrainTOT</s>

<s>/home/fpacini/JOBS/jobsRAL/required/GateMaterials.db</s>

</l></a>

<a n="Environment"><s>GATE_ROOT=/usr/local/gate</s></a>

<a n="rank"><e>(-other.GlueCEStateEstimatedResponseTime)</e></a>

<a n="requirements"><e>(((((((Member("GATE-1.0-3",other.GlueHostApplicationSoftwareRunTimeEnvironment)&&(other.GlueCEStateFreeCPUs>=2))&&(other.GlueCEStateStatus=="Production"))&&(other.GlueCEStateStatus=="Production"))&&(other.GlueCEStateStatus=="Production"))&&(other.GlueCEStateStatus=="Production"))&&(other.GlueCEStateStatus=="Production"))&&(other.GlueCEStateStatus=="Production"))</e></a>

</c></classads>

2.5 Job Description Language (JDL)

Latest Info: Job Description Language Attributes July 2003

Names: Andy Kowalski ?

General Description: JLab’s take on the JDL used by the DataGrid project.

ProjectName = “string”;

JobName = “string”;

Command = “string”;

OSName = “string”;

OSRelease = “string”;

Processor = “string”;

Memory = integer;

DiskSpace = integer;

Application / version = {“string/string”, “string/string”, ...}

CPUTime = integer;

JobType = "string";

CommandFile = “[URL,string]”;

InputFile = “[URL,string]”;

OutputFile = “[URL,string]”;

StatusNotify = “URL”;

Stdin = “URL”;

Stdout = “URL”;

Stderr = “URL”;

UserName = “string”;

ExperimentSpecification = “[URL,uJDL]”;

The JLab JDL also uses the ClassAds XML description.

3) XML Style and Authoring Issues

3.1 Case Sensitivity of XML

(from http://www.wdvl.com/Authoring/Languages/XML/XMLFamily/XMLSyntax/sall2_2.html)

When creating your own XML vocabulary, it would be desirable if there were conventions to explain the use of uppercase, lowercase, mixed case, and underscores, and hyphens. Unfortunately, no such conventions exist in XML 1.0. It is a good idea to adopt your own conventions and to apply them consistently, at least across your project, but ideally throughout your entire organization.

For example, for element names I prefer using what is often called CamelCase because the initial letter of each word in a multiword name is uppercase and all others are lowercase, creating humps like a camel's back. (It's also sometimes called TitleCase because it resembles the title of a book.) For example:

Note that for attributes, I also use CamelCase, except the first word is always begun with a lowercase letter, as in " countryCode". In fact, the terms UpperCamelCase (as I use for elements) and lowerCamelCase (as I use for attributes) are often used to make this distinction more clear. One reason that I favor this convention is that in any context (including documentation), it's easy to distinguish elements from attributes.

3.2 Use of Attributes vs. Elements

(from http://www.w3schools.com/xml/xml_attributes.asp)

Should you avoid using attributes?

Here are some of the problems using attributes:

attributes cannot contain multiple values (child elements can)
attributes are not easily expandable (for future changes)
attributes cannot describe structures (child elements can)
attributes are more difficult to manipulate by program code
attribute values are not easy to test against a Document Type Definition (DTD) - which is used to define the legal elements of an XML document

If you use attributes as containers for data, you end up with documents that are difficult to read and maintain. Try to use elements to describe data. Use attributes only to provide information that is not relevant to the data.

My rule about attributes has one exception:

Sometimes I assign ID references to elements. These ID references can be used to access XML elements in much the same way as the NAME or ID attributes in HTML. This example demonstrates this:

<heading>Reminder</heading>

<body>Don't forget me this weekend!</body>

</note>

<heading>Re: Reminder</heading>

</note>

</messages>

Appendix

A.1 ClassAd Schema

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema">

<xsd:annotation><xsd:documentation xml:lang="en">

Schema for Classified Advertisements (classads) as used in the

Condor High Throughput Computing progject.

</xsd:documentation></xsd:annotation>

<xsd:element name="classads">

<xsd:annotation><xsd:documentation xml:lang="en">

The root document element. It is identical to an "l" element

except for the tag: Its content is simply a sequence of

expressions.

</xsd:documentation></xsd:annotation>

<xsd:complexType>

<xsd:sequence>

<xsd:choice minOccurs="0" maxOccurs="unbounded">

<xsd:element ref="c"/>

<xsd:element ref="l"/>

<xsd:element ref="e"/>

<xsd:element ref="s"/>

<xsd:element ref="i"/>

<xsd:element ref="r"/>

<xsd:element ref="b"/>

<xsd:element ref="er"/>

<xsd:element ref="un"/>

<xsd:element ref="at"/>

<xsd:element ref="rt"/>

</xsd:choice>

</xsd:sequence>

</xsd:complexType>

</xsd:element>

<xsd:element name="s" type="xsd:string">

<xsd:annotation><xsd:documentation xml:lang="en">

A string literal.

Content is the string itself. It may contain character entities,

such as &lt;. It may also contain backslash escapes, which

are interpreted by the application.

</xsd:documentation></xsd:annotation>

</xsd:element>

<xsd:element name="i" type="xsd:int">

<xsd:annotation><xsd:documentation xml:lang="en">

An integer literal.

Content is the value, as a sequence of digits.

</xsd:documentation></xsd:annotation>

</xsd:element>

<xsd:element name="r">

<xsd:annotation><xsd:documentation xml:lang="en">

A real (double-precision floating point) literal.

Content is the value in "scientific" notation corresponding to

the printf %1.5E format, or one of the special values INF, -INF, or NaN.

</xsd:documentation></xsd:annotation>

<xsd:simpleType>

<xsd:restriction base="xsd:double">

<xsd:pattern value="-?INF|NaN|-?\d\.\d{15}E(\+|\-)\d{2,3}"/>

</xsd:restriction>

</xsd:simpleType>

</xsd:element>

<xsd:element name="b">

<xsd:annotation><xsd:documentation xml:lang="en">

A boolean literal. Represented as an element with empty content

and required attribute v="t" or v="f".

Content must be empty.

</xsd:documentation></xsd:annotation>

<xsd:complexType>

<xsd:attribute name="v">

<xsd:simpleType>

<xsd:restriction base="xsd:string">

<xsd:enumeration value="t"/>

<xsd:enumeration value="f"/>

</xsd:restriction>

</xsd:simpleType>

</xsd:attribute>

</xsd:complexType>

</xsd:element>

<xsd:element name="er">

<xsd:annotation><xsd:documentation xml:lang="en">

An error value. The "a" attribute, if present, is an annotation

(reason for the error).

Content must be empty.

</xsd:documentation></xsd:annotation>

<xsd:complexType>

<xsd:attribute name="a" type="xsd:string"/>

</xsd:complexType>

</xsd:element>

<xsd:element name="un">

<xsd:annotation><xsd:documentation xml:lang="en">

An undefined value. The "a" attribute, if present, is an

annotation (perhaps the name of the classad attribute that was

undefined).

Content must be empty.

</xsd:documentation></xsd:annotation>

<xsd:complexType>

<xsd:attribute name="a" type="xsd:string"/>

</xsd:complexType>

</xsd:element>

<xsd:element name="at">

<xsd:annotation><xsd:documentation xml:lang="en">

An absolute time literal.

Content is yyyy:mm:ddThh:mm:ssphh:mm where p is + or -, other

lower-case letter represent arbitrary digits, and the remaining

characters must appear exactly as shown.

Note that the time zone is required and it contains a colon.

</xsd:documentation></xsd:annotation>

<xsd:simpleType>

<xsd:restriction base="xsd:dateTime">

<xsd:pattern value=

"\d{4}-\d\d-\d\dT\d\d:\d\d:\d\d[+\-]\d\d:\d\d"/>

</xsd:restriction>

</xsd:simpleType>

</xsd:element>

<xsd:element name="rt">

<xsd:annotation><xsd:documentation xml:lang="en">

A relative time literal.

Content is -PnDTnHnMn.dddS where "n" represents one or more digits

and "d" represents one digit. The leading sign and individual

components (nD, nH, etc.) may be omitted, but if present, they

must appear in the order indicated. The 'T' is omitted if nD is the

only field. The fractional seconds, together with the decimal

point, may be omitted, but if present there must be exactly three

digits. Note, the pattern below does not completely specify all the

constraints. For example, the T must be omitted if and only if the

H, M, and S fields are all omitted, and the attribute may not be

completely empty.

</xsd:documentation></xsd:annotation>

<xsd:simpleType>

<xsd:restriction base="xsd:duration">

<xsd:pattern value="-?P(\d+D)?T?(\d+H)?(\d+M)?(\d+(\.d\d\d)?S)?"/>

</xsd:restriction>

</xsd:simpleType>

</xsd:element>

<xsd:element name="e" type="xsd:string">

<xsd:annotation><xsd:documentation xml:lang="en">

An expression.

Content is an arbitrary classad expression in the "native syntax".

This element should not be used for literals or if the "top-level

operator" is list or classad.

</xsd:documentation></xsd:annotation>

</xsd:element>

<xsd:element name="l">

<xsd:annotation><xsd:documentation xml:lang="en">

A list. Content is a sequence of arbitrary expressions.

</xsd:documentation></xsd:annotation>

<xsd:complexType>

<xsd:sequence>

<xsd:choice minOccurs="0" maxOccurs="unbounded">

<xsd:element ref="c"/>

<xsd:element ref="l"/>

<xsd:element ref="e"/>

<xsd:element ref="s"/>

<xsd:element ref="i"/>

<xsd:element ref="r"/>

<xsd:element ref="b"/>

<xsd:element ref="er"/>

<xsd:element ref="un"/>

<xsd:element ref="at"/>

<xsd:element ref="rt"/>

</xsd:choice>

</xsd:sequence>

</xsd:complexType>

</xsd:element>

<xsd:element name="c">

<xsd:annotation><xsd:documentation xml:lang="en">

A "classad" (a record). Content is a sequence of "a" elements.

</xsd:documentation></xsd:annotation>

<xsd:complexType>

<xsd:sequence>

<xsd:element name="a" minOccurs="0" maxOccurs="unbounded">

<xsd:complexType>

<xsd:sequence>

<xsd:choice>

<xsd:element ref="c"/>

<xsd:element ref="l"/>

<xsd:element ref="e"/>

<xsd:element ref="s"/>

<xsd:element ref="i"/>

<xsd:element ref="r"/>

<xsd:element ref="b"/>

<xsd:element ref="er"/>

<xsd:element ref="un"/>

<xsd:element ref="at"/>

<xsd:element ref="rt"/>

</xsd:choice>

</xsd:sequence>

<xsd:attribute name="n" type="xsd:string"/>

</xsd:complexType>

</xsd:element>

</xsd:sequence>

</xsd:complexType>

<xsd:unique name="dummy">

<xsd:selector xpath="a"/>

<xsd:field xpath="@n"/>

</xsd:unique>

</xsd:element>

</xsd:schema>