The Queue Monitoring MonALISA Module

Introduction
Installation of the Module
Accessing the Cluster Summaries

Introduction

Information about the status of Local Resource Managment Systems (LRMS) is needed by Resource Brokers for decision making mechanisms and for the implementation of global policies. Resource Brokers, however, do not have access to LRMS and they cannot affect local policies. A mechanism is needed that will provide the aggregate status of a queuing System.

We are developing a Queue Monitoring Module for the MonALISA monitoring Framework. This custom module will provide aggregate status info for the most popular queuing systems (CONDOR, LSF, PBS, SGE) using the same attributes.

We aim to make the provided info compatible with the GLUE Schema. In particular, the Computing Element (CE) of the GLUE Schema represents the entry point to a Queue. There is one Computing Element per Queue.

The following table lists the attributes of the Status object of the CE Element:

Attribute Description
RunningJobs Number of currently running Jobs

WaitingJobs Number of jobs that are in a state other that running

TotalJobs Total Number of Jobs in the CE (Running + Waiting)

Status States a Queue can be in (Production, Closed, Queing, ...)

WorstResponseTime Worst time between job submission till when job starts its execution (in secs)

EstimatedResponseTime Estimated time between job submission till when job starts its execution (in secs)

FreeCPUs Number of Free CPUs available to the Scheduler

Attribute	Description
RunningJobs	Number of currently running Jobs
WaitingJobs	Number of jobs that are in a state other that running
TotalJobs	Total Number of Jobs in the CE (Running + Waiting)
Status	States a Queue can be in (Production, Closed, Queing, ...)
WorstResponseTime	Worst time between job submission till when job starts its execution (in secs)
EstimatedResponseTime	Estimated time between job submission till when job starts its execution (in secs)
FreeCPUs	Number of Free CPUs available to the Scheduler

A presentation with ideas on Queue Monitoring was given at the PPDG collaboration Meeting and is available here .

In developing the ML Queue Monitoring Module we will take into account previously done work by others. In particular, the EDG has made available an MDS information provider that calculates the same attributes. In a way, our goal is to convert the MDS IP into a MonaLisa Monitoring Module.
There is also some work done in providing attributes of the LSF and PBS systems to MonALISA. Although the attrbutes provided are not GLUE-like, we certainly look into this work too.

Installation of the Queue Monitoring ML module

Download the tar file with the ML module queueMonitor.tar and untar it under the usr_code directory:

                     cd $MonaLisa_HOME/Service/usr_code
                     tar -xvf queueMonitor.tar

A new directory, under usr_code, will be created (queueMonitor/) that contains the ML module queueMon.java, an interface BatchInt.java that defines constants, a script comp to compile the module if you make changes to it and a script test_run to run the module from the command line (for testing purposes) before implementing it with the ML monitoring Service.
Edit the BatchInt.java interface to change the parameters, such as the path LSF_BIN, queue names for Condor etc. Then recompile by running the comp script.
As with any other ML custom module, you must include the directory where you installed the module (in our case $MonaLisa_HOME/usr_code/queueMonitor/) in the lia.Monitor.CLASSURLs property. This property is set in the ml.properties file. If you are using other custom modules you must separate the directories with commas. Do not forget the "/" at the end of the directory.
```
lia.Monitor.CLASSURLs=file:${MonaLisa_HOME}/Service/usr_code/LSF/,file:${MonaLisa_HOME}/Service/usr_code/queueMonitor/
                  
```
Edit the < farm_name >.conf config file and add a new cluster:
```
    *QueueMonitor{queueMon, localhost, LSF,CONDOR}%30
             
```
The last two entries in the curly braces (LSF,CONDOR) are the Batch systems that you want to monitor their queues. The available options are LSF, CONDOR, PBS, SGE. Remember, though, that currently we only have implemented queue monitoring for LSF and CONDOR. The names of the Batch Systems are NOT case sensitive and (if you use more that one) they must be separated by commas.

Restart MonALISA:

                   cd $MonaLisa_HOME/Service/CMD
                   ./ML_SER restart

Accessing the Cluster Summaries

Since we run the MonALISA service as a web service you may access the information we publish in MonALISA using the Web Service Clients. You may also access the same informaton and make plots using the Global MonALISA client.

Stratos Efstathiadis - page was last modified