Online Histogram Procedures

(AKA running the Panitkin Plots)

The Online Histograms have three components that need to be running.   Each component runs on the computer "evp.starp.bnl.gov" under the "operator" account.

• The EVP.   This runs as part of DAQ/RTS.
  • The EVP is enabled by default by the run control GUI, but if it is removed via the component tree, no data will be saved for the online histograms.
• The Online Histogram Server  (pplotServer)
  • The Server creates, and save histograms to the STAR Run Log automatically.  If it is running properly there doesn't need to be any intervention by the shift crew.
• The Online Histogram Presenter  (pplotClient)
  • The Histogram Presenter is the user interface which allows the shift crew to see the histograms as they are built.  The presenter can be killed, and restarted, or multiple instances can be running and this does not affect the histograms (for good or ill) the histograms archived in the Run Log.

Starting The Histogram Presenter:

The Histogram Presenter runs on "rts01.starp.bnl.gov".   Typically, it should be up at the start of the shift.  The shift operator doesn't need to control the presenter, except as required to select which plots to look at.

Restart Procedure #1:

1. Double Click the "Online Histograms" icon on the "rts01.starp.bnl.gov" desktop.

Restart Procedure #2:

1. From a terminal type:

         > ssh evp.starp.bnl.gov presenter.csh

Starting the Histogram Server

If all is well, the Histogram Server should never need to be restarted.  However, if necessary (see the debugging tree below) follow these steps:

1. Log on to evp.starp.bnl.gov

        > ssh operator@evp.starp.bnl.gov

2. Kill any existing server processes

        > killall pplotMother.sh

        > killall pplotServer

3. Restart the server

        > pplotMother.sh

4. Check that the server is running

        > ps -ef | grep pplot

      If all is well, you should see something very like:

operator 23427 1 0 11:28 ? 00:00:00 /bin/tcsh pplotMother.sh
operator 23448 23427 0 11:28 ? 00:01:42 /RTS/bin/LINUX/i686/pplotServer -live -start -end -nogui
operator 25691 25675 0 13:14 ? 00:00:07 pplotClient
operator 25826 25803 0 14:21 pts/5 00:00:00 grep pplot
 

Starting EVP

The EVP process runs as part of the DAQ system.  To restart this system select "reboot DAQ" or "reboot ALL" from the run control.

Debugging

• If the Histogram Presenter GUI is unresponsive, it is most likely the presenter.  Simply kill the presenter and restart it following the procedures above.

• If the run number and/or event number does not update in the Histogram Presenter when a new run starts, it is for one of the following reasons:

  1. The presenter is not set to read the live data.  This can happen if the the LIVE button is unclicked by a user to examine histograms without them change, or else if the Histogram Server is restarted while the Presenter is already alive.    To fix this, simply click the LIVE button on the histogram presenter.

  2. The pplotServer is not running.  Perform step #4 of the "Starting the Histogram Server" directions.   If the server is not running, restart it.

  3. There are no events being saved in the event pool.   To check this:

     1. Log on to evp.starp.bnl.gov:

        > ssh evp

     2. Check the disk for the run directory:

        > ls -ltra /a

     3. The last directory should contain the latest run

     4. Check the run directory for events

        > ls -ltra /a/#####/

     5. If there is no run directory, or if there are only a few events in the run, the data is not being saved to the event pool.   Fix this by stopping the run and rebooting DAQ.  

  4. The pplotServer is running but is in a messed-up state.  Restart the server.

• The Histograms are not being saved in the Runlog, but they are displayed on the presenter.  In this case, the problem is either a database problem or a serve problem.  Notify the histogram expert and the database expert by email.  As long as the data are available on disk, the RunLog histograms can be rebuilt, so this is not an immediate emergency.