MAPMT FEE ALARMS/RECOVERY:

  1. Individual box status/alarm

    In general do not stop ongoing runs for individual problems. Take note and begin to investigate as follows (Note: the following actions interrupt the DAQ function of a MAPMT box!):

    1. If the configuration or communication indicator fails (RED) for individual boxes:

      1. Pull up the individual MAPMT box screen and try to reset the box by reloading FPGAs (lower right button) ... follow this by "load", "write" and "read". It may be necessary to repeat a few times.

      2. If errors continue (e.g., HDLC communication) it is very likely that the box has lost its initial configuration download and this will have to be downloaded again.

        • One can check if this is really the case by hitting the "reload FPGA" button on the MAPMT box level GUI display while checking the output to the VxWorks/IOC monitor. If the last line it spits out contains "DONE=0", the box requires download. (equivalently, at the monitor prompt, type: smdReConfig "channel","node").
        • The download (need "magic code") can be accomplished with the "mapmtDownload" buttons at the bottom of the individual MAPMT box screen. In general it is the rdo.bit file that needs downloading and this will be sufficient (a separate "mapmt_fee.bit" is usually not required). The download takes several minutes (one can chart progress on the VxWorks monitor) ... at the GUI, indication the download is done is that the box monitoring process starts again.
        • After download, go back to sequence 1.1 above: "reload FPGAs", "load", "write", "read". Box should now configure without errors and status will turn green with configuration verified.

  2. Multiple box status/alarm or power supply alarm

    1. Check that power to the sector boxes is on as indicated in the left portion of the MAPMT GUI (or via the POWER/CANBus screen). Turn supplies back on if found off:

      • dual supply #97 - MAPMT sectors 12-3
      • dual supply #96 - MAPMT sectors 4-7
      • dual supply #98 - MAPMT sectors 8-11

    2. Table of sector MAPMT boxes and associated dual power supplies:
      Supply #97 Supply #96 Supply #95
      sectors MAPMT boxes sectors MAPMT boxes sectors MAPMT boxes
      12 12S1-3; 12P1 4 4S1-3; 4P1 8 8S1-3; 8P1
      1 1S1-3; 1P1 5 5S1-3; 5P1 9 9S1-3; 9P1
      2 2S1-3; 2P1 6 6S1-3; 6P1 10 10S1-3; 10P1
      3 3S1-3; 3P1 7 7S1-3; 7P1 11 11S1-3; 11P1

    3. Reconfigure all boxes that have had power restored

      1. Do "Master Reload" to restart everything correctly (Note: master reload just the relevant 1/3 of the sectors corresponding to a single supply trip is ok -- see above table).
        -- observe HDLC/config chatter on GUI to chart progress

      2. If all supplies power cycled, use a global MAPMT "Master Reload" -- (it will take ~ 7.5 minutes)
        -- observe HDLC/config chatter on GUI to chart progress

      3. For boxes that give errors on configure -- see details above in 1.1 (Note: similar instructions are in the Normal Ops/mapmt trouble section).

  3. Temperature alarm -- readings of the individual box (water cooled) center plates

    1. If temperature is high for all boxes -- likely water chiller failure (either chiller has shut off or reservoir too low)
      1. Boxes have internal thermal interlock (trip temperature reached ~ 30-40 minutes after cooling loss) so in principle will shut down ... but
      2. (can check chiller reservoir level indicator via remote camera at upper right STAR magnet console -- west side, position 1)
      3. Best to turn off power to crates and investigate with (experts) access ASAP -- if a water leak has developed we want FEE powered down and chiller off

    2. Temperature is high for a (one) box
      1. Either abnormal electronics operation or blockage of individual box cooling
      2. Expert help will be likely needed to secure the affected box while keeping the others powered

  4. Controls (VxWorks IOC) failure/hangup -- GUI "all white"

    If this occurs, likely other systems will be affected as well. See the Controls Recovery & Restart section for the details of the MAPMT FEE recovery.

  5. Note: when MAPMT power supplies are cycled or restored -- for whatever reason

    1. All affected MAPMT boxes (remember the supplies service 16 boxes) must be re-configured -- follow instructions above in this section