Summary of PWG Review Session, December 14, 1999
Update on previous and new computing issues (see previous PWG summaries of May and July).
Summary by T. Trainor and T. Ullrich
Control of the
new migration based on autoQA results and group discussion has significantly improved the working environment. Standardized notification of library changes has been implemented with good result.
Recent confusion about the state of the CVS library should disappear with the migration of the new DST format and corresponding software to
new. There are apparent problems in defining the current state of the developer version (dev) which must be known for useful debugging and QA. State definition is now being addressed by the QA team. In order to minimize the library workload and ensure optimum library uniformity a move to automated script-based library updates is highly desirable. Updates should soon include Online libraries as well.
This effort has been identified as exclusively HPSS related. Thus, impact of GC on disk-resident analysis, the bulk of offline physics analysis in the first year, is seen to be negligible for the foreseeable future. It would be desirable if resources previously directed toward HPSS-related GC development could be temporarily diverted to immediate year-1 software goals, especially the data catalog and TagDB.
Some MySQL data bases are now coming on line, albeit much delayed from expectations formed in May, July and October. DBs continue to be on several software critical paths. It is required now to prioritize DB effort to insure that the most essential elements are in place for first collisions. The TagDB is a critical item for offline analysis, as are calibration and geometry DBs needed for tracking.
Good progress on several fronts: improved consistency between GSTAR model and apparatus, consistent definition of hits for difficult track geometries, updated event menu for MDC3 production, increasing use of mevsim and several standard track densities for QA and general tracking diagnostics.
Great strides in this area in recent months. Script-based monitoring of library software quality has been the major recent emphasis, with autoQA of small nightly productions. In the near future this system must be expanded to general DST production QA with quality-cut criteria defined by experts and PWGs, and to Online QA for monitoring STAR detector performance via DST output. A major near-term goal is to encourage use of the autoQA mechanism routinely in code development by individual workers <before> CVS submissions to speed code development and improve library quality.
Offline analysis production management
PWG operation on CAS requires a job control package which integrates data cataloguing, maker chain configuring and load balancing. An approximation to this facility has recently been implemented by Torre Wenaus, which does not yet incorporate load balancing via LSF.
PWG Common Interests
- Computing Infrastructure
- Catalogued DSTs in StEvent, a common primary-track population extracted to miniDST, possibly common to several PWGs.
- The CAS chain - a general-purpose chain with components from several PWGs.
- Data catalogs and job management on CAS - a unified management facility is required, with catalog for analysis output as well as DST or miniDST input.
- uDST general format - StEvent framework as one option.
- TagDB in MySQL. Jeff Porter needs additional emergency help to bring the DB system up to speed. The DB situation is critical.
- Production autoQA criteria need to be defined in the next couple of months.
- Two weeks starting March 26
- Primary goal - coordinated physics analysis of 100k events on CAS by multiple PWGs, an event-type cocktail with about 10 event classes - several values of global variables, several correlation types, different source sizes, etc., some in 10k units, some in 1k units, depending on `signal' strength and required statistical power.
Separate Working Groups
- Tools for efficiency evaluation are critically needed. Tracking optimization (reduce pathologies) is also very important, as is detailed comparison of simulated and real data when available. A strangeness analysis framework for efficiencies doesn't exist yet. Maker templates do exist from other PWGs. A uDST is needed for efficiency data. The promised 1000/wk of strangeness-competent Geanted events is apparently not happening.
- The TagDB is critically needed in doEvents, as is embedding/evaluation, uDST definitions in persistent StEvent and primary track criteria (depending in detail on each PWG, open at DST production and tightened later in individual analyses). Hit outlier removal should not be in EGR but rather in StEvent. Need capability to refit during analysis; this refit capability may only be required during an initial learning curve and then dropped.
- There is general need for a primary-track selection maker common to several PWGs, but allowing different cut criteria for each group. Several instances already exist (flow, HBT...), with common features. The common primtrk population from the bfc should include all primary tracks suitable for all groups. Cut systems for particular groups should be in a common versioned cut DB accessible for comparison studies (with corresponding cut algorithms in CVS).
- The TagDB is on the critical path, as is a data catalog system for both input and output of CAS analysis tasks and a load-leveling analysis management scheme. The concept of a CAS chain is re-emerging, with combined activities by several PWGs. This cooperative effort should increase. Related to this is adoption of a common self-describing uDST format possibly based on StEvent, with details differing from one PWG to another. There should be a member-function library to include common StEvent methods shared among PWGs.
- Track splitting is a major issue; find and mend split tracks, or mark suspects. Study this with cosmics. TagDB and data catalog are major immediate issues (e.g., EbyE HBT). uDST design and incorporation in a CAS chain with other PWGs, including interface to a common STAR uDST design. Modify mevsim to include unlike-particle correlations and increase yields of some low-yield particles by 10x. Make special events for MDC3 (different source size). Vary collision geometry (centrality), establish high-volume test of HBT infra.
- Manpower - spin is still subcritical. Need new institutions and hires at BNL/LBL, new students. The barrel requires a successful review, strengthened justification for second half. EMC simulation and reconstruction software is still in early stages, urgently needed. Various ongoing hardware projects (RHIC, STAR, ...) need to be monitored so that decisions are not made that are disadvantageous to Spin.
- Trigger is the big issue for PEC; hardware is ok but software and manpower are big questions. MWC commissioning is critical. Info on beam-gas background levels is needed. There are software infra problems regarding small events. Requires by-hand processing. There are still problems with low-multiplicity vertex finding.