ITTF review (September 23/24, 2002)

Review Committee: Jan Balewski, Jerome Baudot, Rene Bellwied (chair), Helen Caines, Yuri Fisyak, Mike Lisa, Maria Mora-Corral, Alan Poppleton, Karel Safarik, Iwona Sakrejda, Alex Suaide, Bernd Surrow, Thomas Ullrich
Ex-officio: Jamie Dunlop, David Hardtke, Morton Kaplan, Tonko Ljubicic, Jerome Lauret, Jim Thomas

Executive Summary


The review committee would like to thank and congratulate the STAR software leader
(Jerome Lauret), the ITTF leader (Claude Pruneau), and his developer crew (Manuel Calderon, Mike Miller, Ben Norman, Andrew Rose) for a productive and well conducted review. We also like to thank the group of testers that presented results during the review. The amount of work that went into the code development and testing over the past few months is truly impressive, and the project is well on its way to the desired results.

The design, implementation and present level of performance of the new STI tracker were presented at the review. The main committee recommendation is the strong endorsement of STI as the future tracking code for STAR. We recommend to officially deploy this code at some point in time (see below) and to stop support for the existing tracker at that point.

The committee was pleased with the efforts and the hard work that the core group of developers has expanded in order to bring the new tracker to its present level. Although the flexibility of the design was not yet demonstrated, which means a careful design study will have to be addressed by a proposed follow-up review, we want to point out that the existing design is well programmed and is comparable to similar designs of other large collider experiments. The implementation of the tracker seems to be on the right track, but certain integration issues that do not pertain to the tracking code to first order will also have to be reviewed in context in the near future.

The present performance features show that the code is not yet ready for deployment. It was not possible to fully evaluate the performance due to contradicting results and obvious bugs that do not seem to be showstoppers, but that have to be addressed before a real performance assessment can be undertaken. It is the impression of the committee that the core groups of developers and testers made significant progress in the weeks leading up to the review, but in order to present physics driven performance results some more basic and systematic tests were left aside. In addition, the speed with which the code had to be developed in the last two months led to coding problems, which negatively affected the performance results. In summary it was not possible to properly evaluate the performance with the existing results.

The code seems to have the capacity to match the performance of the present TPC tracking code, and to allow us to extend the tracking to a truly integrated approach, which in its first installment includes extrapolations to the SVT, EMC, CTB, and RICH. From the existing results it would appear that the code performs comparably to the present STAR tracking code in less CPU time and with no apparent memory leaks. We believe that to fully assess the capabilities and the implementation it is necessary for STAR to enter another dedicated development phase immediately. It is our opinion that if the core developer group can stay together and be fully dedicated, this development phase can be completed by early December. We therefore recommend an in-depth performance review by mid December. Should the STI performance at that time yield the desired results, we recommend that STAR adopt STI as the default analysis program starting at the beginning of the year-3 run in January 2003. The deployment of STI should then also include its application in the fast offline QA. In order to fully address the issue of deployment in STAR at the next review, the STAR software leader should, after further consultation with the STI leader, the STAR reconstruction leader, and the Physics Analysis coordinator, present a deployment plan.

If the collaboration reaches the conclusion that a deployment at that time would be premature we suggest deployment for a second production run in 2003 after a further performance review. The committee stresses the fact that we believe that STAR will not be able to reconstruct data in year-4 with sufficiently high quality to accomplish a successful rare probe and high-pt program without STI. We also point out that even before year-4, the year-2 and 3 pp data and the year-3 dA data need at least a new vertex finder, but would be even better served by this whole new tracker.

Finally, in order to properly address the issue of manpower and continuity for developing and maintaining a new tracking code the committee makes the following manpower recommendations. The core group of developers should be maintained at a larger than 0.5 FTE level for each core developer until the end of this year. The Council members responsible for these individuals should be asked to comply with this recommendation as part of their service work commitments. From January 2003 onwards core developers should be encouraged to stay committed and to serve as consultants for as long as they are STAR members. The STI group leader, Claude Pruneau, should be asked to remain in that capacity, and STAR should commit the equivalent of one FTE (either one person full-time or two persons half-time) to support this effort. We recommend that this person hold a long-term position at a National Laboratory in order to assure continuity. In addition we recommend that STAR turn the maintenance of the new tracking code into an institutional responsibility along the lines of the already existing MoU’s for hardware systems.

The lack of testing and integration manpower should be addressed by requiring each detector subsystem to provide one person and each Physics Working group to provide two persons as liaisons to the STI group for testing, calibration, and integration purposes.
In general, we believe that the integration of the new tracker into the analysis framework is a major task, which can be accomplished in time for year-3 only if sufficient manpower is allocated. We believe that the core STI development group should not be burdened with this task and thus recommend the formation of a STI integration task force immediately. The group should include about five experienced integration and calibration experts from STAR, who could be members of the aforementioned group of liaison physicists. One key task for this group will be to extract the many calibration corrections, presently embedded in TPT maker (e.g. the distortion corrections), and apply them to the new tracker in a better and separated form. The integration effort should also be reviewed as part of the December review.

In the following we will address the specific charges to the committee, make more specific recommendations, and suggest a specific set of tests that should be presented at the December review.

Comments to the General Code Design


  1. The practicality of adding new geometries for new detector sub-systems was not sufficiently demonstrated. The SVT geometry was included in a simplified form compared to the original GEANT files. No comparison of material budgets between the GEANT code and the simplified code were shown. We recommend to show these comparisons in order to demonstrate the equivalence between the two methods. In that regard, a suggested code implementation is to directly link the STI geometries/materials to the GEANT geometry database in order to comply with possible changes by the sub-systems to the GEANT database. Any comparison presented in December should also include the EMC (BEMC and EEMC) in order for the code to be applicable to year-3 data. We further recommend presenting a roadmap in December that explains to future sub-system developers how their geometries are implemented in the code.
  2. We reviewed the specific point of handling of coordinate frames and transformations and we concluded that the method used complies with existing standards and practices and is thus supported by the committee. The new set of simulations for the December review will further demonstrate the feasibility of these transformations and their potential advantage in CPU time and memory usage.
  3. The hit error parameterization needs to be better explained and presented at a future review. Tests of the effects of the hit error parametrization should be performed by the ITTF and the reconstruction groups.
  4. The documentation of the code should grow in parallel to the code development and should be completed by December.
  5. We support the chosen method of a Kalman road finder as the main tracking algorithm. We also support the ‘outside in’ approach, but encourage the group to allow the flexibility of an ‘inside out’ approach if necessary in the future for specific measurements.
  6. The pros and cons of many-to-many hit associations (i.e. hit sharing) should be clearly demonstrated for the next review. The effects of hit merging in different volumes (e.g. SVT, TPC inner sector, TPC outer sector) should be shown. For the standard performance evaluation parameters (e.g. efficiencies, purities etc.) hit-sharing should be disabled.
  7. Continue the evaluation and potential improvement of the seed finder (e.g. do we really need to take out six points for the seed finder?). The present tuning by eye should be replaced with a more objective method.
  8. At the follow-up review the usage of existing code in STI should be documented. For example the existing dE/dx (by Yuri Fisyak) and the existing TPC calibration schemes (by Dave Hardtke) could be re-used. If the developer group decides not to use existing code, it should be explained why this choice was made.

Comments to the Implementation of the Code

  1. The implementation of track extension methods into volumes other than the TPC was not sufficiently addressed. The SVT is incorporated and was part of the review, but track extrapolations to e.g. the EMC (BEMC and EEMC) were not shown. Based on comments by the developers this seems to be a straightforward extension of the Kalman fitter, and code seems to exist already, but the committee recommends demonstrating the procedure by using the EMC as an example for the December review. In terms of the code specific integration of the track extension we recommend that intersection points, errors, and momentum vectors are stored for every hit sub-volume. We further recommend that the extrapolation algorithm is also used an active way by allowing shower max detector (SMD) hits to be used in the seed formation.
  2. We feel that extending the tracker with new tracking models should be a low priority for the group until December. This means for example that the inclusion of the FTPC into STI should be a low priority until the year-3 data run begins.
  3. In order to provide a set of physics results for the current review, key effects like multiple Coulomb scattering and energy loss were purposefully turned off in the present set of simulations. We feel that these effects are the driving force behind the use of Kalman filtering and we therefore strongly suggest to include and address these effects for at least part of the simulations before December. This point will be further addressed in the attached set of recommended performance results. At this point it was not possible to assess the geometry implementations and the treatments of energy loss and scattering.
  4. The existing core group should not attempt integration steps by, for example, changing maker schemes. If such a procedure is necessary, then the newly formed STI integration task force should provide such input.


Comments to the Performance Results


  1. The presented results concerning tracking efficiencies and global tracking performance were not internally consistent and therefore inconclusive. We recognize the potential of the code in the future, but at this point it seems that STI is about 20% less efficient than TPT, uniformly across pt, , and centrality. The code seems to have additional problems at low pt and high . The causes seem to be solvable, but the results have to be stable and comparable to the TPT performance before the next review.
  2. A first attempt at an integrated tracking result (SVT+TPC in pp data from year-2) was made. The code seems to function in principle, but again the performance was not yet close to expectation.
  3. Tracking inefficiencies are not yet fully understood. Bugs had been found in the weeks leading up to the review and during the review (e.g. hit sharing was unintentionally enabled for all data production, which greatly complicated the HBT analysis).
  4. Specific analysis procedures that could be very relevant for the future physics analyses in STAR should be demonstrated. These include for example kink analysis, low pt analysis (integrated and potentially with SVT points alone), energy flow measurements (tracking points plus EMC energy information). Although this is a recommendation these simulations should be the final set of simulations to demonstrate the usefulness of the code, which means a conclusion of the suitability of the code at a December review can be reached without these simulations. Therefore these simulations are listed last in the following list of recommended steps before December.

Recommended list and order of performance simulations:

  1. fix existing known bugs.
  2. show comparative radiation lengths plots for geometry implementations.
  3. run intrinsic tracker benchmark tests with and without MCS and dE/dx.
Intrinsic benchmarks include:
    1. residuals
    2. pulls (full covariance matrix)
    3. - distributions
    4. hit multiplicity in road finder search cone.
  1. run tracking efficiency and purity plots for primary and secondary particles for the TPC alone and the SVT+TPC. Determine ‘matching’ efficiencies and compare to old TPT and TPT/EST simulations.
  2. run physics analyses for primary particle spectra, V0 reconstruction, 3d-HBT, and high pt particles. If time permits also run flow analyses.
  3. run field-off data to show that the code works in that situation. This is a high priority item because it pertains to the design and performance evaluation.
  4. address specifically the issues and inefficiencies of low pt and high tracking.
  5. test the cluster overlap issue in the seed finder in order to provide input to DAQ.
  6. test physics extension capabilities through very low pt and energy flow simulations.
  7. If possible extend the simulations from AA to pp and dA simulations and simulate pp event pileup in the detector.

Regarding the completeness of these tests and the procedure to arrive at a deployment date, we recommend that in particular the physics tests are performed decoupled from each other in various groups in parallel. We recommend that the testers and evaluators get together at least once a week by phone (i.e. a STI phone conference) in order to compare results. If time permits an all encompassing MDC-5 could be run between the December review and the deployment date, but the justification for such an exercise should be addressed again at the next review. Based on the necessity for backward compatibility of STAR results, the new tracker will have to be applied on the year-2 data as well as future year productions in any event. The comparison of the year-2 results with STI and TPT should serve as the final test of the new code. We believe that this task can at best be accomplished in parallel to the year-3 production schedule.