| STAR Computing | STAR HyperNews | |
|
| ||
Hello Gene and QA-er,
A few precisions on this: the idea of having a simple additional
2 pages histograms is mostly driven by a mandate of mine I am afraid
as it became apparent that we would have a hard time to detect when
some key changes appear(ed) in our code. The nightly test suite by
Lidia already catches a lot but we are missing reconstruction
focused details (variable not necessarily based on averages).
In other words, the task is to work toward a set of relevant (to
tracking / reconstruction within the scope MC and real data)
reference histograms we could cross compare. The apparoach is
that I would institute the saving of such histograms along
* each library validation
* each release of dev (reduced sample)
Further is to have such set generated 'on demand' on a larger
statistic samples (Lidia has already implemented the generation
of larger sample statistics and I marginally helped with the
automation part). So, we would also generate those plots on
* on demand larger sample based on "dev"
* on demand larger samples based on .dev (QA-ing of the
development area BEFORE the code is released in dev, providing
one more check of code sanity)
There are two goals set for this effort:
(1) immediate need is a 2 pages (as Gene provides below) one could
generate from a larger sample dataset. In that regard, we need
a way to run the macro not only on one file but a set of
files. Histograms / results would be saved with each library
release as stated above. PDF at first is fine. However, most
welcomed would be a .C dump of the plot as we would like later
to be able to make differential comparisons between
-a- two library release
-b- .DEV and dev for example
(2) longer term: re-establish at a higher priority the project
aimed to provide a QA suite based on a "divergence comparing to
a reference" rather than a strictly histogram based. This
project is not new per-se (discussed this with Lanny in the past,
discussed with Yuri, discussed with Gene) but probably not as
immediately possible as simple words would indicate.
The notion discussed [as a reminder] is that along the line
of the try (Kolmogorov) made in August 2005 by Yuri (for
evaluating Bfield calculation changes), we should aim to
develop a suite which would flag gross divergences (with a
set threshold) in distributions chosen
to monitor library changes. For example:
- a divergence of 1% if the number of globals may not
warrant a flag raised but 10% should. In such case, we
should be able to have such plot flagged and in one
click, get to the before/after and difference plot.
[only those who would divergence within tolerance
would be flagged]
- a change in the number of vertices would be flagged
- a change in eta distribution
- etc ...
A better project description will be written soon
(I hope) so we could proceed with the implementation of the
long term. A priori, and since we would need to have the
ability to see changes between library releases, I would
assume that saving .C (with data points included that is)
would be one implementation detail as it is unlikely we
can save all hist.root files. But there are many possible
implementations.
The short term goal of saving the histos with
the release of each library will be added in the library
release policy.
Cheers,
Gene Van Buren wrote:
> Hello, QA-ers.
>
> The S&C team has asked for a simple set of histograms which can be
> used as an intermediary check of reconstruction performance/
> characteristics when reconstruction code is changed (intermediary
> means simple distributions, which are more detailed than gross numbers
> like total number of tracks, but not as detailed as looking at track
> characteristics with part-per-mil type precision). At this point, the
> focus is on track reconstruction.
>
> The first version is just a couple pages of histograms, the list of
> which are now in CVS (and probably DEV tonight).
>
> For example, the following code, when run in DEV, should quickly
> generate this histogram set as a pdf file for any hist.root files on
> disk (please let it be known if you find otherwise):
>
> >>>>>>
> setenv INPUTFILE /star/data36/reco/2007LowLuminosity/FullField/P08ib/
> 2007/120/8120054/st_physics_8120054_raw_1030024.hist.root
> setenv OUTPUTFILE ${INPUTFILE:t:r}.pdf
> setenv TITLE ${INPUTFILE:t}
> setenv HLIST $STAR/StRoot/St_QA_Maker/QAhlist_Reco.h
>
> root4star -b -q -l bfcread_hist_to_ps.C\(\"$INPUTFILE\",\"EventQA\",
> \"bfcTree\",\"$OUTPUTFILE\",\"$TITLE\",\"$HLIST\"\)
> >>>>>>
>
> There is one caveat which could use some improvement: a separate PDF
> file will currently be created for each trigger type, just as we
> provide during FastOffline QA. This means that for some
> st_physics_blah.hist.root input file, there may be multiple PDFs like
> st_physics_blah.histMB.pdf and st_physics_blah.histHT.pdf. As a next
> step, I'll see if there's a simple way to sum these histograms so that
> this extra confusion needn't be there and just one output file can be
> generated.
>
> Thanks,
> -Gene
>
>
> -------------------------------------------------------------
> Visit this STAR HyperNews message (to reply or unsubscribe) at:
> http://www.star.bnl.gov/HyperNews-star/get/starqa/2017.html
--
,,,,,
( o o )
--m---U---m--
Jerome
|
Messages
Messages : 10000 All 100 200 500 1000 2000 4000 8000
Contact Jerome to set up e-mail posting
Show subscribers
| STAR Computing | STAR HyperNews | Privacy & Security |
|
| ||