ViewVC Help
View File | Revision Log | Show Annotations | Root Listing
root/cvsroot/UserCode/benhoob/cmsnotes/StopSearch/systematics.tex
(Generate patch)

Comparing UserCode/benhoob/cmsnotes/StopSearch/systematics.tex (file contents):
Revision 1.18 by claudioc, Fri Oct 12 04:55:23 2012 UTC vs.
Revision 1.26 by claudioc, Sun Oct 21 03:37:29 2012 UTC

# Line 14 | Line 14 | The calculation is done for each signal
14   region,
15   for electrons and muons separately.
16  
17 < The choice to normalizing to the peak region of $M_T$ has the
17 > The choice to normalize to the peak region of $M_T$ has the
18   advantage that some uncertainties, e.g., luminosity, cancel.
19   It does however introduce complications because it couples
20   some of the uncertainties in non-trivial ways.  For example,
# Line 25 | Line 25 | for example,
25   the $t\bar{t} \to$ dilepton BG estimate because it changes the
26   $t\bar{t}$ normalization to the peak region (because some of the
27   events in the peak region are from rare processes).  These effects
28 < are carefully accounted for.  The contribution to the overall
29 < uncertainty from each BG source is tabulated in
28 > are carefully accounted for. The contribution to the overall
29 > uncertainty from each background source is tabulated in
30   Section~\ref{sec:bgunc-bottomline}.
31 < First, however, we discuss the uncertainties one-by-one and we comment
31 > Here we discuss the uncertainties one-by-one and comment
32   on their impact on the overall result, at least to first order.
33   Second order effects, such as the one described, are also included.
34  
# Line 37 | Line 37 | peak regions}
37   These vary between 2\% and 20\%, depending on the signal region
38   (different
39   signal regions have different \met\ requirements, thus they also have
40 < different $M_T$ regions used as control.
40 > different $M_T$ regions used as control).
41   Since
42 < the major BG, eg, $t\bar{t}$ are normalized to the peak regions, this
42 > the major backgrounds, eg, $t\bar{t}$ are normalized to the peak regions, this
43   fractional uncertainty is pretty much carried through all the way to
44   the end.  There is also an uncertainty from the finite MC event counts
45   in the $M_T$ peak regions.  This is also included, but it is smaller.
# Line 47 | Line 47 | in the $M_T$ peak regions.  This is also
47   Normalizing to the $M_T$ peak has the distinct advantages that
48   uncertainties on luminosity, cross-sections, trigger efficiency,
49   lepton ID, cancel out.
50 < For the low statistics regions with high \met requirements, the
51 < price to pay in terms of event count statistical uncertainties starts
50 > For the low statistics regions with high \met\ requirements, the
51 > price to pay in terms of event count is that statistical uncertainties start
52   to become significant.  In the future we may consider a different
53   normalization startegy in the low statistics regions.
54  
# Line 58 | Line 58 | This choice affects the scale factors of
58   If the $M_T$ peak region is not well modelled, this would introduce an
59   uncertainty.
60  
61 < We have tested this possibility by recalculating the post veto scale factors for a different
61 > We have tested this possibility by recalculating the post-veto scale factors for a different
62   choice
63   of $M_T$ peak region ($40 < M_T < 100$ GeV instead of the default
64 < $50 < M_T < 80$ GeV.  This is shown in Table~\ref{tab:mtpeaksf2}.  
64 > $50 < M_T < 80$ GeV).  This is shown in Table~\ref{tab:mtpeaksf2}.  
65   The two results for the scale factors are very compatible.
66   We do not take any systematic uncertainty for this possible effect.
67  
# Line 111 | Line 111 | e veto \mt-SF      & $0.97 \pm 0.01$ & $
111   \end{table}
112  
113  
114 < \subsection{Uncertainty on the Wjets cross-section and the rare MC cross-sections}
114 > \subsection{Uncertainty on the \wjets\ cross-section and the rare MC cross-sections}
115   These are taken as 50\%, uncorrelated.  
116   The primary effect is to introduce a 50\%
117   uncertainty
# Line 125 | Line 125 | attributed to $t\bar{t}$ goes down, and
125   scaled to the number of $t\bar{t}$ events in the peak, the $t\bar{t}$
126   BG goes down.  
127  
128 < \subsection{Scale factors for the tail-to-peak ratios for lepton +
128 > \subsection{Tail-to-peak ratios for lepton +
129    jets top and W events}
130 < These tail-to-peak ratios are described in Section~\ref{sec:ttp}.
131 < They are studied in CR1 and CR2.  The studies are described
132 < in Sections~\ref{sec:cr1} and~\ref{sec:cr2}), respectively, where
133 < we also give the uncertainty on the scale factors.  See
134 < Tables~\ref{tab:cr1yields}
135 < and~\ref{tab:cr2yields}, scale factors $SFR_{wjet}$ and $SFR_{top})$.
130 > The tail-to-peak ratios $R_{top}$ and $R_{wjet}$ are described in Section~\ref{sec:ttp}.
131 > The data/MC scale factors are studied in CR1 and CR2 (Sections~\ref{sec:cr1} and~\ref{sec:cr2}).
132 > Only the scale factor for \wjets, $SFR_{wjet}$, is used, and its
133 > uncertainty is given in Table~\ref{tab:cr1yields}.
134 > This uncertainty affects both $R_{wjet}$ and $R_{top}$.
135 > The additional systematic uncertainty on $R_{top}$ from the variation between optimistic and pessimistic scenarios is given in Section~\ref{sec:ttp}.
136 >
137  
138   \subsection{Uncertainty on extra jet radiation for dilepton
139    background}
# Line 142 | Line 143 | $t\bar{t} \to$
143   dilepton MC is rescaled by the factors $K_3$ and $K_4$ to make
144   it agree with the data.  The 3\% uncertainties on $K_3$ and $K_4$
145   comes from data/MC statistics.  This  
146 < result directly in a 3\% uncertainty on the dilepton BG, which is by far
146 > results directly in a 3\% uncertainty on the dilepton background, which is by far
147   the most important one.
148  
149   \subsection{Uncertainty from MC statistics}
# Line 153 | Line 154 | is negligible in the low \met\ signal re
154   15\% in SRG.
155  
156  
157 < \subsection{Uncertainty on the \ttll\ Acceptance}
158 <
157 > \subsection{Uncertainty on the \ttll\ Background}
158 > \label{sec:ttdilbkgunc}
159   The \ttbar\ background prediction is obtained from MC, with corrections
160   derived from control samples in data. The uncertainty associated with
161 < the theoretical modeling of the \ttbar\ production and decay is
162 < estimated by comparing the background predictions obtained using
161 > the \ttbar\ background is derived from the level of closure of the
162 > background prediction in CR4 (Table~\ref{tab:cr4yields}) and
163 > CR5 (Table~\ref{tab:cr5yields}). The results from these control region
164 > checks are shown in Figure~\ref{fig:ttdlunc}. The uncertainties assigned
165 > to the \ttdl\ background prediction based on these tests are
166 > 5\% (SRA), 10\% (SRB), 15\% (SRC), 25\% (SRD), 40\% (SRE-G).
167 >
168 > \begin{figure}[hbt]
169 >  \begin{center}
170 >        \includegraphics[width=0.6\linewidth]{plots/ttdilepton_uncertainty.pdf}
171 >        \caption{
172 >          \label{fig:ttdlunc}%\protect
173 >          Results of the comparison of yields in the \mt\ tail comparing the MC prediction (after
174 >          applying SFs) to data for CR4 and CR5 for all the signal
175 >          region requirements considered (A-G). The bands indicate the
176 >          systematic uncertainties assigned based on these tests,
177 >          ranging from $5\%$ for SRA to $40\%$ for SRE-G.}
178 >      \end{center}
179 > \end{figure}
180 >
181 >
182 > \subsubsection{Check of the uncertainty on the \ttll\ Background}
183 >
184 > We check that the systematic uncertainty assigned to the \ttll\ background prediction
185 > covers the uncertainty associated with
186 > the theoretical modeling of the \ttbar\ production and decay
187 > by comparing the background predictions obtained using
188   alternative MC samples. It should be noted that the full analysis is
189   performed with the alternative samples under consideration,
190   including the derivation of the various data-to-MC scale factors.
# Line 166 | Line 192 | The variations considered are
192  
193   \begin{itemize}
194   \item Top mass: The alternative values for the top mass differ
195 <  from the central value by $5~\GeV$: $m_{\mathrm{top}} = 178.5~\GeV$ and $m_{\mathrm{top}}
195 >  from the central value by $6~\GeV$: $m_{\mathrm{top}} = 178.5~\GeV$ and $m_{\mathrm{top}}
196    = 166.5~\GeV$.
197   \item Jet-parton matching scale: This corresponds to variations in the
198    scale at which the Matrix Element partons from Madgraph are matched
# Line 183 | Line 209 | The variations considered are
209    Tauola and is otherwise identical to the Powheg sample.
210    This effect was studied earlier using 7~TeV samples and found to be negligible.
211   \item The PDF uncertainty is estimated following the PDF4LHC
212 <  recommendations[CITE]. The events are reweighted using alternative
212 >  recommendations. The events are reweighted using alternative
213    PDF sets for CT10 and MSTW2008 and the uncertainties for each are derived using the
214 <  alternative eigenvector variations and the ``master equation''. In
215 <  addition, the NNPDF2.1 set with 100 replicas. The central value is
214 >  alternative eigenvector variations and the ``master equation''.
215 >  The NNPDF2.1 set with 100 replicas is also used. The central value is
216    determined from the mean and the uncertainty is derived from the
217    $1\sigma$ range. The overall uncertainty is derived from the envelope of the
218    alternative predictions and their uncertainties.
# Line 254 | Line 280 | on this Figure.
280    statistics.    
281   \item Within the limited statistics, there is no evidence that the
282    situation changes as we go from signal region A to signal region E.
283 < Therefore, we assess a systematic based on the relatively high
284 < statistics
285 < test in signal region A, and apply the same systematic uncertainty
286 < to all other regions.
283 > %Therefore, we assess a systematic based on the relatively high
284 > %statistics
285 > %test in signal region A, and apply the same systematic uncertainty
286 > %to all other regions.
287 > \item In signal regions B and above, the uncertainties assigned in Section~\ref{sec:ttdilbkgunc}
288 > fully cover the alternative MC variations.
289   \item In order to fully (as opposed as 1$\sigma$) cover the
290   alternative MC variations in region A we would have to take a
291   systematic
292 < uncertainty of $\approx 10\%$.  This would be driven by the
292 > uncertainty of $\approx 10\%$ instead of $5\%$.  This would be driven by the
293   scale up/scale down variations, see Table~\ref{tab:fracdiff}.
294   \end{itemize}
295  
# Line 295 | Line 323 | scale up/scale down variations are incon
323   These are described below.
324  
325   The first piece of information is that the jet multiplicity in the scale
326 < up/scale down sample is the most inconsistent with the data.  This can be shown
326 > up/scale down sample is the most inconsistent with the data.  This is shown
327   in Table~\ref{tab:njetskfactors_met100}, where we tabulate the
328 < $K_3$ and $K_4$ factors of Section~\ref{tab:njetskfactors_met100} for
328 > $K_3$ and $K_4$ factors of Section~\ref{sec:jetmultiplicity} for
329   different \ttbar\ MC samples.  The data/MC disagreement in the $N_{jets}$
330   distribution
331   for the scale up/scale down samples is also shown in Fig.~\ref{fig:dileptonnjets_scaleup}
# Line 358 | Line 386 | Then, going back to Table~\ref{tab:fracd
386   up/scale
387   down variations by a factor 2, we can see that a systematic
388   uncertainty
389 < of 6\% would fully cover all of the variations from different MC
390 < samples in SRA and SRB.
391 < {\bf Thus, we take a 6\% systematic uncertainty,  constant as a
392 < function of signal region, as the systematic due to alternative MC
393 < models.}.
394 < Note that this 6\% is also consistent with the level at which we are
395 < able
396 < to test the closure of the method in CR5 for the high statistics
397 < regions
398 < (Table~\ref{tab:hugecr5yields}).
399 <
389 > of 5\% covers the range of reasonable variations from different MC
390 > models in SRA and SRB.
391 > %The alternative MC models indicate that a 6\% systematic uncertainty
392 > %covers the range of reasonable variations.
393 > Note that this 5\% is also consistent with the level at which we are
394 > able to test the closure of the method with alternative samples in CR5 for the high statistics
395 > regions (Table~\ref{tab:hugecr5yields}).
396 > The range of reasonable variations obtained with the alternative
397 > samples are consistent with the uncertainties assigned for
398 > the \ttll\ background based on the closure of the background
399 > predictions and data in CR4 and CR5.
400  
401  
402  
# Line 599 | Line 627 | a second e/$\mu$ or a one prong $\tau \t
627   $P_T > 10$ GeV in $|\eta| < 2.4$.  This fraction is about 1/3, see
628   Table~\ref{tab:trueisotrk}.
629   The uncertainty for these events
630 < is 6\% and is obtained from Tag and Probe studies of Section~\ref{sec:trkveto}
630 > is 6\% and is obtained from tag-and-probe studies, see Section~\ref{sec:trkveto}.
631  
632   \begin{table}[!h]
633   \begin{center}
# Line 648 | Line 676 | decays are well-understood, we currently
676   Second, hadronic tracks may undergo nuclear interactions and hence their tracks may not be reconstructed.
677   As discussed above, independent studies show that the MC reproduces the hadronic tracking efficiency within 4\%,
678   leading to a total background uncertainty of less than 0.5\% (after taking into account the fraction of the total background
679 < due to hadronic $\tau$ decays with \pt\ $>$ 10 GeV tracks), and we hence regard this effect as neglgigible.
679 > due to hadronic $\tau$ decays with \pt\ $>$ 10 GeV tracks), and we hence regard this effect as negligible.
680  
681   The tag-and-probe studies are performed in the full data sample, and compared with the DYJets madgraph sample.
682   All events must contain a tag-probe pair (details below) with opposite-sign and satisfying the Z mass requirement 76--106 GeV.
# Line 705 | Line 733 | The specific criteria for tags and probe
733   The absolute track isolation distributions for passing probes are displayed in Fig.~\ref{fig:tnp}. In general we observe
734   good agreement between data and MC. To be more quantitative, we compare the data vs. MC efficiencies to satisfy
735   absolute track isolation requirements varying from $>$ 1 GeV to $>$ 5 GeV, as summarized in Table~\ref{tab:isotrk}.
736 < In the $\geq$0 and $\geq$1 jet bins where the efficiencies can be tested with statistical precision, the data and MC
736 > In the $\geq 0$ and $\geq 1$ jet bins where the efficiencies can be tested with statistical precision, the data and MC
737   efficiencies agree within 6\%, and we apply this as a systematic uncertainty on the isolated track veto efficiency.
738   For the higher jet multiplicity bins the statistical precision decreases, but we do not observe any evidence for
739   a data vs. MC discrepancy in the isolated track veto efficiency.
# Line 738 | Line 766 | for events with the \njets\ requirement
766  
767   \begin{table}[!ht]
768   \begin{center}
741 \caption{\label{tab:isotrk} Comparison of the data vs. MC efficiencies to satisfy the indicated requirements
742 on the absolute track isolation, and the ratio of these two efficiencies. Results are indicated separately for electrons and muons and for various
743 jet multiplicity requirements.}
769   \begin{tabular}{l|c|c|c|c|c}
770  
771   %Electrons:
# Line 838 | Line 863 | $\mu$ + $\geq$4 jets   &           $>$ 1
863   \hline
864  
865   \end{tabular}
866 + \caption{\label{tab:isotrk} Comparison of the data vs. MC efficiencies to satisfy the indicated requirements
867 + on the absolute track isolation, and the ratio of these two efficiencies. Results are indicated separately for electrons and muons and for various
868 + jet multiplicity requirements.}
869   \end{center}
870   \end{table}
871  
872 + \clearpage
873 + \subsection{Summary of uncertainties}
874 + \label{sec:bgunc-bottomline}
875 +
876 + The contribution from each source to the total uncertainty on the background yield is given in Tables~\ref{tab:relativeuncertaintycomponents} and~\ref{tab:uncertaintycomponents} for the relative and absolute uncertainties, respectively. In the low-\met\ regions the dominant uncertainty comes from the top tail-to-peak ratio, $R_{top}$ (Section~\ref{sec:ttp}), while in the high-\met\ regions the \ttll\ systematic uncertainty dominates (Section~\ref{sec:ttdilbkgunc}).
877 +
878 + \input{uncertainties_table.tex}
879 +
880 +
881 +
882 +
883  
884   %Figure.~\ref{fig:reliso} compares the relative track isolation
885   %for events with a track with $\pt > 10~\GeV$ in addition to a selected
# Line 922 | Line 961 | $\mu$ + $\geq$4 jets   &           $>$ 1
961   %      \end{center}
962   %\end{figure}
963  
964 < % \subsection{Summary of uncertainties}
926 < % \label{sec:bgunc-bottomline}.
964 >
965  
966   % THIS NEEDS TO BE WRITTEN

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines