ViewVC Help
View File | Revision Log | Show Annotations | Root Listing
root/cvsroot/UserCode/claudioc/OSNote2010/datadriven.tex
Revision: 1.32
Committed: Thu Dec 2 17:48:33 2010 UTC (14 years, 5 months ago) by benhoob
Content type: application/x-tex
Branch: MAIN
Changes since 1.31: +2 -2 lines
Log Message:
*** empty log message ***

File Contents

# User Rev Content
1 claudioc 1.1 \section{Data Driven Background Estimation Methods}
2     \label{sec:datadriven}
3     We have developed two data-driven methods to
4     estimate the background in the signal region.
5 benhoob 1.10 The first one exploits the fact that
6 benhoob 1.22 SumJetPt and \met$/\sqrt{\rm SumJetPt}$ are nearly
7 claudioc 1.1 uncorrelated for the $t\bar{t}$ background
8     (Section~\ref{sec:abcd}); the second one
9     is based on the fact that in $t\bar{t}$ the
10     $P_T$ of the dilepton pair is on average
11     nearly the same as the $P_T$ of the pair of neutrinos
12     from $W$-decays, which is reconstructed as \met in the
13     detector.
14    
15 benhoob 1.15
16 claudioc 1.6 %{\color{red} I took these
17     %numbers from the twiki, rescaling from 11.06 to 30/pb.
18     %They seem too large...are they really right?}
19 claudioc 1.1
20    
21     \subsection{ABCD method}
22     \label{sec:abcd}
23    
24 benhoob 1.22 We find that in $t\bar{t}$ events SumJetPt and
25 benhoob 1.16 \met$/\sqrt{\rm SumJetPt}$ are nearly uncorrelated,
26 benhoob 1.31 as demonstrated in Fig.~\ref{fig:uncor}.
27 claudioc 1.1 Thus, we can use an ABCD method in the \met$/\sqrt{\rm SumJetPt}$ vs
28     sumJetPt plane to estimate the background in a data driven way.
29    
30 benhoob 1.24 %\begin{figure}[bht]
31     %\begin{center}
32     %\includegraphics[width=0.75\linewidth]{uncorrelated.pdf}
33     %\caption{\label{fig:uncor}\protect Distributions of SumJetPt
34     %in MC $t\bar{t}$ events for different intervals of
35     %MET$/\sqrt{\rm SumJetPt}$.}
36     %\end{center}
37     %\end{figure}
38    
39 claudioc 1.21 \begin{figure}[bht]
40 claudioc 1.1 \begin{center}
41 benhoob 1.24 \includegraphics[width=0.75\linewidth]{uncor.png}
42 claudioc 1.1 \caption{\label{fig:uncor}\protect Distributions of SumJetPt
43     in MC $t\bar{t}$ events for different intervals of
44 benhoob 1.24 MET$/\sqrt{\rm SumJetPt}$. h1, h2, and h3 refer to the MET$/\sqrt{\rm SumJetPt}$
45     intervals 4.5-6.5, 6.5-8.5 and $>$8.5, respectively.}
46 claudioc 1.1 \end{center}
47     \end{figure}
48    
49 claudioc 1.21 \begin{figure}[tb]
50 claudioc 1.1 \begin{center}
51 benhoob 1.31 \includegraphics[width=0.75\linewidth]{ttdil_uncor_38X.png}
52 benhoob 1.22 \caption{\label{fig:abcdMC}\protect Distributions of MET$/\sqrt{\rm SumJetPt}$ vs.
53 benhoob 1.31 SumJetPt for SM Monte Carlo. Here we also show our choice of ABCD regions. The correlation coefficient
54     ${\rm corr_{XY}}$ is computed for events falling in the ABCD regions.}
55 claudioc 1.1 \end{center}
56     \end{figure}
57    
58    
59     Our choice of ABCD regions is shown in Figure~\ref{fig:abcdMC}.
60     The signal region is region D. The expected number of events
61 benhoob 1.29 in the four regions for the SM Monte Carlo, as well as the background
62     prediction A $\times$ C / B are given in Table~\ref{tab:abcdMC} for an integrated
63 benhoob 1.24 luminosity of 35 pb$^{-1}$. The ABCD method with chosen boundaries is accurate
64 benhoob 1.31 to about 20\%, and we assess a corresponding systematic uncertainty
65     {\bf \color{red} More detail needed here???}
66 benhoob 1.29
67     %As shown in Table~\ref{tab:abcdsyst}, we assess systematic uncertainties
68     %by varying the boundaries by an amount consistent with the hadronic energy scale uncertainty,
69     %which we take as $\pm$5\% for SumJetPt and $\pm$2.5\% for MET/$\sqrt{\rm SumJetPt}$, since the
70     %uncertainty on this quantity partially cancels due to the fact that it is a ratio of correlated
71     %quantities. Based on these studies we assess a correction factor $k_{ABCD} = 1.2 \pm 0.2$ to the
72     %predicted yield using the ABCD method.
73 benhoob 1.24
74    
75 claudioc 1.9 %{\color{red} Avi wants some statement about stability
76     %wrt changes in regions. I am not sure that we have done it and
77     %I am not sure it is necessary (Claudio).}
78 claudioc 1.1
79 claudioc 1.21 \begin{table}[ht]
80 claudioc 1.1 \begin{center}
81     \caption{\label{tab:abcdMC} Expected SM Monte Carlo yields for
82 benhoob 1.13 35 pb$^{-1}$ in the ABCD regions, as well as the predicted yield in
83 benhoob 1.16 the signal region given by A $\times$ C / B. Here `SM other' is the sum
84 benhoob 1.13 of non-dileptonic $t\bar{t}$ decays, $W^{\pm}$+jets, $W^+W^-$,
85     $W^{\pm}Z^0$, $Z^0Z^0$ and single top.}
86 benhoob 1.16 \begin{tabular}{lccccc}
87 benhoob 1.13 \hline
88 benhoob 1.27 sample & A & B & C & D & A $\times$ C / B \\
89 benhoob 1.13 \hline
90 benhoob 1.27 $t\bar{t}\rightarrow \ell^{+}\ell^{-}$ & 8.27 $\pm$ 0.18 & 32.16 $\pm$ 0.35 & 4.69 $\pm$ 0.13 & 1.05 $\pm$ 0.06 & 1.21 $\pm$ 0.04 \\
91     $Z^0 \rightarrow \ell^{+}\ell^{-}$ & 0.22 $\pm$ 0.11 & 1.54 $\pm$ 0.29 & 0.05 $\pm$ 0.05 & 0.16 $\pm$ 0.09 & 0.01 $\pm$ 0.01 \\
92     SM other & 0.54 $\pm$ 0.03 & 2.28 $\pm$ 0.12 & 0.23 $\pm$ 0.03 & 0.07 $\pm$ 0.01 & 0.05 $\pm$ 0.01 \\
93 benhoob 1.25 \hline
94 benhoob 1.27 total SM MC & 9.03 $\pm$ 0.21 & 35.97 $\pm$ 0.46 & 4.97 $\pm$ 0.15 & 1.29 $\pm$ 0.11 & 1.25 $\pm$ 0.05 \\
95 claudioc 1.1 \hline
96     \end{tabular}
97     \end{center}
98     \end{table}
99    
100 benhoob 1.24
101    
102     \begin{table}[ht]
103     \begin{center}
104 benhoob 1.27 \caption{\label{tab:abcdsyst}
105     {\bf \color{red} Do we need this study at all? Observed/predicted is consistent within stat uncertainties as the boundaries are varied- is it enough to simply state this fact in the text??? }
106     Results of the systematic study of the ABCD method by varying the boundaries
107 benhoob 1.24 between the ABCD regions shown in Fig.~\ref{fig:abcdMC}. Here $x_1$ is the lower SumJetPt boundary and
108     $x_2$ is the boundary separating regions A and B from C and D, their nominal values are 125 and 300~GeV,
109     respectively. $y_1$ is the lower MET/$\sqrt{\rm SumJetPt}$ boundary and
110     $y_2$ is the boundary separating regions B and C from A and D, their nominal values are 4.5 and 8.5~GeV$^{1/2}$,
111     respectively.}
112     \begin{tabular}{cccc|c}
113     \hline
114     $x_1$ & $x_2$ & $y_1$ & $y_2$ & Observed/Predicted \\
115     \hline
116 benhoob 1.26 nominal & nominal & nominal & nominal & $1.20 \pm 0.12$ \\
117     +5\% & +5\% & +2.5\% & +2.5\% & $1.38 \pm 0.15$ \\
118     +5\% & +5\% & nominal & nominal & $1.31 \pm 0.14$ \\
119     nominal & nominal & +2.5\% & +2.5\% & $1.25 \pm 0.13$ \\
120     nominal & +5\% & nominal & +2.5\% & $1.32 \pm 0.14$ \\
121     nominal & -5\% & nominal & -2.5\% & $1.16 \pm 0.09$ \\
122     -5\% & -5\% & +2.5\% & +2.5\% & $1.21 \pm 0.11$ \\
123     +5\% & +5\% & -2.5\% & -2.5\% & $1.26 \pm 0.12$ \\
124 benhoob 1.24 \hline
125     \end{tabular}
126     \end{center}
127     \end{table}
128    
129 claudioc 1.2 \subsection{Dilepton $P_T$ method}
130     \label{sec:victory}
131     This method is based on a suggestion by V. Pavlunin\cite{ref:victory},
132     and was investigated by our group in 2009\cite{ref:ourvictory}.
133     The idea is that in dilepton $t\bar{t}$ events the lepton and neutrinos
134     from $W$ decays have the same $P_T$ spectrum (modulo $W$ polarization
135     effects). One can then use the observed
136     $P_T(\ell\ell)$ distribution to model the sum of neutrino $P_T$'s which
137     is identified with the \met.
138    
139     Then, in order to predict the $t\bar{t} \to$ dilepton contribution to a
140     selection with \met$+$X, one applies a cut on $P_T(\ell\ell)+$X instead.
141     In practice one has to rescale the result of the $P_T(\ell\ell)+$X selection
142     to account for the fact that any dilepton selection must include a
143     moderate \met cut in order to reduce Drell Yan backgrounds. This
144     is discussed in Section 5.3 of Reference~\cite{ref:ourvictory}; for a \met
145 benhoob 1.16 cut of 50 GeV, the rescaling factor is obtained from the MC as
146 claudioc 1.2
147     \newcommand{\ptll} {\ensuremath{P_T(\ell\ell)}}
148     \begin{center}
149 benhoob 1.31 $ K = \frac{\int_0^{\infty} {\cal N}(\ptll)~~d\ptll~}{\int_{50}^{\infty} {\cal N}(\ptll)~~d\ptll~} = 1.52$
150 claudioc 1.2 \end{center}
151    
152    
153 benhoob 1.10 %%%TO BE REPLACED
154     %Given the integrated luminosity of the
155     %present dataset, the determination of $K$ in data is severely statistics
156     %limited. Thus, we take $K$ from $t\bar{t}$ Monte Carlo as
157    
158     %\begin{center}
159     %$ K_{MC} = \frac{\int_0^{\infty} {\cal N}(\met)~~d\met~}{\int_{50}^{\infty} {\cal N}(\met)~~d\met~}$
160     %\end{center}
161 claudioc 1.9
162 benhoob 1.10 %\noindent {\color{red} For the 11 pb result we have used $K$ from data.}
163 claudioc 1.2
164     There are several effects that spoil the correspondance between \met and
165     $P_T(\ell\ell)$:
166     \begin{itemize}
167     \item $Ws$ in top events are polarized. Neutrinos are emitted preferentially
168 benhoob 1.22 parallel to the $W$ velocity while charged leptons are emitted prefertially
169     anti-parallel. Thus the $P_T(\nu\nu)$ distribution is harder
170 claudioc 1.2 than the $P_T(\ell\ell)$ distribution for top dilepton events.
171     \item The lepton selections results in $P_T$ and $\eta$ cuts on the individual
172     leptons that have no simple correspondance to the neutrino requirements.
173     \item Similarly, the \met$>$50 GeV cut introduces an asymmetry between leptons and
174     neutrinos which is only partially compensated by the $K$ factor above.
175     \item The \met resolution is much worse than the dilepton $P_T$ resolution.
176 benhoob 1.16 When convoluted with a falling spectrum in the tails of \met, this results
177 claudioc 1.2 in a harder spectrum for \met than the original $P_T(\nu\nu)$.
178     \item The \met response in CMS is not exactly 1. This causes a distortion
179     in the \met distribution that is not present in the $P_T(\ell\ell)$ distribution.
180     \item The $t\bar{t} \to$ dilepton signal includes contributions from
181     $W \to \tau \to \ell$. For these events the arguments about the equivalence
182     of $P_T(\ell\ell)$ and $P_T(\nu\nu)$ do not apply.
183     \item A dilepton selection will include SM events from non $t\bar{t}$
184     sources. These events can affect the background prediction. Particularly
185     dangerous are high $P_T$ Drell Yan events that barely pass the \met$>$ 50
186     GeV selection. They will tend to push the data-driven background prediction up.
187 benhoob 1.16 Therefore we estimate the number of DY events entering the background prediction
188     using the $R_{out/in}$ method as described in Sec.~\ref{sec:othBG}.
189 claudioc 1.2 \end{itemize}
190    
191     We have studied these effects in SM Monte Carlo, using a mixture of generator and
192     reconstruction level studies, putting the various effects in one at a time.
193     For each configuration, we apply the data-driven method and report as figure
194     of merit the ratio of observed and predicted events in the signal region.
195     The results are summarized in Table~\ref{tab:victorybad}.
196    
197     \begin{table}[htb]
198     \begin{center}
199 benhoob 1.27 \caption{\label{tab:victorybad}
200 benhoob 1.32 {\bf \color{red} Should we either update this with 38X MC or remove it?? }
201 benhoob 1.27 Test of the data driven method in Monte Carlo
202 claudioc 1.2 under different assumptions. See text for details.}
203 claudioc 1.6 \begin{tabular}{|l|c|c|c|c|c|c|c|c|}
204 claudioc 1.2 \hline
205 claudioc 1.6 & True $t\bar{t}$ dilepton & $t\to W\to\tau$& other SM & GEN or & Lepton $P_T$ & Z veto & \met $>$ 50& obs/pred \\
206 benhoob 1.28 & included & included & included & RECOSIM & and $\eta$ cuts & & & \\ \hline
207 claudioc 1.6 1&Y & N & N & GEN & N & N & N & 1.90 \\
208     2&Y & N & N & GEN & Y & N & N & 1.64 \\
209     3&Y & N & N & GEN & Y & Y & N & 1.59 \\
210     4&Y & N & N & GEN & Y & Y & Y & 1.55 \\
211     5&Y & N & N & RECOSIM & Y & Y & Y & 1.51 \\
212     6&Y & Y & N & RECOSIM & Y & Y & Y & 1.58 \\
213 benhoob 1.17 7&Y & Y & Y & RECOSIM & Y & Y & Y & 1.38 \\
214 claudioc 1.2 \hline
215     \end{tabular}
216     \end{center}
217     \end{table}
218    
219    
220 benhoob 1.28 \begin{table}[htb]
221     \begin{center}
222     \caption{\label{tab:victorysyst}
223 benhoob 1.30 Summary of uncertainties in $K_C$ due to the MET scale and resolution uncertainty, and to backgrounds other than $t\bar{t} \to$~dilepton.
224 benhoob 1.28 In the first table, `up' and `down' refer to shifting the hadronic energy scale up and down by 5\%. In the second table, the quoted value
225     refers to the amount of additional smearing of the MET, as discussed in the text. In the third table, the normalization of all backgrounds
226     other than $t\bar{t} \to$~dilepton is varied.
227 benhoob 1.30 {\bf \color{red} Should I remove `observed' and `predicted' and show only the ratio? }}
228 benhoob 1.28
229     \begin{tabular}{ lcccc }
230     \hline
231     MET scale & Predicted & Observed & Obs/pred \\
232     \hline
233     nominal & 0.92 $ \pm $ 0.11 & 1.29 $ \pm $ 0.11 & 1.40 $ \pm $ 0.20 \\
234     up & 0.92 $ \pm $ 0.11 & 1.53 $ \pm $ 0.12 & 1.66 $ \pm $ 0.23 \\
235     down & 0.81 $ \pm $ 0.07 & 1.08 $ \pm $ 0.11 & 1.32 $ \pm $ 0.17 \\
236     \hline
237     MET smearing & Predicted & Observed & Obs/pred \\
238     \hline
239     nominal & 0.92 $ \pm $ 0.11 & 1.29 $ \pm $ 0.11 & 1.40 $ \pm $ 0.20 \\
240     10\% & 0.90 $ \pm $ 0.11 & 1.30 $ \pm $ 0.11 & 1.44 $ \pm $ 0.21 \\
241     20\% & 0.84 $ \pm $ 0.07 & 1.36 $ \pm $ 0.11 & 1.61 $ \pm $ 0.19 \\
242     30\% & 1.05 $ \pm $ 0.18 & 1.32 $ \pm $ 0.11 & 1.27 $ \pm $ 0.24 \\
243     40\% & 0.85 $ \pm $ 0.07 & 1.37 $ \pm $ 0.11 & 1.61 $ \pm $ 0.19 \\
244     50\% & 1.08 $ \pm $ 0.18 & 1.36 $ \pm $ 0.11 & 1.26 $ \pm $ 0.24 \\
245     \hline
246     non-$t\bar{t} \to$~dilepton scale factor & Predicted & Observed & Obs/pred \\
247     \hline
248     ttdil only & 0.77 $ \pm $ 0.07 & 1.05 $ \pm $ 0.06 & 1.36 $ \pm $ 0.14 \\
249     nominal & 0.92 $ \pm $ 0.11 & 1.29 $ \pm $ 0.11 & 1.40 $ \pm $ 0.20 \\
250     double non-ttdil yield & 1.06 $ \pm $ 0.18 & 1.52 $ \pm $ 0.20 & 1.43 $ \pm $ 0.30 \\
251     \hline
252     \end{tabular}
253     \end{center}
254     \end{table}
255    
256    
257    
258 claudioc 1.2 The largest discrepancy between prediction and observation occurs on the first
259     line of Table~\ref{tab:victorybad}, {\em i.e.}, at the generator level with no
260     cuts. We have verified that this effect is due to the polarization of
261     the $W$ (we remove the polarization by reweighting the events and we get
262     good agreement between prediction and observation). The kinematical
263 claudioc 1.6 requirements (lines 2,3,4) compensate somewhat for the effect of W polarization.
264     Going from GEN to RECOSIM, the change in observed/predicted is small.
265     % We have tracked this down to the fact that tcMET underestimates the true \met
266     % by $\approx 4\%$\footnote{We find that observed/predicted changes by roughly 0.1
267     %for each 1.5\% change in \met response.}.
268     Finally, contamination from non $t\bar{t}$
269 benhoob 1.16 events can have a significant impact on the BG prediction.
270     %The changes between
271     %lines 6 and 7 of Table~\ref{tab:victorybad} is driven by 3
272     %Drell Yan events that pass the \met selection in Monte Carlo (thus the effect
273     %is statistically not well quantified).
274 claudioc 1.2
275     An additional source of concern is that the CMS Madgraph $t\bar{t}$ MC does
276     not include effects of spin correlations between the two top quarks.
277     We have studied this effect at the generator level using Alpgen. We find
278 claudioc 1.7 that the bias is at the few percent level.
279 claudioc 1.2
280 benhoob 1.31 Based on the results of Table~\ref{tab:victorysyst}, we conclude that the
281 benhoob 1.28 naive data-driven background estimate based on $P_T{(\ell\ell)}$ needs to
282 benhoob 1.31 be corrected by a factor of $ K_C = 1.4 \pm 0.2({\rm stat})$.
283 claudioc 1.2
284 benhoob 1.31 The dominant sources of systematic uncertainty in $K_C$ are due to non-$t\bar{t} \to$~dilepton backgrounds,
285     and the MET scale and resolution uncertainties, as summarized in Table~\ref{tab:victorysyst}.
286     The impact of non-$t\bar{t}$-dilepton background is assessed
287     by varying the yield of all backgrounds except for $t\bar{t} \to$~dilepton.
288     The uncertainty is assessed as the larger of the differences between the nominal $K_C$ value and the values
289 benhoob 1.28 obtained using only $t\bar{t} \to$~dilepton MC and obtained by doubling the non $t\bar{t} \to$~dilepton component,
290     giving an uncertainty of $0.04$.
291    
292     The uncertainty in $K_C$ due to the MET scale uncertainty is assessed by varying the hadronic energy scale using
293 benhoob 1.32 the same method as in~\cite{ref:top}, giving an uncertainty of 0.3. We also assess the impact of the MET resolution
294 benhoob 1.31 uncertainty on $K_C$ by applying a random smearing to the MET. For each event, we determine the expected MET resolution
295     based on the sumJetPt, and smear the MET to simulate an increase in the resolution of 10\%, 20\%, 30\%, 40\% and 50\%.
296     The results show that $K_C$ does not depend strongly on the MET resolution and we therefore do not assess any uncertainty.
297 claudioc 1.2
298 benhoob 1.28 Incorporating all the statistical and systematic uncertainties we find $K_C = 1.4 \pm 0.4$.
299 claudioc 1.6
300 claudioc 1.2 \subsection{Signal Contamination}
301     \label{sec:sigcont}
302    
303 claudioc 1.6 All data-driven methods are in principle subject to signal contaminations
304 claudioc 1.2 in the control regions, and the methods described in
305     Sections~\ref{sec:abcd} and~\ref{sec:victory} are not exceptions.
306     Signal contamination tends to dilute the significance of a signal
307     present in the data by inflating the background prediction.
308    
309     It is hard to quantify how important these effects are because we
310     do not know what signal may be hiding in the data. Having two
311     independent methods (in addition to Monte Carlo ``dead-reckoning'')
312     adds redundancy because signal contamination can have different effects
313     in the different control regions for the two methods.
314     For example, in the extreme case of a
315     new physics signal
316 claudioc 1.6 with $P_T(\ell \ell) = \met$, an excess of events would be seen
317 claudioc 1.2 in the ABCD method but not in the $P_T(\ell \ell)$ method.
318    
319 claudioc 1.4
320 claudioc 1.2 The LM points are benchmarks for SUSY analyses at CMS. The effects
321     of signal contaminations for a couple such points are summarized
322 benhoob 1.14 in Table~\ref{tab:sigcont}. Signal contamination is definitely an important
323 claudioc 1.2 effect for these two LM points, but it does not totally hide the
324     presence of the signal.
325 claudioc 1.1
326    
327 claudioc 1.2 \begin{table}[htb]
328     \begin{center}
329 benhoob 1.14 \caption{\label{tab:sigcont} Effects of signal contamination
330     for the two data-driven background estimates. The three columns give
331     the expected yield in the signal region and the background estimates
332 benhoob 1.20 using the ABCD and $P_T(\ell \ell)$ methods. Results are normalized to 35~pb$^{-1}$.}
333 benhoob 1.14 \begin{tabular}{lccc}
334 claudioc 1.2 \hline
335 benhoob 1.14 & Yield & ABCD & $P_T(\ell \ell)$ \\
336     \hline
337 benhoob 1.27 SM only & 1.29 & 1.25 & 0.92 \\
338     SM + LM0 & 7.57 & 4.44 & 1.96 \\
339     SM + LM1 & 3.85 & 1.60 & 1.43 \\
340 claudioc 1.2 \hline
341     \end{tabular}
342     \end{center}
343     \end{table}
344