ViewVC Help
View File | Revision Log | Show Annotations | Root Listing
root/cvsroot/UserCode/claudioc/OSNote2010/datadriven.tex
Revision: 1.30
Committed: Thu Dec 2 15:07:05 2010 UTC (14 years, 5 months ago) by benhoob
Content type: application/x-tex
Branch: MAIN
Changes since 1.29: +2 -2 lines
Log Message:
Fixed bug=true

File Contents

# Content
1 \section{Data Driven Background Estimation Methods}
2 \label{sec:datadriven}
3 We have developed two data-driven methods to
4 estimate the background in the signal region.
5 The first one exploits the fact that
6 SumJetPt and \met$/\sqrt{\rm SumJetPt}$ are nearly
7 uncorrelated for the $t\bar{t}$ background
8 (Section~\ref{sec:abcd}); the second one
9 is based on the fact that in $t\bar{t}$ the
10 $P_T$ of the dilepton pair is on average
11 nearly the same as the $P_T$ of the pair of neutrinos
12 from $W$-decays, which is reconstructed as \met in the
13 detector.
14
15
16 %{\color{red} I took these
17 %numbers from the twiki, rescaling from 11.06 to 30/pb.
18 %They seem too large...are they really right?}
19
20
21 \subsection{ABCD method}
22 \label{sec:abcd}
23
24 We find that in $t\bar{t}$ events SumJetPt and
25 \met$/\sqrt{\rm SumJetPt}$ are nearly uncorrelated,
26 as demonstrated in Figure~\ref{fig:uncor}.
27 Thus, we can use an ABCD method in the \met$/\sqrt{\rm SumJetPt}$ vs
28 sumJetPt plane to estimate the background in a data driven way.
29
30 %\begin{figure}[bht]
31 %\begin{center}
32 %\includegraphics[width=0.75\linewidth]{uncorrelated.pdf}
33 %\caption{\label{fig:uncor}\protect Distributions of SumJetPt
34 %in MC $t\bar{t}$ events for different intervals of
35 %MET$/\sqrt{\rm SumJetPt}$.}
36 %\end{center}
37 %\end{figure}
38
39 \begin{figure}[bht]
40 \begin{center}
41 \includegraphics[width=0.75\linewidth]{uncor.png}
42 \caption{\label{fig:uncor}\protect Distributions of SumJetPt
43 in MC $t\bar{t}$ events for different intervals of
44 MET$/\sqrt{\rm SumJetPt}$. h1, h2, and h3 refer to the MET$/\sqrt{\rm SumJetPt}$
45 intervals 4.5-6.5, 6.5-8.5 and $>$8.5, respectively.}
46 \end{center}
47 \end{figure}
48
49 \begin{figure}[tb]
50 \begin{center}
51 \includegraphics[width=0.5\linewidth, angle=90]{abcdMC.pdf}
52 \caption{\label{fig:abcdMC}\protect Distributions of MET$/\sqrt{\rm SumJetPt}$ vs.
53 SumJetPt for SM Monte Carlo. Here we also show our choice of ABCD regions.}
54 \end{center}
55 \end{figure}
56
57
58 Our choice of ABCD regions is shown in Figure~\ref{fig:abcdMC}.
59 The signal region is region D. The expected number of events
60 in the four regions for the SM Monte Carlo, as well as the background
61 prediction A $\times$ C / B are given in Table~\ref{tab:abcdMC} for an integrated
62 luminosity of 35 pb$^{-1}$. The ABCD method with chosen boundaries is accurate
63 to about 20\%, and we assess a corresponding systematic uncertainty.
64
65 %As shown in Table~\ref{tab:abcdsyst}, we assess systematic uncertainties
66 %by varying the boundaries by an amount consistent with the hadronic energy scale uncertainty,
67 %which we take as $\pm$5\% for SumJetPt and $\pm$2.5\% for MET/$\sqrt{\rm SumJetPt}$, since the
68 %uncertainty on this quantity partially cancels due to the fact that it is a ratio of correlated
69 %quantities. Based on these studies we assess a correction factor $k_{ABCD} = 1.2 \pm 0.2$ to the
70 %predicted yield using the ABCD method.
71
72
73 %{\color{red} Avi wants some statement about stability
74 %wrt changes in regions. I am not sure that we have done it and
75 %I am not sure it is necessary (Claudio).}
76
77 \begin{table}[ht]
78 \begin{center}
79 \caption{\label{tab:abcdMC} Expected SM Monte Carlo yields for
80 35 pb$^{-1}$ in the ABCD regions, as well as the predicted yield in
81 the signal region given by A $\times$ C / B. Here `SM other' is the sum
82 of non-dileptonic $t\bar{t}$ decays, $W^{\pm}$+jets, $W^+W^-$,
83 $W^{\pm}Z^0$, $Z^0Z^0$ and single top.}
84 \begin{tabular}{lccccc}
85 \hline
86 sample & A & B & C & D & A $\times$ C / B \\
87 \hline
88 $t\bar{t}\rightarrow \ell^{+}\ell^{-}$ & 8.27 $\pm$ 0.18 & 32.16 $\pm$ 0.35 & 4.69 $\pm$ 0.13 & 1.05 $\pm$ 0.06 & 1.21 $\pm$ 0.04 \\
89 $Z^0 \rightarrow \ell^{+}\ell^{-}$ & 0.22 $\pm$ 0.11 & 1.54 $\pm$ 0.29 & 0.05 $\pm$ 0.05 & 0.16 $\pm$ 0.09 & 0.01 $\pm$ 0.01 \\
90 SM other & 0.54 $\pm$ 0.03 & 2.28 $\pm$ 0.12 & 0.23 $\pm$ 0.03 & 0.07 $\pm$ 0.01 & 0.05 $\pm$ 0.01 \\
91 \hline
92 total SM MC & 9.03 $\pm$ 0.21 & 35.97 $\pm$ 0.46 & 4.97 $\pm$ 0.15 & 1.29 $\pm$ 0.11 & 1.25 $\pm$ 0.05 \\
93 \hline
94 \end{tabular}
95 \end{center}
96 \end{table}
97
98
99
100 \begin{table}[ht]
101 \begin{center}
102 \caption{\label{tab:abcdsyst}
103 {\bf \color{red} Do we need this study at all? Observed/predicted is consistent within stat uncertainties as the boundaries are varied- is it enough to simply state this fact in the text??? }
104 Results of the systematic study of the ABCD method by varying the boundaries
105 between the ABCD regions shown in Fig.~\ref{fig:abcdMC}. Here $x_1$ is the lower SumJetPt boundary and
106 $x_2$ is the boundary separating regions A and B from C and D, their nominal values are 125 and 300~GeV,
107 respectively. $y_1$ is the lower MET/$\sqrt{\rm SumJetPt}$ boundary and
108 $y_2$ is the boundary separating regions B and C from A and D, their nominal values are 4.5 and 8.5~GeV$^{1/2}$,
109 respectively.}
110 \begin{tabular}{cccc|c}
111 \hline
112 $x_1$ & $x_2$ & $y_1$ & $y_2$ & Observed/Predicted \\
113 \hline
114 nominal & nominal & nominal & nominal & $1.20 \pm 0.12$ \\
115 +5\% & +5\% & +2.5\% & +2.5\% & $1.38 \pm 0.15$ \\
116 +5\% & +5\% & nominal & nominal & $1.31 \pm 0.14$ \\
117 nominal & nominal & +2.5\% & +2.5\% & $1.25 \pm 0.13$ \\
118 nominal & +5\% & nominal & +2.5\% & $1.32 \pm 0.14$ \\
119 nominal & -5\% & nominal & -2.5\% & $1.16 \pm 0.09$ \\
120 -5\% & -5\% & +2.5\% & +2.5\% & $1.21 \pm 0.11$ \\
121 +5\% & +5\% & -2.5\% & -2.5\% & $1.26 \pm 0.12$ \\
122 \hline
123 \end{tabular}
124 \end{center}
125 \end{table}
126
127 \subsection{Dilepton $P_T$ method}
128 \label{sec:victory}
129 This method is based on a suggestion by V. Pavlunin\cite{ref:victory},
130 and was investigated by our group in 2009\cite{ref:ourvictory}.
131 The idea is that in dilepton $t\bar{t}$ events the lepton and neutrinos
132 from $W$ decays have the same $P_T$ spectrum (modulo $W$ polarization
133 effects). One can then use the observed
134 $P_T(\ell\ell)$ distribution to model the sum of neutrino $P_T$'s which
135 is identified with the \met.
136
137 Then, in order to predict the $t\bar{t} \to$ dilepton contribution to a
138 selection with \met$+$X, one applies a cut on $P_T(\ell\ell)+$X instead.
139 In practice one has to rescale the result of the $P_T(\ell\ell)+$X selection
140 to account for the fact that any dilepton selection must include a
141 moderate \met cut in order to reduce Drell Yan backgrounds. This
142 is discussed in Section 5.3 of Reference~\cite{ref:ourvictory}; for a \met
143 cut of 50 GeV, the rescaling factor is obtained from the MC as
144
145 \newcommand{\ptll} {\ensuremath{P_T(\ell\ell)}}
146 \begin{center}
147 $ K = \frac{\int_0^{\infty} {\cal N}(\ptll)~~d\ptll~}{\int_{50}^{\infty} {\cal N}(\ptll)~~d\ptll~}$
148 \end{center}
149
150
151 Monte Carlo studies give values of $K$ that are typically between 1.5 and 1.6,
152 depending on selection details.
153 %%%TO BE REPLACED
154 %Given the integrated luminosity of the
155 %present dataset, the determination of $K$ in data is severely statistics
156 %limited. Thus, we take $K$ from $t\bar{t}$ Monte Carlo as
157
158 %\begin{center}
159 %$ K_{MC} = \frac{\int_0^{\infty} {\cal N}(\met)~~d\met~}{\int_{50}^{\infty} {\cal N}(\met)~~d\met~}$
160 %\end{center}
161
162 %\noindent {\color{red} For the 11 pb result we have used $K$ from data.}
163
164 There are several effects that spoil the correspondance between \met and
165 $P_T(\ell\ell)$:
166 \begin{itemize}
167 \item $Ws$ in top events are polarized. Neutrinos are emitted preferentially
168 parallel to the $W$ velocity while charged leptons are emitted prefertially
169 anti-parallel. Thus the $P_T(\nu\nu)$ distribution is harder
170 than the $P_T(\ell\ell)$ distribution for top dilepton events.
171 \item The lepton selections results in $P_T$ and $\eta$ cuts on the individual
172 leptons that have no simple correspondance to the neutrino requirements.
173 \item Similarly, the \met$>$50 GeV cut introduces an asymmetry between leptons and
174 neutrinos which is only partially compensated by the $K$ factor above.
175 \item The \met resolution is much worse than the dilepton $P_T$ resolution.
176 When convoluted with a falling spectrum in the tails of \met, this results
177 in a harder spectrum for \met than the original $P_T(\nu\nu)$.
178 \item The \met response in CMS is not exactly 1. This causes a distortion
179 in the \met distribution that is not present in the $P_T(\ell\ell)$ distribution.
180 \item The $t\bar{t} \to$ dilepton signal includes contributions from
181 $W \to \tau \to \ell$. For these events the arguments about the equivalence
182 of $P_T(\ell\ell)$ and $P_T(\nu\nu)$ do not apply.
183 \item A dilepton selection will include SM events from non $t\bar{t}$
184 sources. These events can affect the background prediction. Particularly
185 dangerous are high $P_T$ Drell Yan events that barely pass the \met$>$ 50
186 GeV selection. They will tend to push the data-driven background prediction up.
187 Therefore we estimate the number of DY events entering the background prediction
188 using the $R_{out/in}$ method as described in Sec.~\ref{sec:othBG}.
189 \end{itemize}
190
191 We have studied these effects in SM Monte Carlo, using a mixture of generator and
192 reconstruction level studies, putting the various effects in one at a time.
193 For each configuration, we apply the data-driven method and report as figure
194 of merit the ratio of observed and predicted events in the signal region.
195 The results are summarized in Table~\ref{tab:victorybad}.
196
197 \begin{table}[htb]
198 \begin{center}
199 \caption{\label{tab:victorybad}
200 {\bf \color{red} Need to either update this with 38X MC or remove it }
201 Test of the data driven method in Monte Carlo
202 under different assumptions. See text for details.}
203 \begin{tabular}{|l|c|c|c|c|c|c|c|c|}
204 \hline
205 & True $t\bar{t}$ dilepton & $t\to W\to\tau$& other SM & GEN or & Lepton $P_T$ & Z veto & \met $>$ 50& obs/pred \\
206 & included & included & included & RECOSIM & and $\eta$ cuts & & & \\ \hline
207 1&Y & N & N & GEN & N & N & N & 1.90 \\
208 2&Y & N & N & GEN & Y & N & N & 1.64 \\
209 3&Y & N & N & GEN & Y & Y & N & 1.59 \\
210 4&Y & N & N & GEN & Y & Y & Y & 1.55 \\
211 5&Y & N & N & RECOSIM & Y & Y & Y & 1.51 \\
212 6&Y & Y & N & RECOSIM & Y & Y & Y & 1.58 \\
213 7&Y & Y & Y & RECOSIM & Y & Y & Y & 1.38 \\
214 \hline
215 \end{tabular}
216 \end{center}
217 \end{table}
218
219
220 \begin{table}[htb]
221 \begin{center}
222 \caption{\label{tab:victorysyst}
223 Summary of uncertainties in $K_C$ due to the MET scale and resolution uncertainty, and to backgrounds other than $t\bar{t} \to$~dilepton.
224 In the first table, `up' and `down' refer to shifting the hadronic energy scale up and down by 5\%. In the second table, the quoted value
225 refers to the amount of additional smearing of the MET, as discussed in the text. In the third table, the normalization of all backgrounds
226 other than $t\bar{t} \to$~dilepton is varied.
227 {\bf \color{red} Should I remove `observed' and `predicted' and show only the ratio? }}
228
229 \begin{tabular}{ lcccc }
230 \hline
231 MET scale & Predicted & Observed & Obs/pred \\
232 \hline
233 nominal & 0.92 $ \pm $ 0.11 & 1.29 $ \pm $ 0.11 & 1.40 $ \pm $ 0.20 \\
234 up & 0.92 $ \pm $ 0.11 & 1.53 $ \pm $ 0.12 & 1.66 $ \pm $ 0.23 \\
235 down & 0.81 $ \pm $ 0.07 & 1.08 $ \pm $ 0.11 & 1.32 $ \pm $ 0.17 \\
236 \hline
237
238 \hline
239 MET smearing & Predicted & Observed & Obs/pred \\
240 \hline
241 nominal & 0.92 $ \pm $ 0.11 & 1.29 $ \pm $ 0.11 & 1.40 $ \pm $ 0.20 \\
242 10\% & 0.90 $ \pm $ 0.11 & 1.30 $ \pm $ 0.11 & 1.44 $ \pm $ 0.21 \\
243 20\% & 0.84 $ \pm $ 0.07 & 1.36 $ \pm $ 0.11 & 1.61 $ \pm $ 0.19 \\
244 30\% & 1.05 $ \pm $ 0.18 & 1.32 $ \pm $ 0.11 & 1.27 $ \pm $ 0.24 \\
245 40\% & 0.85 $ \pm $ 0.07 & 1.37 $ \pm $ 0.11 & 1.61 $ \pm $ 0.19 \\
246 50\% & 1.08 $ \pm $ 0.18 & 1.36 $ \pm $ 0.11 & 1.26 $ \pm $ 0.24 \\
247 \hline
248
249 \hline
250 non-$t\bar{t} \to$~dilepton scale factor & Predicted & Observed & Obs/pred \\
251 \hline
252 ttdil only & 0.77 $ \pm $ 0.07 & 1.05 $ \pm $ 0.06 & 1.36 $ \pm $ 0.14 \\
253 nominal & 0.92 $ \pm $ 0.11 & 1.29 $ \pm $ 0.11 & 1.40 $ \pm $ 0.20 \\
254 double non-ttdil yield & 1.06 $ \pm $ 0.18 & 1.52 $ \pm $ 0.20 & 1.43 $ \pm $ 0.30 \\
255 \hline
256 \end{tabular}
257 \end{center}
258 \end{table}
259
260
261
262 The largest discrepancy between prediction and observation occurs on the first
263 line of Table~\ref{tab:victorybad}, {\em i.e.}, at the generator level with no
264 cuts. We have verified that this effect is due to the polarization of
265 the $W$ (we remove the polarization by reweighting the events and we get
266 good agreement between prediction and observation). The kinematical
267 requirements (lines 2,3,4) compensate somewhat for the effect of W polarization.
268 Going from GEN to RECOSIM, the change in observed/predicted is small.
269 % We have tracked this down to the fact that tcMET underestimates the true \met
270 % by $\approx 4\%$\footnote{We find that observed/predicted changes by roughly 0.1
271 %for each 1.5\% change in \met response.}.
272 Finally, contamination from non $t\bar{t}$
273 events can have a significant impact on the BG prediction.
274 %The changes between
275 %lines 6 and 7 of Table~\ref{tab:victorybad} is driven by 3
276 %Drell Yan events that pass the \met selection in Monte Carlo (thus the effect
277 %is statistically not well quantified).
278
279 An additional source of concern is that the CMS Madgraph $t\bar{t}$ MC does
280 not include effects of spin correlations between the two top quarks.
281 We have studied this effect at the generator level using Alpgen. We find
282 that the bias is at the few percent level.
283
284 Based on the results of Table~\ref{tab:victorybad}, we conclude that the
285 naive data-driven background estimate based on $P_T{(\ell\ell)}$ needs to
286 be corrected by a factor of $ K_C = 1.4 \pm 0.2(stat)$.
287
288 The 2 dominant sources of systematic uncertainty in $K_C$ are due to non-$t\bar{t} \to$~dilepton backgrounds,
289 and the MET scale and resolution uncertainties. The impact of non-$t\bar{t}$-dilepton background is assessed
290 by varying the yield of all backgrounds except for $t\bar{t} \to$~dilepton, as shown in Table~\ref{table_kc}.
291 The systematic is assessed as the larger of the differences between the nominal $K_C$ value and the values
292 obtained using only $t\bar{t} \to$~dilepton MC and obtained by doubling the non $t\bar{t} \to$~dilepton component,
293 giving an uncertainty of $0.04$.
294
295 The uncertainty in $K_C$ due to the MET scale uncertainty is assessed by varying the hadronic energy scale using
296 the same method as in~\ref{} and checking how much $K_C$ changes, as summarized in Table~\ref{tab:victorysyst}.
297 This gives an uncertainty of 0.3. We also assess the impact of the MET resolution uncertainty on $K_C$ by applying
298 a random smearing to the MET. For each event, we determine the expected MET resolution based on the sumJetPt, and
299 smear the MET to simulate an increase in the resolution of 10\%, 20\%, 30\%, 40\% and 50\%. The results show that
300 $K_C$ does not depend strongly on the MET resolution and we therefore do not assess any uncertainty.
301
302 Incorporating all the statistical and systematic uncertainties we find $K_C = 1.4 \pm 0.4$.
303
304 \subsection{Signal Contamination}
305 \label{sec:sigcont}
306
307 All data-driven methods are in principle subject to signal contaminations
308 in the control regions, and the methods described in
309 Sections~\ref{sec:abcd} and~\ref{sec:victory} are not exceptions.
310 Signal contamination tends to dilute the significance of a signal
311 present in the data by inflating the background prediction.
312
313 It is hard to quantify how important these effects are because we
314 do not know what signal may be hiding in the data. Having two
315 independent methods (in addition to Monte Carlo ``dead-reckoning'')
316 adds redundancy because signal contamination can have different effects
317 in the different control regions for the two methods.
318 For example, in the extreme case of a
319 new physics signal
320 with $P_T(\ell \ell) = \met$, an excess of events would be seen
321 in the ABCD method but not in the $P_T(\ell \ell)$ method.
322
323
324 The LM points are benchmarks for SUSY analyses at CMS. The effects
325 of signal contaminations for a couple such points are summarized
326 in Table~\ref{tab:sigcont}. Signal contamination is definitely an important
327 effect for these two LM points, but it does not totally hide the
328 presence of the signal.
329
330
331 \begin{table}[htb]
332 \begin{center}
333 \caption{\label{tab:sigcont} Effects of signal contamination
334 for the two data-driven background estimates. The three columns give
335 the expected yield in the signal region and the background estimates
336 using the ABCD and $P_T(\ell \ell)$ methods. Results are normalized to 35~pb$^{-1}$.}
337 \begin{tabular}{lccc}
338 \hline
339 & Yield & ABCD & $P_T(\ell \ell)$ \\
340 \hline
341 SM only & 1.29 & 1.25 & 0.92 \\
342 SM + LM0 & 7.57 & 4.44 & 1.96 \\
343 SM + LM1 & 3.85 & 1.60 & 1.43 \\
344 \hline
345 \end{tabular}
346 \end{center}
347 \end{table}
348