ViewVC Help
View File | Revision Log | Show Annotations | Root Listing
root/cvsroot/UserCode/claudioc/OSNote2010/datadriven.tex
Revision: 1.39
Committed: Tue Dec 14 05:53:07 2010 UTC (14 years, 4 months ago) by claudioc
Content type: application/x-tex
Branch: MAIN
CVS Tags: v3
Changes since 1.38: +51 -2 lines
Log Message:
version 3

File Contents

# Content
1 \section{Data Driven Background Estimation Methods}
2 \label{sec:datadriven}
3 We have developed two data-driven methods to
4 estimate the background in the signal region.
5 The first one exploits the fact that
6 SumJetPt and \met$/\sqrt{\rm SumJetPt}$ are nearly
7 uncorrelated for the $t\bar{t}$ background
8 (Section~\ref{sec:abcd}); the second one
9 is based on the fact that in $t\bar{t}$ the
10 $P_T$ of the dilepton pair is on average
11 nearly the same as the $P_T$ of the pair of neutrinos
12 from $W$-decays, which is reconstructed as \met in the
13 detector.
14
15
16 %{\color{red} I took these
17 %numbers from the twiki, rescaling from 11.06 to 30/pb.
18 %They seem too large...are they really right?}
19
20
21 \subsection{ABCD method}
22 \label{sec:abcd}
23
24 We find that in $t\bar{t}$ events SumJetPt and
25 \met$/\sqrt{\rm SumJetPt}$ are nearly uncorrelated,
26 as demonstrated in Fig.~\ref{fig:uncor}.
27 Thus, we can use an ABCD method in the \met$/\sqrt{\rm SumJetPt}$ vs
28 sumJetPt plane to estimate the background in a data driven way.
29
30 %\begin{figure}[bht]
31 %\begin{center}
32 %\includegraphics[width=0.75\linewidth]{uncorrelated.pdf}
33 %\caption{\label{fig:uncor}\protect Distributions of SumJetPt
34 %in MC $t\bar{t}$ events for different intervals of
35 %MET$/\sqrt{\rm SumJetPt}$.}
36 %\end{center}
37 %\end{figure}
38
39 \begin{figure}[bht]
40 \begin{center}
41 \includegraphics[width=0.75\linewidth]{uncor.png}
42 \caption{\label{fig:uncor}\protect Distributions of SumJetPt
43 in MC $t\bar{t}$ events for different intervals of
44 MET$/\sqrt{\rm SumJetPt}$. h1, h2, and h3 refer to the MET$/\sqrt{\rm SumJetPt}$
45 intervals 4.5-6.5, 6.5-8.5 and $>$8.5, respectively. }
46 \end{center}
47 \end{figure}
48
49 \begin{figure}[tb]
50 \begin{center}
51 \includegraphics[width=0.75\linewidth]{ttdil_uncor_38X.png}
52 \caption{\label{fig:abcdMC}\protect Distributions of MET$/\sqrt{\rm SumJetPt}$ vs.
53 SumJetPt for SM Monte Carlo. Here we also show our choice of ABCD regions. The correlation coefficient
54 ${\rm corr_{XY}}$ is computed for events falling in the ABCD regions.}
55 \end{center}
56 \end{figure}
57
58
59 Our choice of ABCD regions is shown in Figure~\ref{fig:abcdMC}.
60 The signal region is region D. The expected number of events
61 in the four regions for the SM Monte Carlo, as well as the background
62 prediction A $\times$ C / B are given in Table~\ref{tab:abcdMC} for an integrated
63 luminosity of 34.0 pb$^{-1}$. In Table~\ref{tab:abcdsyst}, we test the stability of
64 observed/predicted with respect to variations in the ABCD boundaries.
65 Based on the results in Tables~\ref{tab:abcdMC} and~\ref{tab:abcdsyst}, we assess
66 a systematic uncertainty of 20\% on the prediction of the ABCD method.
67
68 %As shown in Table~\ref{tab:abcdsyst}, we assess systematic uncertainties
69 %by varying the boundaries by an amount consistent with the hadronic energy scale uncertainty,
70 %which we take as $\pm$5\% for SumJetPt and $\pm$2.5\% for MET/$\sqrt{\rm SumJetPt}$, since the
71 %uncertainty on this quantity partially cancels due to the fact that it is a ratio of correlated
72 %quantities. Based on these studies we assess a correction factor $k_{ABCD} = 1.2 \pm 0.2$ to the
73 %predicted yield using the ABCD method.
74
75
76 %{\color{red} Avi wants some statement about stability
77 %wrt changes in regions. I am not sure that we have done it and
78 %I am not sure it is necessary (Claudio).}
79
80 \begin{table}[ht]
81 \begin{center}
82 \caption{\label{tab:abcdMC} Expected SM Monte Carlo yields for
83 34.0~pb$^{-1}$ in the ABCD regions, as well as the predicted yield in
84 the signal region given by A $\times$ C / B. Here `SM other' is the sum
85 of non-dileptonic $t\bar{t}$ decays, $W^{\pm}$+jets, $W^+W^-$,
86 $W^{\pm}Z^0$, $Z^0Z^0$ and single top.}
87 \begin{tabular}{lccccc}
88 %%%official json v3, 38X MC (D6T ttbar and DY)
89 \hline
90 sample & A & B & C & D & PRED \\
91 \hline
92 $t\bar{t}\rightarrow \ell^{+}\ell^{-}$ & 8.44 $\pm$ 0.18 & 32.83 $\pm$ 0.35 & 4.78 $\pm$ 0.14 & 1.07 $\pm$ 0.06 & 1.23 $\pm$ 0.05 \\
93 $Z^0 \rightarrow \ell^{+}\ell^{-}$ & 0.17 $\pm$ 0.08 & 1.18 $\pm$ 0.22 & 0.04 $\pm$ 0.04 & 0.12 $\pm$ 0.07 & 0.01 $\pm$ 0.01 \\
94 SM other & 0.53 $\pm$ 0.03 & 2.26 $\pm$ 0.11 & 0.23 $\pm$ 0.03 & 0.07 $\pm$ 0.01 & 0.05 $\pm$ 0.01 \\
95 \hline
96 total SM MC & 9.14 $\pm$ 0.20 & 36.26 $\pm$ 0.43 & 5.05 $\pm$ 0.14 & 1.27 $\pm$ 0.10 & 1.27 $\pm$ 0.05 \\
97 \hline
98 \end{tabular}
99 \end{center}
100 \end{table}
101
102
103
104 \begin{table}[ht]
105 \begin{center}
106 \caption{\label{tab:abcdsyst}
107 Results of the systematic study of the ABCD method by varying the boundaries
108 between the ABCD regions shown in Fig.~\ref{fig:abcdMC}. Here $x_1$ is the lower SumJetPt boundary and
109 $x_2$ is the boundary separating regions A and B from C and D, their nominal values are 125 and 300~GeV,
110 respectively. $y_1$ is the lower MET/$\sqrt{\rm SumJetPt}$ boundary and
111 $y_2$ is the boundary separating regions B and C from A and D, their nominal values are 4.5 and 8.5~GeV$^{1/2}$,
112 respectively.}
113 \begin{tabular}{cccc|c}
114 \hline
115 $x_1$ & $x_2$ & $y_1$ & $y_2$ & Observed/Predicted \\
116 \hline
117
118 nominal & nominal & nominal & nominal & $1.00 \pm 0.08$ \\
119
120 +5\% & +5\% & +2.5\% & +2.5\% & $1.08 \pm 0.11$ \\
121
122 +5\% & +5\% & nominal & nominal & $1.04 \pm 0.10$ \\
123
124 nominal & nominal & +2.5\% & +2.5\% & $1.03 \pm 0.09$ \\
125
126 nominal & +5\% & nominal & +2.5\% & $1.05 \pm 0.10$ \\
127
128 nominal & -5\% & nominal & -2.5\% & $0.95 \pm 0.07$ \\
129
130 -5\% & -5\% & +2.5\% & +2.5\% & $1.00 \pm 0.08$ \\
131
132 +5\% & +5\% & -2.5\% & -2.5\% & $0.98 \pm 0.09$ \\
133 \hline
134 \end{tabular}
135 \end{center}
136 \end{table}
137
138
139 \clearpage
140
141 \subsection{Dilepton $P_T$ method}
142 \label{sec:victory}
143 This method is based on a suggestion by V. Pavlunin\cite{ref:victory},
144 and was investigated by our group in 2009\cite{ref:ourvictory}.
145 The idea is that in dilepton $t\bar{t}$ events the lepton and neutrinos
146 from $W$ decays have the same $P_T$ spectrum (modulo $W$ polarization
147 effects). One can then use the observed
148 $P_T(\ell\ell)$ distribution to model the sum of neutrino $P_T$'s which
149 is identified with the \met.
150
151 Then, in order to predict the $t\bar{t} \to$ dilepton contribution to a
152 selection with \met$+$X, one applies a cut on $P_T(\ell\ell)+$X instead.
153 In practice one has to rescale the result of the $P_T(\ell\ell)+$X selection
154 to account for the fact that any dilepton selection must include a
155 moderate \met cut in order to reduce Drell Yan backgrounds. This
156 is discussed in Section 5.3 of Reference~\cite{ref:ourvictory}; for a \met
157 cut of 50 GeV, the rescaling factor is obtained from the MC as
158
159 \newcommand{\ptll} {\ensuremath{P_T(\ell\ell)}}
160 \begin{center}
161 $ K = \frac{\int_0^{\infty} {\cal N}(\ptll)~~d\ptll~}{\int_{50}^{\infty} {\cal N}(\ptll)~~d\ptll~} = 1.5$
162 \end{center}
163
164
165 %%%TO BE REPLACED
166 %Given the integrated luminosity of the
167 %present dataset, the determination of $K$ in data is severely statistics
168 %limited. Thus, we take $K$ from $t\bar{t}$ Monte Carlo as
169
170 %\begin{center}
171 %$ K_{MC} = \frac{\int_0^{\infty} {\cal N}(\met)~~d\met~}{\int_{50}^{\infty} {\cal N}(\met)~~d\met~}$
172 %\end{center}
173
174 %\noindent {\color{red} For the 11 pb result we have used $K$ from data.}
175
176
177 \begin{figure}[bht]
178 \begin{center}
179 \includegraphics[width=0.75\linewidth]{genvictory_Dec13.png}
180 \caption{\label{fig:genvictory}\protect Distributions $P_T(\ell \ell)$
181 and $P_T(\nu \nu)$ (aka {\it genmet})
182 in $t\bar{t} \to$ dilepton Monte Carlo at the
183 generator level. Events with $W \to \tau \to \ell$ are not included.
184 No kinematical requirements have been made.}
185 \end{center}
186 \end{figure}
187
188
189 There are several effects that spoil the correspondance between \met and
190 $P_T(\ell\ell)$:
191 \begin{itemize}
192 \item $Ws$ in top events are polarized. Neutrinos are emitted preferentially
193 parallel to the $W$ velocity while charged leptons are emitted prefertially
194 anti-parallel. Thus the $P_T(\nu\nu)$ distribution is harder
195 than the $P_T(\ell\ell)$ distribution for top dilepton events.
196 This turns out to be the dominant effect and it is illustrated in
197 Figure~\ref{fig:genvictory}.
198 \item The lepton selections results in $P_T$ and $\eta$ cuts on the individual
199 leptons that have no simple correspondance to the neutrino requirements.
200 \item Similarly, the \met$>$50 GeV cut introduces an asymmetry between leptons and
201 neutrinos which is only partially compensated by the $K$ factor above.
202 \item The \met resolution is much worse than the dilepton $P_T$ resolution.
203 When convoluted with a falling spectrum in the tails of \met, this results
204 in a harder spectrum for \met than the original $P_T(\nu\nu)$.
205 \item The \met response in CMS is not exactly 1. This causes a distortion
206 in the \met distribution that is not present in the $P_T(\ell\ell)$ distribution.
207 \item The $t\bar{t} \to$ dilepton signal includes contributions from
208 $W \to \tau \to \ell$. For these events the arguments about the equivalence
209 of $P_T(\ell\ell)$ and $P_T(\nu\nu)$ do not apply.
210 \item A dilepton selection will include SM events from non $t\bar{t}$
211 sources. These events can affect the background prediction. Particularly
212 dangerous are high $P_T$ Drell Yan events that barely pass the \met$>$ 50
213 GeV selection. They will tend to push the data-driven background prediction up.
214 Therefore we estimate the number of DY events entering the background prediction
215 using the $R_{out/in}$ method as described in Sec.~\ref{sec:othBG}.
216 \end{itemize}
217
218 We have studied these effects in SM Monte Carlo, using a mixture of generator and
219 reconstruction level studies, putting the various effects in one at a time.
220 For each configuration, we apply the data-driven method and report as figure
221 of merit the ratio of observed and predicted events in the signal region.
222 The figure of merit is calculated as follows
223 \begin{itemize}
224 \item We construct \met/$\sqrt{{\rm sumJetPt}}$
225 and $P_T(\ell \ell)/\sqrt{{\rm sumJetPt}}$ (rescaled by the factor $K$ defined
226 above) distributions.
227 \item The distributions are constructed using either
228 GEN or RECO, and including or excluding various effects ({\em e.g.:}
229 $t \to W \to \tau \to \ell$).
230 \item In all cases the $N_{jets} \ge 2$ and
231 sumJetPt $>$ 300 GeV requirements are applied.
232 \item ``observed events'' is the integral of the \met/$\sqrt{{\rm sumJetPt}}$ distribution
233 above 8.5.
234 \item ``predicted events'' is the integral of the $P_T(\ell \ell)/\sqrt{{\rm sumJetPt}}$ distribution
235 above 8.5.
236 \end{itemize}
237 The results are summarized in Table~\ref{tab:victorybad}. Distributions corresponding to
238 lines 4 and 5 of Table~\ref{tab:victorybad} are shown in Figure~\ref{fig:victorybad}.
239
240 \begin{table}[htb]
241 \begin{center}
242 \caption{\label{tab:victorybad}
243 Test of the data driven method in Monte Carlo
244 under different assumptions, evaluated using Spring10 MC. See text for details.}
245 \begin{tabular}{|l|c|c|c|c|c|c|c|c|}
246 \hline
247 & True $t\bar{t}$ dilepton & $t\to W\to\tau$& other SM & GEN or & Lepton $P_T$ & Z veto & \met $>$ 50& obs/pred \\
248 & included & included & included & RECOSIM & and $\eta$ cuts & & & \\ \hline
249 1&Y & N & N & GEN & N & N & N & 1.90 \\
250 2&Y & N & N & GEN & Y & N & N & 1.64 \\
251 3&Y & N & N & GEN & Y & Y & N & 1.59 \\
252 4&Y & N & N & GEN & Y & Y & Y & 1.55 \\
253 5&Y & N & N & RECOSIM & Y & Y & Y & 1.51 \\
254 6&Y & Y & N & RECOSIM & Y & Y & Y & 1.58 \\
255 7&Y & Y & Y & RECOSIM & Y & Y & Y & 1.38 \\
256 \hline
257 \end{tabular}
258 \end{center}
259 \end{table}
260
261
262 \begin{figure}[bht]
263 \begin{center}
264 \includegraphics[width=0.48\linewidth]{genvictory_sqrtHt_Dec13.png}
265 \includegraphics[width=0.48\linewidth]{victory_Dec13.png}
266 \caption{\label{fig:victorybad}\protect Distributions
267 of MET/$\sqrt{{\rm sumJetPt}}$ (black) and $P_T(\ell \ell)/\sqrt{{\rm sumJetPt}}$
268 (red) in $t\bar{t} \to$ dilepton Monte Carlo
269 after lepton kinematical cuts, $N_{jets} \ge 2$, and
270 sumJetPt $>$ 300 GeV. The left (right) plot is at the GEN (RECO) level
271 and corresponds to line 4 (5) of Table~\ref{tab:victorybad}.}
272 \end{center}
273 \end{figure}
274
275
276
277 \begin{table}[htb]
278 \begin{center}
279 \caption{\label{tab:victorysyst}
280 Summary of variations in $K_C$ due to the MET scale and resolution uncertainty, and to backgrounds other than $t\bar{t} \to$~dilepton.
281 In the first table, `up' and `down' refer to shifting the hadronic energy scale up and down by 5\%. In the second table, the quoted value
282 refers to the amount of additional smearing of the MET, as discussed in the text. In the third table, the normalization of all backgrounds
283 other than $t\bar{t} \to$~dilepton is varied. }
284 \begin{tabular}{ lcccc }
285 \hline
286 MET scale & Predicted & Observed & Obs/pred \\
287 \hline
288 nominal & 0.92 $ \pm $ 0.09 & 1.27 $ \pm $ 0.10 & 1.39 $ \pm $ 0.18 \\
289 up & 0.90 $ \pm $ 0.09 & 1.58 $ \pm $ 0.10 & 1.75 $ \pm $ 0.21 \\
290 down & 0.70 $ \pm $ 0.06 & 0.96 $ \pm $ 0.09 & 1.37 $ \pm $ 0.18 \\
291 \hline
292 MET smearing & Predicted & Observed & Obs/pred \\
293 \hline
294 nominal & 0.92 $ \pm $ 0.09 & 1.27 $ \pm $ 0.10 & 1.39 $ \pm $ 0.18 \\
295 10\% & 0.88 $ \pm $ 0.09 & 1.28 $ \pm $ 0.10 & 1.47 $ \pm $ 0.19 \\
296 20\% & 0.87 $ \pm $ 0.09 & 1.26 $ \pm $ 0.10 & 1.44 $ \pm $ 0.19 \\
297 30\% & 1.03 $ \pm $ 0.17 & 1.33 $ \pm $ 0.10 & 1.29 $ \pm $ 0.23 \\
298 40\% & 0.88 $ \pm $ 0.09 & 1.36 $ \pm $ 0.10 & 1.55 $ \pm $ 0.20 \\
299 50\% & 0.80 $ \pm $ 0.07 & 1.39 $ \pm $ 0.10 & 1.73 $ \pm $ 0.19 \\
300 \hline
301 non-$t\bar{t} \to$~dilepton bkg & Predicted & Observed & Obs/pred \\
302 \hline
303 ttdil only & 0.79 $ \pm $ 0.07 & 1.07 $ \pm $ 0.06 & 1.36 $ \pm $ 0.14 \\
304 nominal & 0.92 $ \pm $ 0.09 & 1.27 $ \pm $ 0.10 & 1.39 $ \pm $ 0.18 \\
305 double non-ttdil yield & 1.04 $ \pm $ 0.15 & 1.47 $ \pm $ 0.16 & 1.40 $ \pm $ 0.25 \\
306 \hline
307 \end{tabular}
308 \end{center}
309 \end{table}
310
311 The largest discrepancy between prediction and observation occurs on the first
312 line of Table~\ref{tab:victorybad}, {\em i.e.}, at the generator level with no
313 cuts. We have verified that this effect is due to the polarization of
314 the $W$ (we remove the polarization by reweighting the events and we get
315 good agreement between prediction and observation). The kinematical
316 requirements (lines 2,3,4) compensate somewhat for the effect of W polarization.
317 Going from GEN to RECOSIM, the change in observed/predicted is small.
318 % We have tracked this down to the fact that tcMET underestimates the true \met
319 % by $\approx 4\%$\footnote{We find that observed/predicted changes by roughly 0.1
320 %for each 1.5\% change in \met response.}.
321 Finally, contamination from non $t\bar{t}$
322 events can have a significant impact on the BG prediction.
323 %The changes between
324 %lines 6 and 7 of Table~\ref{tab:victorybad} is driven by 3
325 %Drell Yan events that pass the \met selection in Monte Carlo (thus the effect
326 %is statistically not well quantified).
327
328 An additional source of concern is that the CMS Madgraph $t\bar{t}$ MC does
329 not include effects of spin correlations between the two top quarks.
330 We have studied this effect at the generator level using Alpgen. We find
331 that the bias is (at most) at the few percent level.
332
333 Based on the results of Table~\ref{tab:victorysyst}, we conclude that the
334 naive data-driven background estimate based on $P_T{(\ell\ell)}$ needs to
335 be corrected by a factor of $ K_C = 1.4 \pm 0.2({\rm stat})$.
336
337 The dominant sources of systematic uncertainty in $K_C$ are due to non-$t\bar{t} \to$~dilepton backgrounds,
338 and the MET scale and resolution uncertainties, as summarized in Table~\ref{tab:victorysyst}.
339 The impact of non-$t\bar{t}$-dilepton background is assessed
340 by varying the yield of all backgrounds except for $t\bar{t} \to$~dilepton.
341 The uncertainty is assessed as the larger of the differences between the nominal $K_C$ value and the values
342 obtained using only $t\bar{t} \to$~dilepton MC and obtained by doubling the non $t\bar{t} \to$~dilepton component,
343 giving an uncertainty of $0.03$.
344
345 The uncertainty in $K_C$ due to the MET scale uncertainty is assessed by varying the hadronic energy scale using
346 the same method as in~\cite{ref:top}, giving an uncertainty of 0.36.
347 We also assess the impact of the MET resolution
348 uncertainty on $K_C$ by applying a random smearing to the MET. For each event, we determine the expected MET resolution
349 based on the sumJetPt, and smear the MET to simulate an increase in the resolution of 10\%, 20\%, 30\%, 40\% and 50\%.
350 The results show that $K_C$ does not depend strongly on the MET resolution and we therefore do not assess any uncertainty.
351
352 Incorporating all the statistical and systematic uncertainties we find $K_C = 1.4 \pm 0.4$.
353
354 \subsection{Signal Contamination}
355 \label{sec:sigcont}
356
357 All data-driven methods are in principle subject to signal contaminations
358 in the control regions, and the methods described in
359 Sections~\ref{sec:abcd} and~\ref{sec:victory} are not exceptions.
360 Signal contamination tends to dilute the significance of a signal
361 present in the data by inflating the background prediction.
362
363 It is hard to quantify how important these effects are because we
364 do not know what signal may be hiding in the data. Having two
365 independent methods (in addition to Monte Carlo ``dead-reckoning'')
366 adds redundancy because signal contamination can have different effects
367 in the different control regions for the two methods.
368 For example, in the extreme case of a
369 new physics signal
370 with $P_T(\ell \ell) = \met$, an excess of events would be seen
371 in the ABCD method but not in the $P_T(\ell \ell)$ method.
372
373
374 The LM points are benchmarks for SUSY analyses at CMS. The effects
375 of signal contaminations for a couple such points are summarized
376 in Table~\ref{tab:sigcont}. Signal contamination is definitely an important
377 effect for these two LM points, but it does not totally hide the
378 presence of the signal.
379
380
381 \begin{table}[htb]
382 \begin{center}
383 \caption{\label{tab:sigcont} Effects of signal contamination
384 for the two data-driven background estimates. The three columns give
385 the expected yield in the signal region and the background estimates
386 using the ABCD and $P_T(\ell \ell)$ methods. Results are normalized to 34.0~pb$^{-1}$.}
387 \begin{tabular}{lccc}
388 \hline
389 & Yield & ABCD & $P_T(\ell \ell)$ \\
390 \hline
391 SM only & 1.3 & 1.3 & 0.9 \\
392 SM + LM0 & 7.4 & 4.4 & 1.9 \\
393 SM + LM1 & 3.8 & 1.6 & 1.4 \\
394 %SM only & 1.27 & 1.27 & 0.92 \\
395 %SM + LM0 & 7.39 & 4.38 & 1.93 \\
396 %SM + LM1 & 3.77 & 1.62 & 1.41 \\
397 \hline
398 \end{tabular}
399 \end{center}
400 \end{table}
401