ViewVC Help
View File | Revision Log | Show Annotations | Root Listing
root/cvsroot/UserCode/claudioc/OSNote2010/datadriven.tex
Revision: 1.38
Committed: Wed Dec 8 12:18:30 2010 UTC (14 years, 5 months ago) by benhoob
Content type: application/x-tex
Branch: MAIN
Changes since 1.37: +1 -1 lines
Log Message:
Minor updates

File Contents

# User Rev Content
1 claudioc 1.1 \section{Data Driven Background Estimation Methods}
2     \label{sec:datadriven}
3     We have developed two data-driven methods to
4     estimate the background in the signal region.
5 benhoob 1.10 The first one exploits the fact that
6 benhoob 1.22 SumJetPt and \met$/\sqrt{\rm SumJetPt}$ are nearly
7 claudioc 1.1 uncorrelated for the $t\bar{t}$ background
8     (Section~\ref{sec:abcd}); the second one
9     is based on the fact that in $t\bar{t}$ the
10     $P_T$ of the dilepton pair is on average
11     nearly the same as the $P_T$ of the pair of neutrinos
12     from $W$-decays, which is reconstructed as \met in the
13     detector.
14    
15 benhoob 1.15
16 claudioc 1.6 %{\color{red} I took these
17     %numbers from the twiki, rescaling from 11.06 to 30/pb.
18     %They seem too large...are they really right?}
19 claudioc 1.1
20    
21     \subsection{ABCD method}
22     \label{sec:abcd}
23    
24 benhoob 1.22 We find that in $t\bar{t}$ events SumJetPt and
25 benhoob 1.16 \met$/\sqrt{\rm SumJetPt}$ are nearly uncorrelated,
26 benhoob 1.31 as demonstrated in Fig.~\ref{fig:uncor}.
27 claudioc 1.1 Thus, we can use an ABCD method in the \met$/\sqrt{\rm SumJetPt}$ vs
28     sumJetPt plane to estimate the background in a data driven way.
29    
30 benhoob 1.24 %\begin{figure}[bht]
31     %\begin{center}
32     %\includegraphics[width=0.75\linewidth]{uncorrelated.pdf}
33     %\caption{\label{fig:uncor}\protect Distributions of SumJetPt
34     %in MC $t\bar{t}$ events for different intervals of
35     %MET$/\sqrt{\rm SumJetPt}$.}
36     %\end{center}
37     %\end{figure}
38    
39 claudioc 1.21 \begin{figure}[bht]
40 claudioc 1.1 \begin{center}
41 benhoob 1.24 \includegraphics[width=0.75\linewidth]{uncor.png}
42 claudioc 1.1 \caption{\label{fig:uncor}\protect Distributions of SumJetPt
43     in MC $t\bar{t}$ events for different intervals of
44 benhoob 1.24 MET$/\sqrt{\rm SumJetPt}$. h1, h2, and h3 refer to the MET$/\sqrt{\rm SumJetPt}$
45 benhoob 1.37 intervals 4.5-6.5, 6.5-8.5 and $>$8.5, respectively. }
46 claudioc 1.1 \end{center}
47     \end{figure}
48    
49 claudioc 1.21 \begin{figure}[tb]
50 claudioc 1.1 \begin{center}
51 benhoob 1.31 \includegraphics[width=0.75\linewidth]{ttdil_uncor_38X.png}
52 benhoob 1.22 \caption{\label{fig:abcdMC}\protect Distributions of MET$/\sqrt{\rm SumJetPt}$ vs.
53 benhoob 1.31 SumJetPt for SM Monte Carlo. Here we also show our choice of ABCD regions. The correlation coefficient
54     ${\rm corr_{XY}}$ is computed for events falling in the ABCD regions.}
55 claudioc 1.1 \end{center}
56     \end{figure}
57    
58    
59     Our choice of ABCD regions is shown in Figure~\ref{fig:abcdMC}.
60     The signal region is region D. The expected number of events
61 benhoob 1.29 in the four regions for the SM Monte Carlo, as well as the background
62     prediction A $\times$ C / B are given in Table~\ref{tab:abcdMC} for an integrated
63 benhoob 1.37 luminosity of 34.0 pb$^{-1}$. In Table~\ref{tab:abcdsyst}, we test the stability of
64 benhoob 1.33 observed/predicted with respect to variations in the ABCD boundaries.
65     Based on the results in Tables~\ref{tab:abcdMC} and~\ref{tab:abcdsyst}, we assess
66     a systematic uncertainty of 20\% on the prediction of the ABCD method.
67 benhoob 1.29
68     %As shown in Table~\ref{tab:abcdsyst}, we assess systematic uncertainties
69     %by varying the boundaries by an amount consistent with the hadronic energy scale uncertainty,
70     %which we take as $\pm$5\% for SumJetPt and $\pm$2.5\% for MET/$\sqrt{\rm SumJetPt}$, since the
71     %uncertainty on this quantity partially cancels due to the fact that it is a ratio of correlated
72     %quantities. Based on these studies we assess a correction factor $k_{ABCD} = 1.2 \pm 0.2$ to the
73     %predicted yield using the ABCD method.
74 benhoob 1.24
75    
76 claudioc 1.9 %{\color{red} Avi wants some statement about stability
77     %wrt changes in regions. I am not sure that we have done it and
78     %I am not sure it is necessary (Claudio).}
79 claudioc 1.1
80 claudioc 1.21 \begin{table}[ht]
81 claudioc 1.1 \begin{center}
82     \caption{\label{tab:abcdMC} Expected SM Monte Carlo yields for
83 benhoob 1.37 34.0~pb$^{-1}$ in the ABCD regions, as well as the predicted yield in
84 benhoob 1.16 the signal region given by A $\times$ C / B. Here `SM other' is the sum
85 benhoob 1.13 of non-dileptonic $t\bar{t}$ decays, $W^{\pm}$+jets, $W^+W^-$,
86     $W^{\pm}Z^0$, $Z^0Z^0$ and single top.}
87 benhoob 1.16 \begin{tabular}{lccccc}
88 benhoob 1.37 %%%official json v3, 38X MC (D6T ttbar and DY)
89 benhoob 1.13 \hline
90 benhoob 1.37 sample & A & B & C & D & PRED \\
91 benhoob 1.13 \hline
92 benhoob 1.37 $t\bar{t}\rightarrow \ell^{+}\ell^{-}$ & 8.44 $\pm$ 0.18 & 32.83 $\pm$ 0.35 & 4.78 $\pm$ 0.14 & 1.07 $\pm$ 0.06 & 1.23 $\pm$ 0.05 \\
93     $Z^0 \rightarrow \ell^{+}\ell^{-}$ & 0.17 $\pm$ 0.08 & 1.18 $\pm$ 0.22 & 0.04 $\pm$ 0.04 & 0.12 $\pm$ 0.07 & 0.01 $\pm$ 0.01 \\
94     SM other & 0.53 $\pm$ 0.03 & 2.26 $\pm$ 0.11 & 0.23 $\pm$ 0.03 & 0.07 $\pm$ 0.01 & 0.05 $\pm$ 0.01 \\
95 benhoob 1.25 \hline
96 benhoob 1.37 total SM MC & 9.14 $\pm$ 0.20 & 36.26 $\pm$ 0.43 & 5.05 $\pm$ 0.14 & 1.27 $\pm$ 0.10 & 1.27 $\pm$ 0.05 \\
97 claudioc 1.1 \hline
98     \end{tabular}
99     \end{center}
100     \end{table}
101    
102 benhoob 1.24
103    
104     \begin{table}[ht]
105     \begin{center}
106 benhoob 1.27 \caption{\label{tab:abcdsyst}
107     Results of the systematic study of the ABCD method by varying the boundaries
108 benhoob 1.24 between the ABCD regions shown in Fig.~\ref{fig:abcdMC}. Here $x_1$ is the lower SumJetPt boundary and
109     $x_2$ is the boundary separating regions A and B from C and D, their nominal values are 125 and 300~GeV,
110     respectively. $y_1$ is the lower MET/$\sqrt{\rm SumJetPt}$ boundary and
111     $y_2$ is the boundary separating regions B and C from A and D, their nominal values are 4.5 and 8.5~GeV$^{1/2}$,
112     respectively.}
113     \begin{tabular}{cccc|c}
114     \hline
115     $x_1$ & $x_2$ & $y_1$ & $y_2$ & Observed/Predicted \\
116     \hline
117 benhoob 1.37
118     nominal & nominal & nominal & nominal & $1.00 \pm 0.08$ \\
119    
120     +5\% & +5\% & +2.5\% & +2.5\% & $1.08 \pm 0.11$ \\
121    
122     +5\% & +5\% & nominal & nominal & $1.04 \pm 0.10$ \\
123    
124     nominal & nominal & +2.5\% & +2.5\% & $1.03 \pm 0.09$ \\
125    
126     nominal & +5\% & nominal & +2.5\% & $1.05 \pm 0.10$ \\
127    
128     nominal & -5\% & nominal & -2.5\% & $0.95 \pm 0.07$ \\
129    
130     -5\% & -5\% & +2.5\% & +2.5\% & $1.00 \pm 0.08$ \\
131    
132     +5\% & +5\% & -2.5\% & -2.5\% & $0.98 \pm 0.09$ \\
133 benhoob 1.24 \hline
134     \end{tabular}
135     \end{center}
136     \end{table}
137    
138 claudioc 1.2 \subsection{Dilepton $P_T$ method}
139     \label{sec:victory}
140     This method is based on a suggestion by V. Pavlunin\cite{ref:victory},
141     and was investigated by our group in 2009\cite{ref:ourvictory}.
142     The idea is that in dilepton $t\bar{t}$ events the lepton and neutrinos
143     from $W$ decays have the same $P_T$ spectrum (modulo $W$ polarization
144     effects). One can then use the observed
145     $P_T(\ell\ell)$ distribution to model the sum of neutrino $P_T$'s which
146     is identified with the \met.
147    
148     Then, in order to predict the $t\bar{t} \to$ dilepton contribution to a
149     selection with \met$+$X, one applies a cut on $P_T(\ell\ell)+$X instead.
150     In practice one has to rescale the result of the $P_T(\ell\ell)+$X selection
151     to account for the fact that any dilepton selection must include a
152     moderate \met cut in order to reduce Drell Yan backgrounds. This
153     is discussed in Section 5.3 of Reference~\cite{ref:ourvictory}; for a \met
154 benhoob 1.16 cut of 50 GeV, the rescaling factor is obtained from the MC as
155 claudioc 1.2
156     \newcommand{\ptll} {\ensuremath{P_T(\ell\ell)}}
157     \begin{center}
158 benhoob 1.37 $ K = \frac{\int_0^{\infty} {\cal N}(\ptll)~~d\ptll~}{\int_{50}^{\infty} {\cal N}(\ptll)~~d\ptll~} = 1.5$
159 claudioc 1.2 \end{center}
160    
161    
162 benhoob 1.10 %%%TO BE REPLACED
163     %Given the integrated luminosity of the
164     %present dataset, the determination of $K$ in data is severely statistics
165     %limited. Thus, we take $K$ from $t\bar{t}$ Monte Carlo as
166    
167     %\begin{center}
168     %$ K_{MC} = \frac{\int_0^{\infty} {\cal N}(\met)~~d\met~}{\int_{50}^{\infty} {\cal N}(\met)~~d\met~}$
169     %\end{center}
170 claudioc 1.9
171 benhoob 1.10 %\noindent {\color{red} For the 11 pb result we have used $K$ from data.}
172 claudioc 1.2
173     There are several effects that spoil the correspondance between \met and
174     $P_T(\ell\ell)$:
175     \begin{itemize}
176     \item $Ws$ in top events are polarized. Neutrinos are emitted preferentially
177 benhoob 1.22 parallel to the $W$ velocity while charged leptons are emitted prefertially
178     anti-parallel. Thus the $P_T(\nu\nu)$ distribution is harder
179 claudioc 1.2 than the $P_T(\ell\ell)$ distribution for top dilepton events.
180     \item The lepton selections results in $P_T$ and $\eta$ cuts on the individual
181     leptons that have no simple correspondance to the neutrino requirements.
182     \item Similarly, the \met$>$50 GeV cut introduces an asymmetry between leptons and
183     neutrinos which is only partially compensated by the $K$ factor above.
184     \item The \met resolution is much worse than the dilepton $P_T$ resolution.
185 benhoob 1.16 When convoluted with a falling spectrum in the tails of \met, this results
186 claudioc 1.2 in a harder spectrum for \met than the original $P_T(\nu\nu)$.
187     \item The \met response in CMS is not exactly 1. This causes a distortion
188     in the \met distribution that is not present in the $P_T(\ell\ell)$ distribution.
189     \item The $t\bar{t} \to$ dilepton signal includes contributions from
190     $W \to \tau \to \ell$. For these events the arguments about the equivalence
191     of $P_T(\ell\ell)$ and $P_T(\nu\nu)$ do not apply.
192     \item A dilepton selection will include SM events from non $t\bar{t}$
193     sources. These events can affect the background prediction. Particularly
194     dangerous are high $P_T$ Drell Yan events that barely pass the \met$>$ 50
195     GeV selection. They will tend to push the data-driven background prediction up.
196 benhoob 1.16 Therefore we estimate the number of DY events entering the background prediction
197     using the $R_{out/in}$ method as described in Sec.~\ref{sec:othBG}.
198 claudioc 1.2 \end{itemize}
199    
200     We have studied these effects in SM Monte Carlo, using a mixture of generator and
201     reconstruction level studies, putting the various effects in one at a time.
202     For each configuration, we apply the data-driven method and report as figure
203     of merit the ratio of observed and predicted events in the signal region.
204     The results are summarized in Table~\ref{tab:victorybad}.
205    
206     \begin{table}[htb]
207     \begin{center}
208 benhoob 1.27 \caption{\label{tab:victorybad}
209     Test of the data driven method in Monte Carlo
210 benhoob 1.37 under different assumptions, evaluated using Spring10 MC. See text for details.}
211 claudioc 1.6 \begin{tabular}{|l|c|c|c|c|c|c|c|c|}
212 claudioc 1.2 \hline
213 claudioc 1.6 & True $t\bar{t}$ dilepton & $t\to W\to\tau$& other SM & GEN or & Lepton $P_T$ & Z veto & \met $>$ 50& obs/pred \\
214 benhoob 1.28 & included & included & included & RECOSIM & and $\eta$ cuts & & & \\ \hline
215 claudioc 1.6 1&Y & N & N & GEN & N & N & N & 1.90 \\
216     2&Y & N & N & GEN & Y & N & N & 1.64 \\
217     3&Y & N & N & GEN & Y & Y & N & 1.59 \\
218     4&Y & N & N & GEN & Y & Y & Y & 1.55 \\
219     5&Y & N & N & RECOSIM & Y & Y & Y & 1.51 \\
220     6&Y & Y & N & RECOSIM & Y & Y & Y & 1.58 \\
221 benhoob 1.17 7&Y & Y & Y & RECOSIM & Y & Y & Y & 1.38 \\
222 claudioc 1.2 \hline
223     \end{tabular}
224     \end{center}
225     \end{table}
226    
227    
228 benhoob 1.28 \begin{table}[htb]
229     \begin{center}
230     \caption{\label{tab:victorysyst}
231 benhoob 1.35 Summary of variations in $K_C$ due to the MET scale and resolution uncertainty, and to backgrounds other than $t\bar{t} \to$~dilepton.
232 benhoob 1.28 In the first table, `up' and `down' refer to shifting the hadronic energy scale up and down by 5\%. In the second table, the quoted value
233     refers to the amount of additional smearing of the MET, as discussed in the text. In the third table, the normalization of all backgrounds
234 benhoob 1.38 other than $t\bar{t} \to$~dilepton is varied. }
235 benhoob 1.28 \begin{tabular}{ lcccc }
236     \hline
237     MET scale & Predicted & Observed & Obs/pred \\
238     \hline
239 benhoob 1.37 nominal & 0.92 $ \pm $ 0.09 & 1.27 $ \pm $ 0.10 & 1.39 $ \pm $ 0.18 \\
240     up & 0.90 $ \pm $ 0.09 & 1.58 $ \pm $ 0.10 & 1.75 $ \pm $ 0.21 \\
241     down & 0.70 $ \pm $ 0.06 & 0.96 $ \pm $ 0.09 & 1.37 $ \pm $ 0.18 \\
242     \hline
243     MET smearing & Predicted & Observed & Obs/pred \\
244     \hline
245     nominal & 0.92 $ \pm $ 0.09 & 1.27 $ \pm $ 0.10 & 1.39 $ \pm $ 0.18 \\
246     10\% & 0.88 $ \pm $ 0.09 & 1.28 $ \pm $ 0.10 & 1.47 $ \pm $ 0.19 \\
247     20\% & 0.87 $ \pm $ 0.09 & 1.26 $ \pm $ 0.10 & 1.44 $ \pm $ 0.19 \\
248     30\% & 1.03 $ \pm $ 0.17 & 1.33 $ \pm $ 0.10 & 1.29 $ \pm $ 0.23 \\
249     40\% & 0.88 $ \pm $ 0.09 & 1.36 $ \pm $ 0.10 & 1.55 $ \pm $ 0.20 \\
250     50\% & 0.80 $ \pm $ 0.07 & 1.39 $ \pm $ 0.10 & 1.73 $ \pm $ 0.19 \\
251     \hline
252     non-$t\bar{t} \to$~dilepton bkg & Predicted & Observed & Obs/pred \\
253     \hline
254     ttdil only & 0.79 $ \pm $ 0.07 & 1.07 $ \pm $ 0.06 & 1.36 $ \pm $ 0.14 \\
255     nominal & 0.92 $ \pm $ 0.09 & 1.27 $ \pm $ 0.10 & 1.39 $ \pm $ 0.18 \\
256     double non-ttdil yield & 1.04 $ \pm $ 0.15 & 1.47 $ \pm $ 0.16 & 1.40 $ \pm $ 0.25 \\
257 benhoob 1.28 \hline
258     \end{tabular}
259     \end{center}
260     \end{table}
261    
262 claudioc 1.2 The largest discrepancy between prediction and observation occurs on the first
263     line of Table~\ref{tab:victorybad}, {\em i.e.}, at the generator level with no
264     cuts. We have verified that this effect is due to the polarization of
265     the $W$ (we remove the polarization by reweighting the events and we get
266     good agreement between prediction and observation). The kinematical
267 claudioc 1.6 requirements (lines 2,3,4) compensate somewhat for the effect of W polarization.
268     Going from GEN to RECOSIM, the change in observed/predicted is small.
269     % We have tracked this down to the fact that tcMET underestimates the true \met
270     % by $\approx 4\%$\footnote{We find that observed/predicted changes by roughly 0.1
271     %for each 1.5\% change in \met response.}.
272     Finally, contamination from non $t\bar{t}$
273 benhoob 1.16 events can have a significant impact on the BG prediction.
274     %The changes between
275     %lines 6 and 7 of Table~\ref{tab:victorybad} is driven by 3
276     %Drell Yan events that pass the \met selection in Monte Carlo (thus the effect
277     %is statistically not well quantified).
278 claudioc 1.2
279     An additional source of concern is that the CMS Madgraph $t\bar{t}$ MC does
280     not include effects of spin correlations between the two top quarks.
281     We have studied this effect at the generator level using Alpgen. We find
282 claudioc 1.7 that the bias is at the few percent level.
283 claudioc 1.2
284 benhoob 1.31 Based on the results of Table~\ref{tab:victorysyst}, we conclude that the
285 benhoob 1.28 naive data-driven background estimate based on $P_T{(\ell\ell)}$ needs to
286 benhoob 1.31 be corrected by a factor of $ K_C = 1.4 \pm 0.2({\rm stat})$.
287 claudioc 1.2
288 benhoob 1.31 The dominant sources of systematic uncertainty in $K_C$ are due to non-$t\bar{t} \to$~dilepton backgrounds,
289     and the MET scale and resolution uncertainties, as summarized in Table~\ref{tab:victorysyst}.
290     The impact of non-$t\bar{t}$-dilepton background is assessed
291     by varying the yield of all backgrounds except for $t\bar{t} \to$~dilepton.
292     The uncertainty is assessed as the larger of the differences between the nominal $K_C$ value and the values
293 benhoob 1.28 obtained using only $t\bar{t} \to$~dilepton MC and obtained by doubling the non $t\bar{t} \to$~dilepton component,
294 benhoob 1.37 giving an uncertainty of $0.03$.
295 benhoob 1.28
296     The uncertainty in $K_C$ due to the MET scale uncertainty is assessed by varying the hadronic energy scale using
297 benhoob 1.37 the same method as in~\cite{ref:top}, giving an uncertainty of 0.36.
298     We also assess the impact of the MET resolution
299 benhoob 1.31 uncertainty on $K_C$ by applying a random smearing to the MET. For each event, we determine the expected MET resolution
300     based on the sumJetPt, and smear the MET to simulate an increase in the resolution of 10\%, 20\%, 30\%, 40\% and 50\%.
301     The results show that $K_C$ does not depend strongly on the MET resolution and we therefore do not assess any uncertainty.
302 claudioc 1.2
303 benhoob 1.28 Incorporating all the statistical and systematic uncertainties we find $K_C = 1.4 \pm 0.4$.
304 claudioc 1.6
305 claudioc 1.2 \subsection{Signal Contamination}
306     \label{sec:sigcont}
307    
308 claudioc 1.6 All data-driven methods are in principle subject to signal contaminations
309 claudioc 1.2 in the control regions, and the methods described in
310     Sections~\ref{sec:abcd} and~\ref{sec:victory} are not exceptions.
311     Signal contamination tends to dilute the significance of a signal
312     present in the data by inflating the background prediction.
313    
314     It is hard to quantify how important these effects are because we
315     do not know what signal may be hiding in the data. Having two
316     independent methods (in addition to Monte Carlo ``dead-reckoning'')
317     adds redundancy because signal contamination can have different effects
318     in the different control regions for the two methods.
319     For example, in the extreme case of a
320     new physics signal
321 claudioc 1.6 with $P_T(\ell \ell) = \met$, an excess of events would be seen
322 claudioc 1.2 in the ABCD method but not in the $P_T(\ell \ell)$ method.
323    
324 claudioc 1.4
325 claudioc 1.2 The LM points are benchmarks for SUSY analyses at CMS. The effects
326     of signal contaminations for a couple such points are summarized
327 benhoob 1.14 in Table~\ref{tab:sigcont}. Signal contamination is definitely an important
328 claudioc 1.2 effect for these two LM points, but it does not totally hide the
329     presence of the signal.
330 claudioc 1.1
331    
332 claudioc 1.2 \begin{table}[htb]
333     \begin{center}
334 benhoob 1.14 \caption{\label{tab:sigcont} Effects of signal contamination
335     for the two data-driven background estimates. The three columns give
336     the expected yield in the signal region and the background estimates
337 benhoob 1.37 using the ABCD and $P_T(\ell \ell)$ methods. Results are normalized to 34.0~pb$^{-1}$.}
338 benhoob 1.14 \begin{tabular}{lccc}
339 claudioc 1.2 \hline
340 benhoob 1.14 & Yield & ABCD & $P_T(\ell \ell)$ \\
341     \hline
342 benhoob 1.37 SM only & 1.3 & 1.3 & 0.9 \\
343     SM + LM0 & 7.4 & 4.4 & 1.9 \\
344     SM + LM1 & 3.8 & 1.6 & 1.4 \\
345     %SM only & 1.27 & 1.27 & 0.92 \\
346     %SM + LM0 & 7.39 & 4.38 & 1.93 \\
347     %SM + LM1 & 3.77 & 1.62 & 1.41 \\
348 claudioc 1.2 \hline
349     \end{tabular}
350     \end{center}
351     \end{table}
352