ViewVC Help
View File | Revision Log | Show Annotations | Root Listing
root/cvsroot/UserCode/claudioc/OSNote2010/datadriven.tex
Revision: 1.28
Committed: Thu Dec 2 14:27:31 2010 UTC (14 years, 5 months ago) by benhoob
Content type: application/x-tex
Branch: MAIN
Changes since 1.27: +60 -25 lines
Log Message:
Updates

File Contents

# User Rev Content
1 claudioc 1.1 \section{Data Driven Background Estimation Methods}
2     \label{sec:datadriven}
3     We have developed two data-driven methods to
4     estimate the background in the signal region.
5 benhoob 1.10 The first one exploits the fact that
6 benhoob 1.22 SumJetPt and \met$/\sqrt{\rm SumJetPt}$ are nearly
7 claudioc 1.1 uncorrelated for the $t\bar{t}$ background
8     (Section~\ref{sec:abcd}); the second one
9     is based on the fact that in $t\bar{t}$ the
10     $P_T$ of the dilepton pair is on average
11     nearly the same as the $P_T$ of the pair of neutrinos
12     from $W$-decays, which is reconstructed as \met in the
13     detector.
14    
15 benhoob 1.15
16 claudioc 1.6 %{\color{red} I took these
17     %numbers from the twiki, rescaling from 11.06 to 30/pb.
18     %They seem too large...are they really right?}
19 claudioc 1.1
20    
21     \subsection{ABCD method}
22     \label{sec:abcd}
23    
24 benhoob 1.22 We find that in $t\bar{t}$ events SumJetPt and
25 benhoob 1.16 \met$/\sqrt{\rm SumJetPt}$ are nearly uncorrelated,
26     as demonstrated in Figure~\ref{fig:uncor}.
27 claudioc 1.1 Thus, we can use an ABCD method in the \met$/\sqrt{\rm SumJetPt}$ vs
28     sumJetPt plane to estimate the background in a data driven way.
29    
30 benhoob 1.24 %\begin{figure}[bht]
31     %\begin{center}
32     %\includegraphics[width=0.75\linewidth]{uncorrelated.pdf}
33     %\caption{\label{fig:uncor}\protect Distributions of SumJetPt
34     %in MC $t\bar{t}$ events for different intervals of
35     %MET$/\sqrt{\rm SumJetPt}$.}
36     %\end{center}
37     %\end{figure}
38    
39 claudioc 1.21 \begin{figure}[bht]
40 claudioc 1.1 \begin{center}
41 benhoob 1.24 \includegraphics[width=0.75\linewidth]{uncor.png}
42 claudioc 1.1 \caption{\label{fig:uncor}\protect Distributions of SumJetPt
43     in MC $t\bar{t}$ events for different intervals of
44 benhoob 1.24 MET$/\sqrt{\rm SumJetPt}$. h1, h2, and h3 refer to the MET$/\sqrt{\rm SumJetPt}$
45     intervals 4.5-6.5, 6.5-8.5 and $>$8.5, respectively.}
46 claudioc 1.1 \end{center}
47     \end{figure}
48    
49 claudioc 1.21 \begin{figure}[tb]
50 claudioc 1.1 \begin{center}
51 claudioc 1.3 \includegraphics[width=0.5\linewidth, angle=90]{abcdMC.pdf}
52 benhoob 1.22 \caption{\label{fig:abcdMC}\protect Distributions of MET$/\sqrt{\rm SumJetPt}$ vs.
53     SumJetPt for SM Monte Carlo. Here we also show our choice of ABCD regions.}
54 claudioc 1.1 \end{center}
55     \end{figure}
56    
57    
58     Our choice of ABCD regions is shown in Figure~\ref{fig:abcdMC}.
59     The signal region is region D. The expected number of events
60     in the four regions for the SM Monte Carlo, as well as the BG
61 claudioc 1.2 prediction AC/B are given in Table~\ref{tab:abcdMC} for an integrated
62 benhoob 1.24 luminosity of 35 pb$^{-1}$. The ABCD method with chosen boundaries is accurate
63     to about 20\%. As shown in Table~\ref{tab:abcdsyst}, we assess systematic uncertainties
64     by varying the boundaries by an amount consistent with the hadronic energy scale uncertainty,
65     which we take as $\pm$5\% for SumJetPt and $\pm$2.5\% for MET/$\sqrt{\rm SumJetPt}$, since the
66     uncertainty on this quantity partially cancels due to the fact that it is a ratio of correlated
67     quantities. Based on these studies we assess a correction factor $k_{ABCD} = 1.2 \pm 0.2$ to the
68     predicted yield using the ABCD method.
69    
70    
71 claudioc 1.9 %{\color{red} Avi wants some statement about stability
72     %wrt changes in regions. I am not sure that we have done it and
73     %I am not sure it is necessary (Claudio).}
74 claudioc 1.1
75 claudioc 1.21 \begin{table}[ht]
76 claudioc 1.1 \begin{center}
77     \caption{\label{tab:abcdMC} Expected SM Monte Carlo yields for
78 benhoob 1.13 35 pb$^{-1}$ in the ABCD regions, as well as the predicted yield in
79 benhoob 1.16 the signal region given by A $\times$ C / B. Here `SM other' is the sum
80 benhoob 1.13 of non-dileptonic $t\bar{t}$ decays, $W^{\pm}$+jets, $W^+W^-$,
81     $W^{\pm}Z^0$, $Z^0Z^0$ and single top.}
82 benhoob 1.16 \begin{tabular}{lccccc}
83 benhoob 1.13 \hline
84 benhoob 1.27 sample & A & B & C & D & A $\times$ C / B \\
85 benhoob 1.13 \hline
86 benhoob 1.27 $t\bar{t}\rightarrow \ell^{+}\ell^{-}$ & 8.27 $\pm$ 0.18 & 32.16 $\pm$ 0.35 & 4.69 $\pm$ 0.13 & 1.05 $\pm$ 0.06 & 1.21 $\pm$ 0.04 \\
87     $Z^0 \rightarrow \ell^{+}\ell^{-}$ & 0.22 $\pm$ 0.11 & 1.54 $\pm$ 0.29 & 0.05 $\pm$ 0.05 & 0.16 $\pm$ 0.09 & 0.01 $\pm$ 0.01 \\
88     SM other & 0.54 $\pm$ 0.03 & 2.28 $\pm$ 0.12 & 0.23 $\pm$ 0.03 & 0.07 $\pm$ 0.01 & 0.05 $\pm$ 0.01 \\
89 benhoob 1.25 \hline
90 benhoob 1.27 total SM MC & 9.03 $\pm$ 0.21 & 35.97 $\pm$ 0.46 & 4.97 $\pm$ 0.15 & 1.29 $\pm$ 0.11 & 1.25 $\pm$ 0.05 \\
91 claudioc 1.1 \hline
92     \end{tabular}
93     \end{center}
94     \end{table}
95    
96 benhoob 1.24
97    
98     \begin{table}[ht]
99     \begin{center}
100 benhoob 1.27 \caption{\label{tab:abcdsyst}
101     {\bf \color{red} Do we need this study at all? Observed/predicted is consistent within stat uncertainties as the boundaries are varied- is it enough to simply state this fact in the text??? }
102     Results of the systematic study of the ABCD method by varying the boundaries
103 benhoob 1.24 between the ABCD regions shown in Fig.~\ref{fig:abcdMC}. Here $x_1$ is the lower SumJetPt boundary and
104     $x_2$ is the boundary separating regions A and B from C and D, their nominal values are 125 and 300~GeV,
105     respectively. $y_1$ is the lower MET/$\sqrt{\rm SumJetPt}$ boundary and
106     $y_2$ is the boundary separating regions B and C from A and D, their nominal values are 4.5 and 8.5~GeV$^{1/2}$,
107     respectively.}
108     \begin{tabular}{cccc|c}
109     \hline
110     $x_1$ & $x_2$ & $y_1$ & $y_2$ & Observed/Predicted \\
111     \hline
112 benhoob 1.26 nominal & nominal & nominal & nominal & $1.20 \pm 0.12$ \\
113     +5\% & +5\% & +2.5\% & +2.5\% & $1.38 \pm 0.15$ \\
114     +5\% & +5\% & nominal & nominal & $1.31 \pm 0.14$ \\
115     nominal & nominal & +2.5\% & +2.5\% & $1.25 \pm 0.13$ \\
116     nominal & +5\% & nominal & +2.5\% & $1.32 \pm 0.14$ \\
117     nominal & -5\% & nominal & -2.5\% & $1.16 \pm 0.09$ \\
118     -5\% & -5\% & +2.5\% & +2.5\% & $1.21 \pm 0.11$ \\
119     +5\% & +5\% & -2.5\% & -2.5\% & $1.26 \pm 0.12$ \\
120 benhoob 1.24 \hline
121     \end{tabular}
122     \end{center}
123     \end{table}
124    
125 claudioc 1.2 \subsection{Dilepton $P_T$ method}
126     \label{sec:victory}
127     This method is based on a suggestion by V. Pavlunin\cite{ref:victory},
128     and was investigated by our group in 2009\cite{ref:ourvictory}.
129     The idea is that in dilepton $t\bar{t}$ events the lepton and neutrinos
130     from $W$ decays have the same $P_T$ spectrum (modulo $W$ polarization
131     effects). One can then use the observed
132     $P_T(\ell\ell)$ distribution to model the sum of neutrino $P_T$'s which
133     is identified with the \met.
134    
135     Then, in order to predict the $t\bar{t} \to$ dilepton contribution to a
136     selection with \met$+$X, one applies a cut on $P_T(\ell\ell)+$X instead.
137     In practice one has to rescale the result of the $P_T(\ell\ell)+$X selection
138     to account for the fact that any dilepton selection must include a
139     moderate \met cut in order to reduce Drell Yan backgrounds. This
140     is discussed in Section 5.3 of Reference~\cite{ref:ourvictory}; for a \met
141 benhoob 1.16 cut of 50 GeV, the rescaling factor is obtained from the MC as
142 claudioc 1.2
143     \newcommand{\ptll} {\ensuremath{P_T(\ell\ell)}}
144     \begin{center}
145     $ K = \frac{\int_0^{\infty} {\cal N}(\ptll)~~d\ptll~}{\int_{50}^{\infty} {\cal N}(\ptll)~~d\ptll~}$
146     \end{center}
147    
148    
149     Monte Carlo studies give values of $K$ that are typically between 1.5 and 1.6,
150 benhoob 1.10 depending on selection details.
151     %%%TO BE REPLACED
152     %Given the integrated luminosity of the
153     %present dataset, the determination of $K$ in data is severely statistics
154     %limited. Thus, we take $K$ from $t\bar{t}$ Monte Carlo as
155    
156     %\begin{center}
157     %$ K_{MC} = \frac{\int_0^{\infty} {\cal N}(\met)~~d\met~}{\int_{50}^{\infty} {\cal N}(\met)~~d\met~}$
158     %\end{center}
159 claudioc 1.9
160 benhoob 1.10 %\noindent {\color{red} For the 11 pb result we have used $K$ from data.}
161 claudioc 1.2
162     There are several effects that spoil the correspondance between \met and
163     $P_T(\ell\ell)$:
164     \begin{itemize}
165     \item $Ws$ in top events are polarized. Neutrinos are emitted preferentially
166 benhoob 1.22 parallel to the $W$ velocity while charged leptons are emitted prefertially
167     anti-parallel. Thus the $P_T(\nu\nu)$ distribution is harder
168 claudioc 1.2 than the $P_T(\ell\ell)$ distribution for top dilepton events.
169     \item The lepton selections results in $P_T$ and $\eta$ cuts on the individual
170     leptons that have no simple correspondance to the neutrino requirements.
171     \item Similarly, the \met$>$50 GeV cut introduces an asymmetry between leptons and
172     neutrinos which is only partially compensated by the $K$ factor above.
173     \item The \met resolution is much worse than the dilepton $P_T$ resolution.
174 benhoob 1.16 When convoluted with a falling spectrum in the tails of \met, this results
175 claudioc 1.2 in a harder spectrum for \met than the original $P_T(\nu\nu)$.
176     \item The \met response in CMS is not exactly 1. This causes a distortion
177     in the \met distribution that is not present in the $P_T(\ell\ell)$ distribution.
178     \item The $t\bar{t} \to$ dilepton signal includes contributions from
179     $W \to \tau \to \ell$. For these events the arguments about the equivalence
180     of $P_T(\ell\ell)$ and $P_T(\nu\nu)$ do not apply.
181     \item A dilepton selection will include SM events from non $t\bar{t}$
182     sources. These events can affect the background prediction. Particularly
183     dangerous are high $P_T$ Drell Yan events that barely pass the \met$>$ 50
184     GeV selection. They will tend to push the data-driven background prediction up.
185 benhoob 1.16 Therefore we estimate the number of DY events entering the background prediction
186     using the $R_{out/in}$ method as described in Sec.~\ref{sec:othBG}.
187 claudioc 1.2 \end{itemize}
188    
189     We have studied these effects in SM Monte Carlo, using a mixture of generator and
190     reconstruction level studies, putting the various effects in one at a time.
191     For each configuration, we apply the data-driven method and report as figure
192     of merit the ratio of observed and predicted events in the signal region.
193     The results are summarized in Table~\ref{tab:victorybad}.
194    
195     \begin{table}[htb]
196     \begin{center}
197 benhoob 1.27 \caption{\label{tab:victorybad}
198 benhoob 1.28 {\bf \color{red} Need to either update this with 38X MC or remove it }
199 benhoob 1.27 Test of the data driven method in Monte Carlo
200 claudioc 1.2 under different assumptions. See text for details.}
201 claudioc 1.6 \begin{tabular}{|l|c|c|c|c|c|c|c|c|}
202 claudioc 1.2 \hline
203 claudioc 1.6 & True $t\bar{t}$ dilepton & $t\to W\to\tau$& other SM & GEN or & Lepton $P_T$ & Z veto & \met $>$ 50& obs/pred \\
204 benhoob 1.28 & included & included & included & RECOSIM & and $\eta$ cuts & & & \\ \hline
205 claudioc 1.6 1&Y & N & N & GEN & N & N & N & 1.90 \\
206     2&Y & N & N & GEN & Y & N & N & 1.64 \\
207     3&Y & N & N & GEN & Y & Y & N & 1.59 \\
208     4&Y & N & N & GEN & Y & Y & Y & 1.55 \\
209     5&Y & N & N & RECOSIM & Y & Y & Y & 1.51 \\
210     6&Y & Y & N & RECOSIM & Y & Y & Y & 1.58 \\
211 benhoob 1.17 7&Y & Y & Y & RECOSIM & Y & Y & Y & 1.38 \\
212 claudioc 1.2 \hline
213     \end{tabular}
214     \end{center}
215     \end{table}
216    
217    
218 benhoob 1.28 \begin{table}[htb]
219     \begin{center}
220     \caption{\label{tab:victorysyst}
221     {Summary of uncertainties in $K_C$ due to the MET scale and resolution uncertainty, and to backgrounds other than $t\bar{t} \to$~dilepton.
222     In the first table, `up' and `down' refer to shifting the hadronic energy scale up and down by 5\%. In the second table, the quoted value
223     refers to the amount of additional smearing of the MET, as discussed in the text. In the third table, the normalization of all backgrounds
224     other than $t\bar{t} \to$~dilepton is varied.
225     {\bf \color{ref} Should I remove `observed' and `predicted' and show only the ratio? }}
226    
227     \begin{tabular}{ lcccc }
228     \hline
229     MET scale & Predicted & Observed & Obs/pred \\
230     \hline
231     nominal & 0.92 $ \pm $ 0.11 & 1.29 $ \pm $ 0.11 & 1.40 $ \pm $ 0.20 \\
232     up & 0.92 $ \pm $ 0.11 & 1.53 $ \pm $ 0.12 & 1.66 $ \pm $ 0.23 \\
233     down & 0.81 $ \pm $ 0.07 & 1.08 $ \pm $ 0.11 & 1.32 $ \pm $ 0.17 \\
234     \hline
235    
236     \hline
237     MET smearing & Predicted & Observed & Obs/pred \\
238     \hline
239     nominal & 0.92 $ \pm $ 0.11 & 1.29 $ \pm $ 0.11 & 1.40 $ \pm $ 0.20 \\
240     10\% & 0.90 $ \pm $ 0.11 & 1.30 $ \pm $ 0.11 & 1.44 $ \pm $ 0.21 \\
241     20\% & 0.84 $ \pm $ 0.07 & 1.36 $ \pm $ 0.11 & 1.61 $ \pm $ 0.19 \\
242     30\% & 1.05 $ \pm $ 0.18 & 1.32 $ \pm $ 0.11 & 1.27 $ \pm $ 0.24 \\
243     40\% & 0.85 $ \pm $ 0.07 & 1.37 $ \pm $ 0.11 & 1.61 $ \pm $ 0.19 \\
244     50\% & 1.08 $ \pm $ 0.18 & 1.36 $ \pm $ 0.11 & 1.26 $ \pm $ 0.24 \\
245     \hline
246    
247     \hline
248     non-$t\bar{t} \to$~dilepton scale factor & Predicted & Observed & Obs/pred \\
249     \hline
250     ttdil only & 0.77 $ \pm $ 0.07 & 1.05 $ \pm $ 0.06 & 1.36 $ \pm $ 0.14 \\
251     nominal & 0.92 $ \pm $ 0.11 & 1.29 $ \pm $ 0.11 & 1.40 $ \pm $ 0.20 \\
252     double non-ttdil yield & 1.06 $ \pm $ 0.18 & 1.52 $ \pm $ 0.20 & 1.43 $ \pm $ 0.30 \\
253     \hline
254     \end{tabular}
255     \end{center}
256     \end{table}
257    
258    
259    
260 claudioc 1.2 The largest discrepancy between prediction and observation occurs on the first
261     line of Table~\ref{tab:victorybad}, {\em i.e.}, at the generator level with no
262     cuts. We have verified that this effect is due to the polarization of
263     the $W$ (we remove the polarization by reweighting the events and we get
264     good agreement between prediction and observation). The kinematical
265 claudioc 1.6 requirements (lines 2,3,4) compensate somewhat for the effect of W polarization.
266     Going from GEN to RECOSIM, the change in observed/predicted is small.
267     % We have tracked this down to the fact that tcMET underestimates the true \met
268     % by $\approx 4\%$\footnote{We find that observed/predicted changes by roughly 0.1
269     %for each 1.5\% change in \met response.}.
270     Finally, contamination from non $t\bar{t}$
271 benhoob 1.16 events can have a significant impact on the BG prediction.
272     %The changes between
273     %lines 6 and 7 of Table~\ref{tab:victorybad} is driven by 3
274     %Drell Yan events that pass the \met selection in Monte Carlo (thus the effect
275     %is statistically not well quantified).
276 claudioc 1.2
277     An additional source of concern is that the CMS Madgraph $t\bar{t}$ MC does
278     not include effects of spin correlations between the two top quarks.
279     We have studied this effect at the generator level using Alpgen. We find
280 claudioc 1.7 that the bias is at the few percent level.
281 claudioc 1.2
282     Based on the results of Table~\ref{tab:victorybad}, we conclude that the
283 benhoob 1.28 naive data-driven background estimate based on $P_T{(\ell\ell)}$ needs to
284     be corrected by a factor of $ K_C = 1.4 \pm 0.2(stat)$.
285 claudioc 1.2
286 benhoob 1.28 The 2 dominant sources of systematic uncertainty in $K_C$ are due to non-$t\bar{t} \to$~dilepton backgrounds,
287     and the MET scale and resolution uncertainties. The impact of non-$t\bar{t}$-dilepton background is assessed
288     by varying the yield of all backgrounds except for $t\bar{t} \to$~dilepton, as shown in Table~\ref{table_kc}.
289     The systematic is assessed as the larger of the differences between the nominal $K_C$ value and the values
290     obtained using only $t\bar{t} \to$~dilepton MC and obtained by doubling the non $t\bar{t} \to$~dilepton component,
291     giving an uncertainty of $0.04$.
292    
293     The uncertainty in $K_C$ due to the MET scale uncertainty is assessed by varying the hadronic energy scale using
294     the same method as in~\ref{} and checking how much $K_C$ changes, as summarized in Table~\ref{tab:victorysyst}.
295     This gives an uncertainty of 0.3. We also assess the impact of the MET resolution uncertainty on $K_C$ by applying
296     a random smearing to the MET. For each event, we determine the expected MET resolution based on the sumJetPt, and
297     smear the MET to simulate an increase in the resolution of 10\%, 20\%, 30\%, 40\% and 50\%. The results show that
298     $K_C$ does not depend strongly on the MET resolution and we therefore do not assess any uncertainty.
299 claudioc 1.2
300 benhoob 1.28 Incorporating all the statistical and systematic uncertainties we find $K_C = 1.4 \pm 0.4$.
301 claudioc 1.6
302 claudioc 1.2 \subsection{Signal Contamination}
303     \label{sec:sigcont}
304    
305 claudioc 1.6 All data-driven methods are in principle subject to signal contaminations
306 claudioc 1.2 in the control regions, and the methods described in
307     Sections~\ref{sec:abcd} and~\ref{sec:victory} are not exceptions.
308     Signal contamination tends to dilute the significance of a signal
309     present in the data by inflating the background prediction.
310    
311     It is hard to quantify how important these effects are because we
312     do not know what signal may be hiding in the data. Having two
313     independent methods (in addition to Monte Carlo ``dead-reckoning'')
314     adds redundancy because signal contamination can have different effects
315     in the different control regions for the two methods.
316     For example, in the extreme case of a
317     new physics signal
318 claudioc 1.6 with $P_T(\ell \ell) = \met$, an excess of events would be seen
319 claudioc 1.2 in the ABCD method but not in the $P_T(\ell \ell)$ method.
320    
321 claudioc 1.4
322 claudioc 1.2 The LM points are benchmarks for SUSY analyses at CMS. The effects
323     of signal contaminations for a couple such points are summarized
324 benhoob 1.14 in Table~\ref{tab:sigcont}. Signal contamination is definitely an important
325 claudioc 1.2 effect for these two LM points, but it does not totally hide the
326     presence of the signal.
327 claudioc 1.1
328    
329 claudioc 1.2 \begin{table}[htb]
330     \begin{center}
331 benhoob 1.14 \caption{\label{tab:sigcont} Effects of signal contamination
332     for the two data-driven background estimates. The three columns give
333     the expected yield in the signal region and the background estimates
334 benhoob 1.20 using the ABCD and $P_T(\ell \ell)$ methods. Results are normalized to 35~pb$^{-1}$.}
335 benhoob 1.14 \begin{tabular}{lccc}
336 claudioc 1.2 \hline
337 benhoob 1.14 & Yield & ABCD & $P_T(\ell \ell)$ \\
338     \hline
339 benhoob 1.27 SM only & 1.29 & 1.25 & 0.92 \\
340     SM + LM0 & 7.57 & 4.44 & 1.96 \\
341     SM + LM1 & 3.85 & 1.60 & 1.43 \\
342 claudioc 1.2 \hline
343     \end{tabular}
344     \end{center}
345     \end{table}
346