ViewVC Help
View File | Revision Log | Show Annotations | Root Listing
root/cvsroot/UserCode/claudioc/OSNote2010/datadriven.tex
Revision: 1.26
Committed: Sat Nov 20 22:56:58 2010 UTC (14 years, 5 months ago) by benhoob
Content type: application/x-tex
Branch: MAIN
Changes since 1.25: +8 -13 lines
Log Message:
Added errors to ABCD syst study

File Contents

# User Rev Content
1 claudioc 1.1 \section{Data Driven Background Estimation Methods}
2     \label{sec:datadriven}
3     We have developed two data-driven methods to
4     estimate the background in the signal region.
5 benhoob 1.10 The first one exploits the fact that
6 benhoob 1.22 SumJetPt and \met$/\sqrt{\rm SumJetPt}$ are nearly
7 claudioc 1.1 uncorrelated for the $t\bar{t}$ background
8     (Section~\ref{sec:abcd}); the second one
9     is based on the fact that in $t\bar{t}$ the
10     $P_T$ of the dilepton pair is on average
11     nearly the same as the $P_T$ of the pair of neutrinos
12     from $W$-decays, which is reconstructed as \met in the
13     detector.
14    
15 benhoob 1.15
16 claudioc 1.6 %{\color{red} I took these
17     %numbers from the twiki, rescaling from 11.06 to 30/pb.
18     %They seem too large...are they really right?}
19 claudioc 1.1
20    
21     \subsection{ABCD method}
22     \label{sec:abcd}
23    
24 benhoob 1.22 We find that in $t\bar{t}$ events SumJetPt and
25 benhoob 1.16 \met$/\sqrt{\rm SumJetPt}$ are nearly uncorrelated,
26     as demonstrated in Figure~\ref{fig:uncor}.
27 claudioc 1.1 Thus, we can use an ABCD method in the \met$/\sqrt{\rm SumJetPt}$ vs
28     sumJetPt plane to estimate the background in a data driven way.
29    
30 benhoob 1.24 %\begin{figure}[bht]
31     %\begin{center}
32     %\includegraphics[width=0.75\linewidth]{uncorrelated.pdf}
33     %\caption{\label{fig:uncor}\protect Distributions of SumJetPt
34     %in MC $t\bar{t}$ events for different intervals of
35     %MET$/\sqrt{\rm SumJetPt}$.}
36     %\end{center}
37     %\end{figure}
38    
39 claudioc 1.21 \begin{figure}[bht]
40 claudioc 1.1 \begin{center}
41 benhoob 1.24 \includegraphics[width=0.75\linewidth]{uncor.png}
42 claudioc 1.1 \caption{\label{fig:uncor}\protect Distributions of SumJetPt
43     in MC $t\bar{t}$ events for different intervals of
44 benhoob 1.24 MET$/\sqrt{\rm SumJetPt}$. h1, h2, and h3 refer to the MET$/\sqrt{\rm SumJetPt}$
45     intervals 4.5-6.5, 6.5-8.5 and $>$8.5, respectively.}
46 claudioc 1.1 \end{center}
47     \end{figure}
48    
49 claudioc 1.21 \begin{figure}[tb]
50 claudioc 1.1 \begin{center}
51 claudioc 1.3 \includegraphics[width=0.5\linewidth, angle=90]{abcdMC.pdf}
52 benhoob 1.22 \caption{\label{fig:abcdMC}\protect Distributions of MET$/\sqrt{\rm SumJetPt}$ vs.
53     SumJetPt for SM Monte Carlo. Here we also show our choice of ABCD regions.}
54 claudioc 1.1 \end{center}
55     \end{figure}
56    
57    
58     Our choice of ABCD regions is shown in Figure~\ref{fig:abcdMC}.
59     The signal region is region D. The expected number of events
60     in the four regions for the SM Monte Carlo, as well as the BG
61 claudioc 1.2 prediction AC/B are given in Table~\ref{tab:abcdMC} for an integrated
62 benhoob 1.24 luminosity of 35 pb$^{-1}$. The ABCD method with chosen boundaries is accurate
63     to about 20\%. As shown in Table~\ref{tab:abcdsyst}, we assess systematic uncertainties
64     by varying the boundaries by an amount consistent with the hadronic energy scale uncertainty,
65     which we take as $\pm$5\% for SumJetPt and $\pm$2.5\% for MET/$\sqrt{\rm SumJetPt}$, since the
66     uncertainty on this quantity partially cancels due to the fact that it is a ratio of correlated
67     quantities. Based on these studies we assess a correction factor $k_{ABCD} = 1.2 \pm 0.2$ to the
68     predicted yield using the ABCD method.
69    
70    
71 claudioc 1.9 %{\color{red} Avi wants some statement about stability
72     %wrt changes in regions. I am not sure that we have done it and
73     %I am not sure it is necessary (Claudio).}
74 claudioc 1.1
75 claudioc 1.21 \begin{table}[ht]
76 claudioc 1.1 \begin{center}
77     \caption{\label{tab:abcdMC} Expected SM Monte Carlo yields for
78 benhoob 1.13 35 pb$^{-1}$ in the ABCD regions, as well as the predicted yield in
79 benhoob 1.16 the signal region given by A $\times$ C / B. Here `SM other' is the sum
80 benhoob 1.13 of non-dileptonic $t\bar{t}$ decays, $W^{\pm}$+jets, $W^+W^-$,
81     $W^{\pm}Z^0$, $Z^0Z^0$ and single top.}
82 benhoob 1.16 \begin{tabular}{lccccc}
83 benhoob 1.13 \hline
84 benhoob 1.16 sample & A & B & C & D & A $\times$ C / B \\
85 benhoob 1.13 \hline
86 benhoob 1.25 $t\bar{t}\rightarrow \ell^{+}\ell^{-}$ & 7.96 $\pm$ 0.17 & 33.07 $\pm$ 0.35 & 4.81 $\pm$ 0.13 & 1.20 $\pm$ 0.07 & 1.16 $\pm$ 0.04 \\
87     $Z^0 \rightarrow \ell^{+}\ell^{-}$ & 0.03 $\pm$ 0.03 & 1.47 $\pm$ 0.38 & 0.10 $\pm$ 0.10 & 0.10 $\pm$ 0.10 & 0.00 $\pm$ 0.00 \\
88     SM other & 0.65 $\pm$ 0.06 & 2.31 $\pm$ 0.13 & 0.17 $\pm$ 0.03 & 0.14 $\pm$ 0.03 & 0.05 $\pm$ 0.01 \\
89     \hline
90     total SM MC & 8.63 $\pm$ 0.18 & 36.85 $\pm$ 0.53 & 5.07 $\pm$ 0.17 & 1.43 $\pm$ 0.12 & 1.19 $\pm$ 0.05 \\
91 claudioc 1.1 \hline
92     \end{tabular}
93     \end{center}
94     \end{table}
95    
96 benhoob 1.24
97    
98     \begin{table}[ht]
99     \begin{center}
100     \caption{\label{tab:abcdsyst} Results of the systematic study of the ABCD method by varying the boundaries
101     between the ABCD regions shown in Fig.~\ref{fig:abcdMC}. Here $x_1$ is the lower SumJetPt boundary and
102     $x_2$ is the boundary separating regions A and B from C and D, their nominal values are 125 and 300~GeV,
103     respectively. $y_1$ is the lower MET/$\sqrt{\rm SumJetPt}$ boundary and
104     $y_2$ is the boundary separating regions B and C from A and D, their nominal values are 4.5 and 8.5~GeV$^{1/2}$,
105     respectively.}
106     \begin{tabular}{cccc|c}
107     \hline
108     $x_1$ & $x_2$ & $y_1$ & $y_2$ & Observed/Predicted \\
109     \hline
110 benhoob 1.26 nominal & nominal & nominal & nominal & $1.20 \pm 0.12$ \\
111     +5\% & +5\% & +2.5\% & +2.5\% & $1.38 \pm 0.15$ \\
112     +5\% & +5\% & nominal & nominal & $1.31 \pm 0.14$ \\
113     nominal & nominal & +2.5\% & +2.5\% & $1.25 \pm 0.13$ \\
114     nominal & +5\% & nominal & +2.5\% & $1.32 \pm 0.14$ \\
115     nominal & -5\% & nominal & -2.5\% & $1.16 \pm 0.09$ \\
116     -5\% & -5\% & +2.5\% & +2.5\% & $1.21 \pm 0.11$ \\
117     +5\% & +5\% & -2.5\% & -2.5\% & $1.26 \pm 0.12$ \\
118 benhoob 1.24 \hline
119     \end{tabular}
120     \end{center}
121     \end{table}
122    
123 claudioc 1.2 \subsection{Dilepton $P_T$ method}
124     \label{sec:victory}
125     This method is based on a suggestion by V. Pavlunin\cite{ref:victory},
126     and was investigated by our group in 2009\cite{ref:ourvictory}.
127     The idea is that in dilepton $t\bar{t}$ events the lepton and neutrinos
128     from $W$ decays have the same $P_T$ spectrum (modulo $W$ polarization
129     effects). One can then use the observed
130     $P_T(\ell\ell)$ distribution to model the sum of neutrino $P_T$'s which
131     is identified with the \met.
132    
133     Then, in order to predict the $t\bar{t} \to$ dilepton contribution to a
134     selection with \met$+$X, one applies a cut on $P_T(\ell\ell)+$X instead.
135     In practice one has to rescale the result of the $P_T(\ell\ell)+$X selection
136     to account for the fact that any dilepton selection must include a
137     moderate \met cut in order to reduce Drell Yan backgrounds. This
138     is discussed in Section 5.3 of Reference~\cite{ref:ourvictory}; for a \met
139 benhoob 1.16 cut of 50 GeV, the rescaling factor is obtained from the MC as
140 claudioc 1.2
141     \newcommand{\ptll} {\ensuremath{P_T(\ell\ell)}}
142     \begin{center}
143     $ K = \frac{\int_0^{\infty} {\cal N}(\ptll)~~d\ptll~}{\int_{50}^{\infty} {\cal N}(\ptll)~~d\ptll~}$
144     \end{center}
145    
146    
147     Monte Carlo studies give values of $K$ that are typically between 1.5 and 1.6,
148 benhoob 1.10 depending on selection details.
149     %%%TO BE REPLACED
150     %Given the integrated luminosity of the
151     %present dataset, the determination of $K$ in data is severely statistics
152     %limited. Thus, we take $K$ from $t\bar{t}$ Monte Carlo as
153    
154     %\begin{center}
155     %$ K_{MC} = \frac{\int_0^{\infty} {\cal N}(\met)~~d\met~}{\int_{50}^{\infty} {\cal N}(\met)~~d\met~}$
156     %\end{center}
157 claudioc 1.9
158 benhoob 1.10 %\noindent {\color{red} For the 11 pb result we have used $K$ from data.}
159 claudioc 1.2
160     There are several effects that spoil the correspondance between \met and
161     $P_T(\ell\ell)$:
162     \begin{itemize}
163     \item $Ws$ in top events are polarized. Neutrinos are emitted preferentially
164 benhoob 1.22 parallel to the $W$ velocity while charged leptons are emitted prefertially
165     anti-parallel. Thus the $P_T(\nu\nu)$ distribution is harder
166 claudioc 1.2 than the $P_T(\ell\ell)$ distribution for top dilepton events.
167     \item The lepton selections results in $P_T$ and $\eta$ cuts on the individual
168     leptons that have no simple correspondance to the neutrino requirements.
169     \item Similarly, the \met$>$50 GeV cut introduces an asymmetry between leptons and
170     neutrinos which is only partially compensated by the $K$ factor above.
171     \item The \met resolution is much worse than the dilepton $P_T$ resolution.
172 benhoob 1.16 When convoluted with a falling spectrum in the tails of \met, this results
173 claudioc 1.2 in a harder spectrum for \met than the original $P_T(\nu\nu)$.
174     \item The \met response in CMS is not exactly 1. This causes a distortion
175     in the \met distribution that is not present in the $P_T(\ell\ell)$ distribution.
176     \item The $t\bar{t} \to$ dilepton signal includes contributions from
177     $W \to \tau \to \ell$. For these events the arguments about the equivalence
178     of $P_T(\ell\ell)$ and $P_T(\nu\nu)$ do not apply.
179     \item A dilepton selection will include SM events from non $t\bar{t}$
180     sources. These events can affect the background prediction. Particularly
181     dangerous are high $P_T$ Drell Yan events that barely pass the \met$>$ 50
182     GeV selection. They will tend to push the data-driven background prediction up.
183 benhoob 1.16 Therefore we estimate the number of DY events entering the background prediction
184     using the $R_{out/in}$ method as described in Sec.~\ref{sec:othBG}.
185 claudioc 1.2 \end{itemize}
186    
187     We have studied these effects in SM Monte Carlo, using a mixture of generator and
188     reconstruction level studies, putting the various effects in one at a time.
189     For each configuration, we apply the data-driven method and report as figure
190     of merit the ratio of observed and predicted events in the signal region.
191     The results are summarized in Table~\ref{tab:victorybad}.
192    
193     \begin{table}[htb]
194     \begin{center}
195     \caption{\label{tab:victorybad} Test of the data driven method in Monte Carlo
196     under different assumptions. See text for details.}
197 claudioc 1.6 \begin{tabular}{|l|c|c|c|c|c|c|c|c|}
198 claudioc 1.2 \hline
199 claudioc 1.6 & True $t\bar{t}$ dilepton & $t\to W\to\tau$& other SM & GEN or & Lepton $P_T$ & Z veto & \met $>$ 50& obs/pred \\
200     & included & included & included & RECOSIM & and $\eta$ cuts & & & \\ \hline
201     1&Y & N & N & GEN & N & N & N & 1.90 \\
202     2&Y & N & N & GEN & Y & N & N & 1.64 \\
203     3&Y & N & N & GEN & Y & Y & N & 1.59 \\
204     4&Y & N & N & GEN & Y & Y & Y & 1.55 \\
205     5&Y & N & N & RECOSIM & Y & Y & Y & 1.51 \\
206     6&Y & Y & N & RECOSIM & Y & Y & Y & 1.58 \\
207 benhoob 1.17 7&Y & Y & Y & RECOSIM & Y & Y & Y & 1.38 \\
208 benhoob 1.16 %%%NOTE: updated value 1.18 -> 1.46 since 2/3 DY events have been removed by updated analysis selections,
209     %%%dpt/pt cut and general lepton veto
210 claudioc 1.2 \hline
211     \end{tabular}
212     \end{center}
213     \end{table}
214    
215    
216     The largest discrepancy between prediction and observation occurs on the first
217     line of Table~\ref{tab:victorybad}, {\em i.e.}, at the generator level with no
218     cuts. We have verified that this effect is due to the polarization of
219     the $W$ (we remove the polarization by reweighting the events and we get
220     good agreement between prediction and observation). The kinematical
221 claudioc 1.6 requirements (lines 2,3,4) compensate somewhat for the effect of W polarization.
222     Going from GEN to RECOSIM, the change in observed/predicted is small.
223     % We have tracked this down to the fact that tcMET underestimates the true \met
224     % by $\approx 4\%$\footnote{We find that observed/predicted changes by roughly 0.1
225     %for each 1.5\% change in \met response.}.
226     Finally, contamination from non $t\bar{t}$
227 benhoob 1.16 events can have a significant impact on the BG prediction.
228     %The changes between
229     %lines 6 and 7 of Table~\ref{tab:victorybad} is driven by 3
230     %Drell Yan events that pass the \met selection in Monte Carlo (thus the effect
231     %is statistically not well quantified).
232 claudioc 1.2
233     An additional source of concern is that the CMS Madgraph $t\bar{t}$ MC does
234     not include effects of spin correlations between the two top quarks.
235     We have studied this effect at the generator level using Alpgen. We find
236 claudioc 1.7 that the bias is at the few percent level.
237 claudioc 1.2
238 benhoob 1.10 %%%TO BE REPLACED
239     %Based on the results of Table~\ref{tab:victorybad}, we conclude that the
240     %naive data driven background estimate based on $P_T{(\ell\ell)}$ needs to
241     %be corrected by a factor of {\color{red} $ K_{\rm{fudge}} =1.2 \pm 0.3$
242     %(We still need to settle on thie exact value of this.
243     %For the 11 pb analysis it is taken as =1.)} . The quoted
244     %uncertainty is based on the stability of the Monte Carlo tests under
245     %variations of event selections, choices of \met algorithm, etc.
246     %For example, we find that observed/predicted changes by roughly 0.1
247     %for each 1.5\% change in the average \met response.
248    
249 claudioc 1.2 Based on the results of Table~\ref{tab:victorybad}, we conclude that the
250 claudioc 1.6 naive data driven background estimate based on $P_T{(\ell\ell)}$ needs to
251 benhoob 1.11 be corrected by a factor of $ K_C = X \pm Y$.
252 benhoob 1.10 The value of this correction factor as well as the systematic uncertainty
253     will be assessed using 38X ttbar madgraph MC. In the following we use
254 benhoob 1.11 $K_C = 1$ for simplicity. Based on previous MC studies we foresee a correction
255 benhoob 1.14 factor of $K_C \approx 1.2 - 1.5$, and we will assess an uncertainty
256 benhoob 1.10 based on the stability of the Monte Carlo tests under
257 claudioc 1.2 variations of event selections, choices of \met algorithm, etc.
258 claudioc 1.8 For example, we find that observed/predicted changes by roughly 0.1
259 benhoob 1.10 for each 1.5\% change in the average \met response.
260 claudioc 1.2
261    
262 claudioc 1.6
263 claudioc 1.2 \subsection{Signal Contamination}
264     \label{sec:sigcont}
265    
266 claudioc 1.6 All data-driven methods are in principle subject to signal contaminations
267 claudioc 1.2 in the control regions, and the methods described in
268     Sections~\ref{sec:abcd} and~\ref{sec:victory} are not exceptions.
269     Signal contamination tends to dilute the significance of a signal
270     present in the data by inflating the background prediction.
271    
272     It is hard to quantify how important these effects are because we
273     do not know what signal may be hiding in the data. Having two
274     independent methods (in addition to Monte Carlo ``dead-reckoning'')
275     adds redundancy because signal contamination can have different effects
276     in the different control regions for the two methods.
277     For example, in the extreme case of a
278     new physics signal
279 claudioc 1.6 with $P_T(\ell \ell) = \met$, an excess of events would be seen
280 claudioc 1.2 in the ABCD method but not in the $P_T(\ell \ell)$ method.
281    
282 claudioc 1.4
283 claudioc 1.2 The LM points are benchmarks for SUSY analyses at CMS. The effects
284     of signal contaminations for a couple such points are summarized
285 benhoob 1.14 in Table~\ref{tab:sigcont}. Signal contamination is definitely an important
286 claudioc 1.2 effect for these two LM points, but it does not totally hide the
287     presence of the signal.
288 claudioc 1.1
289    
290 claudioc 1.2 \begin{table}[htb]
291     \begin{center}
292 benhoob 1.14 \caption{\label{tab:sigcont} Effects of signal contamination
293     for the two data-driven background estimates. The three columns give
294     the expected yield in the signal region and the background estimates
295 benhoob 1.20 using the ABCD and $P_T(\ell \ell)$ methods. Results are normalized to 35~pb$^{-1}$.}
296 benhoob 1.14 \begin{tabular}{lccc}
297 claudioc 1.2 \hline
298 benhoob 1.14 & Yield & ABCD & $P_T(\ell \ell)$ \\
299     \hline
300 benhoob 1.17 SM only & 1.43 & 1.19 & 1.03 \\
301 benhoob 1.19 SM + LM0 & 7.90 & 4.23 & 2.35 \\
302     SM + LM1 & 4.00 & 1.53 & 1.51 \\
303 claudioc 1.2 \hline
304     \end{tabular}
305     \end{center}
306     \end{table}
307    
308 benhoob 1.14
309    
310     %\begin{table}[htb]
311     %\begin{center}
312     %\caption{\label{tab:sigcontABCD} Effects of signal contamination
313     %for the background predictions of the ABCD method including LM0 or
314     %LM1. Results
315     %are normalized to 30 pb$^{-1}$.}
316     %\begin{tabular}{|c|c||c|c||c|c|}
317     %\hline
318     %SM & BG Prediction & SM$+$LM0 & BG Prediction & SM$+$LM1 & BG Prediction \\
319     %Background & SM Only & Contribution & Including LM0 & Contribution & Including LM1 \\ \hline
320     %1.2 & 1.0 & 6.8 & 3.7 & 3.4 & 1.3 \\
321     %\hline
322     %\end{tabular}
323     %\end{center}
324     %\end{table}
325    
326     %\begin{table}[htb]
327     %\begin{center}
328     %\caption{\label{tab:sigcontPT} Effects of signal contamination
329     %for the background predictions of the $P_T(\ell\ell)$ method including LM0 or
330     %LM1. Results
331     %are normalized to 30 pb$^{-1}$.}
332     %\begin{tabular}{|c|c||c|c||c|c|}
333     %\hline
334     %SM & BG Prediction & SM$+$LM0 & BG Prediction & SM$+$LM1 & BG Prediction \\
335     %Background & SM Only & Contribution & Including LM0 & Contribution & Including LM1 \\ \hline
336     %1.2 & 1.0 & 6.8 & 2.2 & 3.4 & 1.5 \\
337     %\hline
338     %\end{tabular}
339     %\end{center}
340     %\end{table}
341 claudioc 1.1