ViewVC Help
View File | Revision Log | Show Annotations | Root Listing
root/cvsroot/UserCode/claudioc/OSNote2010/datadriven.tex
(Generate patch)

Comparing UserCode/claudioc/OSNote2010/datadriven.tex (file contents):
Revision 1.1 by claudioc, Thu Oct 28 05:28:08 2010 UTC vs.
Revision 1.38 by benhoob, Wed Dec 8 12:18:30 2010 UTC

# Line 2 | Line 2
2   \label{sec:datadriven}
3   We have developed two data-driven methods to
4   estimate the background in the signal region.
5 < The first one explouts the fact that
6 < \met and \met$/\sqrt{\rm SumJetPt}$ are nearly
5 > The first one exploits the fact that
6 > SumJetPt and \met$/\sqrt{\rm SumJetPt}$ are nearly
7   uncorrelated for the $t\bar{t}$ background
8   (Section~\ref{sec:abcd});  the second one
9   is based on the fact that in $t\bar{t}$ the
# Line 12 | Line 12 | nearly the same as the $P_T$ of the pair
12   from $W$-decays, which is reconstructed as \met in the
13   detector.
14  
15 < in 30 pb$^{-1}$ we expect $\approx$ 1 SM event in
16 < the signal region.  The expectations from the LMO
17 < and LM1 SUSY benchmark points are {\color{red} XX} and
18 < {\color{red} XX} events respectively.
15 >
16 > %{\color{red} I took these
17 > %numbers from the twiki, rescaling from 11.06 to 30/pb.
18 > %They seem too large...are they really right?}
19  
20  
21   \subsection{ABCD method}
22   \label{sec:abcd}
23  
24 < We find that in $t\bar{t}$ events \met and
25 < \met$/\sqrt{\rm SumJetPt}$ are nearly uncorrelated.
26 < This is demonstrated in Figure~\ref{fig:uncor}.
24 > We find that in $t\bar{t}$ events SumJetPt and
25 > \met$/\sqrt{\rm SumJetPt}$ are nearly uncorrelated,
26 > as demonstrated in Fig.~\ref{fig:uncor}.
27   Thus, we can use an ABCD method in the \met$/\sqrt{\rm SumJetPt}$ vs
28   sumJetPt plane to estimate the background in a data driven way.
29  
30 < \begin{figure}[htb]
30 > %\begin{figure}[bht]
31 > %\begin{center}
32 > %\includegraphics[width=0.75\linewidth]{uncorrelated.pdf}
33 > %\caption{\label{fig:uncor}\protect Distributions of SumJetPt
34 > %in MC $t\bar{t}$ events for different intervals of
35 > %MET$/\sqrt{\rm SumJetPt}$.}
36 > %\end{center}
37 > %\end{figure}
38 >
39 > \begin{figure}[bht]
40   \begin{center}
41 < \includegraphics[width=0.75\linewidth]{uncorrelated.pdf}
41 > \includegraphics[width=0.75\linewidth]{uncor.png}
42   \caption{\label{fig:uncor}\protect Distributions of SumJetPt
43   in MC $t\bar{t}$ events for different intervals of
44 < MET$/\sqrt{\rm SumJetPt}$.}
44 > MET$/\sqrt{\rm SumJetPt}$. h1, h2, and h3 refer to the MET$/\sqrt{\rm SumJetPt}$
45 > intervals 4.5-6.5, 6.5-8.5 and $>$8.5, respectively. }
46   \end{center}
47   \end{figure}
48  
49 < \begin{figure}[htb]
49 > \begin{figure}[tb]
50   \begin{center}
51 < \includegraphics[width=0.75\linewidth]{abcdMC.jpg}
52 < \caption{\label{fig:abcdMC}\protect Distributions of SumJetPt
53 < vs. MET$/\sqrt{\rm SumJetPt}$ for SM Monte Carlo.  Here we also
54 < show our choice of ABCD regions. {\color{red} We need a better
45 < picture with the letters A-B-C-D and with the numerical values
46 < of the boundaries clearly indicated.}}
51 > \includegraphics[width=0.75\linewidth]{ttdil_uncor_38X.png}
52 > \caption{\label{fig:abcdMC}\protect Distributions of MET$/\sqrt{\rm SumJetPt}$ vs.
53 > SumJetPt for SM Monte Carlo.  Here we also show our choice of ABCD regions. The correlation coefficient
54 > ${\rm corr_{XY}}$ is computed for events falling in the ABCD regions.}
55   \end{center}
56   \end{figure}
57  
58  
59   Our choice of ABCD regions is shown in Figure~\ref{fig:abcdMC}.
60   The signal region is region D.  The expected number of events
61 < in the four regions for the SM Monte Carlo, as well as the BG
62 < prediction AC/B is given in Table~\ref{tab:abcdMC} for an integrated
63 < luminosity of 30 pb$^{-1}$.  The ABCD method is accurate
64 < to about 10\%.
61 > in the four regions for the SM Monte Carlo, as well as the background
62 > prediction A $\times$ C / B are given in Table~\ref{tab:abcdMC} for an integrated
63 > luminosity of 34.0 pb$^{-1}$. In Table~\ref{tab:abcdsyst}, we test the stability of
64 > observed/predicted with respect to variations in the ABCD boundaries.
65 > Based on the results in Tables~\ref{tab:abcdMC} and~\ref{tab:abcdsyst}, we assess
66 > a systematic uncertainty of 20\% on the prediction of the ABCD method.
67 >
68 > %As shown in Table~\ref{tab:abcdsyst}, we assess systematic uncertainties
69 > %by varying the boundaries by an amount consistent with the hadronic energy scale uncertainty,
70 > %which we take as $\pm$5\% for SumJetPt and $\pm$2.5\% for MET/$\sqrt{\rm SumJetPt}$, since the
71 > %uncertainty on this quantity partially cancels due to the fact that it is a ratio of correlated
72 > %quantities. Based on these studies we assess a correction factor $k_{ABCD} = 1.2 \pm 0.2$ to the
73 > %predicted yield using the ABCD method.
74 >
75 >
76 > %{\color{red} Avi wants some statement about stability
77 > %wrt changes in regions.  I am not sure that we have done it and
78 > %I am not sure it is necessary (Claudio).}
79  
80 < \begin{table}[htb]
80 > \begin{table}[ht]
81   \begin{center}
82   \caption{\label{tab:abcdMC} Expected SM Monte Carlo yields for
83 < 30 pb$^{-1}$ in the ABCD regions.}
84 < \begin{tabular}{|l|c|c|c|c||c|}
83 > 34.0~pb$^{-1}$ in the ABCD regions, as well as the predicted yield in
84 > the signal region given by A $\times$ C / B. Here `SM other' is the sum
85 > of non-dileptonic $t\bar{t}$ decays, $W^{\pm}$+jets, $W^+W^-$,
86 > $W^{\pm}Z^0$, $Z^0Z^0$ and single top.}
87 > \begin{tabular}{lccccc}
88 > %%%official json v3, 38X MC (D6T ttbar and DY)
89 > \hline
90 >              sample                     &                   A   &                   B   &                   C   &                   D   &                PRED  \\
91 > \hline
92 > $t\bar{t}\rightarrow \ell^{+}\ell^{-}$   &   8.44  $\pm$  0.18   &  32.83  $\pm$  0.35   &   4.78  $\pm$  0.14   &   1.07  $\pm$  0.06   &   1.23  $\pm$  0.05  \\
93 > $Z^0 \rightarrow \ell^{+}\ell^{-}$       &   0.17  $\pm$  0.08   &   1.18  $\pm$  0.22   &   0.04  $\pm$  0.04   &   0.12  $\pm$  0.07   &   0.01  $\pm$  0.01  \\
94 >            SM other                     &   0.53  $\pm$  0.03   &   2.26  $\pm$  0.11   &   0.23  $\pm$  0.03   &   0.07  $\pm$  0.01   &   0.05  $\pm$  0.01  \\
95 > \hline
96 >         total SM MC                     &   9.14  $\pm$  0.20   &  36.26  $\pm$  0.43   &   5.05  $\pm$  0.14   &   1.27  $\pm$  0.10   &   1.27  $\pm$  0.05  \\
97 > \hline
98 > \end{tabular}
99 > \end{center}
100 > \end{table}
101 >
102 >
103 >
104 > \begin{table}[ht]
105 > \begin{center}
106 > \caption{\label{tab:abcdsyst}
107 > Results of the systematic study of the ABCD method by varying the boundaries
108 > between the ABCD regions shown in Fig.~\ref{fig:abcdMC}. Here $x_1$ is the lower SumJetPt boundary and
109 > $x_2$ is the boundary separating regions A and B from C and D, their nominal values are 125 and 300~GeV,
110 > respectively. $y_1$ is the lower MET/$\sqrt{\rm SumJetPt}$ boundary and
111 > $y_2$ is the boundary separating regions B and C from A and D, their nominal values are 4.5 and 8.5~GeV$^{1/2}$,
112 > respectively.}
113 > \begin{tabular}{cccc|c}
114 > \hline
115 > $x_1$   &   $x_2$ & $y_1$   &   $y_2$ & Observed/Predicted \\
116 > \hline
117 >
118 > nominal & nominal & nominal & nominal & $1.00 \pm 0.08$    \\
119 >
120 > +5\%    & +5\%    & +2.5\%  & +2.5\%  & $1.08 \pm 0.11$    \\
121 >
122 > +5\%    & +5\%    & nominal & nominal & $1.04 \pm 0.10$    \\
123 >
124 > nominal & nominal & +2.5\%  & +2.5\%  & $1.03 \pm 0.09$    \\
125 >
126 > nominal & +5\%    & nominal & +2.5\%  & $1.05 \pm 0.10$    \\
127 >
128 > nominal & -5\%    & nominal & -2.5\%  & $0.95 \pm 0.07$    \\
129 >
130 > -5\%    & -5\%    & +2.5\%  & +2.5\%  & $1.00 \pm 0.08$    \\
131 >
132 > +5\%    & +5\%    & -2.5\%  & -2.5\%  & $0.98 \pm 0.09$    \\
133 > \hline
134 > \end{tabular}
135 > \end{center}
136 > \end{table}
137 >
138 > \subsection{Dilepton $P_T$ method}
139 > \label{sec:victory}
140 > This method is based on a suggestion by V. Pavlunin\cite{ref:victory},
141 > and was investigated by our group in 2009\cite{ref:ourvictory}.
142 > The idea is that in dilepton $t\bar{t}$ events the lepton and neutrinos
143 > from $W$ decays have the same $P_T$ spectrum (modulo $W$ polarization
144 > effects).  One can then use the observed
145 > $P_T(\ell\ell)$ distribution to model the sum of neutrino $P_T$'s which
146 > is identified with the \met.
147 >
148 > Then, in order to predict the $t\bar{t} \to$ dilepton contribution to a
149 > selection with \met$+$X, one applies a cut on $P_T(\ell\ell)+$X instead.
150 > In practice one has to rescale the result of the $P_T(\ell\ell)+$X selection
151 > to account for the fact that any dilepton selection must include a
152 > moderate \met cut in order to reduce Drell Yan backgrounds.  This
153 > is discussed in Section 5.3 of Reference~\cite{ref:ourvictory}; for a \met
154 > cut of 50 GeV, the rescaling factor is obtained from the MC as
155 >
156 > \newcommand{\ptll} {\ensuremath{P_T(\ell\ell)}}
157 > \begin{center}
158 > $ K = \frac{\int_0^{\infty} {\cal N}(\ptll)~~d\ptll~}{\int_{50}^{\infty} {\cal N}(\ptll)~~d\ptll~} = 1.5$
159 > \end{center}
160 >
161 >
162 > %%%TO BE REPLACED
163 > %Given the integrated luminosity of the
164 > %present dataset, the determination of $K$ in data is severely statistics
165 > %limited.  Thus, we take $K$ from $t\bar{t}$ Monte Carlo as
166 >
167 > %\begin{center}
168 > %$ K_{MC} = \frac{\int_0^{\infty} {\cal N}(\met)~~d\met~}{\int_{50}^{\infty} {\cal N}(\met)~~d\met~}$
169 > %\end{center}
170 >
171 > %\noindent {\color{red} For the 11 pb result we have used $K$ from data.}
172 >
173 > There are several effects that spoil the correspondance between \met and
174 > $P_T(\ell\ell)$:
175 > \begin{itemize}
176 > \item $Ws$ in top events are polarized.  Neutrinos are emitted preferentially
177 > parallel to the $W$ velocity while charged leptons are emitted prefertially
178 > anti-parallel. Thus the $P_T(\nu\nu)$ distribution is harder
179 > than the $P_T(\ell\ell)$ distribution for top dilepton events.
180 > \item The lepton selections results in $P_T$ and $\eta$ cuts on the individual
181 > leptons that have no simple correspondance to the neutrino requirements.
182 > \item Similarly, the \met$>$50 GeV cut introduces an asymmetry between leptons and
183 > neutrinos which is only partially compensated by the $K$ factor above.
184 > \item The \met resolution is much worse than the dilepton $P_T$ resolution.
185 > When convoluted with a falling spectrum in the tails of \met, this results
186 > in a harder spectrum for \met than the original $P_T(\nu\nu)$.
187 > \item The \met response in CMS is not exactly 1.  This causes a distortion
188 > in the \met distribution that is not present in the $P_T(\ell\ell)$ distribution.
189 > \item The $t\bar{t} \to$ dilepton signal includes contributions from
190 > $W \to \tau \to \ell$.  For these events the arguments about the equivalence
191 > of $P_T(\ell\ell)$ and $P_T(\nu\nu)$ do not apply.
192 > \item A dilepton selection will include SM events from non $t\bar{t}$
193 > sources.  These events can affect the background prediction.  Particularly
194 > dangerous are high $P_T$ Drell Yan events that barely pass the \met$>$ 50
195 > GeV selection.  They will tend to push the data-driven background prediction up.
196 > Therefore we estimate the number of DY events entering the background prediction
197 > using the $R_{out/in}$ method as described in Sec.~\ref{sec:othBG}.
198 > \end{itemize}
199 >
200 > We have studied these effects in SM Monte Carlo, using a mixture of generator and
201 > reconstruction level studies, putting the various effects in one at a time.
202 > For each configuration, we apply the data-driven method and report as figure
203 > of merit the ratio of observed and predicted events in the signal region.
204 > The results are summarized in Table~\ref{tab:victorybad}.
205 >
206 > \begin{table}[htb]
207 > \begin{center}
208 > \caption{\label{tab:victorybad}
209 > Test of the data driven method in Monte Carlo
210 > under different assumptions, evaluated using Spring10 MC.  See text for details.}
211 > \begin{tabular}{|l|c|c|c|c|c|c|c|c|}
212 > \hline
213 > & True $t\bar{t}$ dilepton & $t\to W\to\tau$& other SM & GEN or  & Lepton $P_T$    & Z veto & \met $>$ 50& obs/pred \\
214 > & included                 & included       & included & RECOSIM & and $\eta$ cuts &        &            &       \\ \hline
215 > 1&Y                        &     N          &   N      &  GEN    &   N             &   N    & N          & 1.90  \\
216 > 2&Y                        &     N          &   N      &  GEN    &   Y             &   N    & N          & 1.64  \\
217 > 3&Y                        &     N          &   N      &  GEN    &   Y             &   Y    & N          & 1.59  \\
218 > 4&Y                        &     N          &   N      &  GEN    &   Y             &   Y    & Y          & 1.55  \\
219 > 5&Y                        &     N          &   N      & RECOSIM &   Y             &   Y    & Y          & 1.51  \\
220 > 6&Y                        &     Y          &   N      & RECOSIM &   Y             &   Y    & Y          & 1.58  \\
221 > 7&Y                        &     Y          &   Y      & RECOSIM &   Y             &   Y    & Y          & 1.38  \\
222 > \hline
223 > \end{tabular}
224 > \end{center}
225 > \end{table}
226 >
227 >
228 > \begin{table}[htb]
229 > \begin{center}
230 > \caption{\label{tab:victorysyst}
231 > Summary of variations in $K_C$ due to the MET scale and resolution uncertainty, and to backgrounds other than $t\bar{t} \to$~dilepton.
232 > In the first table, `up' and `down' refer to shifting the hadronic energy scale up and down by 5\%. In the second table, the quoted value
233 > refers to the amount of additional smearing of the MET, as discussed in the text. In the third table, the normalization of all backgrounds
234 > other than $t\bar{t} \to$~dilepton is varied. }
235 > \begin{tabular}{ lcccc }
236 > \hline
237 >       MET scale  &      Predicted       &       Observed       &       Obs/pred       \\
238 > \hline
239 >        nominal   &  0.92 $ \pm $ 0.09   &  1.27 $ \pm $ 0.10   &   1.39 $ \pm $ 0.18  \\
240 >            up    &  0.90 $ \pm $ 0.09   &  1.58 $ \pm $ 0.10   &   1.75 $ \pm $ 0.21  \\
241 >          down    &  0.70 $ \pm $ 0.06   &  0.96 $ \pm $ 0.09   &   1.37 $ \pm $ 0.18  \\
242 > \hline
243 >   MET smearing   &      Predicted       &       Observed       &       Obs/pred       \\
244 > \hline
245 >        nominal   &  0.92 $ \pm $ 0.09   &  1.27 $ \pm $ 0.10   &   1.39 $ \pm $ 0.18  \\
246 >           10\%   &  0.88 $ \pm $ 0.09   &  1.28 $ \pm $ 0.10   &   1.47 $ \pm $ 0.19  \\
247 >           20\%   &  0.87 $ \pm $ 0.09   &  1.26 $ \pm $ 0.10   &   1.44 $ \pm $ 0.19  \\
248 >           30\%   &  1.03 $ \pm $ 0.17   &  1.33 $ \pm $ 0.10   &   1.29 $ \pm $ 0.23  \\
249 >           40\%   &  0.88 $ \pm $ 0.09   &  1.36 $ \pm $ 0.10   &   1.55 $ \pm $ 0.20  \\
250 >           50\%   &  0.80 $ \pm $ 0.07   &  1.39 $ \pm $ 0.10   &   1.73 $ \pm $ 0.19  \\
251 > \hline
252 >  non-$t\bar{t} \to$~dilepton bkg   &       Predicted   &           Observed   &           Obs/pred   \\
253 > \hline
254 >   ttdil only                       &   0.79 $ \pm $ 0.07   &   1.07 $ \pm $ 0.06   &   1.36 $ \pm $ 0.14   \\
255 >   nominal                          &   0.92 $ \pm $ 0.09   &   1.27 $ \pm $ 0.10   &   1.39 $ \pm $ 0.18   \\
256 >   double non-ttdil yield           &   1.04 $ \pm $ 0.15   &   1.47 $ \pm $ 0.16   &   1.40 $ \pm $ 0.25   \\
257   \hline
64 Sample   & A   & B    & C   & D   & AC/D \\ \hline
65 ttdil    & 6.4 & 28.4 & 4.2 & 1.0 & 0.9  \\
66 Zjets    & 0.0 & 1.3  & 0.2 & 0.0 & 0.0  \\
67 Other SM & 0.6 & 2.1  & 0.2 & 0.1 & 0.0  \\ \hline
68 total MC & 7.0 & 31.8 & 4.5 & 1.1 & 1.0 \\ \hline
258   \end{tabular}
259   \end{center}
260   \end{table}
261  
262 + The largest discrepancy between prediction and observation occurs on the first
263 + line of Table~\ref{tab:victorybad}, {\em i.e.}, at the generator level with no
264 + cuts.  We have verified that this effect is due to the polarization of
265 + the $W$ (we remove the polarization by reweighting the events and we get
266 + good agreement between prediction and observation).  The kinematical
267 + requirements (lines 2,3,4) compensate somewhat for the effect of W polarization.
268 + Going from GEN to RECOSIM, the change in observed/predicted is small.  
269 + % We have tracked this down to the fact that tcMET underestimates the true \met
270 + % by $\approx 4\%$\footnote{We find that observed/predicted changes by roughly 0.1
271 + %for each 1.5\% change in \met response.}.  
272 + Finally, contamination from non $t\bar{t}$
273 + events can have a significant impact on the BG prediction.  
274 + %The changes between
275 + %lines 6 and 7 of Table~\ref{tab:victorybad} is driven by 3
276 + %Drell Yan events that pass the \met selection in Monte Carlo (thus the effect
277 + %is statistically not well quantified).
278 +
279 + An additional source of concern is that the CMS Madgraph $t\bar{t}$ MC does
280 + not include effects of spin correlations between the two top quarks.  
281 + We have studied this effect at the generator level using Alpgen.  We find
282 + that the bias is at the few percent level.
283 +
284 + Based on the results of Table~\ref{tab:victorysyst}, we conclude that the
285 + naive data-driven background estimate based on $P_T{(\ell\ell)}$ needs to
286 + be corrected by a factor of $ K_C = 1.4 \pm 0.2({\rm stat})$.
287 +
288 + The dominant sources of systematic uncertainty in $K_C$ are due to non-$t\bar{t} \to$~dilepton backgrounds,
289 + and the MET scale and resolution uncertainties, as summarized in Table~\ref{tab:victorysyst}.
290 + The impact of non-$t\bar{t}$-dilepton background is assessed
291 + by varying the yield of all backgrounds except for $t\bar{t} \to$~dilepton.
292 + The uncertainty is assessed as the larger of the differences between the nominal $K_C$ value and the values
293 + obtained using only $t\bar{t} \to$~dilepton MC and obtained by doubling the non $t\bar{t} \to$~dilepton component,
294 + giving an uncertainty of $0.03$.
295 +
296 + The uncertainty in $K_C$ due to the MET scale uncertainty is assessed by varying the hadronic energy scale using
297 + the same method as in~\cite{ref:top}, giving an uncertainty of 0.36.
298 + We also assess the impact of the MET resolution
299 + uncertainty on $K_C$ by applying a random smearing to the MET. For each event, we determine the expected MET resolution
300 + based on the sumJetPt, and smear the MET to simulate an increase in the resolution of 10\%, 20\%, 30\%, 40\% and 50\%.
301 + The results show that $K_C$ does not depend strongly on the MET resolution and we therefore do not assess any uncertainty.
302 +
303 + Incorporating all the statistical and systematic uncertainties we find $K_C = 1.4 \pm 0.4$.
304 +
305 + \subsection{Signal Contamination}
306 + \label{sec:sigcont}
307 +
308 + All data-driven methods are in principle subject to signal contaminations
309 + in the control regions, and the methods described in
310 + Sections~\ref{sec:abcd} and~\ref{sec:victory} are not exceptions.
311 + Signal contamination tends to dilute the significance of a signal
312 + present in the data by inflating the background prediction.
313 +
314 + It is hard to quantify how important these effects are because we
315 + do not know what signal may be hiding in the data.  Having two
316 + independent methods (in addition to Monte Carlo ``dead-reckoning'')
317 + adds redundancy because signal contamination can have different effects
318 + in the different control regions for the two methods.
319 + For example, in the extreme case of a
320 + new physics signal
321 + with $P_T(\ell \ell) = \met$, an excess of events would be seen
322 + in the ABCD method but not in the $P_T(\ell \ell)$ method.
323 +
324 +
325 + The LM points are benchmarks for SUSY analyses at CMS.  The effects
326 + of signal contaminations for a couple such points are summarized
327 + in Table~\ref{tab:sigcont}. Signal contamination is definitely an important
328 + effect for these two LM points, but it does not totally hide the
329 + presence of the signal.
330  
331  
332 + \begin{table}[htb]
333 + \begin{center}
334 + \caption{\label{tab:sigcont} Effects of signal contamination
335 + for the two data-driven background estimates. The three columns give
336 + the expected yield in the signal region and the background estimates
337 + using the ABCD and $P_T(\ell \ell)$ methods. Results are normalized to 34.0~pb$^{-1}$.}
338 + \begin{tabular}{lccc}
339 + \hline
340 +            &      Yield      &      ABCD    & $P_T(\ell \ell)$  \\
341 + \hline
342 + SM only     &       1.3      &      1.3    &       0.9        \\
343 + SM + LM0    &       7.4      &      4.4    &       1.9        \\
344 + SM + LM1    &       3.8      &      1.6    &       1.4        \\
345 + %SM only     &       1.27      &      1.27    &       0.92        \\
346 + %SM + LM0    &       7.39      &      4.38    &       1.93        \\
347 + %SM + LM1    &       3.77      &      1.62    &       1.41        \\
348 + \hline
349 + \end{tabular}
350 + \end{center}
351 + \end{table}
352  

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines