ViewVC Help
View File | Revision Log | Show Annotations | Root Listing
root/cvsroot/UserCode/claudioc/OSNote2010/datadriven.tex
(Generate patch)

Comparing UserCode/claudioc/OSNote2010/datadriven.tex (file contents):
Revision 1.8 by claudioc, Fri Nov 5 15:53:10 2010 UTC vs.
Revision 1.28 by benhoob, Thu Dec 2 14:27:31 2010 UTC

# Line 2 | Line 2
2   \label{sec:datadriven}
3   We have developed two data-driven methods to
4   estimate the background in the signal region.
5 < The first one explouts the fact that
6 < \met and \met$/\sqrt{\rm SumJetPt}$ are nearly
5 > The first one exploits the fact that
6 > SumJetPt and \met$/\sqrt{\rm SumJetPt}$ are nearly
7   uncorrelated for the $t\bar{t}$ background
8   (Section~\ref{sec:abcd});  the second one
9   is based on the fact that in $t\bar{t}$ the
# Line 12 | Line 12 | nearly the same as the $P_T$ of the pair
12   from $W$-decays, which is reconstructed as \met in the
13   detector.
14  
15 < In 30 pb$^{-1}$ we expect $\approx$ 1 SM event in
16 < the signal region.  The expectations from the LMO
17 < and LM1 SUSY benchmark points are 5.6 and
18 < 2.2 events respectively.
15 >
16   %{\color{red} I took these
17   %numbers from the twiki, rescaling from 11.06 to 30/pb.
18   %They seem too large...are they really right?}
# Line 24 | Line 21 | and LM1 SUSY benchmark points are 5.6 an
21   \subsection{ABCD method}
22   \label{sec:abcd}
23  
24 < We find that in $t\bar{t}$ events \met and
25 < \met$/\sqrt{\rm SumJetPt}$ are nearly uncorrelated.
26 < This is demonstrated in Figure~\ref{fig:uncor}.
24 > We find that in $t\bar{t}$ events SumJetPt and
25 > \met$/\sqrt{\rm SumJetPt}$ are nearly uncorrelated,
26 > as demonstrated in Figure~\ref{fig:uncor}.
27   Thus, we can use an ABCD method in the \met$/\sqrt{\rm SumJetPt}$ vs
28   sumJetPt plane to estimate the background in a data driven way.
29  
30 < \begin{figure}[tb]
30 > %\begin{figure}[bht]
31 > %\begin{center}
32 > %\includegraphics[width=0.75\linewidth]{uncorrelated.pdf}
33 > %\caption{\label{fig:uncor}\protect Distributions of SumJetPt
34 > %in MC $t\bar{t}$ events for different intervals of
35 > %MET$/\sqrt{\rm SumJetPt}$.}
36 > %\end{center}
37 > %\end{figure}
38 >
39 > \begin{figure}[bht]
40   \begin{center}
41 < \includegraphics[width=0.75\linewidth]{uncorrelated.pdf}
41 > \includegraphics[width=0.75\linewidth]{uncor.png}
42   \caption{\label{fig:uncor}\protect Distributions of SumJetPt
43   in MC $t\bar{t}$ events for different intervals of
44 < MET$/\sqrt{\rm SumJetPt}$.}
44 > MET$/\sqrt{\rm SumJetPt}$. h1, h2, and h3 refer to the MET$/\sqrt{\rm SumJetPt}$
45 > intervals 4.5-6.5, 6.5-8.5 and $>$8.5, respectively.}
46   \end{center}
47   \end{figure}
48  
49 < \begin{figure}[bt]
49 > \begin{figure}[tb]
50   \begin{center}
51   \includegraphics[width=0.5\linewidth, angle=90]{abcdMC.pdf}
52 < \caption{\label{fig:abcdMC}\protect Distributions of SumJetPt
53 < vs. MET$/\sqrt{\rm SumJetPt}$ for SM Monte Carlo.  Here we also
47 < show our choice of ABCD regions.}
52 > \caption{\label{fig:abcdMC}\protect Distributions of MET$/\sqrt{\rm SumJetPt}$ vs.
53 > SumJetPt for SM Monte Carlo.  Here we also show our choice of ABCD regions.}
54   \end{center}
55   \end{figure}
56  
# Line 53 | Line 59 | Our choice of ABCD regions is shown in F
59   The signal region is region D.  The expected number of events
60   in the four regions for the SM Monte Carlo, as well as the BG
61   prediction AC/B are given in Table~\ref{tab:abcdMC} for an integrated
62 < luminosity of 30 pb$^{-1}$.  The ABCD method is accurate
63 < to about 10\%. {\color{red} Avi wants some statement about stability
64 < wrt changes in regions.  I am not sure that we have done it and
65 < I am not sure it is necessary (Claudio).}
62 > luminosity of 35 pb$^{-1}$.  The ABCD method with chosen boundaries is accurate
63 > to about 20\%. As shown in Table~\ref{tab:abcdsyst}, we assess systematic uncertainties
64 > by varying the boundaries by an amount consistent with the hadronic energy scale uncertainty,
65 > which we take as $\pm$5\% for SumJetPt and $\pm$2.5\% for MET/$\sqrt{\rm SumJetPt}$, since the
66 > uncertainty on this quantity partially cancels due to the fact that it is a ratio of correlated
67 > quantities. Based on these studies we assess a correction factor $k_{ABCD} = 1.2 \pm 0.2$ to the
68 > predicted yield using the ABCD method.
69  
70 < \begin{table}[htb]
70 >
71 > %{\color{red} Avi wants some statement about stability
72 > %wrt changes in regions.  I am not sure that we have done it and
73 > %I am not sure it is necessary (Claudio).}
74 >
75 > \begin{table}[ht]
76   \begin{center}
77   \caption{\label{tab:abcdMC} Expected SM Monte Carlo yields for
78 < 30 pb$^{-1}$ in the ABCD regions.}
79 < \begin{tabular}{|l|c|c|c|c||c|}
78 > 35 pb$^{-1}$ in the ABCD regions, as well as the predicted yield in
79 > the signal region given by A $\times$ C / B. Here `SM other' is the sum
80 > of non-dileptonic $t\bar{t}$ decays, $W^{\pm}$+jets, $W^+W^-$,
81 > $W^{\pm}Z^0$, $Z^0Z^0$ and single top.}
82 > \begin{tabular}{lccccc}
83 > \hline
84 >              sample   &                   A   &                   B   &                   C   &                   D   &                      A $\times$ C / B  \\
85 > \hline
86 > $t\bar{t}\rightarrow \ell^{+}\ell^{-}$   &   8.27  $\pm$  0.18   &  32.16  $\pm$  0.35   &   4.69  $\pm$  0.13   &   1.05  $\pm$  0.06   &   1.21  $\pm$  0.04  \\
87 > $Z^0 \rightarrow \ell^{+}\ell^{-}$       &   0.22  $\pm$  0.11   &   1.54  $\pm$  0.29   &   0.05  $\pm$  0.05   &   0.16  $\pm$  0.09   &   0.01  $\pm$  0.01  \\
88 >            SM other                     &   0.54  $\pm$  0.03   &   2.28  $\pm$  0.12   &   0.23  $\pm$  0.03   &   0.07  $\pm$  0.01   &   0.05  $\pm$  0.01  \\
89 > \hline
90 >         total SM MC                     &   9.03  $\pm$  0.21   &  35.97  $\pm$  0.46   &   4.97  $\pm$  0.15   &   1.29  $\pm$  0.11   &   1.25  $\pm$  0.05  \\
91 > \hline
92 > \end{tabular}
93 > \end{center}
94 > \end{table}
95 >
96 >
97 >
98 > \begin{table}[ht]
99 > \begin{center}
100 > \caption{\label{tab:abcdsyst}
101 > {\bf \color{red} Do we need this study at all? Observed/predicted is consistent within stat uncertainties as the boundaries are varied- is it enough to simply state this fact in the text??? }
102 > Results of the systematic study of the ABCD method by varying the boundaries
103 > between the ABCD regions shown in Fig.~\ref{fig:abcdMC}. Here $x_1$ is the lower SumJetPt boundary and
104 > $x_2$ is the boundary separating regions A and B from C and D, their nominal values are 125 and 300~GeV,
105 > respectively. $y_1$ is the lower MET/$\sqrt{\rm SumJetPt}$ boundary and
106 > $y_2$ is the boundary separating regions B and C from A and D, their nominal values are 4.5 and 8.5~GeV$^{1/2}$,
107 > respectively.}
108 > \begin{tabular}{cccc|c}
109 > \hline
110 > $x_1$   &   $x_2$ & $y_1$   &   $y_2$ & Observed/Predicted \\
111 > \hline
112 > nominal & nominal & nominal & nominal & $1.20 \pm 0.12$    \\
113 > +5\%    & +5\%    & +2.5\%  & +2.5\%  & $1.38 \pm 0.15$    \\
114 > +5\%    & +5\%    & nominal & nominal & $1.31 \pm 0.14$    \\
115 > nominal & nominal & +2.5\%  & +2.5\%  & $1.25 \pm 0.13$    \\
116 > nominal & +5\%    & nominal & +2.5\%  & $1.32 \pm 0.14$    \\
117 > nominal & -5\%    & nominal & -2.5\%  & $1.16 \pm 0.09$    \\
118 > -5\%    & -5\%    & +2.5\%  & +2.5\%  & $1.21 \pm 0.11$    \\
119 > +5\%    & +5\%    & -2.5\%  & -2.5\%  & $1.26 \pm 0.12$    \\
120   \hline
67 Sample   & A   & B    & C   & D   & AC/D \\ \hline
68 ttdil    & 6.9 & 28.6 & 4.2 & 1.0 & 1.0  \\
69 Zjets    & 0.0 & 1.3  & 0.1 & 0.1 & 0.0  \\
70 Other SM & 0.5 & 2.0  & 0.1 & 0.1 & 0.0  \\ \hline
71 total MC & 7.4 & 31.9 & 4.4 & 1.2 & 1.0 \\ \hline
121   \end{tabular}
122   \end{center}
123   \end{table}
# Line 89 | Line 138 | In practice one has to rescale the resul
138   to account for the fact that any dilepton selection must include a
139   moderate \met cut in order to reduce Drell Yan backgrounds.  This
140   is discussed in Section 5.3 of Reference~\cite{ref:ourvictory}; for a \met
141 < cut of 50 GeV, the rescaling factor is obtained from the data as
141 > cut of 50 GeV, the rescaling factor is obtained from the MC as
142  
143   \newcommand{\ptll} {\ensuremath{P_T(\ell\ell)}}
144   \begin{center}
# Line 98 | Line 147 | $ K = \frac{\int_0^{\infty} {\cal N}(\pt
147  
148  
149   Monte Carlo studies give values of $K$ that are typically between 1.5 and 1.6,
150 < depending on selection details.
150 > depending on selection details.  
151 > %%%TO BE REPLACED
152 > %Given the integrated luminosity of the
153 > %present dataset, the determination of $K$ in data is severely statistics
154 > %limited.  Thus, we take $K$ from $t\bar{t}$ Monte Carlo as
155 >
156 > %\begin{center}
157 > %$ K_{MC} = \frac{\int_0^{\infty} {\cal N}(\met)~~d\met~}{\int_{50}^{\infty} {\cal N}(\met)~~d\met~}$
158 > %\end{center}
159 >
160 > %\noindent {\color{red} For the 11 pb result we have used $K$ from data.}
161  
162   There are several effects that spoil the correspondance between \met and
163   $P_T(\ell\ell)$:
164   \begin{itemize}
165   \item $Ws$ in top events are polarized.  Neutrinos are emitted preferentially
166 < forward in the $W$ rest frame, thus the $P_T(\nu\nu)$ distribution is harder
166 > parallel to the $W$ velocity while charged leptons are emitted prefertially
167 > anti-parallel. Thus the $P_T(\nu\nu)$ distribution is harder
168   than the $P_T(\ell\ell)$ distribution for top dilepton events.
169   \item The lepton selections results in $P_T$ and $\eta$ cuts on the individual
170   leptons that have no simple correspondance to the neutrino requirements.
171   \item Similarly, the \met$>$50 GeV cut introduces an asymmetry between leptons and
172   neutrinos which is only partially compensated by the $K$ factor above.
173   \item The \met resolution is much worse than the dilepton $P_T$ resolution.
174 < When convoluted with a falling spectrum in the tails of \met, this result
174 > When convoluted with a falling spectrum in the tails of \met, this results
175   in a harder spectrum for \met than the original $P_T(\nu\nu)$.
176   \item The \met response in CMS is not exactly 1.  This causes a distortion
177   in the \met distribution that is not present in the $P_T(\ell\ell)$ distribution.
# Line 122 | Line 182 | of $P_T(\ell\ell)$ and $P_T(\nu\nu)$ do
182   sources.  These events can affect the background prediction.  Particularly
183   dangerous are high $P_T$ Drell Yan events that barely pass the \met$>$ 50
184   GeV selection.  They will tend to push the data-driven background prediction up.
185 + Therefore we estimate the number of DY events entering the background prediction
186 + using the $R_{out/in}$ method as described in Sec.~\ref{sec:othBG}.
187   \end{itemize}
188  
189   We have studied these effects in SM Monte Carlo, using a mixture of generator and
# Line 132 | Line 194 | The results are summarized in Table~\ref
194  
195   \begin{table}[htb]
196   \begin{center}
197 < \caption{\label{tab:victorybad} Test of the data driven method in Monte Carlo
197 > \caption{\label{tab:victorybad}
198 > {\bf \color{red} Need to either update this with 38X MC  or remove it }
199 > Test of the data driven method in Monte Carlo
200   under different assumptions.  See text for details.}
201   \begin{tabular}{|l|c|c|c|c|c|c|c|c|}
202   \hline
203   & True $t\bar{t}$ dilepton & $t\to W\to\tau$& other SM & GEN or  & Lepton $P_T$    & Z veto & \met $>$ 50& obs/pred \\
204 < & included                 & included       & included & RECOSIM & and $\eta$ cuts &        &            &  \\ \hline
204 > & included                 & included       & included & RECOSIM & and $\eta$ cuts &        &            &       \\ \hline
205   1&Y                        &     N          &   N      &  GEN    &   N             &   N    & N          & 1.90  \\
206   2&Y                        &     N          &   N      &  GEN    &   Y             &   N    & N          & 1.64  \\
207   3&Y                        &     N          &   N      &  GEN    &   Y             &   Y    & N          & 1.59  \\
208   4&Y                        &     N          &   N      &  GEN    &   Y             &   Y    & Y          & 1.55  \\
209   5&Y                        &     N          &   N      & RECOSIM &   Y             &   Y    & Y          & 1.51  \\
210   6&Y                        &     Y          &   N      & RECOSIM &   Y             &   Y    & Y          & 1.58  \\
211 < 7&Y                        &     Y          &   Y      & RECOSIM &   Y             &   Y    & Y          & 1.18  \\
211 > 7&Y                        &     Y          &   Y      & RECOSIM &   Y             &   Y    & Y          & 1.38  \\
212 > \hline
213 > \end{tabular}
214 > \end{center}
215 > \end{table}
216 >
217 >
218 > \begin{table}[htb]
219 > \begin{center}
220 > \caption{\label{tab:victorysyst}
221 > {Summary of uncertainties in $K_C$ due to the MET scale and resolution uncertainty, and to backgrounds other than $t\bar{t} \to$~dilepton.
222 > In the first table, `up' and `down' refer to shifting the hadronic energy scale up and down by 5\%. In the second table, the quoted value
223 > refers to the amount of additional smearing of the MET, as discussed in the text. In the third table, the normalization of all backgrounds
224 > other than $t\bar{t} \to$~dilepton is varied.
225 > {\bf \color{ref} Should I remove `observed' and `predicted' and show only the ratio? }}
226 >
227 > \begin{tabular}{ lcccc }
228 > \hline
229 >       MET scale  &      Predicted       &       Observed       &       Obs/pred       \\
230 > \hline
231 >        nominal   &  0.92 $ \pm $ 0.11   &  1.29 $ \pm $ 0.11   &  1.40 $ \pm $ 0.20   \\
232 >            up    &  0.92 $ \pm $ 0.11   &  1.53 $ \pm $ 0.12   &  1.66 $ \pm $ 0.23   \\
233 >          down    &  0.81 $ \pm $ 0.07   &  1.08 $ \pm $ 0.11   &  1.32 $ \pm $ 0.17   \\
234 > \hline
235 >
236 > \hline
237 >   MET smearing   &      Predicted       &       Observed        &       Obs/pred      \\
238 > \hline
239 >        nominal   &  0.92 $ \pm $ 0.11   &  1.29 $ \pm $ 0.11   &  1.40 $ \pm $ 0.20   \\
240 >           10\%   &  0.90 $ \pm $ 0.11   &  1.30 $ \pm $ 0.11   &  1.44 $ \pm $ 0.21   \\
241 >           20\%   &  0.84 $ \pm $ 0.07   &  1.36 $ \pm $ 0.11   &  1.61 $ \pm $ 0.19   \\
242 >           30\%   &  1.05 $ \pm $ 0.18   &  1.32 $ \pm $ 0.11   &  1.27 $ \pm $ 0.24   \\
243 >           40\%   &  0.85 $ \pm $ 0.07   &  1.37 $ \pm $ 0.11   &  1.61 $ \pm $ 0.19   \\
244 >           50\%   &  1.08 $ \pm $ 0.18   &  1.36 $ \pm $ 0.11   &  1.26 $ \pm $ 0.24   \\
245 > \hline
246 >
247 > \hline
248 >  non-$t\bar{t} \to$~dilepton scale factor   &          Predicted   &           Observed   &           Obs/pred   \\
249 > \hline
250 >   ttdil only                                &  0.77 $ \pm $ 0.07   &  1.05 $ \pm $ 0.06   &  1.36 $ \pm $ 0.14   \\
251 >   nominal                                   &  0.92 $ \pm $ 0.11   &  1.29 $ \pm $ 0.11   &  1.40 $ \pm $ 0.20   \\
252 >   double non-ttdil yield                    &  1.06 $ \pm $ 0.18   &  1.52 $ \pm $ 0.20   &  1.43 $ \pm $ 0.30   \\
253   \hline
254   \end{tabular}
255   \end{center}
256   \end{table}
257  
258  
259 +
260   The largest discrepancy between prediction and observation occurs on the first
261   line of Table~\ref{tab:victorybad}, {\em i.e.}, at the generator level with no
262   cuts.  We have verified that this effect is due to the polarization of
# Line 162 | Line 268 | Going from GEN to RECOSIM, the change in
268   % by $\approx 4\%$\footnote{We find that observed/predicted changes by roughly 0.1
269   %for each 1.5\% change in \met response.}.  
270   Finally, contamination from non $t\bar{t}$
271 < events can have a significant impact on the BG prediction.  The changes between
272 < lines 6 and 7 of Table~\ref{tab:victorybad} is driven by 3
273 < Drell Yan events that pass the \met selection in Monte Carlo (thus the effect
274 < is statistically not well quantified).
271 > events can have a significant impact on the BG prediction.  
272 > %The changes between
273 > %lines 6 and 7 of Table~\ref{tab:victorybad} is driven by 3
274 > %Drell Yan events that pass the \met selection in Monte Carlo (thus the effect
275 > %is statistically not well quantified).
276  
277   An additional source of concern is that the CMS Madgraph $t\bar{t}$ MC does
278   not include effects of spin correlations between the two top quarks.  
# Line 173 | Line 280 | We have studied this effect at the gener
280   that the bias is at the few percent level.
281  
282   Based on the results of Table~\ref{tab:victorybad}, we conclude that the
283 < naive data driven background estimate based on $P_T{(\ell\ell)}$ needs to
284 < be corrected by a factor of {\color{red} $1.2 \pm 0.3$ (We need to talk
178 < about this)} . The quoted
179 < uncertainty is based on the stability of the Monte Carlo tests under
180 < variations of event selections, choices of \met algorithm, etc.
181 < For example, we find that observed/predicted changes by roughly 0.1
182 < for each 1.5\% change in the average \met response.  
283 > naive data-driven background estimate based on $P_T{(\ell\ell)}$ needs to
284 > be corrected by a factor of $ K_C = 1.4 \pm 0.2(stat)$.
285  
286 + The 2 dominant sources of systematic uncertainty in $K_C$ are due to non-$t\bar{t} \to$~dilepton backgrounds,
287 + and the MET scale and resolution uncertainties. The impact of non-$t\bar{t}$-dilepton background is assessed
288 + by varying the yield of all backgrounds except for $t\bar{t} \to$~dilepton, as shown in Table~\ref{table_kc}.
289 + The systematic is assessed as the larger of the differences between the nominal $K_C$ value and the values
290 + obtained using only $t\bar{t} \to$~dilepton MC and obtained by doubling the non $t\bar{t} \to$~dilepton component,
291 + giving an uncertainty of $0.04$.
292 +
293 + The uncertainty in $K_C$ due to the MET scale uncertainty is assessed by varying the hadronic energy scale using
294 + the same method as in~\ref{} and checking how much $K_C$ changes, as summarized in Table~\ref{tab:victorysyst}.
295 + This gives an uncertainty of 0.3. We also assess the impact of the MET resolution uncertainty on $K_C$ by applying
296 + a random smearing to the MET. For each event, we determine the expected MET resolution based on the sumJetPt, and
297 + smear the MET to simulate an increase in the resolution of 10\%, 20\%, 30\%, 40\% and 50\%. The results show that
298 + $K_C$ does not depend strongly on the MET resolution and we therefore do not assess any uncertainty.
299  
300 + Incorporating all the statistical and systematic uncertainties we find $K_C = 1.4 \pm 0.4$.
301  
302   \subsection{Signal Contamination}
303   \label{sec:sigcont}
# Line 205 | Line 321 | in the ABCD method but not in the $P_T(\
321  
322   The LM points are benchmarks for SUSY analyses at CMS.  The effects
323   of signal contaminations for a couple such points are summarized
324 < in Table~\ref{tab:sigcontABCD} and~\ref{tab:sigcontPT}.
209 < Signal contamination is definitely an important
324 > in Table~\ref{tab:sigcont}. Signal contamination is definitely an important
325   effect for these two LM points, but it does not totally hide the
326   presence of the signal.
327  
328  
329   \begin{table}[htb]
330   \begin{center}
331 < \caption{\label{tab:sigcontABCD} Effects of signal contamination
332 < for the background predictions of the ABCD method including LM0 or
333 < LM1.  Results
334 < are normalized to 30 pb$^{-1}$.}
335 < \begin{tabular}{|c||c|c||c|c|}
221 < \hline
222 < SM         & SM$+$LM0    & BG Prediction & Sm$+$LM1     & BG Prediction \\
223 < Background & Contribution& Including LM0 & Contribution & Including LM1  \\ \hline
224 < 1.2        & 6.8         & 3.7           & 3.4          & 1.3 \\
331 > \caption{\label{tab:sigcont} Effects of signal contamination
332 > for the two data-driven background estimates. The three columns give
333 > the expected yield in the signal region and the background estimates
334 > using the ABCD and $P_T(\ell \ell)$ methods. Results are normalized to 35~pb$^{-1}$.}
335 > \begin{tabular}{lccc}
336   \hline
337 < \end{tabular}
338 < \end{center}
339 < \end{table}
340 <
341 < \begin{table}[htb]
231 < \begin{center}
232 < \caption{\label{tab:sigcontPT} Effects of signal contamination
233 < for the background predictions of the $P_T(\ell\ell)$ method including LM0 or
234 < LM1.  Results
235 < are normalized to 30 pb$^{-1}$. {\color{red} Does this BG prediction include
236 < the fudge factor of 1.4 or watever because the method is not perfect.}}
237 < \begin{tabular}{|c||c|c||c|c|}
238 < \hline
239 < SM         & SM$+$LM0    & BG Prediction & Sm$+$LM1     & BG Prediction \\
240 < Background & Contribution& Including LM0 & Contribution & Including LM1  \\ \hline
241 < 1.2        & 6.8         & 2.2           & 3.4          & 1.5 \\
337 >            &      Yield      &      ABCD    & $P_T(\ell \ell)$  \\
338 > \hline
339 > SM only     &       1.29      &      1.25    &           0.92    \\
340 > SM + LM0    &       7.57      &      4.44    &           1.96    \\
341 > SM + LM1    &       3.85      &      1.60    &           1.43    \\
342   \hline
343   \end{tabular}
344   \end{center}

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines