ViewVC Help
View File | Revision Log | Show Annotations | Root Listing
root/cvsroot/UserCode/benhoob/cmsnotes/StopSearch/systematics.tex
(Generate patch)

Comparing UserCode/benhoob/cmsnotes/StopSearch/systematics.tex (file contents):
Revision 1.14 by vimartin, Wed Oct 10 04:03:33 2012 UTC vs.
Revision 1.22 by vimartin, Sat Oct 20 19:57:24 2012 UTC

# Line 1 | Line 1
1   %\section{Systematics Uncertainties on the Background Prediction}
2   %\label{sec:systematics}
3  
4 [DESCRIBE HERE ONE BY ONE THE UNCERTAINTIES THAT ARE PRESENT IN THE SPREADSHHET
5 FROM WHICH WE CALCULATE THE TOTAL UNCERTAINTY. WE KNOW HOW TO DO THIS
6 AND
7 WE HAVE THE TECHNOLOGY FROM THE 7 TEV ANALYSIS TO PROPAGATE ALL
8 UNCERTAINTIES
9 CORRECTLY THROUGH.  WE WILL DO IT ONCE WE HAVE SETTLED ON THE
10 INDIVIDUAL PIECES WHICH ARE STILL IN FLUX]
11
4   In this Section we discuss the systematic uncertainty on the BG
5   prediction.  This prediction is assembled from the event
6   counts in the peak region of the transverse mass distribution as
# Line 22 | Line 14 | The calculation is done for each signal
14   region,
15   for electrons and muons separately.
16  
17 < The choice to normalizing to the peak region of $M_T$ has the
17 > The choice to normalize to the peak region of $M_T$ has the
18   advantage that some uncertainties, e.g., luminosity, cancel.
19   It does however introduce complications because it couples
20   some of the uncertainties in non-trivial ways.  For example,
# Line 33 | Line 25 | for example,
25   the $t\bar{t} \to$ dilepton BG estimate because it changes the
26   $t\bar{t}$ normalization to the peak region (because some of the
27   events in the peak region are from rare processes).  These effects
28 < are carefully accounted for.  The contribution to the overall
29 < uncertainty from each BG source is tabulated in
28 > are carefully accounted for. The contribution to the overall
29 > uncertainty from each background source is tabulated in
30   Section~\ref{sec:bgunc-bottomline}.
31 < First, however, we discuss the uncertainties one-by-one and we comment
31 > Here we discuss the uncertainties one-by-one and comment
32   on their impact on the overall result, at least to first order.
33   Second order effects, such as the one described, are also included.
34  
35   \subsection{Statistical uncertainties on the event counts in the $M_T$
36   peak regions}
37 < These vary between XX and XX \%, depending on the signal region
37 > These vary between 2\% and 20\%, depending on the signal region
38   (different
39   signal regions have different \met\ requirements, thus they also have
40 < different $M_T$ regions used as control.
40 > different $M_T$ regions used as control).
41   Since
42 < the major BG, eg, $t\bar{t}$ are normalized to the peak regions, this
42 > the major backgrounds, eg, $t\bar{t}$ are normalized to the peak regions, this
43   fractional uncertainty is pretty much carried through all the way to
44   the end.  There is also an uncertainty from the finite MC event counts
45   in the $M_T$ peak regions.  This is also included, but it is smaller.
46  
47 + Normalizing to the $M_T$ peak has the distinct advantages that
48 + uncertainties on luminosity, cross-sections, trigger efficiency,
49 + lepton ID, cancel out.
50 + For the low statistics regions with high \met\ requirements, the
51 + price to pay in terms of event count is that statistical uncertainties start
52 + to become significant.  In the future we may consider a different
53 + normalization startegy in the low statistics regions.
54 +
55   \subsection{Uncertainty from the choice of $M_T$ peak region}
56 IN 7 TEV DATA WE HAD SOME SHAPE DIFFERENCES IN THE MTRANS REGION THAT
57 LED US TO CONSERVATIVELY INCLUDE THIS UNCERTAINTY.  WE NEED TO LOOK
58 INTO THIS AGAIN
56  
57 < \subsection{Uncertainty on the Wjets cross-section and the rare MC cross-sections}
57 > This choice affects the scale factors of Table~\ref{tab:mtpeaksf}.  
58 > If the $M_T$ peak region is not well modelled, this would introduce an
59 > uncertainty.
60 >
61 > We have tested this possibility by recalculating the post-veto scale factors for a different
62 > choice
63 > of $M_T$ peak region ($40 < M_T < 100$ GeV instead of the default
64 > $50 < M_T < 80$ GeV).  This is shown in Table~\ref{tab:mtpeaksf2}.  
65 > The two results for the scale factors are very compatible.
66 > We do not take any systematic uncertainty for this possible effect.
67 >
68 > \begin{table}[!h]
69 > \begin{center}
70 > {\footnotesize
71 > \begin{tabular}{l||c|c|c|c|c|c|c}
72 > \hline
73 > Sample              & SRA & SRB & SRC & SRD & SRE & SRF & SRG\\
74 > \hline
75 > \hline
76 > \multicolumn{8}{c}{$50 \leq \mt \leq 80$} \\
77 > \hline
78 > $\mu$ pre-veto \mt-SF      & $1.02 \pm 0.02$ & $0.95 \pm 0.03$ & $0.90 \pm 0.05$ & $0.98 \pm 0.08$ & $0.97 \pm 0.13$ & $0.85 \pm 0.18$ & $0.92 \pm 0.31$ \\
79 > $\mu$ post-veto \mt-SF     & $1.00 \pm 0.02$ & $0.95 \pm 0.03$ & $0.91 \pm 0.05$ & $1.00 \pm 0.09$ & $0.99 \pm 0.13$ & $0.85 \pm 0.18$ & $0.96 \pm 0.31$ \\
80 > \hline
81 > $\mu$ veto \mt-SF          & $0.98 \pm 0.01$ & $0.99 \pm 0.01$ & $1.01 \pm 0.02$ & $1.02 \pm 0.04$ & $1.02 \pm 0.06$ & $1.00 \pm 0.09$ & $1.04 \pm 0.11$ \\
82 > \hline
83 > \hline
84 > e pre-veto \mt-SF          & $0.95 \pm 0.02$ & $0.95 \pm 0.03$ & $0.94 \pm 0.06$ & $0.85 \pm 0.09$ & $0.84 \pm 0.13$ & $1.05 \pm 0.23$ & $1.04 \pm 0.33$ \\
85 > e post-veto \mt-SF         & $0.92 \pm 0.02$ & $0.91 \pm 0.03$ & $0.91 \pm 0.06$ & $0.74 \pm 0.08$ & $0.75 \pm 0.13$ & $0.91 \pm 0.22$ & $1.01 \pm 0.33$ \\
86 > \hline
87 > e veto \mt-SF      & $0.97 \pm 0.01$ & $0.96 \pm 0.02$ & $0.97 \pm 0.03$ & $0.87 \pm 0.05$ & $0.89 \pm 0.08$ & $0.86 \pm 0.11$ & $0.97 \pm 0.14$ \\
88 > \hline
89 > \hline
90 > \multicolumn{8}{c}{$40 \leq \mt \leq 100$} \\
91 > \hline
92 > $\mu$ pre-veto \mt-SF      & $1.02 \pm 0.01$ & $0.97 \pm 0.02$ & $0.91 \pm 0.05$ & $0.95 \pm 0.06$ & $0.97 \pm 0.10$ & $0.80 \pm 0.14$ & $0.74 \pm 0.22$ \\
93 > $\mu$ post-veto \mt-SF     & $1.00 \pm 0.01$ & $0.96 \pm 0.02$ & $0.90 \pm 0.04$ & $0.98 \pm 0.07$ & $1.00 \pm 0.11$ & $0.80 \pm 0.15$ & $0.81 \pm 0.24$ \\
94 > \hline
95 > $\mu$ veto \mt-SF          & $0.98 \pm 0.01$ & $0.99 \pm 0.01$ & $0.99 \pm 0.02$ & $1.03 \pm 0.03$ & $1.03 \pm 0.05$ & $1.01 \pm 0.08$ & $1.09 \pm 0.09$ \\
96 > \hline
97 > \hline
98 > e pre-veto \mt-SF          & $0.97 \pm 0.01$ & $0.93 \pm 0.02$ & $0.94 \pm 0.04$ & $0.81 \pm 0.06$ & $0.86 \pm 0.10$ & $0.95 \pm 0.17$ & $1.06 \pm 0.26$ \\
99 > e post-veto \mt-SF         & $0.94 \pm 0.01$ & $0.91 \pm 0.02$ & $0.91 \pm 0.04$ & $0.71 \pm 0.06$ & $0.82 \pm 0.10$ & $0.93 \pm 0.17$ & $1.09 \pm 0.27$ \\
100 > \hline
101 > e veto \mt-SF      & $0.97 \pm 0.01$ & $0.98 \pm 0.01$ & $0.97 \pm 0.02$ & $0.88 \pm 0.04$ & $0.95 \pm 0.06$ & $0.98 \pm 0.08$ & $1.03 \pm 0.09$ \\
102 > \hline
103 > \end{tabular}}
104 > \caption{ \mt\ peak Data/MC scale factors. The pre-veto SFs are applied to the
105 >  \ttdl\ sample, while the post-veto SFs are applied to the single
106 >  lepton samples. The veto SF is shown for comparison across channels.
107 >  The raw MC is used for backgrounds from rare processes.
108 >  The uncertainties are statistical only.
109 > \label{tab:mtpeaksf2}}
110 > \end{center}
111 > \end{table}
112 >
113 >
114 > \subsection{Uncertainty on the \wjets\ cross-section and the rare MC cross-sections}
115   These are taken as 50\%, uncorrelated.  
116   The primary effect is to introduce a 50\%
117   uncertainty
# Line 71 | Line 125 | attributed to $t\bar{t}$ goes down, and
125   scaled to the number of $t\bar{t}$ events in the peak, the $t\bar{t}$
126   BG goes down.  
127  
128 < \subsection{Scale factors for the tail-to-peak ratios for lepton +
128 > \subsection{Tail-to-peak ratios for lepton +
129    jets top and W events}
130 < These tail-to-peak ratios are described in Section~\ref{sec:ttp}.
131 < They are studied in CR1 and CR2.  The studies are described
132 < in Sections~\ref{sec:cr1} and~\ref{sec:cr2}), respectively, where
133 < we also give the uncertainty on the scale factors.
130 > The tail-to-peak ratios $R_{top}$ and $R_{wjet}$ are described in Section~\ref{sec:ttp}.
131 > The data/MC scale factors are studied in CR1 and CR2 (Sections~\ref{sec:cr1} and~\ref{sec:cr2}).
132 > Only the scale factor for \wjets, $SFR_{wjet}$, is used, and its uncertainty is given in Table~\ref{tab:cr1yields}). This uncertainty affects both $R_{wjet}$ and $R_{top}$.
133 > The additional systematic uncertainty on $R_{top}$ from the variation between optimistic and pessimistic scenarios is given in Section~\ref{sec:ttp}.
134 >
135  
136   \subsection{Uncertainty on extra jet radiation for dilepton
137    background}
# Line 84 | Line 139 | As discussed in Section~\ref{sec:jetmult
139   jet distribution in
140   $t\bar{t} \to$
141   dilepton MC is rescaled by the factors $K_3$ and $K_4$ to make
142 < it agree with the data.  The XX\% uncertainties on $K_3$ and $K_4$
142 > it agree with the data.  The 3\% uncertainties on $K_3$ and $K_4$
143   comes from data/MC statistics.  This  
144 < result directly in a XX\% uncertainty on the dilepton BG, which is by far
144 > results directly in a 3\% uncertainty on the dilepton background, which is by far
145   the most important one.
146  
147 + \subsection{Uncertainty from MC statistics}
148 + This affects mostly the \ttll\ background estimate, which is taken
149 + from
150 + Monte Carlo with appropriate correction factors.  This uncertainty
151 + is negligible in the low \met\ signal regions, and grows to about
152 + 15\% in SRG.
153 +
154  
155 < \subsection{Uncertainty on the \ttll\ Acceptance}
155 > \subsection{Uncertainty on the \ttll\ Background}
156  
157   The \ttbar\ background prediction is obtained from MC, with corrections
158   derived from control samples in data. The uncertainty associated with
159 + the \ttbar\ background is derived from the level of closure of the
160 + background prediction in CR4 (Table~\ref{tab:cr4yields}) and
161 + CR5 (Table~\ref{tab:cr5yields}). The results from these control region
162 + checks are shown in Figure~\ref{fig:ttdlunc}. The uncertainties assigned
163 + to the \ttdl\ background prediction based on these tests are
164 + 5\% (SRA), 10\% (SRB), 15\% (SRC), 25\% (SRD), 40\% (SRE-G).
165 +
166 + \begin{figure}[hbt]
167 +  \begin{center}
168 +        \includegraphics[width=0.6\linewidth]{plots/ttdilepton_uncertainty.pdf}
169 +        \caption{
170 +          \label{fig:ttdlunc}%\protect
171 +          Results of the comparison of yields in the \mt\ tail comparing the MC prediction (after
172 +          applying SFs) to data for CR4 and CR5 for all the signal
173 +          region requirements considered (A-G). The bands indicate the
174 +          systematic uncertainties assigned based on these tests,
175 +          ranging from $5\%$ for SRA to $40\%$ for SRE-G.}
176 +      \end{center}
177 + \end{figure}
178 +
179 +
180 + \subsubsection{Check of the uncertainty on the \ttll\ Acceptance}
181 +
182 + The uncertainty associated with
183   the theoretical modeling of the \ttbar\ production and decay is
184 < estimated by comparing the background predictions obtained using
184 > checked by comparing the background predictions obtained using
185   alternative MC samples. It should be noted that the full analysis is
186   performed with the alternative samples under consideration,
187   including the derivation of the various data-to-MC scale factors.
# Line 103 | Line 189 | The variations considered are
189  
190   \begin{itemize}
191   \item Top mass: The alternative values for the top mass differ
192 <  from the central value by $5~\GeV$: $m_{\mathrm{top}} = 178.5~\GeV$ and $m_{\mathrm{top}}
192 >  from the central value by $6~\GeV$: $m_{\mathrm{top}} = 178.5~\GeV$ and $m_{\mathrm{top}}
193    = 166.5~\GeV$.
194   \item Jet-parton matching scale: This corresponds to variations in the
195    scale at which the Matrix Element partons from Madgraph are matched
# Line 115 | Line 201 | The variations considered are
201    value for the scale used is $Q^2 = m_{\mathrm{top}}^2 +
202    \sum_{\mathrm{jets}} \pt^2$.
203   \item Alternative generators: Samples produced with different
204 <  generators include MC@NLO and Powheg (NLO generators) and
119 <  Pythia (LO). It may also be noted that MC@NLO uses Herwig6 for the
120 <  hadronisation, while POWHEG uses Pythia6.
204 >  generators, Powheg (our default) and Madgraph.
205   \item Modeling of taus: The alternative sample does not include
206    Tauola and is otherwise identical to the Powheg sample.
207    This effect was studied earlier using 7~TeV samples and found to be negligible.
208   \item The PDF uncertainty is estimated following the PDF4LHC
209 <  recommendations[CITE]. The events are reweighted using alternative
209 >  recommendations. The events are reweighted using alternative
210    PDF sets for CT10 and MSTW2008 and the uncertainties for each are derived using the
211 <  alternative eigenvector variations and the ``master equation''. In
212 <  addition, the NNPDF2.1 set with 100 replicas. The central value is
211 >  alternative eigenvector variations and the ``master equation''.
212 >  The NNPDF2.1 set with 100 replicas is also used. The central value is
213    determined from the mean and the uncertainty is derived from the
214    $1\sigma$ range. The overall uncertainty is derived from the envelope of the
215    alternative predictions and their uncertainties.
216    This effect was studied earlier using 7~TeV samples and found to be negligible.
217    \end{itemize}
218  
219 <
220 < \begin{table}[!h]
221 < \begin{center}
222 < {\footnotesize
223 < \begin{tabular}{l||c||c|c|c|c|c|c|c}
224 < \hline
225 < Sample              & Powheg & Madgraph & Mass Up & Mass Down & Scale Up & Scale Down &
226 < Match Up & Match Down \\
227 < \hline
228 < \hline
229 < SRA      & $579 \pm 10$ & $569 \pm 16$ & $591 \pm 18$ & $610 \pm 22$ & $651 \pm 22$ & $537 \pm 16$ & $578 \pm 18$ & $570 \pm 17$  \\
230 < \hline
231 < SRB      & $328 \pm 7$ & $307 \pm 11$ & $329 \pm 13$ & $348 \pm 15$ & $344 \pm 15$ & $287 \pm 10$ & $313 \pm 13$ & $307 \pm 12$  \\
232 < \hline
233 < SRC      & $111 \pm 4$ & $99 \pm 5$ & $107 \pm 7$ & $113 \pm 8$ & $124 \pm 8$ & $95 \pm 6$ & $93 \pm 6$ & $106 \pm 6$  \\
234 < \hline
235 < SRD      & $39 \pm 2$ & $35 \pm 3$ & $41 \pm 4$ & $41 \pm 5$ & $47 \pm 5$ & $33 \pm 3$ & $31 \pm 3$ & $39 \pm 4$  \\
236 < \hline
237 < SRE      & $14 \pm 1$ & $15 \pm 2$ & $17 \pm 3$ & $12 \pm 3$ & $15 \pm 3$ & $13 \pm 2$ & $12 \pm 2$ & $16 \pm 2$  \\
154 < \hline
155 < \end{tabular}}
156 < \caption{ \ttdl\ predictions for alternative MC samples. The uncertainties are statistical only.
157 < \label{tab:ttdlalt}}
158 < \end{center}
159 < \end{table}
219 > \begin{figure}[hbt]
220 >  \begin{center}
221 >        \includegraphics[width=0.5\linewidth]{plots/n_dl_comp_SRA.pdf}%
222 >        \includegraphics[width=0.5\linewidth]{plots/n_dl_comp_SRB.pdf}
223 >        \includegraphics[width=0.5\linewidth]{plots/n_dl_comp_SRC.pdf}%
224 >        \includegraphics[width=0.5\linewidth]{plots/n_dl_comp_SRD.pdf}
225 >        \includegraphics[width=0.5\linewidth]{plots/n_dl_comp_SRE.pdf}
226 >        \caption{
227 >          \label{fig:ttllsyst}\protect
228 >          Comparison of the \ttll\ central prediction with those using
229 >          alternative MC samples. The blue band corresponds to the
230 >          total statistical error for all data and MC samples. The
231 >          alternative sample predictions are indicated by the
232 >          datapoints. The uncertainties on the alternative predictions
233 >          correspond to the uncorrelated statistical uncertainty from
234 >          the size of the alternative sample only.  Note the
235 >          suppressed vertical scales.}
236 >      \end{center}
237 >    \end{figure}
238  
239  
240   \begin{table}[!h]
# Line 172 | Line 250 | SRA     & $2$ & $2$ & $5$ & $12$ & $7$ & $
250   \hline
251   SRB      & $6$ & $0$ & $6$ & $5$ & $12$ & $5$ & $6$  \\
252   \hline
253 < SRC      & $10$ & $3$ & $2$ & $12$ & $14$ & $16$ & $4$  \\
254 < \hline
255 < SRD      & $10$ & $6$ & $6$ & $21$ & $15$ & $19$ & $0$  \\
256 < \hline
257 < SRE      & $6$ & $17$ & $15$ & $2$ & $12$ & $17$ & $8$  \\
253 > % SRC    & $10$ & $3$ & $2$ & $12$ & $14$ & $16$ & $4$  \\
254 > % \hline
255 > % SRD    & $10$ & $6$ & $6$ & $21$ & $15$ & $19$ & $0$  \\
256 > % \hline
257 > % SRE    & $6$ & $17$ & $15$ & $2$ & $12$ & $17$ & $8$  \\
258   \hline
259   \end{tabular}}
260 < \caption{ Relative difference in \ttdl\ predictions for alternative MC samples.
260 > \caption{ Relative difference in \ttdl\ predictions for alternative MC
261 >  samples in
262 > the higher statistics regions SRA and SRB.  These differences
263 > are based on the central values of the predictions.  For a fuller
264 > picture
265 > of the situation, including statistical uncertainites, see Fig.~\ref{fig:ttllsyst}.
266   \label{tab:fracdiff}}
267   \end{center}
268   \end{table}
269  
270  
271 < \begin{table}[!h]
272 < \begin{center}
273 < {\footnotesize
191 < \begin{tabular}{l||c|c|c|c|c|c|c}
192 < \hline
193 < $N \sigma$     & Madgraph & Mass Up & Mass Down & Scale Up & Scale Down &
194 < Match Up & Match Down \\
195 < \hline
196 < \hline
197 < SRA      & $0.38$ & $0.42$ & $1.02$ & $2.34$ & $1.58$ & $0.01$ & $0.33$  \\
198 < \hline
199 < SRB      & $1.17$ & $0.07$ & $0.98$ & $0.76$ & $2.29$ & $0.78$ & $1.11$  \\
200 < \hline
201 < SRC      & $1.33$ & $0.37$ & $0.26$ & $1.24$ & $1.82$ & $1.97$ & $0.54$  \\
202 < \hline
203 < SRD      & $0.82$ & $0.46$ & $0.38$ & $1.32$ & $1.27$ & $1.47$ & $0.00$  \\
204 < \hline
205 < SRE      & $0.32$ & $0.75$ & $0.66$ & $0.07$ & $0.66$ & $0.83$ & $0.38$  \\
206 < \hline
207 < \end{tabular}}
208 < \caption{ N $\sigma$ difference in \ttdl\ predictions for alternative MC samples.
209 < \label{tab:nsig}}
210 < \end{center}
211 < \end{table}
271 > In Fig.~\ref{fig:ttllsyst} we compare the alternate MC \ttll\ background predictions
272 > for regions A through E.  We can make the following observations based
273 > on this Figure.
274  
275 + \begin{itemize}
276 + \item In the tighter signal regions we are running out of
277 +  statistics.    
278 + \item Within the limited statistics, there is no evidence that the
279 +  situation changes as we go from signal region A to signal region E.
280 + Therefore, we assess a systematic based on the relatively high
281 + statistics
282 + test in signal region A, and apply the same systematic uncertainty
283 + to all other regions.
284 + \item In order to fully (as opposed as 1$\sigma$) cover the
285 + alternative MC variations in region A we would have to take a
286 + systematic
287 + uncertainty of $\approx 10\%$.  This would be driven by the
288 + scale up/scale down variations, see Table~\ref{tab:fracdiff}.
289 + \end{itemize}
290  
291 < \begin{table}[!h]
291 > \begin{table}[!ht]
292   \begin{center}
293 < \begin{tabular}{l||c|c|c|c}
217 < \hline
218 < Av. $\Delta$ Evt.     & Alt. Gen. & $\Delta$ Mass & $\Delta$ Scale
219 < & $\Delta$ Match \\
220 < \hline
221 < \hline
222 < SRA      & $5.0$ ($1\%$) & $9.6$ ($2\%$) & $56.8$ ($10\%$) & $4.4$ ($1\%$)  \\
293 > \begin{tabular}{l|c|c}
294   \hline
295 < SRB      & $10.4$ ($3\%$) & $9.6$ ($3\%$) & $28.2$ ($9\%$) & $2.8$ ($1\%$)  \\
295 >            Sample
296 >            &                K3   & K4\\
297   \hline
226 SRC      & $5.7$ ($5\%$) & $3.1$ ($3\%$) & $14.5$ ($13\%$) & $6.4$ ($6\%$)  \\
298   \hline
299 < SRD      & $1.9$ ($5\%$) & $0.1$ ($0\%$) & $6.9$ ($18\%$) & $3.6$ ($9\%$)  \\
300 < \hline
301 < SRE      & $0.5$ ($3\%$) & $2.3$ ($16\%$) & $1.0$ ($7\%$) & $1.8$ ($12\%$)  \\
299 > Powheg     & $1.01 \pm 0.03$ & $0.93 \pm 0.04$ \\
300 > Madgraph  & $1.01 \pm 0.04$ & $0.92 \pm 0.04$ \\
301 > Mass Up    & $1.00 \pm 0.04$ & $0.92 \pm 0.04$ \\
302 > Mass Down    & $1.06 \pm 0.04$ & $0.99 \pm 0.05$ \\
303 > Scale Up    & $1.14 \pm 0.04$ & $1.23 \pm 0.06$ \\
304 > Scale Down    & $0.89 \pm 0.03$ & $0.74 \pm 0.03$ \\
305 > Match Up    & $1.02 \pm 0.04$ & $0.97 \pm 0.04$ \\
306 > Match Down    & $1.02 \pm 0.04$ & $0.91 \pm 0.04$ \\
307   \hline
308   \end{tabular}
309 < \caption{ Av. difference in \ttdl\ events for alternative sample pairs.
310 < \label{tab:devt}}
309 > \caption{$\met>100$ GeV: Data/MC scale factors used to account for differences in the
310 >  fraction of events with additional hard jets from radiation in
311 >  \ttll\ events. \label{tab:njetskfactors_met100}}
312   \end{center}
313   \end{table}
314  
315  
316 + However, we have two pieces of information indicating that the
317 + scale up/scale down variations are inconsistent with the data.
318 + These are described below.
319 +
320 + The first piece of information is that the jet multiplicity in the scale
321 + up/scale down sample is the most inconsistent with the data.  This is shown
322 + in Table~\ref{tab:njetskfactors_met100}, where we tabulate the
323 + $K_3$ and $K_4$ factors of Section~\ref{sec:jetmultiplicity} for
324 + different \ttbar\ MC samples.  The data/MC disagreement in the $N_{jets}$
325 + distribution
326 + for the scale up/scale down samples is also shown in Fig.~\ref{fig:dileptonnjets_scaleup}
327 + and~\ref{fig:dileptonnjets_scaledw}.  This should be compared with the
328 + equivalent $N_{jets}$ plots for the default Powheg MC, see
329 + Fig.~\ref{fig:dileptonnjets}, which agrees much better with data.
330 +
331   \begin{figure}[hbt]
332    \begin{center}
333 <        \includegraphics[width=0.5\linewidth]{plots/n_dl_comp_SRA.pdf}%
334 <        \includegraphics[width=0.5\linewidth]{plots/n_dl_comp_SRB.pdf}
335 <        \includegraphics[width=0.5\linewidth]{plots/n_dl_comp_SRC.pdf}%
336 <        \includegraphics[width=0.5\linewidth]{plots/n_dl_comp_SRD.pdf}
337 <        \includegraphics[width=0.5\linewidth]{plots/n_dl_comp_SRE.pdf}
338 <        \caption{
339 <          \label{fig:ttllsyst}\protect
248 <          Comparison of the \ttll\ central prediction with those using
249 <          alternative MC samples. The blue band corresponds to the
250 <          total statistical error for all data and MC samples. The
251 <          alternative sample predictions are indicated by the
252 <          datapoints. The uncertainties on the alternative predictions
253 <          correspond to the uncorrelated statistical uncertainty from
254 <          the size of the alternative sample only.
255 <        [TO BE UPDATED WITH THE LATEST SELECTION AND SFS]}
333 >        \includegraphics[width=0.5\linewidth]{plots/njets_all_met50_mueg_scaleup.pdf}
334 >        \includegraphics[width=0.5\linewidth]{plots/njets_all_met50_diel_scaleup.pdf}%
335 >        \includegraphics[width=0.5\linewidth]{plots/njets_all_met50_dimu_scaleup.pdf}
336 >        \caption{
337 >          \label{fig:dileptonnjets_scaleup}%\protect
338 >          SCALE UP: Comparison of the jet multiplicity distribution in data and MC for dilepton events in the \E-\M\
339 >          (top), \E-\E\ (bottom left) and \M-\M\ (bottom right) channels.}  
340        \end{center}
341 <    \end{figure}
341 > \end{figure}
342 >
343 > \begin{figure}[hbt]
344 >  \begin{center}
345 >        \includegraphics[width=0.5\linewidth]{plots/njets_all_met50_mueg_scaledw.pdf}
346 >        \includegraphics[width=0.5\linewidth]{plots/njets_all_met50_diel_scaledw.pdf}%
347 >        \includegraphics[width=0.5\linewidth]{plots/njets_all_met50_dimu_scaledw.pdf}
348 >        \caption{
349 >          \label{fig:dileptonnjets_scaledw}%\protect
350 >          SCALE DOWN: Comparison of the jet multiplicity distribution in data and MC for dilepton events in the \E-\M\
351 >          (top), \E-\E\ (bottom left) and \M-\M\ (bottom right) channels.}  
352 >      \end{center}
353 > \end{figure}
354 >
355 >
356 > \clearpage
357 >
358 > The second piece of information is that we have performed closure
359 > tests in CR5 using the alternative MC samples.  These are exactly
360 > the same tests as the one performed in Section~\ref{sec:CR5} on the
361 > Powheg sample.  As we argued previously, this is a very powerful
362 > test of the background calculation.
363 > The results of this test are summarized in Table~\ref{tab:hugecr5yields}.
364 > Concentrating on the relatively high statistics CR5A region, we see
365 > for all \ttbar\ MC samples except scale up/scale down we obtain
366 > closure within 1$\sigma$.  The scale up/scale down tests closes
367 > worse, only within 2$\sigma$.  This again is evidence that the
368 > scale up/scale down variations are in disagreement with the data.
369 >
370 > \input{hugeCR5Table.tex}
371 >
372 > Based on the two observations above, we argue that the MC
373 > scale up/scale down variations are too extreme.  We feel that
374 > a reasonable choice would be to take one-half of the scale up/scale
375 > down variations in our MC.  This factor of 1/2 would then bring
376 > the discrepancy in the closure test of
377 > Table~\ref{tab:hugecr5yields} for the scale up/scale down variations
378 > from about 2$\sigma$ to about 1$\sigma$.
379 >
380 > Then, going back to Table~\ref{tab:fracdiff}, and reducing the scale
381 > up/scale
382 > down variations by a factor 2, we can see that a systematic
383 > uncertainty
384 > of 6\% would fully cover all of the variations from different MC
385 > samples in SRA and SRB.
386 > The alternative MC models indicate that a 6\% systematic uncertainty to
387 > cover the range of reasonable variations.
388 > Note that this 6\% is also consistent with the level at which we are
389 > able to test the closure of the method with alternative samples in CR5 for the high statistics
390 > regions (Table~\ref{tab:hugecr5yields}).
391 > The range of reasonable variations obtained with the alternative
392 > samples are consistent with the uncertainties assigned for
393 > the \ttll\ background based on the closure of the background
394 > predictions and data in CR4 and CR5.
395 >
396 >
397 >
398 >
399 >
400 > %\begin{table}[!h]
401 > %\begin{center}
402 > %{\footnotesize
403 > %\begin{tabular}{l||c||c|c|c|c|c|c|c}
404 > %\hline
405 > %Sample              & Powheg & Madgraph & Mass Up & Mass Down & Scale
406 > %Up & Scale Down &
407 > %Match Up & Match Down \\
408 > %\hline
409 > %\hline
410 > %SRA     & $579 \pm 10$ & $569 \pm 16$ & $591 \pm 18$ & $610 \pm 22$ & $651 \pm 22$ & $537 \pm 16$ & $578 \pm 18$ & $570 \pm 17$  \\
411 > %\hline
412 > %SRB     & $328 \pm 7$ & $307 \pm 11$ & $329 \pm 13$ & $348 \pm 15$ & $344 \pm 15$ & $287 \pm 10$ & $313 \pm 13$ & $307 \pm 12$  \\
413 > %\hline
414 > %SRC     & $111 \pm 4$ & $99 \pm 5$ & $107 \pm 7$ & $113 \pm 8$ & $124 \pm 8$ & $95 \pm 6$ & $93 \pm 6$ & $106 \pm 6$  \\
415 > %\hline
416 > %SRD     & $39 \pm 2$ & $35 \pm 3$ & $41 \pm 4$ & $41 \pm 5$ & $47 \pm 5$ & $33 \pm 3$ & $31 \pm 3$ & $39 \pm 4$  \\
417 > %\hline
418 > %SRE     & $14 \pm 1$ & $15 \pm 2$ & $17 \pm 3$ & $12 \pm 3$ & $15 \pm 3$ & $13 \pm 2$ & $12 \pm 2$ & $16 \pm 2$  \\
419 > %\hline
420 > %\end{tabular}}
421 > %\caption{ \ttdl\ predictions for alternative MC samples. The uncertainties are statistical only.
422 > %\label{tab:ttdlalt}}
423 > %\end{center}
424 > %\end{table}
425 >
426 >
427 >
428 >
429 > %\begin{table}[!h]
430 > %\begin{center}
431 > %{\footnotesize
432 > %\begin{tabular}{l||c|c|c|c|c|c|c}
433 > %\hline
434 > %$N \sigma$     & Madgraph & Mass Up & Mass Down & Scale Up & Scale Down &
435 > %Match Up & Match Down \\
436 > %\hline
437 > %\hline
438 > %SRA     & $0.38$ & $0.42$ & $1.02$ & $2.34$ & $1.58$ & $0.01$ & $0.33$  \\
439 > %\hline
440 > %SRB     & $1.17$ & $0.07$ & $0.98$ & $0.76$ & $2.29$ & $0.78$ & $1.11$  \\
441 > %\hline
442 > %SRC     & $1.33$ & $0.37$ & $0.26$ & $1.24$ & $1.82$ & $1.97$ & $0.54$  \\
443 > %\hline
444 > %SRD     & $0.82$ & $0.46$ & $0.38$ & $1.32$ & $1.27$ & $1.47$ & $0.00$  \\
445 > %\hline
446 > %SRE     & $0.32$ & $0.75$ & $0.66$ & $0.07$ & $0.66$ & $0.83$ & $0.38$  \\
447 > %\hline
448 > %\end{tabular}}
449 > %\caption{ N $\sigma$ difference in \ttdl\ predictions for alternative MC samples.
450 > %\label{tab:nsig}}
451 > %\end{center}
452 > %\end{table}
453 >
454 >
455 > %\begin{table}[!h]
456 > %\begin{center}
457 > %\begin{tabular}{l||c|c|c|c}
458 > %\hline
459 > %Av. $\Delta$ Evt.     & Alt. Gen. & $\Delta$ Mass & $\Delta$ Scale
460 > %& $\Delta$ Match \\
461 > %\hline
462 > %\hline
463 > %SRA     & $5.0$ ($1\%$) & $9.6$ ($2\%$) & $56.8$ ($10\%$) & $4.4$ ($1\%$)  \\
464 > %\hline
465 > %SRB     & $10.4$ ($3\%$) & $9.6$ ($3\%$) & $28.2$ ($9\%$) & $2.8$ ($1\%$)  \\
466 > %\hline
467 > %SRC     & $5.7$ ($5\%$) & $3.1$ ($3\%$) & $14.5$ ($13\%$) & $6.4$ ($6\%$)  \\
468 > %\hline
469 > %SRD     & $1.9$ ($5\%$) & $0.1$ ($0\%$) & $6.9$ ($18\%$) & $3.6$ ($9\%$)  \\
470 > %\hline
471 > %SRE     & $0.5$ ($3\%$) & $2.3$ ($16\%$) & $1.0$ ($7\%$) & $1.8$ ($12\%$)  \\
472 > %\hline
473 > %\end{tabular}
474 > %\caption{ Av. difference in \ttdl\ events for alternative sample pairs.
475 > %\label{tab:devt}}
476 > %\end{center}
477 > %\end{table}
478 >
479 >
480  
481   \clearpage
482  
# Line 397 | Line 619 | This is the uncertainty associated with
619   veto performance is modeled by the Monte Carlo.  This uncertainty
620   only applies to the fraction of dilepton BG events that have
621   a second e/$\mu$ or a one prong $\tau \to h$, with
622 < $P_T > 10$ GeV in $|\eta| < 2.4$.  This fraction is 1/3 (THIS WAS THE
623 < 7 TEV NUMBER, CHECK).  The uncertainty for these events
624 < is XX\% and is obtained from Tag and Probe studies of Section~\ref{sec:trkveto}
625 <
404 < \subsubsection{Isolated Track Veto: Tag and Probe Studies}
405 < \label{sec:trkveto}
406 <
407 < [EVERYTHING IS 7TEV HERE, UPDATE WITH NEW RESULTS \\
408 < ADD TABLE WITH FRACTION OF EVENTS THAT HAVE A TRUE ISOLATED TRACK]
622 > $P_T > 10$ GeV in $|\eta| < 2.4$.  This fraction is about 1/3, see
623 > Table~\ref{tab:trueisotrk}.
624 > The uncertainty for these events
625 > is 6\% and is obtained from tag-and-probe studies, see Section~\ref{sec:trkveto}.
626  
627   \begin{table}[!h]
628   \begin{center}
# Line 426 | Line 643 | e Frac. \ttdl\ with true iso. trk.      & $
643   \end{center}
644   \end{table}
645  
646 + \subsubsection{Isolated Track Veto: Tag and Probe Studies}
647 + \label{sec:trkveto}
648 +
649  
650   In this section we compare the performance of the isolated track veto in data and MC using tag-and-probe studies
651   with samples of Z$\to$ee and Z$\to\mu\mu$. The purpose of these studies is to demonstrate that the efficiency
652   to satisfy the isolated track veto requirements is well-reproduced in the MC, since if this were not the case
653 < we would need to apply a data-to-MC scale factor in order to correctly predict the \ttll\ background. This study
653 > we would need to apply a data-to-MC scale factor in order to correctly
654 > predict the \ttll\ background.
655 >
656 > This study
657   addresses possible data vs. MC discrepancies for the {\bf efficiency} to identify (and reject) events with a
658   second {\bf genuine} lepton (e, $\mu$, or $\tau\to$1-prong). It does not address possible data vs. MC discrepancies
659   in the fake rate for rejecting events without a second genuine lepton; this is handled separately in the top normalization
660   procedure by scaling the \ttlj\ contribution to match the data in the \mt\ peak after applying the isolated track veto.
661 +
662   Furthermore, we test the data and MC
663   isolated track veto efficiencies for electrons and muons since we are using a Z tag-and-probe technique, but we do not
664   directly test the performance for hadronic tracks from $\tau$ decays. The performance for hadronic $\tau$ decay products
# Line 447 | Line 671 | decays are well-understood, we currently
671   Second, hadronic tracks may undergo nuclear interactions and hence their tracks may not be reconstructed.
672   As discussed above, independent studies show that the MC reproduces the hadronic tracking efficiency within 4\%,
673   leading to a total background uncertainty of less than 0.5\% (after taking into account the fraction of the total background
674 < due to hadronic $\tau$ decays with \pt\ $>$ 10 GeV tracks), and we hence regard this effect as neglgigible.
674 > due to hadronic $\tau$ decays with \pt\ $>$ 10 GeV tracks), and we hence regard this effect as negligible.
675  
676 < The tag-and-probe studies are performed in the full 2011 data sample, and compared with the DYJets madgraph sample.
676 > The tag-and-probe studies are performed in the full data sample, and compared with the DYJets madgraph sample.
677   All events must contain a tag-probe pair (details below) with opposite-sign and satisfying the Z mass requirement 76--106 GeV.
678   We compare the distributions of absolute track isolation for probe electrons/muons in data vs. MC. The contributions to
679   this isolation sum are from ambient energy in the event from underlying event, pile-up and jet activitiy, and hence do
# Line 504 | Line 728 | The specific criteria for tags and probe
728   The absolute track isolation distributions for passing probes are displayed in Fig.~\ref{fig:tnp}. In general we observe
729   good agreement between data and MC. To be more quantitative, we compare the data vs. MC efficiencies to satisfy
730   absolute track isolation requirements varying from $>$ 1 GeV to $>$ 5 GeV, as summarized in Table~\ref{tab:isotrk}.
731 < In the $\geq$0 and $\geq$1 jet bins where the efficiencies can be tested with statistical precision, the data and MC
731 > In the $\geq 0$ and $\geq 1$ jet bins where the efficiencies can be tested with statistical precision, the data and MC
732   efficiencies agree within 6\%, and we apply this as a systematic uncertainty on the isolated track veto efficiency.
733   For the higher jet multiplicity bins the statistical precision decreases, but we do not observe any evidence for
734   a data vs. MC discrepancy in the isolated track veto efficiency.
# Line 537 | Line 761 | for events with the \njets\ requirement
761  
762   \begin{table}[!ht]
763   \begin{center}
540 \caption{\label{tab:isotrk} Comparison of the data vs. MC efficiencies to satisfy the indicated requirements
541 on the absolute track isolation, and the ratio of these two efficiencies. Results are indicated separately for electrons and muons and for various
542 jet multiplicity requirements.}
764   \begin{tabular}{l|c|c|c|c|c}
765  
766   %Electrons:
# Line 637 | Line 858 | $\mu$ + $\geq$4 jets   &           $>$ 1
858   \hline
859  
860   \end{tabular}
861 + \caption{\label{tab:isotrk} Comparison of the data vs. MC efficiencies to satisfy the indicated requirements
862 + on the absolute track isolation, and the ratio of these two efficiencies. Results are indicated separately for electrons and muons and for various
863 + jet multiplicity requirements.}
864   \end{center}
865   \end{table}
866  
867 + \clearpage
868 + \subsection{Summary of uncertainties}
869 + \label{sec:bgunc-bottomline}.
870 + \input{uncertainties_table.tex}
871  
872   %Figure.~\ref{fig:reliso} compares the relative track isolation
873   %for events with a track with $\pt > 10~\GeV$ in addition to a selected
# Line 691 | Line 919 | $\mu$ + $\geq$4 jets   &           $>$ 1
919   %END SECTION TO WRITE OUT
920  
921  
922 < {\bf fix me: What you have written in the next paragraph does not explain how $\epsilon_{fake}$ is measured.
923 < Why not measure $\epsilon_{fake}$ in the b-veto region?}
922 > %{\bf fix me: What you have written in the next paragraph does not
923 > %explain how $\epsilon_{fake}$ is measured.
924 > %Why not measure $\epsilon_{fake}$ in the b-veto region?}
925  
926   %A measurement of the $\epsilon_{fake}$ in data is non-trivial. However, it is
927   %possible to correct for differences in the $\epsilon_{fake}$ between data and MC by
# Line 720 | Line 949 | Why not measure $\epsilon_{fake}$ in the
949   %      \end{center}
950   %\end{figure}
951  
723 \subsection{Summary of uncertainties}
724 \label{sec:bgunc-bottomline}.
952  
953 < THIS NEEDS TO BE WRITTEN
953 >
954 > % THIS NEEDS TO BE WRITTEN

Diff Legend

Removed lines
+ Added lines
< Changed lines
> Changed lines