TancNote/note/tanc_nn_training.tex

The samples used to train the TaNC neural networks are typical of the signals
and backgrounds found in common physics analyses using taus.  The signal--type
training sample is composed of reconstucted tau--candidates that are matched to
generator level hadronic tau decays coming from simulated $Z \rightarrow
\tau^{+}\tau^{-}$ events.  The background training sample consists of reconstructed
tau--candidates in simulated QCD $2\rightarrow2$ hard scattering events.  The
QCD $P_T$ spectrum is steeply falling, and to obtain sufficient statistics
across a broad range of $P_T$ the sample is split into different $\hat P_{T}$
bins.  Each QCD sub-sample imposes a generator level cut on the transverse
energy of the hard interaction.  


both signal and background samples, 20\% of the events are reserved as a
statistically independent sample to evaluate the performance of the neural nets
after the training is completed.  The TaNC uses the ``MLP'' neural network
implementation provided by the TMVA software package, described in ~\cite{TMVA}.

The signal and background samples are split into five subsamples corresponding
to each reconstructed decay mode.  An additional selection is applied to each
subsample by requiring a ``leading pion'': either a charged hadron or gamma
candidate with transverse momentum greater than 5 GeV$/c$.  A large number of
QCD training events is required as the leading pion selection and the
requirement that the decay mode match one of the dominant modes given in table
~\ref{tab:decay_modes} are both effective discriminants.  For each subsample,
10000 signal and background tau--candidates are reserved to be used internally
by the TMVA software to test for over--training. The number of signal and
background entries used for each decay mode subsample is given in table
~\ref{tab:trainingEvents}.

%Chained 100 signal files.
%Chained 208 background files.
%Total signal entries: 874266
%Total background entries: 9526176
%Pruning non-relevant entries.
%After pruning, 584895 signal and 644315 background entries remain.
%**********************************************************************************
%*********************************** Summary **************************************
%**********************************************************************************
%*     NumEvents with weight > 0 (Total NumEvents)                                *
%*--------------------------------------------------------------------------------*
%*shrinkingConePFTauDecayModeProducer   ThreeProngNoPiZero: Signal:      53257(53271)            Background:155793(155841)
%*shrinkingConePFTauDecayModeProducer  ThreeProngOnePiZero: Signal:      13340(13342)            Background:135871(135942)
%*shrinkingConePFTauDecayModeProducer    OneProngTwoPiZero: Signal:      34780(34799)            Background:51181(51337)
%*shrinkingConePFTauDecayModeProducer    OneProngOnePiZero: Signal:      136464(138171)          Background:137739(139592)
%*shrinkingConePFTauDecayModeProducer     OneProngNoPiZero: Signal:      300951(345312)          Background:144204(161603)

\begin{table}
   \centering
   \begin{tabular}{lcc}
      %\multirow{2}{*}{}                         & \multicolumn{2}{c}{Events}    \\
                                                & Signal        & Background    \\
      \hline
      Total number of tau--candidates           & 874266        & 9526176       \\
      Tau--candidates passing preselection      & 584895        & 644315        \\
      Tau--candidates with $W(P_T,\eta)>0$      & 538792        & 488917        \\
      \hline
      Decay Mode                        & \multicolumn{2}{c}{Training Events}   \\
      \hline
      $\pi^{-}$                         & 300951   & 144204                     \\
      $\pi^{-}\pi^0$                    & 135464   & 137739                     \\
      $\pi^{-}\pi^0\pi^0$               & 34780    & 51181                      \\
      $\pi^{-}\pi^{-}\pi^{+}$           & 53247    & 155793                     \\
      $\pi^{-}\pi^{-}\pi^{+}\pi^0$      & 13340    & 135871                     \\
   \end{tabular}
   \label{tab:trainingEvents}
   \caption{Number of events used for neural network training for each
   selected decay mode.}
\end{table}


The neural nets uses as input variables the transverse momentum and $\eta$ of the
tau--candidates.  These variables are included as their correlations with other
observables can increase the separation power of the ensemble of observables.
For example, the opening angle in $\Delta R$ for signal tau--candidates is
inversely related to the transverse momentum, while for background events the
correlation is very small (see~\cite{DavisTau}). In the training signal and
background samples, there is significant discrimination power in the $P_T$
spectrum.   However, it is desirable to eliminate any systematic dependence of
the neural network output on $P_T$ and $\eta$, as in use the TaNC will be
presented with tau--candidates whose $P_T-\eta$ spectrum will be analysis
dependent. The dependence on $P_T$ and $\eta$ is removed by applying a $P_T$ and
$\eta$ dependent weight to the tau--candidates when training the neural nets.  

The weights are defined such that in any region in $P_T-\eta$ where the signal
and background probability density function are different, the sample with
higher probability density is weighted such that the samples have identical
$P_T-\eta$ probability distributions.  This removes regions of $P_T-\eta$ space
where the training sample is exclusively signal or background.  The weights are
computed by
\begin{align*}
   W(P_T, \eta) &=  {\rm less}(p_{sig}(P_T, \eta), p_{bkg}(P_T, \eta))\\
   w_{sig}(P_T, \eta) &=  W(P_T, \eta)/p_{sig}(P_T, \eta) \\
   w_{bkg}(P_T, \eta) &=  W(P_T, \eta)/p_{bkg}(P_T, \eta) 
\end{align*}
where $p_{sig}(P_T,\eta)$ and $p_{bkg}(P_T,\eta)$ are the probility densities of
the signal and background samples after the ``leading pion'' and decay mode
selections. Figure~\ref{fig:nnTrainingWeights} shows the signal and background
training $P_T$ distributions before and after the weighting is applied.


\begin{figure}[t]
\setlength{\unitlength}{1mm}
\begin{center}
\begin{picture}(150,60)(0,0)
\put(10.5, 2){
\mbox{\includegraphics*[height=58mm]{figures/training_weights_unweighted.pdf}}}
\put(86.0, 2){
\mbox{\includegraphics*[height=58mm]{figures/training_weights_weighted.pdf}}}
%\put(-5.5, 112.5){\small (a)}
%\put(72.0, 112.5){\small (b)}
%\put(-5.5, 54.5){\small (c)}
%\put(72.0, 54.5){\small (d)}
\end{picture}
\caption{\captiontext Transverse momentum spectrum of signal and background
tau--candidates used in neural net training before (left) and after (right) the
application of $P_T-\eta$ dependent weight function.  Application of the weights
lowers the training significance of tau--candidates in regions of $P_T-\eta$
phase space where either the signal or background samples has an excess of
events. }
\label{fig:nnTrainingWeights}
\end{center}
\end{figure} 

The 


Revision:	1.2
Committed:	Tue Apr 6 17:11:02 2010 UTC (15 years, 1 month ago) by friis
Content type:	application/x-tex
Branch:	MAIN
Changes since 1.1:	+83 -27 lines
Log Message:	* empty log message *
#	User	Rev	Content
1	friis	1.1	The samples used to train the TaNC neural networks are typical of the signals
2			and backgrounds found in common physics analyses using taus. The signal--type
3			training sample is composed of reconstucted tau--candidates that are matched to
4			generator level hadronic tau decays coming from simulated $Z \rightarrow
5			\tau^{+}\tau^{-}$ events. The background training sample consists of reconstructed
6	friis	1.2	tau--candidates in simulated QCD $2\rightarrow2$ hard scattering events. The
7			QCD $P_T$ spectrum is steeply falling, and to obtain sufficient statistics
8			across a broad range of $P_T$ the sample is split into different $\hat P_{T}$
9			bins. Each QCD sub-sample imposes a generator level cut on the transverse
10			energy of the hard interaction.
11
12
13	friis	1.1	both signal and background samples, 20\% of the events are reserved as a
14			statistically independent sample to evaluate the performance of the neural nets
15			after the training is completed. The TaNC uses the ``MLP'' neural network
16			implementation provided by the TMVA software package, described in ~\cite{TMVA}.
17
18			The signal and background samples are split into five subsamples corresponding
19			to each reconstructed decay mode. An additional selection is applied to each
20			subsample by requiring a ``leading pion'': either a charged hadron or gamma
21			candidate with transverse momentum greater than 5 GeV$/c$. A large number of
22			QCD training events is required as the leading pion selection and the
23			requirement that the decay mode match one of the dominant modes given in table
24			~\ref{tab:decay_modes} are both effective discriminants. For each subsample,
25			10000 signal and background tau--candidates are reserved to be used internally
26			by the TMVA software to test for over--training. The number of signal and
27			background entries used for each decay mode subsample is given in table
28			~\ref{tab:trainingEvents}.
29
30	friis	1.2	%Chained 100 signal files.
31			%Chained 208 background files.
32			%Total signal entries: 874266
33			%Total background entries: 9526176
34			%Pruning non-relevant entries.
35			%After pruning, 584895 signal and 644315 background entries remain.
36			%**********************************************************************************
37			%********************************* Summary ************************************
38			%**********************************************************************************
39			%* NumEvents with weight > 0 (Total NumEvents) *
40			%--------------------------------------------------------------------------------
41			%*shrinkingConePFTauDecayModeProducer ThreeProngNoPiZero: Signal: 53257(53271) Background:155793(155841)
42			%*shrinkingConePFTauDecayModeProducer ThreeProngOnePiZero: Signal: 13340(13342) Background:135871(135942)
43			%*shrinkingConePFTauDecayModeProducer OneProngTwoPiZero: Signal: 34780(34799) Background:51181(51337)
44			%*shrinkingConePFTauDecayModeProducer OneProngOnePiZero: Signal: 136464(138171) Background:137739(139592)
45			%*shrinkingConePFTauDecayModeProducer OneProngNoPiZero: Signal: 300951(345312) Background:144204(161603)
46
47	friis	1.1	\begin{table}
48			\centering
49	friis	1.2	\begin{tabular}{lcc}
50			%\multirow{2}{*}{} & \multicolumn{2}{c}{Events} \\
51			& Signal & Background \\
52	friis	1.1	\hline
53	friis	1.2	Total number of tau--candidates & 874266 & 9526176 \\
54			Tau--candidates passing preselection & 584895 & 644315 \\
55			Tau--candidates with $W(P_T,\eta)>0$ & 538792 & 488917 \\
56	friis	1.1	\hline
57	friis	1.2	Decay Mode & \multicolumn{2}{c}{Training Events} \\
58	friis	1.1	\hline
59	friis	1.2	$\pi^{-}$ & 300951 & 144204 \\
60			$\pi^{-}\pi^0$ & 135464 & 137739 \\
61			$\pi^{-}\pi^0\pi^0$ & 34780 & 51181 \\
62			$\pi^{-}\pi^{-}\pi^{+}$ & 53247 & 155793 \\
63			$\pi^{-}\pi^{-}\pi^{+}\pi^0$ & 13340 & 135871 \\
64	friis	1.1	\end{tabular}
65			\label{tab:trainingEvents}
66			\caption{Number of events used for neural network training for each
67			selected decay mode.}
68			\end{table}
69
70
71	friis	1.2	The neural nets uses as input variables the transverse momentum and $\eta$ of the
72	friis	1.1	tau--candidates. These variables are included as their correlations with other
73			observables can increase the separation power of the ensemble of observables.
74			For example, the opening angle in $\Delta R$ for signal tau--candidates is
75			inversely related to the transverse momentum, while for background events the
76			correlation is very small (see~\cite{DavisTau}). In the training signal and
77			background samples, there is significant discrimination power in the $P_T$
78			spectrum. However, it is desirable to eliminate any systematic dependence of
79	friis	1.2	the neural network output on $P_T$ and $\eta$, as in use the TaNC will be
80			presented with tau--candidates whose $P_T-\eta$ spectrum will be analysis
81			dependent. The dependence on $P_T$ and $\eta$ is removed by applying a $P_T$ and
82			$\eta$ dependent weight to the tau--candidates when training the neural nets.
83
84			The weights are defined such that in any region in $P_T-\eta$ where the signal
85			and background probability density function are different, the sample with
86			higher probability density is weighted such that the samples have identical
87			$P_T-\eta$ probability distributions. This removes regions of $P_T-\eta$ space
88			where the training sample is exclusively signal or background. The weights are
89			computed by
90			\begin{align*}
91			W(P_T, \eta) &= {\rm less}(p_{sig}(P_T, \eta), p_{bkg}(P_T, \eta))\\
92			w_{sig}(P_T, \eta) &= W(P_T, \eta)/p_{sig}(P_T, \eta) \\
93			w_{bkg}(P_T, \eta) &= W(P_T, \eta)/p_{bkg}(P_T, \eta)
94			\end{align*}
95			where $p_{sig}(P_T,\eta)$ and $p_{bkg}(P_T,\eta)$ are the probility densities of
96			the signal and background samples after the ``leading pion'' and decay mode
97			selections. Figure~\ref{fig:nnTrainingWeights} shows the signal and background
98			training $P_T$ distributions before and after the weighting is applied.
99
100
101			\begin{figure}[t]
102			\setlength{\unitlength}{1mm}
103			\begin{center}
104			\begin{picture}(150,60)(0,0)
105			\put(10.5, 2){
106			\mbox{\includegraphics*[height=58mm]{figures/training_weights_unweighted.pdf}}}
107			\put(86.0, 2){
108			\mbox{\includegraphics*[height=58mm]{figures/training_weights_weighted.pdf}}}
109			%\put(-5.5, 112.5){\small (a)}
110			%\put(72.0, 112.5){\small (b)}
111			%\put(-5.5, 54.5){\small (c)}
112			%\put(72.0, 54.5){\small (d)}
113			\end{picture}
114			\caption{\captiontext Transverse momentum spectrum of signal and background
115			tau--candidates used in neural net training before (left) and after (right) the
116			application of $P_T-\eta$ dependent weight function. Application of the weights
117			lowers the training significance of tau--candidates in regions of $P_T-\eta$
118			phase space where either the signal or background samples has an excess of
119			events. }
120			\label{fig:nnTrainingWeights}
121			\end{center}
122			\end{figure}
123
124			The
125	friis	1.1
126