71 lines
2.5 KiB
TeX
71 lines
2.5 KiB
TeX
We instantiate the KDE gradient~\eqref{eq:kde-grad-combined} for the logistic sigmoid
|
|
kernel used in our implementation.
|
|
|
|
\paragraph{Kernel definition.}
|
|
|
|
\begin{equation}
|
|
\sigma(x ; v) = \frac{1}{1 + \exp(-vx)},
|
|
\qquad
|
|
\sigma'(x ; v) = \frac{v\,\exp(-v|x|)}{\bigl(1 + \exp(-v|x|)\bigr)^{2}}
|
|
\label{eq:sigmoid-kernel}
|
|
\end{equation}
|
|
|
|
where $v > 0$ is the bandwidth parameter. The derivative uses $|x|$ rather than $x$:
|
|
since $\sigma'$ is even, both forms are mathematically identical, but $\exp(-v|x|) \to 0$
|
|
as $|x| \to \infty$ whereas $\exp(-vx) \to \infty$ for $x \to -\infty$, so the $|x|$
|
|
form avoids floating-point overflow when scores fall well below $\tau$.
|
|
|
|
\paragraph{Forward pass.}
|
|
|
|
Substituting \eqref{eq:sigmoid-kernel} into the KDE CDF:
|
|
%
|
|
\begin{equation}
|
|
\hat{F}_{k}(\tau ; \mathbf{s}^{(k)}) =
|
|
\frac{1}{|\mathcal{B}_k|}
|
|
\sum_{\mathbf{x}_j \in \mathcal{B}_k}
|
|
\frac{1}{1 + \exp\!\bigl(-v_k\bigl(\tau - f_\theta(\mathbf{x}_j)\bigr)\bigr)}
|
|
\label{eq:kde-sigmoid-cdf}
|
|
\end{equation}
|
|
%
|
|
The derivative of $\hat{F}_k$ with respect to $\tau$ — used in Newton--Raphson inversion
|
|
and in the gradient — is:
|
|
%
|
|
\begin{equation}
|
|
\frac{\partial \hat{F}_{k}(\tau)}{\partial \tau}
|
|
=
|
|
\frac{1}{|\mathcal{B}_k|}
|
|
\sum_{\mathbf{x}_j \in \mathcal{B}_k}
|
|
\frac{v_k\,\exp\!\bigl(-v_k\bigl|\tau - f_\theta(\mathbf{x}_j)\bigr|\bigr)}
|
|
{\Bigl(1 + \exp\!\bigl(-v_k\bigl|\tau - f_\theta(\mathbf{x}_j)\bigr|\bigr)\Bigr)^{2}}
|
|
\label{eq:kde-sigmoid-pdf}
|
|
\end{equation}
|
|
|
|
\paragraph{Gradient.}
|
|
|
|
Define the shorthand $\varphi_k(u) = \sigma'(u ; v_k)$:
|
|
\begin{equation}
|
|
\varphi_k(u) \;=\;
|
|
\frac{v_k\,\exp(-v_k|u|)}{\bigl(1+\exp(-v_k|u|)\bigr)^{2}}
|
|
\label{eq:sigmoid-phi}
|
|
\end{equation}
|
|
|
|
Substituting \eqref{eq:sigmoid-kernel} into \eqref{eq:kde-grad-combined}:
|
|
|
|
\begin{equation}
|
|
\frac{\partial \mathcal{L}_{\text{ROLL-TPR@FPR}}}{\partial f_\theta(\mathbf{x}_i)} =
|
|
\begin{cases}
|
|
-\dfrac{\varphi_1\!\left(\tau - f_\theta(\mathbf{x}_i)\right)}{|\mathcal{B}_1|}
|
|
& \text{if } y_i = 1\\[14pt]
|
|
+\,\dfrac{\displaystyle\sum_{\mathbf{x}_j \in \mathcal{B}_1}
|
|
\varphi_1\!\left(\tau - f_\theta(\mathbf{x}_j)\right)}{|\mathcal{B}_1|}
|
|
\;\cdot\;
|
|
\dfrac{\varphi_0\!\left(\tau-f_\theta(\mathbf{x}_i)\right)}
|
|
{\displaystyle\sum_{\mathbf{x}_j \in \mathcal{B}_0}
|
|
\varphi_0\!\left(\tau-f_\theta(\mathbf{x}_j)\right)}
|
|
& \text{if } y_i = 0
|
|
\end{cases}
|
|
\label{eq:kde-sigmoid-grad}
|
|
\end{equation}
|
|
|
|
where $\tau = \hat{F}_0^{-1}(1-\alpha ; \mathcal{B}_0)$.
|