220 lines
11 KiB
TeX
220 lines
11 KiB
TeX
\Cref{sec:roll-kde-backward} derives \eqref{eq:kde-grad-combined} by treating the
|
|
bandwidths $v_0, v_1$ as constants during differentiation. This appendix makes explicit
|
|
what is given up by that choice: we derive the gradient that results from
|
|
differentiating through the bandwidths as well (\Cref{sec:bandwidth-freeze-general}),
|
|
then give two counterexamples (\Cref{sec:bandwidth-freeze-cex1,sec:bandwidth-freeze-cex2})
|
|
showing that the resulting correction is not a vanishing refinement of
|
|
\eqref{eq:kde-grad-combined} but can cancel it exactly or reverse its sign entirely.
|
|
|
|
\section{The Fully Adaptive Gradient}
|
|
\label{sec:bandwidth-freeze-general}
|
|
|
|
In order to derive the gradient w.r.t.\ the bandwidth parameter, we now write
|
|
$v_k = v_k(\mathbf{s}^{(k)})$ for the bandwidth parameter of class $k$, now a
|
|
differentiable function of that class's scores $\mathbf{s}^{(k)} =
|
|
\bigl(f_\theta(\mathbf{x}_j)\bigr)_{\mathbf{x}_j \in \mathcal{B}_k}$, as
|
|
\Cref{sec:kde-bandwidth} in fact computes it (via ISJ or the Silverman fallback) on every
|
|
forward pass. Differentiating the sigmoid kernel~\eqref{eq:sigmoid-kernel} with respect
|
|
to $v_k$ and applying the identity~\eqref{eq:sigmoid-identity} gives the
|
|
bandwidth-sensitivity term
|
|
|
|
\begin{equation}
|
|
\rho_k \;:=\; \frac{\partial \hat{F}_k(\tau)}{\partial v_k}
|
|
\;=\; \frac{1}{v_k\,|\mathcal{B}_k|}\sum_{\mathbf{x}_j \in \mathcal{B}_k}
|
|
\bigl(\tau - f_\theta(\mathbf{x}_j)\bigr)\,\sigma_k'\bigl(\tau - f_\theta(\mathbf{x}_j)\bigr).
|
|
\label{eq:bandwidth-sensitivity}
|
|
\end{equation}
|
|
|
|
\paragraph{Case $y_i = 1$.}
|
|
$\tau$ does not depend on $\mathcal{B}_1$, so the chain rule adds a single term to the
|
|
direct contribution already derived in~\eqref{eq:kde-grad-y1}:
|
|
\begin{equation}
|
|
\frac{dL}{df_\theta(\mathbf{x}_i)}\bigg|_{y_i=1}
|
|
= \underbrace{-\frac{1}{|\mathcal{B}_1|}\,\sigma_1'\bigl(\tau - f_\theta(\mathbf{x}_i)\bigr)}
|
|
_{\text{\eqref{eq:kde-grad-y1}}}
|
|
\;+\; \rho_1 \cdot \frac{dv_1}{df_\theta(\mathbf{x}_i)}.
|
|
\label{eq:bandwidth-freeze-y1}
|
|
\end{equation}
|
|
|
|
\paragraph{Case $y_i = 0$.}
|
|
Here $\tau$ is defined implicitly by $\hat{F}_0(\tau) = 1-\alpha$, and $v_0$ is itself a
|
|
function of $\mathbf{x}_i$'s class. Totally differentiating the defining equation and
|
|
comparing to~\eqref{eq:kde-grad-tau} shows the fixed-bandwidth result picks up the same
|
|
kind of correction:
|
|
\begin{equation}
|
|
\frac{d\tau}{df_\theta(\mathbf{x}_i)}
|
|
= \underbrace{\frac{\sigma_0'\bigl(\tau-f_\theta(\mathbf{x}_i)\bigr)}
|
|
{\sum_{\mathbf{x}_j \in \mathcal{B}_0}\sigma_0'\bigl(\tau-f_\theta(\mathbf{x}_j)\bigr)}}
|
|
_{\text{\eqref{eq:kde-grad-tau}}}
|
|
\;-\; \frac{|\mathcal{B}_0|\,\rho_0}
|
|
{\sum_{\mathbf{x}_j \in \mathcal{B}_0}\sigma_0'\bigl(\tau-f_\theta(\mathbf{x}_j)\bigr)}
|
|
\cdot \frac{dv_0}{df_\theta(\mathbf{x}_i)},
|
|
\label{eq:bandwidth-freeze-tau}
|
|
\end{equation}
|
|
and chaining through $\partial \hat{F}_1(\tau)/\partial\tau$ from~\eqref{eq:kde-dF1-dtau}:
|
|
\begin{equation}
|
|
\frac{dL}{df_\theta(\mathbf{x}_i)}\bigg|_{y_i=0}
|
|
= \underbrace{\eqref{eq:kde-grad-combined}\big|_{y_i=0}}_{\text{unchanged}}
|
|
\;-\; \left(\frac{1}{|\mathcal{B}_1|}\sum_{\mathbf{x}_j\in\mathcal{B}_1}
|
|
\sigma_1'\bigl(\tau-f_\theta(\mathbf{x}_j)\bigr)\right)
|
|
\frac{|\mathcal{B}_0|\,\rho_0}
|
|
{\sum_{\mathbf{x}_j \in \mathcal{B}_0}\sigma_0'\bigl(\tau-f_\theta(\mathbf{x}_j)\bigr)}
|
|
\cdot \frac{dv_0}{df_\theta(\mathbf{x}_i)}.
|
|
\label{eq:bandwidth-freeze-y0}
|
|
\end{equation}
|
|
|
|
In both cases, the fully adaptive gradient equals~\eqref{eq:kde-grad-combined} plus
|
|
a correction proportional to $dv_k/df_\theta(\mathbf{x}_i)$. For the Silverman fallback of
|
|
\Cref{sec:kde-bandwidth}, with class mean $\bar{f}_k$ and class variance
|
|
$\hat\sigma_k^2$,
|
|
\begin{equation}
|
|
\frac{dv_k}{df_\theta(\mathbf{x}_i)}
|
|
= -\frac{v_k\,\bigl(f_\theta(\mathbf{x}_i) - \bar{f}_k\bigr)}{|\mathcal{B}_k|\,\hat\sigma_k^2},
|
|
\qquad \mathbf{x}_i \in \mathcal{B}_k,
|
|
\label{eq:silverman-dv-df}
|
|
\end{equation}
|
|
which the two counterexamples below instantiate directly.
|
|
|
|
Notably, ISJ has no closed form, and so would require re-derivation through its numerical computation.
|
|
The Silverman instance already suffices to establish our claim.
|
|
|
|
\section{Counterexample 1: Exact Cancellation}
|
|
\label{sec:bandwidth-freeze-cex1}
|
|
|
|
Take $\mathcal{B}_0 = \{-1, 1\}$ with any bandwidth $v_0$, and
|
|
$\mathcal{B}_1 = \{0, b\}$ for a free scalar $b > 0$, with $\alpha = 1/2$.
|
|
|
|
By symmetry of $\mathcal{B}_0$ about $0$ and evenness of the sigmoid
|
|
kernel~\eqref{eq:sigmoid-kernel}, $\hat{F}_0(0) = \tfrac{1}{2}\bigl[\sigma(1;v_0) +
|
|
\sigma(-1;v_0)\bigr] = \tfrac{1}{2}$ for \emph{every} $v_0$, so $\tau = 0$ exactly,
|
|
independent of the bandwidth.
|
|
|
|
Silverman's rule on $\mathcal{B}_1 = \{0,b\}$ ($|\mathcal{B}_1|=2$,
|
|
$\hat\sigma_1 = b/2$) gives $h_1 = \kappa b$ with $\kappa := 1.06 \cdot 2^{-1/5}/2$, i.e.\
|
|
$v_1 = 1/(\kappa b)$: the bandwidth scales linearly with $b$.
|
|
|
|
\paragraph{True adaptive loss.} Substituting $v_1 = 1/(\kappa b)$,
|
|
\[
|
|
L(b) = \hat{F}_1(0) = \tfrac{1}{2}\bigl[\sigma(0;v_1) + \sigma(-b;v_1)\bigr]
|
|
= \tfrac{1}{2}\Bigl[\tfrac{1}{2} + \sigma(-1/\kappa)\Bigr]
|
|
= \tfrac{1}{4} + \tfrac{1}{2}\sigma(-1/\kappa),
|
|
\]
|
|
using $v_1 b = 1/\kappa$, a constant. The loss does not depend on $b$ at all:
|
|
\[
|
|
\frac{dL}{db} = 0 \quad\text{exactly, for every } b > 0.
|
|
\]
|
|
Rescaling $b$ rescales its own bandwidth identically, so the standardized KDE shape at
|
|
$\tau = 0$ never moves.
|
|
|
|
A loss that derives to a parameter gradient of zero is a serious failure mode for
|
|
gradient-based learning: it gives optimization nothing to act on, regardless of where
|
|
$b$ is initialized. In this example, once $b > 1$, this batch's negative-class scores
|
|
$\{-1,1\}$ are confined to $[-1,1]$, so scores above $1$ are a regime reserved solely
|
|
for class $1$, a more separated region of score space.
|
|
Under the fully adaptive gradient, $b$ would never move toward it: $dL/db = 0$
|
|
identically, so gradient descent stalls wherever $b$ starts, however far from that
|
|
regime.
|
|
|
|
\paragraph{What \eqref{eq:kde-grad-combined} gives instead.} Plugging directly into the
|
|
frozen-bandwidth term of~\eqref{eq:bandwidth-freeze-y1} (equivalently
|
|
\eqref{eq:kde-grad-y1}),
|
|
\[
|
|
\frac{dL}{db}\bigg|_{\text{\eqref{eq:kde-grad-combined}}}
|
|
= -\tfrac{1}{2}\,\sigma_1'(-b; v_1),
|
|
\]
|
|
which is strictly negative for every finite $b, v_1 > 0$, since $\sigma_1' > 0$
|
|
everywhere. \Cref{eq:kde-grad-combined} therefore reports a nonzero, consistently signed
|
|
gradient.
|
|
We see that using the frozen-bandwidth term, eventually, $b$ would be pushed
|
|
higher, approaching $b>1$, and achieving better separability.
|
|
|
|
\section{Counterexample 2: Persistent Sign Reversal}
|
|
\label{sec:bandwidth-freeze-cex2}
|
|
|
|
Take the same $\mathcal{B}_0 = \{-1,1\}$ (so again $\tau = 0$ for any $v_0$), and
|
|
$\mathcal{B}_1 = \{1+\varepsilon,\ b\}$ for a small fixed $\varepsilon > 0$ and a free
|
|
scalar $b > 1+\varepsilon$. We define $\delta := b - (1+\varepsilon) > 0$.
|
|
Silverman's rule again gives $h_1 = \kappa\delta$, $v_1 = 1/(\kappa\delta)$, with $\kappa$ as above.
|
|
|
|
\paragraph{What \eqref{eq:kde-grad-combined} gives.} Plugging into
|
|
\eqref{eq:kde-grad-y1} and simplifying with the sigmoid-identity~\eqref{eq:sigmoid-identity},
|
|
\begin{equation}
|
|
\frac{dL}{db}\bigg|_{\text{\eqref{eq:kde-grad-combined}}}
|
|
= -\tfrac{1}{2}\,\sigma_1'(-b;v_1)
|
|
= -\frac{1}{8\kappa\delta}\operatorname{sech}^2\!\left(\frac{b}{2\kappa\delta}\right)
|
|
\;<\; 0 \qquad \text{for every } \delta > 0.
|
|
\label{eq:cex2-frozen}
|
|
\end{equation}
|
|
This is the intuitively correct sign. $\mathbf{x}_i$ already scores above $\tau$, so
|
|
increasing $b$ further should only increase separability, therefore decreasing the loss.
|
|
|
|
\paragraph{The fully adaptive gradient.} Substituting the same configuration into
|
|
\eqref{eq:bandwidth-freeze-y1} requires $\rho_1$ and $dv_1/db$. Since
|
|
$v_1 = 1/(\kappa\delta)$ depends on $b$ only through $\delta = b-(1+\varepsilon)$,
|
|
\begin{equation}
|
|
\frac{dv_1}{db} = -\frac{1}{\kappa\delta^2} = -\frac{v_1}{\delta}.
|
|
\label{eq:cex2-dv1-db}
|
|
\end{equation}
|
|
Expanding \eqref{eq:bandwidth-sensitivity} for $\mathcal{B}_1 = \{1+\varepsilon, b\}$
|
|
and again applying the sigmoid-identity~\eqref{eq:sigmoid-identity},
|
|
\begin{equation}
|
|
\rho_1 = -\frac{1}{8}\left[(1+\varepsilon)\operatorname{sech}^2\!\left(\frac{(1+\varepsilon)v_1}{2}\right)
|
|
+ b\operatorname{sech}^2\!\left(\frac{bv_1}{2}\right)\right].
|
|
\label{eq:cex2-rho1}
|
|
\end{equation}
|
|
Substituting \eqref{eq:cex2-dv1-db} and \eqref{eq:cex2-rho1} into \eqref{eq:bandwidth-freeze-y1}
|
|
alongside \eqref{eq:cex2-frozen}, the $\operatorname{sech}^2(b/(2\kappa\delta))$ terms
|
|
combine and $b - \delta = 1+\varepsilon$ cancels the remainder, leaving:
|
|
\begin{equation}
|
|
\frac{dL}{db}\bigg|_{\text{full}}
|
|
= \frac{1+\varepsilon}{8\kappa\delta^2}\left[
|
|
\operatorname{sech}^2\!\left(\frac{b}{2\kappa\delta}\right)
|
|
+ \operatorname{sech}^2\!\left(\frac{1+\varepsilon}{2\kappa\delta}\right)
|
|
\right].
|
|
\label{eq:cex2-full}
|
|
\end{equation}
|
|
Since $\operatorname{sech}^2(x) > 0$ for every finite $x$ and $1+\varepsilon,\, \kappa,\, \delta^2 > 0$, \eqref{eq:cex2-full} is
|
|
strictly positive for every $\delta > 0$, the opposite sign to
|
|
\eqref{eq:cex2-frozen}, established here for the entire domain $b > 1+\varepsilon$.
|
|
|
|
\Cref{tab:cex2-values} confirms \eqref{eq:cex2-full} numerically (cross-checked against
|
|
an independent finite-difference derivative of $L(b)$) across an exponential range of
|
|
$\delta$, from just off the degenerate boundary $\delta \to 0^+$ to well-separated, for
|
|
$\varepsilon = 0.05$ ($\kappa \approx 0.4614$):
|
|
|
|
\begin{center}
|
|
\begin{tabular}{rrrr}
|
|
\toprule
|
|
$\delta$ & $b$ & \eqref{eq:kde-grad-combined} (frozen) & fully adaptive, \eqref{eq:cex2-full} \\
|
|
\midrule
|
|
$10^{-2}$ & 1.06 & $-1.82\times 10^{-98}$ & $+1.86\times 10^{-95}$ \\
|
|
$10^{-1}$ & 1.15 & $-1.62\times 10^{-10}$ & $+1.66\times 10^{-8}$ \\
|
|
$10^{0}$ & 2.05 & $-1.24\times 10^{-2}$ & $+1.09\times 10^{-1}$ \\
|
|
$10^{1}$ & 11.05 & $-8.30\times 10^{-3}$ & $+3.68\times 10^{-3}$ \\
|
|
$10^{2}$ & 101.05 & $-9.81\times 10^{-4}$ & $+3.87\times 10^{-5}$ \\
|
|
\bottomrule
|
|
\end{tabular}
|
|
\label{tab:cex2-values}
|
|
\end{center}
|
|
|
|
The fully adaptive gradient is positive at every finite
|
|
$\delta > 0$, never matching \eqref{eq:cex2-frozen}'s sign. Gradient
|
|
descent on the fully adaptive loss would push an already well-separated positive sample
|
|
back toward the decision threshold, throughout its entire domain, weakening separability
|
|
against what we are trying to achieve.
|
|
|
|
\section{Discussion}
|
|
\label{sec:bandwidth-freeze-discussion}
|
|
|
|
The two counterexamples isolate the same correction term,
|
|
$\rho_k \cdot dv_k/df_\theta(\mathbf{x}_i)$, in two regimes: in
|
|
\Cref{sec:bandwidth-freeze-cex1} it exactly cancels the gradient we need to continue learning,
|
|
and in \Cref{sec:bandwidth-freeze-cex2} it inverts it entirely. In neither case is it
|
|
simply a small correction to \eqref{eq:kde-grad-combined}. It is not bounded by, or shrinking
|
|
relative to, the term it is added to. With this term added, we could no longer guarantee
|
|
gradient sign correctness (\Cref{obs:gradient-sign-correctness}): \Cref{sec:bandwidth-freeze-cex2}
|
|
gives a sample with $y_i=1$ whose gradient is positive throughout its entire domain, in
|
|
direct violation of the guarantee. \Cref{sec:roll-kde-backward} therefore adopts
|
|
\eqref{eq:kde-grad-combined}, with $v_0, v_1$ frozen at their forward-pass values, as the
|
|
definition of the ROLL gradient.
|