142 lines
7.4 KiB
TeX
142 lines
7.4 KiB
TeX
This appendix proves \Cref{prop:normalization-invariance} (\Cref{sec:score-normalization-proof}),
|
|
shows numerically that ISJ does not satisfy its scale-equivariance hypothesis
|
|
(\Cref{sec:score-normalization-isj}), and derives the chain-rule term that a frozen
|
|
normalisation constant $\hat\sigma_\mathcal{B}$ omits, together with a concrete instance
|
|
where omitting it is necessary to preserve \Cref{obs:gradient-sign-correctness}
|
|
(\Cref{sec:score-normalization-missing-term}).
|
|
|
|
\section{Proof of Proposition~\ref{prop:normalization-invariance}}
|
|
\label{sec:score-normalization-proof}
|
|
|
|
Write $s$ for the raw batch scores, $c := \hat\sigma_\mathcal{B}(s) > 0$, and
|
|
$\tilde{s} = s/c$. Let $v_k^A := 1/\hat{h}(s^{(k)})$ (raw path) and
|
|
$v_k^B := 1/\hat{h}(\tilde{s}^{(k)})$ (normalised path), and let $\tau^A, \tau^B$ solve
|
|
$\hat{F}_0(\tau; s^{(0)}, v_0^A) = 1-\alpha$ and
|
|
$\hat{F}_0(\tau; \tilde{s}^{(0)}, v_0^B) = 1-\alpha$ respectively.
|
|
|
|
By hypothesis $\hat{h}(\tilde{s}^{(k)}) = \hat{h}(s^{(k)}/c) = \hat{h}(s^{(k)})/c$, so
|
|
\begin{equation}
|
|
v_k^B = c \cdot v_k^A.
|
|
\label{eq:sn-v-scale}
|
|
\end{equation}
|
|
Substituting $\tau = \tau^A/c$ into $\hat{F}_0(\tau; \tilde{s}^{(0)}, v_0^B)$:
|
|
\[
|
|
v_0^B\bigl(\tau^A/c - \tilde{s}_j^{(0)}\bigr)
|
|
= c v_0^A\Bigl(\frac{\tau^A - s_j^{(0)}}{c}\Bigr)
|
|
= v_0^A\bigl(\tau^A - s_j^{(0)}\bigr)
|
|
\]
|
|
for every $j$, so $\hat{F}_0(\tau^A/c; \tilde{s}^{(0)}, v_0^B) = \hat{F}_0(\tau^A; s^{(0)}, v_0^A) = 1-\alpha$
|
|
i.e.\ $\tau^A/c$ solves the normalised-path equation, and by uniqueness of the
|
|
bisection root, $\tau^B = \tau^A/c$ exactly. The same substitution gives, for every $j$,
|
|
\begin{equation}
|
|
v_k^B\bigl(\tau^B - \tilde{s}_j^{(k)}\bigr) = v_k^A\bigl(\tau^A - s_j^{(k)}\bigr),
|
|
\label{eq:sn-argument-match}
|
|
\end{equation}
|
|
i.e.\ every sigmoid-kernel argument is identical between the two paths. Consequently
|
|
$\sigma(\cdot)$ evaluates identically in both paths (so the loss value matches:
|
|
$\hat{F}_1(\tau^B;\tilde{s}^{(1)},v_1^B) = \hat{F}_1(\tau^A;s^{(1)},v_1^A)$), while
|
|
$\sigma_k'(u;v) = v\,\sigma(u;v)(1-\sigma(u;v))$ picks up exactly a factor of
|
|
$v_k^B/v_k^A = c$ by \eqref{eq:sn-v-scale}:
|
|
\begin{equation}
|
|
\sigma_k'\bigl(\tau^B - \tilde{s}_j^{(k)}; v_k^B\bigr)
|
|
= c \cdot \sigma_k'\bigl(\tau^A - s_j^{(k)}; v_k^A\bigr).
|
|
\label{eq:sn-derivative-scale}
|
|
\end{equation}
|
|
|
|
\paragraph{Case $y_i=1$.} By \eqref{eq:kde-grad-y1} and the frozen-$c$ chain rule
|
|
$\partial/\partial s_i = (1/c)\,\partial/\partial\tilde{s}_i$:
|
|
\[
|
|
\frac{\partial L^B}{\partial s_i}
|
|
= \frac{1}{c}\left(-\frac{1}{|\mathcal{B}_1|}\sigma_1'\bigl(\tau^B-\tilde{s}_i;v_1^B\bigr)\right)
|
|
\overset{\eqref{eq:sn-derivative-scale}}{=}
|
|
\frac{1}{c}\left(-\frac{c}{|\mathcal{B}_1|}\sigma_1'\bigl(\tau^A-s_i;v_1^A\bigr)\right)
|
|
= \frac{\partial L^A}{\partial s_i}.
|
|
\]
|
|
|
|
\paragraph{Case $y_i=0$.} By \eqref{eq:kde-grad-combined}, the gradient is
|
|
$\frac{1}{c}\cdot D_1^B \cdot
|
|
\dfrac{\sigma_0'(\tau^B-\tilde{s}_i;v_0^B)}{\sum_j \sigma_0'(\tau^B-\tilde{s}_j;v_0^B)}$,
|
|
where $D_1^B := \frac{1}{|\mathcal{B}_1|}\sum_j\sigma_1'(\tau^B-\tilde{s}_j;v_1^B) = c\cdot D_1^A$
|
|
by \eqref{eq:sn-derivative-scale}. The ratio's numerator and denominator both pick up the
|
|
same factor $c$ (again by \eqref{eq:sn-derivative-scale}), so the ratio itself is
|
|
unchanged:
|
|
\[
|
|
\frac{\partial L^B}{\partial s_i}
|
|
= \frac{1}{c}\cdot c D_1^A \cdot
|
|
\frac{\sigma_0'(\tau^A-s_i;v_0^A)}{\sum_j\sigma_0'(\tau^A-s_j;v_0^A)}
|
|
= D_1^A \cdot \frac{\sigma_0'(\tau^A-s_i;v_0^A)}{\sum_j\sigma_0'(\tau^A-s_j;v_0^A)}
|
|
= \frac{\partial L^A}{\partial s_i}.
|
|
\]
|
|
|
|
In both cases the outer $1/c$ from the chain rule is exactly cancelled by the $c$
|
|
introduced into $v_k^B$, confirming \Cref{prop:normalization-invariance}. (Verified
|
|
numerically to float32 precision: $\max_i|\partial L^A/\partial s_i - \partial L^B/\partial s_i|
|
|
\approx 5.6\times10^{-9}$ on a random batch, using Silverman's rule for $\hat{h}$.)
|
|
|
|
\section{ISJ Does Not Satisfy the Hypothesis}
|
|
\label{sec:score-normalization-isj}
|
|
|
|
Silverman's rule satisfies $\hat{h}(a\cdot x)=a\cdot\hat{h}(x)$ exactly, since
|
|
$\hat{h}\propto\hat\sigma(x)$ and $\hat\sigma(ax)=a\hat\sigma(x)$. ISJ, computed via
|
|
\texttt{KDEpy}'s \texttt{improved\_sheather\_jones} (the primary estimator,
|
|
\Cref{sec:kde-bandwidth}), does not: rescaling a fixed sample by $a$ and re-running ISJ
|
|
gives $\hat{h}(ax)/a$ ratios that vary with $a$ rather than staying constant:
|
|
|
|
\begin{center}
|
|
\begin{tabular}{rrr}
|
|
\toprule
|
|
scale $a$ & $\hat{h}_{\text{Silverman}}(ax)/a$ & $\hat{h}_{\text{ISJ}}(ax)/a$ \\
|
|
\midrule
|
|
$10^{-6}$ & $0.353732$ & $0.001157$ \\
|
|
$10^{-3}$ & $0.353732$ & $0.001155$ \\
|
|
$10^{0}$ & $0.353732$ & $0.116810$ \\
|
|
$10^{3}$ & $0.353732$ & $0.217671$ \\
|
|
$10^{6}$ & $0.353732$ & $0.217671$ \\
|
|
\bottomrule
|
|
\end{tabular}
|
|
\end{center}
|
|
|
|
Consequently, for a batch where ISJ (rather than the Silverman fallback) is used,
|
|
neither the loss value nor the gradient is invariant to normalisation: a directly
|
|
measured example (random batch, $n{=}10$) gives loss $0.734$ on raw scores versus
|
|
$0.767$ on normalised scores, and a maximum per-sample gradient difference of $0.014$,
|
|
both far from the exact invariance \Cref{prop:normalization-invariance} guarantees
|
|
for Silverman. This is the genuine numerical-stability role of normalisation for the
|
|
method as actually used: not ``preventing overflow via an unchanged, merely-rescaled
|
|
argument'' (\Cref{prop:normalization-invariance} shows that mechanism is a no-op under
|
|
exact scale-equivariance), but pinning ISJ's input to the fixed scale regime (roughly
|
|
unit variance) it behaves consistently at, rather than leaving it to whatever scale the
|
|
model's raw outputs happen to have at a given point in training.
|
|
|
|
\section{The Missing Chain-Rule Term for a Differentiated $\hat\sigma_\mathcal{B}$}
|
|
\label{sec:score-normalization-missing-term}
|
|
|
|
\Cref{sec:roll-numerical-stability} treats $c=\hat\sigma_\mathcal{B}$ as frozen:
|
|
gradients with respect to normalised scores are multiplied by $1/c$ alone. Since $c$ is
|
|
itself a function of every score in the batch, $\tilde{s}_j = s_j/c(s_1,\ldots,s_n)$,
|
|
the fully differentiated chain rule is
|
|
\begin{equation}
|
|
\frac{\partial L}{\partial s_i}
|
|
= \sum_j \frac{\partial L}{\partial \tilde{s}_j}\frac{\partial \tilde{s}_j}{\partial s_i}
|
|
= \frac{1}{c}\frac{\partial L}{\partial \tilde{s}_i}
|
|
\;-\; \frac{1}{c^2}\frac{\partial c}{\partial s_i}\sum_j s_j\, q_j,
|
|
\qquad q_j := \frac{\partial L}{\partial \tilde{s}_j},
|
|
\label{eq:sn-missing-term}
|
|
\end{equation}
|
|
using $\partial \tilde{s}_j/\partial s_i = \delta_{ij}/c - (s_j/c^2)\,\partial c/\partial s_i$.
|
|
For the sample standard deviation used in \Cref{sec:roll-numerical-stability},
|
|
$\partial c/\partial s_i = (s_i-\bar{s})/((n-1)c)$. The frozen-$c$ convention sets
|
|
$\partial c/\partial s_i := 0$ by declaration, which is exactly why the implemented
|
|
gradient (\Cref{sec:roll-numerical-stability}) has no such term.
|
|
|
|
If instead $\hat\sigma_\mathcal{B}$ is differentiated through, the term in
|
|
\eqref{eq:sn-missing-term} does not generally vanish. There is no gaurantee
|
|
that this term does not overwhelm the first, flipping the sign and rendering the loss
|
|
useless. Numerically (random batch, $n{=}10$, one $y_i{=}1$ sample):
|
|
the frozen-$c$ gradient for that sample
|
|
is $-0.00181$, correctly satisfying \Cref{obs:gradient-sign-correctness} ($\leq 0$
|
|
required), while the fully-differentiated-$c$ gradient for the same sample is
|
|
$+0.00020$, violating it. Freezing $c$ is therefore adopted for the same reason as
|
|
freezing $v_0,v_1$ (Appendix~\ref{appendix:bandwidth-freeze}): the alternative is not
|
|
merely more complex, it can produce gradients of the wrong sign.
|