thesis/content/appendixs/appendix_score_normalization.tex

This appendix proves \Cref{prop:normalization-invariance} (\Cref{sec:score-normalization-proof}),
shows numerically that ISJ does not satisfy its scale-equivariance hypothesis
(\Cref{sec:score-normalization-isj}), and derives the chain-rule term that a frozen
normalisation constant $\hat\sigma_\mathcal{B}$ omits, together with a concrete instance
where omitting it is necessary to preserve \Cref{obs:gradient-sign-correctness}
(\Cref{sec:score-normalization-missing-term}).

\section{Proof of Proposition~\ref{prop:normalization-invariance}}
\label{sec:score-normalization-proof}

Write $s$ for the raw batch scores, $c := \hat\sigma_\mathcal{B}(s) > 0$, and
$\tilde{s} = s/c$. Let $v_k^A := 1/\hat{h}(s^{(k)})$ (raw path) and
$v_k^B := 1/\hat{h}(\tilde{s}^{(k)})$ (normalised path), and let $\tau^A, \tau^B$ solve
$\hat{F}_0(\tau; s^{(0)}, v_0^A) = 1-\alpha$ and
$\hat{F}_0(\tau; \tilde{s}^{(0)}, v_0^B) = 1-\alpha$ respectively.

By hypothesis $\hat{h}(\tilde{s}^{(k)}) = \hat{h}(s^{(k)}/c) = \hat{h}(s^{(k)})/c$, so
\begin{equation}
  v_k^B = c \cdot v_k^A.
  \label{eq:sn-v-scale}
\end{equation}
Substituting $\tau = \tau^A/c$ into $\hat{F}_0(\tau; \tilde{s}^{(0)}, v_0^B)$:
\[
  v_0^B\bigl(\tau^A/c - \tilde{s}_j^{(0)}\bigr)
  = c v_0^A\Bigl(\frac{\tau^A - s_j^{(0)}}{c}\Bigr)
  = v_0^A\bigl(\tau^A - s_j^{(0)}\bigr)
\]
for every $j$, so $\hat{F}_0(\tau^A/c; \tilde{s}^{(0)}, v_0^B) = \hat{F}_0(\tau^A; s^{(0)}, v_0^A) = 1-\alpha$
— i.e.\ $\tau^A/c$ solves the normalised-path equation, and by uniqueness of the
bisection root, $\tau^B = \tau^A/c$ exactly. The same substitution gives, for every $j$,
\begin{equation}
  v_k^B\bigl(\tau^B - \tilde{s}_j^{(k)}\bigr) = v_k^A\bigl(\tau^A - s_j^{(k)}\bigr),
  \label{eq:sn-argument-match}
\end{equation}
i.e.\ every sigmoid-kernel argument is identical between the two paths. Consequently
$\sigma(\cdot)$ evaluates identically in both paths (so the loss value matches:
$\hat{F}_1(\tau^B;\tilde{s}^{(1)},v_1^B) = \hat{F}_1(\tau^A;s^{(1)},v_1^A)$), while
$\sigma_k'(u;v) = v\,\sigma(u;v)(1-\sigma(u;v))$ picks up exactly a factor of
$v_k^B/v_k^A = c$ by \eqref{eq:sn-v-scale}:
\begin{equation}
  \sigma_k'\bigl(\tau^B - \tilde{s}_j^{(k)}; v_k^B\bigr)
  = c \cdot \sigma_k'\bigl(\tau^A - s_j^{(k)}; v_k^A\bigr).
  \label{eq:sn-derivative-scale}
\end{equation}

\paragraph{Case $y_i=1$.} By \eqref{eq:kde-grad-y1} and the frozen-$c$ chain rule
$\partial/\partial s_i = (1/c)\,\partial/\partial\tilde{s}_i$:
\[
  \frac{\partial L^B}{\partial s_i}
  = \frac{1}{c}\left(-\frac{1}{|\mathcal{B}_1|}\sigma_1'\bigl(\tau^B-\tilde{s}_i;v_1^B\bigr)\right)
  \overset{\eqref{eq:sn-derivative-scale}}{=}
  \frac{1}{c}\left(-\frac{c}{|\mathcal{B}_1|}\sigma_1'\bigl(\tau^A-s_i;v_1^A\bigr)\right)
  = \frac{\partial L^A}{\partial s_i}.
\]

\paragraph{Case $y_i=0$.} By \eqref{eq:kde-grad-combined}, the gradient is
$\frac{1}{c}\cdot D_1^B \cdot
\dfrac{\sigma_0'(\tau^B-\tilde{s}_i;v_0^B)}{\sum_j \sigma_0'(\tau^B-\tilde{s}_j;v_0^B)}$,
where $D_1^B := \frac{1}{|\mathcal{B}_1|}\sum_j\sigma_1'(\tau^B-\tilde{s}_j;v_1^B) = c\cdot D_1^A$
by \eqref{eq:sn-derivative-scale}. The ratio's numerator and denominator both pick up the
same factor $c$ (again by \eqref{eq:sn-derivative-scale}), so the ratio itself is
unchanged:
\[
  \frac{\partial L^B}{\partial s_i}
  = \frac{1}{c}\cdot c D_1^A \cdot
    \frac{\sigma_0'(\tau^A-s_i;v_0^A)}{\sum_j\sigma_0'(\tau^A-s_j;v_0^A)}
  = D_1^A \cdot \frac{\sigma_0'(\tau^A-s_i;v_0^A)}{\sum_j\sigma_0'(\tau^A-s_j;v_0^A)}
  = \frac{\partial L^A}{\partial s_i}.
\]

In both cases the outer $1/c$ from the chain rule is exactly cancelled by the $c$
introduced into $v_k^B$, confirming \Cref{prop:normalization-invariance}. (Verified
numerically to float32 precision — $\max_i|\partial L^A/\partial s_i - \partial L^B/\partial s_i|
\approx 5.6\times10^{-9}$ on a random batch, using Silverman's rule for $\hat{h}$.)

\section{ISJ Does Not Satisfy the Hypothesis}
\label{sec:score-normalization-isj}

Silverman's rule satisfies $\hat{h}(a\cdot x)=a\cdot\hat{h}(x)$ exactly, since
$\hat{h}\propto\hat\sigma(x)$ and $\hat\sigma(ax)=a\hat\sigma(x)$. ISJ, computed via
\texttt{KDEpy}'s \texttt{improved\_sheather\_jones} (the primary estimator,
\Cref{sec:kde-bandwidth}), does not: rescaling a fixed sample by $a$ and re-running ISJ
gives $\hat{h}(ax)/a$ ratios that vary with $a$ rather than staying constant:

\begin{center}
\begin{tabular}{rrr}
\toprule
scale $a$ & $\hat{h}_{\text{Silverman}}(ax)/a$ & $\hat{h}_{\text{ISJ}}(ax)/a$ \\
\midrule
$10^{-6}$ & $0.353732$ & $0.001157$ \\
$10^{-3}$ & $0.353732$ & $0.001155$ \\
$10^{0}$  & $0.353732$ & $0.116810$ \\
$10^{3}$  & $0.353732$ & $0.217671$ \\
$10^{6}$  & $0.353732$ & $0.217671$ \\
\bottomrule
\end{tabular}
\end{center}

Consequently, for a batch where ISJ (rather than the Silverman fallback) is used,
neither the loss value nor the gradient is invariant to normalisation: a directly
measured example (random batch, $n{=}10$) gives loss $0.734$ on raw scores versus
$0.767$ on normalised scores, and a maximum per-sample gradient difference of $0.014$
— both far from the exact invariance \Cref{prop:normalization-invariance} guarantees
for Silverman. This is the genuine numerical-stability role of normalisation for the
method as actually used: not ``preventing overflow via an unchanged, merely-rescaled
argument'' (\Cref{prop:normalization-invariance} shows that mechanism is a no-op under
exact scale-equivariance), but pinning ISJ's input to the fixed scale regime — roughly
unit variance — it behaves consistently at, rather than leaving it to whatever scale the
model's raw outputs happen to have at a given point in training.

\section{The Missing Chain-Rule Term for a Differentiated $\hat\sigma_\mathcal{B}$}
\label{sec:score-normalization-missing-term}

\Cref{sec:roll-numerical-stability} treats $c=\hat\sigma_\mathcal{B}$ as frozen:
gradients with respect to normalised scores are multiplied by $1/c$ alone. Since $c$ is
itself a function of every score in the batch, $\tilde{s}_j = s_j/c(s_1,\ldots,s_n)$,
the fully differentiated chain rule is
\begin{equation}
  \frac{\partial L}{\partial s_i}
  = \sum_j \frac{\partial L}{\partial \tilde{s}_j}\frac{\partial \tilde{s}_j}{\partial s_i}
  = \frac{1}{c}\frac{\partial L}{\partial \tilde{s}_i}
    \;-\; \frac{1}{c^2}\frac{\partial c}{\partial s_i}\sum_j s_j\, q_j,
  \qquad q_j := \frac{\partial L}{\partial \tilde{s}_j},
  \label{eq:sn-missing-term}
\end{equation}
using $\partial \tilde{s}_j/\partial s_i = \delta_{ij}/c - (s_j/c^2)\,\partial c/\partial s_i$.
For the sample standard deviation used in \Cref{sec:roll-numerical-stability},
$\partial c/\partial s_i = (s_i-\bar{s})/((n-1)c)$. The frozen-$c$ convention sets
$\partial c/\partial s_i := 0$ by declaration, which is exactly why the implemented
gradient (\Cref{sec:roll-numerical-stability}) has no such term.

If instead $\hat\sigma_\mathcal{B}$ is differentiated through, the term in
\eqref{eq:sn-missing-term} does not generally vanish. Unlike the bandwidth case, there
is no exact-cancellation argument available here: the symmetry that would zero it out
via Euler's identity ($\sum_j \tilde{s}_j\, q_j = 0$, from $L$ being homogeneous of
degree $0$ in $\tilde{s}$) requires the bandwidth to be differentiated through the same
rescaling too — which \Cref{sec:roll-kde-backward} deliberately does not do. Numerically
(random batch, $n{=}10$, one $y_i{=}1$ sample): the frozen-$c$ gradient for that sample
is $-0.00181$ — correctly satisfying \Cref{obs:gradient-sign-correctness} ($\leq 0$
required) — while the fully-differentiated-$c$ gradient for the same sample is
$+0.00020$, violating it. Freezing $c$ is therefore adopted for the same reason as
freezing $v_0,v_1$ (Appendix~\ref{appendix:bandwidth-freeze}): the alternative is not
merely more complex, it can produce gradients of the wrong sign.