101 lines
5.1 KiB
TeX
101 lines
5.1 KiB
TeX
We describe the bisection procedure used to compute $\tau = \hat{F}_0^{-1}(1-\alpha;\mathcal{B}_0)$
|
|
in the KDE-ROLL forward pass (\Cref{sec:roll-kde-forward}).
|
|
|
|
\paragraph{Bracket initialisation.}
|
|
A fixed margin of $10/v_0 = 10h_0$ beyond the data range,
|
|
\[
|
|
\mathrm{lo} = \min(\mathbf{s}^{(0)}) - 10h_0,
|
|
\qquad
|
|
\mathrm{hi} = \max(\mathbf{s}^{(0)}) + 10h_0,
|
|
\]
|
|
only guarantees $\hat{F}_0(\mathrm{lo}) < 1-\alpha < \hat{F}_0(\mathrm{hi})$ for
|
|
$1-\alpha \in (\sigma(-10;1),\,\sigma(10;1)) \approx (4.5\times10^{-5},\,1-4.5\times10^{-5})$,
|
|
since that is the range the sigmoid tails have decayed into at distance $10h_0$. This
|
|
does not hold true for
|
|
every $\alpha \in (0,1)$ as a fixed margin might suggest.
|
|
|
|
Instead we use an exact, target-dependent bracket. For any $p \in (0,1)$, define
|
|
\[
|
|
\mathrm{lo}(p) = \min(\mathbf{s}^{(0)}) + h_0\log\!\frac{p}{1-p},
|
|
\qquad
|
|
\mathrm{hi}(p) = \max(\mathbf{s}^{(0)}) + h_0\log\!\frac{p}{1-p}.
|
|
\]
|
|
|
|
\begin{proposition}[Exact bracket containment]
|
|
\label{prop:bisection-bracket}
|
|
If $\min(\mathbf{s}^{(0)}) < \max(\mathbf{s}^{(0)})$, then for every $p \in (0,1)$,
|
|
\[
|
|
\hat{F}_0(\mathrm{lo}(p)) < p < \hat{F}_0(\mathrm{hi}(p)).
|
|
\]
|
|
\end{proposition}
|
|
|
|
\begin{proof}
|
|
$\sigma(u;v_0)$ is strictly increasing in $u$ (its derivative $v_0\,\sigma(1-\sigma) > 0$).
|
|
Direct substitution, using $v_0 h_0 = 1$, gives
|
|
$\sigma\bigl(\mathrm{lo}(p) - \min(\mathbf{s}^{(0)});\,v_0\bigr)
|
|
= \sigma\bigl(\log\tfrac{p}{1-p};\,1\bigr)
|
|
= \bigl(1+\tfrac{1-p}{p}\bigr)^{-1} = p$
|
|
exactly. For any other score $s_j > \min(\mathbf{s}^{(0)})$,
|
|
$\mathrm{lo}(p) - s_j < \mathrm{lo}(p) - \min(\mathbf{s}^{(0)})$, so monotonicity gives
|
|
$\sigma(\mathrm{lo}(p)-s_j;v_0) < p$. Averaging over $\mathcal{B}_0$, with at least one
|
|
score exceeding the minimum, $\hat{F}_0(\mathrm{lo}(p)) < p$ strictly. The symmetric
|
|
argument at $\max(\mathbf{s}^{(0)})$ gives $\hat{F}_0(\mathrm{hi}(p)) > p$.
|
|
\end{proof}
|
|
|
|
Setting $p = 1-\alpha$ gives a bracket valid for every $\alpha \in (0,1)$, unlike the
|
|
fixed-margin version above. The degenerate case $\min(\mathbf{s}^{(0)}) = \max(\mathbf{s}^{(0)})$.
|
|
This represents a batch with constant negative-class scores. I is excluded from
|
|
\Cref{prop:bisection-bracket} above; it also breaks the Silverman bandwidth estimate
|
|
itself ($h_0 = 0$), and is addressed as part of the degenerate-batch handling still
|
|
outstanding for \Cref{sec:kde-bandwidth}.
|
|
|
|
\paragraph{Bisection iterations.}
|
|
At each step we evaluate the midpoint,
|
|
\[
|
|
\tau_{\mathrm{mid}} = \tfrac{1}{2}(\mathrm{lo} + \mathrm{hi}),
|
|
\]
|
|
and update the bracket:
|
|
\[
|
|
\begin{cases}
|
|
\mathrm{lo} \leftarrow \tau_{\mathrm{mid}} & \text{if } \hat{F}_0(\tau_{\mathrm{mid}};\,\mathcal{B}_0) < 1-\alpha, \\
|
|
\mathrm{hi} \leftarrow \tau_{\mathrm{mid}} & \text{otherwise.}
|
|
\end{cases}
|
|
\]
|
|
\paragraph{Stopping criterion.}
|
|
A residual check on $\hat{F}_0$ alone, $\lvert\hat{F}_0(\tau_{\mathrm{mid}})-(1-\alpha)\rvert < \varepsilon$,
|
|
certifies that the \emph{threshold} $\tau$ has been located accurately, but not that the
|
|
\emph{loss} $\hat{F}_1(\tau)$ has: if $\hat{F}_0$ is comparatively flat near the target
|
|
(e.g.\ $v_0 \ll v_1$, or the negative-class kernel density is low at $\tau$), a wide range
|
|
of $\tau$ values can satisfy the residual bound while $\hat{F}_1(\tau)$ still varies
|
|
substantially across that range.
|
|
|
|
Instead, we exploit that $\hat{F}_1$ is itself a CDF (monotonically increasing) and
|
|
that the bracket invariant $\hat{F}_0(\mathrm{lo}) < 1-\alpha < \hat{F}_0(\mathrm{hi})$
|
|
keeps the true root $\tau^\star$ inside $[\mathrm{lo},\mathrm{hi}]$ throughout. Hence
|
|
\[
|
|
\hat{F}_1(\mathrm{lo}) \;\leq\; \hat{F}_1(\tau^\star) \;\leq\; \hat{F}_1(\mathrm{hi})
|
|
\]
|
|
regardless of exactly where $\tau^\star$ falls in the bracket. Iteration continues until
|
|
\begin{equation}
|
|
\hat{F}_1(\mathrm{hi};\,\mathcal{B}_1) - \hat{F}_1(\mathrm{lo};\,\mathcal{B}_1) < \varepsilon_L
|
|
\label{eq:bisection-loss-cert}
|
|
\end{equation}
|
|
(with $\varepsilon_L = 10^{-4}$), or until a maximum iteration count is reached. Unlike a
|
|
Lipschitz-based bound (e.g.\ using $\max_u \sigma'(u;v_1) = v_1/4$ to bound $\hat{F}_1$'s
|
|
slope and stopping purely on bracket width), \eqref{eq:bisection-loss-cert} is exact
|
|
rather than a worst-case estimate, at the cost of two extra $\hat{F}_1$ evaluations per
|
|
iteration.
|
|
|
|
\paragraph{Why the residual-only rule fails.} Take $\mathcal{B}_0 = \{-30,-25,5,45,60\}$
|
|
with $v_0 = 0.01$ (wide, smooth kernel) and $\alpha=\tfrac12$; the residual-only rule with
|
|
$\varepsilon=10^{-4}$ stops after 8 iterations at $\tau \approx 10.918$, versus the
|
|
converged root $\tau^\star \approx 10.932$, a gap of only $0.014$ in $\tau$. But with a
|
|
sharp positive-class kernel ($v_1 = 1000$) and a cluster of $\mathcal{B}_1$ sitting inside
|
|
that gap, $\hat{F}_1(\tau)$ differs by $0.40$ between the two: the reported loss is wrong
|
|
by nearly half its range, despite the threshold residual being $8\times$ tighter than
|
|
$\varepsilon$. Certifying via \eqref{eq:bisection-loss-cert} instead catches this
|
|
directly, converging to $\hat{F}_1$ accurate to $3\times 10^{-5}$.
|
|
|
|
The method converges unconditionally and requires no derivative evaluation, in contrast
|
|
to the Newton--Raphson alternative documented in Appendix~\ref{appendix:kde-nr}.
|