next up previous [pdf]

Next: THE PRECONDITIONED SOLVER Up: PRECONDITIONING THE REGULARIZATION Previous: You better make your

Choice of a unitless epsilon

The parameter epsilon $ \epsilon$ strikes the balance between our data-fitting goal and our model-styling goal. These two regression systems typically have differing physical units; therefore, the numerical value of $ \epsilon$ is accidental, for example comparing milliseconds to meters.

\begin{displaymath}\begin{array}{rcl} \bold 0  &\approx& \bold r_d = \bold W (...
...\bold 0  &\approx& \bold r_m = \epsilon  \bold p \end{array}\end{displaymath} (12)

The numerical value of $ \epsilon$ is meaningless before we learn to express the idea in a unitless (dimensionless) manner. Without pretending we are doing physics, let us use some of the language of thermodynamics, a physical field that does deal with equilibria and random fluctuations. Define an energy ratio $ u$ and a volume ratio $ v$ that can be used to bring $ \epsilon$ to unitless form. Naturally, the square roots arise, because we are minimizing quadratic functions of residuals.

$\displaystyle u = {\rm energy ratio} \quad=\quad  \sqrt{
\frac{\bold r_d\cdot \bold r_d}
{\bold p\cdot\bold p}
}
$

$\displaystyle v = {\rm volume ratio} \quad=\quad  \sqrt{
\frac{n_{r_d}}{n_p}
}
$

Can we really think of ``volume'' as related to the number $ n_p$ of components in the model space? Perhaps. Likewise the data space? Less likely. And, is the energy measure really an appropriate one? Maybe. What is the goal of these speculative thoughts? The goal is to give you a starting numerical value for $ \epsilon$ , say $ \epsilon=1$ . Your final guide is your own experimental experience. Try either one of these next two regressions:


$\displaystyle \bold 0  \approx \
\bold r_m$ $\displaystyle =$ $\displaystyle \epsilon_{\rm extrinsic}  u  \bold p$ (13)
$\displaystyle \bold 0  \approx \
\bold r_m$ $\displaystyle =$ $\displaystyle \epsilon_{\rm intrinsic} (u/v)  \bold p$ (14)


next up previous [pdf]

Next: THE PRECONDITIONED SOLVER Up: PRECONDITIONING THE REGULARIZATION Previous: You better make your

2015-05-07