Model fitting by least squares

Measuring success

In image estimation we have data $\bold d$, residual $\bold r$, operator $\bold F$, and final solution, the image $\bold m$. We minimize the norm of $\bold r(\bold m) = \bold F\bold m-\bold d$ by variation of $\bold m$. With least squares, minimizing the norm squared is equivalent to minimizing the norm. We measure the success of the model fitting the data by

$${\rm Fitting~success} = \ 1 - \vert\bold r\vert/\vert\bold d\vert$$ (84)

where the norm of any vector, say $\bold r$, is $\vert\bold r\vert=\sqrt{\bold r \cdot \bold r}$.

Since with image estimation applications the number of unknowns (dimension of $\bold m$) is usually hopelessly large, we can never iterate long enough to actually solve the normal equations $\bold F\T \bold r=\bold 0$ (which is the same as iterating until the gradient $\bold F\T\bold r$ vanishes) so we are obliged to report this fact. I advocate this:

$${\rm Computational~success} = \ 1 - \vert\bold F\T\bold r\vert/\vert\bold F\T\bold d\vert$$ (85)

There are three ways to think about this: First $\bold F\T\bold r$ is the gradient which should vanish. Second, it is the data fitting residual transformed into model space. Third, $\bold F\T\bold r$ is the final $\Delta \bold m$, while $\bold F\T\bold d$ is the first estimated model $\bold m_1$ (often called the adjoint model). Although it seems like the gradient should diminish monotonically in size (it certainly would in one dimension), I am not able to prove it here.

Model fitting by least squares