Model fitting by least squares

Physical nonlinearity

When standard methods of physics relate theoretical data $\bold d_{\rm theor}$ to model parameters $\bold m$, they often use a nonlinear relation, say $\bold d_{\rm theor} =\bold f(\bold m)$. The power-series approach then leads to representing theoretical data as

$$\bold d_{\rm theor} \eq \bold f(\bold m_0 + \Delta \bold m)... ...\approx\quad \bold f\bold (\bold m_0) + \bold F\Delta \bold m$$ (71)

where $\bold F$ is the matrix of partial derivatives of data values by model parameters, say $\partial d_i /\partial m_j$, evaluated at $\bold m_0$. The theoretical data $\bold d_{\rm theor}$ minus the observed data $\bold d_{\rm obs}$ is the residual we minimize.

$$\bold 0 \quad\approx\quad \bold d_{\rm theor} - \bold d_{\rm obs} = \bold F\bold \Delta\bold m +[\bold f(\bold m_0) - \bold d_{\rm obs}]$$ (72)

$$\bold r_{\rm new} = \bold F\bold \Delta\bold m + \bold r_{\rm old}$$ (73)

It is worth noticing that the residual updating (2.73) in a nonlinear problem is the same as that in a linear problem (2.47). If you make a large step $\Delta \bold m$, however, the new residual will be different from that expected by (2.73). Thus you should always re-evaluate the residual vector at the new location, and if you are reasonably cautious, you should be sure the residual norm has actually decreased before you accept a large step.

The pathway of inversion with physical nonlinearity is well developed in the academic literature and Bill Symes at Rice University has a particularly active group.

Model fitting by least squares