Model fitting by least squares

Differentiation by a complex vector

Complex numbers frequently arise in physical problems, particularly those with Fourier series. Let us extend the multivariable least-squares theory to the use of complex-valued unknowns $\bold x$. First recall how complex numbers were handled with single-variable least squares; i.e., as in the discussion leading up to equation (2.5). Use a prime, such as $\bold x'$, to denote the complex conjugate of the transposed vector $\bold x$. Now write the positive quadratic form as

$$Q(\bold x', \bold x) \eq (\bold F\bold x - \bold d)' (\bold ... ... \eq (\bold x' \bold F' - \bold d') (\bold F\bold x - \bold d)$$ (37)

After equation (2.4), we minimized a quadratic form $Q(\bar X,X)$ by setting to zero both $\partial Q/\partial \bar X$ and $\partial Q/\partial X$. We noted that only one of $\partial Q/\partial \bar X$ and $\partial Q/\partial X$ is necessarily zero because they are conjugates of each other. Now take the derivative of $Q$ with respect to the (possibly complex, row) vector $\bold x'$. Notice that $\partial Q/\partial \bold x'$ is the complex conjugate transpose of $\partial Q/\partial \bold x$. Thus, setting one to zero sets the other also to zero. Setting $\partial Q/\partial \bold x' =\bold 0$ gives the normal equations:

$$\bold 0 \eq {\partial Q \over \partial \bold x'} \eq \bold F' (\bold F\bold x - \bold d)$$ (38)

The result is merely the complex form of our earlier result (2.35). Therefore, differentiating by a complex vector is an abstract concept, but it gives the same set of equations as differentiating by each scalar component, and it saves much clutter.

Model fitting by least squares