Least-square inversion with inexact adjoints. Method of conjugate directions: A tutorial

Induction

Continuing by induction the process of adding a linear combination of the previous steps to the arbitrarily chosen direction ${\bf c}_n$ (known in mathematics as the Gram-Schmidt orthogonalization process), we finally arrive at the complete definition of the new step ${\bf s}_n$, as follows:

$${\bf s}_n = {\bf s}_n^{(1)} = {\bf c}_{n} + \sum_{j=1}^{j=n-1} \beta_n^{(j)} {\bf s}_{j}\;.$$ (22)

Here the coefficients $\beta_n^{(j)}$ are defined by equations

$$\beta_n^{(j)} = - {{\left({\bf A c}_n, {\bf A s}_{j}\right)} \over {\Vert{\bf A s}_{j}\Vert^2}}\;,$$ (23)

which correspond to the orthogonality principles

$$\left({\bf A s}_n, {\bf A s}_{j}\right) = 0\;,\;\;1 \leq j \leq n-1$$ (24)

and

$$\left({\bf r}_{n}, {\bf A s}_{j}\right) = 0\;,\;1 \leq j \leq n\;.$$ (25)

It is these orthogonality properties that allowed us to optimize the search parameters one at a time instead of solving the $n$-dimensional system of optimization equations for $\alpha_n$ and $\beta_n^{(j)}$.

Least-square inversion with inexact adjoints. Method of conjugate directions: A tutorial