Least-square inversion with inexact adjoints. Method of conjugate directions: A tutorial

IN SEARCH OF THE DIRECTION

Let's suppose we have a generator that provides particular search directions at each step. The new direction can be the gradient of the objective function (as in the method of steepest descent), some other operator applied on the residual from the previous step, or, generally speaking, any arbitrary vector in the model space. Let us denote the automatically generated direction by ${\bf c}_n$. According to formula (8), the residual decreases as a result of choosing this direction by

$$\Vert{\bf r}_{n-1}\Vert^2 - \Vert{\bf r}_{n}\Vert^2 = {{\le... ..., {\bf A c}_n\right)^2} \over {\Vert{\bf A c}_n\Vert^2}}\;.$$ (9)

How can we improve on this result?