Model fitting by least squares

Unknown input: deconvolution with a known filter

For solving the unknown-input problem, we put the known filter $f_t$ in a matrix of downshifted columns $\bold F$. Our statement of wishes is now to find $x_t$ so that $\bold y \approx \bold F \bold x$. We can expect to have trouble finding unknown inputs $x_t$ when we are dealing with certain kinds of filters, such as bandpass filters. If the output is zero in a frequency band, we will never be able to find the input in that band and will need to prevent $x_t$ from diverging there. We do this by the statement that we wish $\bold 0\approx\epsilon \bold x$, where $\epsilon$ is a parameter that is small and whose exact size will be chosen by experimentation. Putting both wishes into a single, partitioned matrix equation gives

$$\left[ \begin{array}{c} \bold 0 \\ \bold 0 \end{array... ... \begin{array}{c} \bold y \\ \bold 0 \end{array} \right]$$ (40)

To minimize the residuals $\bold r_1$ and $\bold r_2$, we can minimize the scalar $\bold r' \bold r = \bold {r'}_1\bold r_1 + \bold {r'}_2\bold r_2$. This is

$$Q(\bold x', \bold x) = (\bold F \bold x - \bold y)' (\bold F\bold x-\bold y) + \epsilon^2 \bold x' \bold x$$

$$= (\bold x' \bold F' - \bold y') (\bold F\bold x-\bold y) + \epsilon^2 \bold x' \bold x$$ (41)

We solved this minimization in the frequency domain (beginning from equation (2.4)).

Formally the solution is found just as with equation (2.38), but this solution looks unappealing in practice because there are so many unknowns and because the problem can be solved much more quickly in the Fourier domain. To motivate ourselves to solve this problem in the time domain, we need either to find an approximate solution method that is much faster, or to discover that constraints or time-variable weighting functions are required in some applications. This is an issue we must be continuously alert to, whether the cost of a method is justified by its need.

EXERCISES:

1. In 1695, 150 years before Lord Kelvin's absolute temperature scale, 120 years before Sadi Carnot's PhD thesis, 40 years before Anders Celsius, and 20 years before Gabriel Farenheit, the French physicist Guillaume Amontons, deaf since birth, took a mercury manometer (pressure gauge) and sealed it inside a glass pipe (a constant volume of air). He heated it to the boiling point of water at $100^\circ$C. As he lowered the temperature to freezing at $0^\circ$ C, he observed the pressure dropped by 25% . He could not drop the temperature any further but he supposed that if he could drop it further by a factor of three, the pressure would drop to zero (the lowest possible pressure) and the temperature would have been the lowest possible temperature. Had he lived after Anders Celsius he might have calculated this temperature to be $-300^\circ$ C (Celsius). Absolute zero is now known to be $-273^\circ$ C.

   It is your job to be Amontons' lab assistant. Your $i$th measurement of temperature $T_i$ you make with Issac Newton's thermometer and you measure pressure $P_i$ and volume $V_i$ in the metric system. Amontons needs you to fit his data with the regression $0\approx \alpha(T_i-T_0)-P_i V_i$ and calculate the temperature shift $T_0$ that Newton should have made when he defined his temperature scale. Do not solve this problem! Instead, cast it in the form of equation (2.14), identifying the data $\bold d$ and the two column vectors $\bold f_1$ and $\bold f_2$ that are the fitting functions. Relate the model parameters $x_1$ and $x_2$ to the physical parameters $\alpha$ and $T_0$. Suppose you make ALL your measurements at room temperature, can you find $T_0$? Why or why not?

Model fitting by least squares