Model fitting by least squares

Sign convention

On the last day of the survey, a storm blew up, the sea got rough, and the receivers drifted further downwind. The data recorded that day had a larger than usual difference from that predicted by the final model. We could call $(\bold d-\bold F\bold m)$ the experimental error. (Here $\bold d$ is data, $\bold m$ is model parameters, and $\bold F$ is their linear relation).

The alternate view is that our theory was too simple. It lacked model parameters for the waves and the drifting cables. Because of this model oversimplification we had a modeling error of the opposite polarity $(\bold F\bold m-\bold d)$.

A strong experimentalist prefers to think of the error as experimental error, something for him or her to work out. Likewise a strong analyst likes to think of the error as a theoretical problem. (Weaker investigators might be inclined to take the opposite view.)

Regardless of the above, and opposite to common practice, I define the sign convention for the error (or residual) as $(\bold F\bold m-\bold d)$. When we choose this sign convention, our hazard for analysis errors will be reduced because $\bold F$ is often complicated and formed by combining many parts.

Beginners often feel disappointment when the data does not fit the model very well. They see it as a defect in the data instead of an opportunity to design a stronger theory.

Model fitting by least squares