Linear Fits

This chapter discusses one of the most used functions of EDA: fitting data to linear models. Calling the dependent variable y and the independent one x, a general representation of such a model can be given.

y = f[x] =

Here the a[k] are the parameters to be fit, and X[x,k] are called the "basis" functions.

By far the most common choice of basis functions are polynomials. Imagine we are trying to fit y versus x to a straight line.

We are trying to determine a[0] and a[1], and the two basis functions are 1 and x.

Imagine we are fitting the data to a second-order polynomial.

Now we are trying to fit to a[0], a[1], and a[2], and the basis functions are 1, x, and x^2.

As we shall see, the whole topic of fitting data to models is often surprisingly subtle.

The use of the word "linear" is sometimes confusing in the context of fitting. It means that the model being fit is linear in the parameters to which we are fitting (i.e. the a[i] in the notation just introduced).

There is no such constraint on the basis functions. For example we can fit y versus x data to trigonometric functions.

This is a linear fit, and the techniques discussed in this chapter may be used. The fact that the basis functions are not linear has no relevance in this context.

Imagine we are fitting to an exponential.

This is not a linear fit, since the parameter a[2] is nonlinear. Note that, in this example, the relationship can be made linear by transforming.

Writing aprime[1] = Log[a[1]] makes the relation a bit clearer.

Thus, fitting the logarithm of y versus x to a straight line effectively fits to the original equation and is linear. A small point about this sort of transformation is that it introduces biases into the parameters but often those biases can be ignored; this is discussed is Section 8.2.

Imagine we are fitting to a more complex exponential.

There is no simple transformation that will linearize this form. The techniques discussed in Chapter 5 on nonlinear techniques are required.