By default, LinearFit uses SingularValues on the "design matrix". This is not the way least-squares fits are commonly performed. Typically, they involve solving the "normal equations" of the system.
The problem with the common technique is that sometimes the data does not allow one to distinguish between two or more of the basis functions. When this occurs the normal equations become nearly singular and the solutions are unstable.
When the Method option is set to LUD, the common algorithm is used, involving LU decomposition of the normal equations.
For example, we can fit some thermocouple calibration data to a cubic polynomial using the default method. The numerical results of the fit are:
| A0 = -0.876 ±
0.037 A1 = -0.338 ± 0.033 A2 = -0.000097 ± 0.000077 A3 = -2.5 × 10-7 ± 5.1 × 10-7 Chi Squared = 0.769614 for 17 degrees of freedom (Chi-squared probability: 100.) |
A couple of points to note about the fit:
We repeat the fit using the LUD method:
Examining the progress of the fit gives:
21 data points, each with 3 variables. |
Iteration 1 using LUD routines. |
| LUDecomposition::luc: Result for LUDecomposition of badly conditioned matrix {{8400., 420000., 2.87 107 , 2.205 109 }, <<3>>} may contain significant numerical errors. |
Chi-squared = 0.769614 |
-7
Parameter values: {-0.875698, 0.0337949, 0.000096794, -2.46708 10 }
|
Calculating covariance matrix. |
| Inverse::luc: Result for Inverse of
badly conditioned matrix {{8400., 420000., 2.87 107 , 2.205 109 }, <<3>>} may contain significant numerical errors. |
Adjusting significant figures of parameters. |
In this case, despite the warnings from LUDecomposition, the result of the fit came out the same. This will not necessarily always occur.
Regarding the complaint from Inverse in this fit, see the discussion of the InverseMethod Option.
Often, as in the example shown, the reason for the complaint from LUDecomposition is that inherent in the least-squares algorithm is an assumption that the basis functions are orthogonal. If this is not true, then one or more of the pivot elements of the "normal equations" of the problem will be very small. More often than many experimenters would like to admit, the data does not allow one to justify the assumption that the basis functions are orthogonal.
Finally, if we are fitting data to a line with the slope forced to zero, the default SVD method does not work. In this case LinearFit automatically changes to LUD; although you may not notice that this has occured unless you examine the progress of the fit.
This help document is Copyright © 1999 David M. Harrison. The sample screens are Copyright © 1999 Solomon R.C. Douglas and David M. Harrison. This is version 1.1 of the help document, date (m/d/y) 11/25/99.