THE TOTAL LEAST SQUARES PROBLEM: ANALYSIS, COMPUTATIONS AND APPLICATIONS

Abstract

Every linear parameter estimation problem gives rise to an overdetermined system of equations Am•n x = b. The total least squares (TLS) method is a generalization of the ordinary least squares (LS) method to solve the overdetermined system. In fact, the analysis ofLS is based on the assumption that the inconsistency of the system is due to errors only in the observation vector b while TLS assumes the possibility of the existence of errors in the data matrix A as well as in b . Therefore, TLS is appropriate whenever errors occur in all data and it amounts to fitting a best subspace to it. The problem was first studied by G. H. Golub and C.F. Van Loan in 1980, and perfected by S. Van Huffel and J. Vandewalle in 1991. A new version ofTLS specialized in data fitting was introduced by Y. Nievergelt in 1994 and totally analyzed by P. De Groen in 1996. It was shown that to a comparable amount of computations, mainly based on the singular value decomposition (SVD) of matrices, TLS could obtain more accurate results than LS. Solving the system in the TLS sense while keeping a special structure for the matrix of coefficients is known as structured total least squares (STLS) method. The objective of this thesis is to compare LS and TLS both analytically and geometrically and give a detailed analysis ofTLS (including the version specialized in data fitting) and STLS showing the importance of the SVD to estimate the solution. It also presents the TLS problem if additional constraints are imposed on the solution vector such as nonnegativity or inequality constraints . Besides, we illustrate the use of STLS in numerous applications. Numerical comparison is also given when applying LS, TLS, and STLS to solving practical examples in systems and control.