Total Least Squares (TLS)

In this contribution the fitting of a straight line to 3D point data is considered, with Cartesian coordinates xi , yi , zi as observations subject to random errors. A direct solution for the case of equally weighted and uncorrelated coordinate components was already presented almost forty years ago. For more general weighting cases, iterative algorithms, e.g., by means of an iteratively linearized Gauss-Helmert (GH) model, have been proposed in the literature. In this investigation, a new direct solution for the case of pointwise weights is derived. In the terminology of total least squares (TLS), this solution is a direct weighted total least squares (WTLS) approach. For the most general weighting case, considering a full dispersion matrix of the observations that can even be singular to some extent, a new iterative solution based on the ordinary iteration method is developed. The latter is a new iterative WTLS algorithm, since no linearization of the problem by Taylor series is performed at any step. Using a numerical example it is demonstrated how the newly developed WTLS approaches can be applied for 3D straight line fitting considering different weighting cases. The solutions are compared with results from the literature and with those obtained from an iteratively linearized GH model.

The traditional way of solving non-linear least squares (LS) problems in Geodesy includes a linearization of the functional model and iterative solution of a non-linear equation system. Direct solutions for a class of non-linear adjustment problems have been presented by the mathematical community since the 1980s, based on total least squares (TLS) algorithms and involving the use of singular value decomposition (SVD). However, direct LS solutions for this class of problems have been developed in the past also by geodesists. In this contribution we attempt to establish a systematic approach for direct solutions of non-linear LS problems from a "geodetic" point of view. Therefore, four non-linear adjustment problems are investigated: the fit of a straight line to given points in 2D and in 3D, the fit of a plane in 3D and the 2D symmetric similarity transformation of coordinates. For all these problems a direct LS solution is derived using the same methodology by transforming the problem to the solution of a quadratic or cubic algebraic equation. Furthermore, by applying TLS all these four problems can be transformed to solving the respective characteristic eigenvalue equations. It is demonstrated that the algebraic equations obtained in this way are identical with those resulting from the LS approach. As a by-product of this research two novel approaches are presented for the TLS solutions of fitting a straight line to 3D and the 2D similarity transformation of coordinates. The derived direct solutions of the four considered problems are illustrated on examples from the literature and also numerically compared to published iterative solutions.

This dissertation deals with a class of nonlinear adjustment problems that has a direct least squares solutionfor certain weighting cases.  In the literature of mathematical statistics these problems are expressed in anonlinear model called Errors-In-Variables (EIV) and their solution became popular as total least squares(TLS). The TLS solution is direct and involves the use of singular value decomposition (SVD), presented inmost cases for adjustment problems with equally weighted and uncorrelated measurements.  Additionally,several weighted total least squares (WTLS) algorithms have been published in the last years for derivingiterative solutions, when more general weighting cases have to be taken into account and without linearizingthe problem in any step of the solution process.This research provides  rstly a well de ned mathematical relationship between TLS and direct least squaressolutions.  As a by-product, a systematic approach for the direct solution of these adjustments is established,using a consistent and complete mathematical formalization.  By transforming the problem to the solutionof a quadratic or cubic algebraic equation, which is identical with those resulting from TLS, it will be shownthat TLS is an algorithmic approach already known to the geodetic community and not a new method.A second contribution of this work is the clear overview of weighted least squares solutions for the discussedclass of problems, i.e.  the WTLS solution in the terminology of the statistical community.  It will be shownthat for certain weighting cases a direct solution still exists, for which two new solution strategies will beproposed. Further, stochastic models with more general weight matrices are examined, including correlationsbetween the measurements or even singular cofactor matrices.  New algorithms are developed and presented,that provide iterative weighted least squares solutions without linearizing the original nonlinear problem.The aim of this work is the popularization of the TLS approach, by presenting a complete framework forobtaining a (weighted) least squares solution for the investigated class of nonlinear adjustment problems.The proposed approaches and the implemented algorithms can be employed for obtaining direct solutionsin engineering tasks for which efficiency is important, while iterative solutions can be derived for stochasticmodels with more general weights.

In the problem of least squares min || Ax-b || with A ∈ Rm × n, m ≥ n and b ∈ Rm,
It is often assumed that the matrix is accurate and the vector b is tainted by errors. It is
hypothesis is not always realistic because existen problems where the matrix is also subject to errors.
One way to deal with such problems is by method of Least Squares Total (TLS). In this paper we use the TLS method to solve the problem spectroscopy
magnetic resonance and present a comparison of results obtained with the method of Least Squares (LS).

A strategy for adaptive control and energetic optimization of aerobic fermentors was implemented, with both air flow and agitation speed as manipulated variables. This strategy is separable in its components: control, optimization, estimation. We optimized parameter’s estimation (from the usual KLa correlation) using sinusoidal excitation of air flow and agitation speed. We have implemented parameter’s estimation trough recursive least squares algorithm with forgetting factor. We carried separate essays on control, optimization and estimation algorithms. We carried our essays using an original computational simulation environment, with noise and delay generating facilities for data sampling and filtering.

Our results show the convergence and robustness of the estimation algorithm used, improved with use of both forgetting factor and KLa dead-band facilities. Control algorithm used in our work compares favorably with PID using the integrated area criteria for deviation between oxygen molarity and critical molarity (set point). Optimization algorithm clearly reduces energetic consumption, respecting critical molarity. Integration of control, optimization and adaptive algorithms was implemented, but future work is needed for stability. Methods were defined and implemented for stability improvement. We have implemented data acquisition and computer manipulation of air flow and agitation speed for actual fermentors.