Skip to main content

Trond Steihaug

University of Bergen, Department of Informatics, Faculty Member

Followers

15

Following

7

Co-authors

7

Public Views

Address: Snail-mail: Department of Informatics, University of Bergen, Postboks 7803, N-5020 Bergen
Physical address: HIB - Thormøhlensgt. 55

less

University of North Carolina at Chapel Hill

The University of Manchester

University of Edinburgh

Леонид Решетов

Bruce Christianson

University of Hertfordshire

Ohio University

Martin Plešinger

Technical University of Liberec

Daniel Kressner

Interests

Uploads

Papers by Trond Steihaug

Are Three Parameters Enough to Represent a Parallel Computer Architecture

by Yasemin Yalcinkaya and Trond Steihaug

Sixth Meeting of the Nordic Section of the Mathematical Programming Society. Proceedings. Erik Dotzauer, Mattias Björkman, Kenneth Holmström (Eds.) Opuscula, Mälardalen University, Västerås, Sweden. ISSN 1400 – 5468, 1999

The purpose of the paper is to build an analytical model to predict the performance of an iterati... more The purpose of the paper is to build an analytical model to predict the performance of an iterative linear least squares solver on a class of architectures using master-slave strategy. The model is used to analyze the behavior of the algorithm on varying number of processors, problem sizes and different architectures. Though it is based on only three architecture parameters, it gives fair predictions of execution time. Experimental results on SGI Origin 2000 and Cray T3E are included.

An Analytical Model for a Class of Architectures under Master-Slave Paradigm

by Yasemin Yalcinkaya and Trond Steihaug

We build an analytical model for an application utilizing master-slave paradigm. In the model, on... more We build an analytical model for an application utilizing master-slave paradigm. In the model, only three architecture parameters are used: latency, bandwidth and flop rate. Instead of using the vendor supplied or experimentally determined values, these parameters are estimated using the analytical model itself. Experimental results on Cray T3E and SGI Origin 2000 indicate that this simple model can give fair predictions.

Reducing the Deteriorating Effect of Old History in Asynchronous Iterations

by Yasemin Yalcinkaya and Trond Steihaug

Lecture Notes in Computer Science Volume 3149, 2004, pp 698-707, 2004

The deteriorating effect of old history in asynchronous iterations is investigated on an applicat... more The deteriorating effect of old history in asynchronous iterations is investigated on an application based on the specialization of parallel variable distribution approach of Ferris and Mangasarian [4] to linear least squares problem. A partially asynchronous algorithm is developed which employs a combination of synchronization, a relaxation parameter and a certain form of overlap between subproblems. It is shown by numerical experiments that this combined effort to decrease the effect of old history is more effective than the single attempts considered in [9,11].

Aggregating Variables for Asynchronous Iterations

by Yasemin Yalcinkaya and Trond Steihaug

The deteriorating effect of old history in asynchronous implementations of Jacobi-type iterative ... more The deteriorating effect of old history in asynchronous implementations of Jacobi-type iterative methods applied to linear least squares problem is well documented. A partially asynchronous algorithm is developed which employs a combination of synchronization, a relaxation parameter and an aggregation of variables. It is shown by numerical experiments that this combined effort to decrease the effect of old history is effective.

Asynchronous Methods and Least Squares: An Example of Deteriorating Convergence

by Yasemin Yalcinkaya and Trond Steihaug

Achim Sydow (ed.): "15th IMACS World Congress on Scientific Computation, Modeling and Applied Mathematics, Berlin, August 1997, Volume 1 Computational Mathematics", Wissenschaft & Technik Verlag, pp. 535-540, Aug 1997

We use a block iterative method for solving linear least squares problems. The subproblems are so... more We use a block iterative method for solving linear least squares problems. The subproblems are solved asynchronously on a distributed memory multiprocessor. It is observed that increased number of processors results in deteriorating convergence. We illustrate the deteriorating convergence by some numerical experiments. The deterioration of the convergence can be explained by contamination of the residual by old information. Our purpose is to reduce the effect of old information. The issues investigated here are the effect of the number of processors, the role of "essential neighbors" [7] and heterogeneous processors. We include two heuristics to identify the information to be discarded and reduce the effect of old information: a relaxation factor and synchronization. The characterization of old information remains as an open problem.

Deteriorating Convergence for Asynchronous Methods on Linear Least Squares Problems

by Yasemin Yalcinkaya and Trond Steihaug

Christian Lengauer, Martin Griebl, and Sergei Gorlatch (eds.): "Euro-Par'97 Parallel Processing", LNCS Vol. 1300, Springer Verlag, pp. 750-759, 1997

A block iterative method is used for solving linear least squares problems. The subproblems are s... more A block iterative method is used for solving linear least squares problems. The subproblems are solved asynchronously on a distributed memory multiprocessor. It is observed that an increased number of processors results in deteriorating rate of convergence. This deteriorating convergence is illustrated by numerical experiments. The deterioration of the convergence can be explained by contamination of the residual. Our purpose is to show that the residual is contaminated by old information. The issues investigated here are the effect of the number of processors, the role of essential neighbors, and synchronization. The characterization of old information remains an open problem.

On computing with general sparse third derivatives in unconstrained optimization

In this paper it is shown that when the sparsity structure of the problem is utilized higher orde... more In this paper it is shown that when the sparsity structure of the problem is utilized higher order methods are competitive to second order methods (Newton), for solving large unconstrained optimization problems when the objective function is three times continuously differentiable. It is also shown how to arrange the computations of general sparse third derivatives.

On the final steps of Newton and higher order methods

by Sara Suleiman and Trond Steihaug

Optimization Letters, 2015

An Analytical Model for a Class of Architectures under Master-Slave Paradigm

Lecture Notes in Computer Science, 2000

We build an analytical model for an application utilizing master-slave paradigm. In the model, on... more We build an analytical model for an application utilizing master-slave paradigm. In the model, only three architecture parameters are used: latency, bandwidth and flop rate. Instead of using the vendor supplied or experimentally determined values, these parameters are estimated using the analytical model itself. Experimental results on Cray T3E and SGI Origin 2000 indicate that this simple model can give fair predictions.

Pattern Graphs for Sparse Matrices1

The excellent research in the area of sparse derivative matrix determination over the last three ... more The excellent research in the area of sparse derivative matrix determination over the last three decades culminated into some highly innovative and efficient determination techniques. Graph theory has been employed as an important tool to study the complexity of the problems and to design "good" heuristics for solving them. Consequently, different graph models that have been proposed are rather dependent on specific properties e.g., symmetry, that the methods try to exploit. We introduce a new graph model that is a natural representation of the zero-nonzero structure of a sparse matrix . This new representation is also economical with respect to the storage space compared with the "Element Isolation (EI)" graph . Furthermore, the unidirectional (see ) and the bidirectional determination problems (see ) can now be stated using a single graph in a natural way and the connection between the two problems can be clearly stated as compared with the bipartite graph .

Graph coloring and the estimation of sparse Jacobian matrices with segmented columns

It is well known that a sparse Jacobian matrix can be estimated by fewer function evaluations tha... more It is well known that a sparse Jacobian matrix can be estimated by fewer function evaluations than the number of columns by using the CPR technique. An example shows that if the rows of the matrix are partitioned into two blocks then fewer function evaluations are needed. In this paper we show the relationship between estimating the Jacobian matrix by grouping together both rows and columns and the graph coloring problem. We give an easy implementation of the element isolation principle.

The CPR Method and Beyond : Prologue

In 1974 A.R. Curtis, M.J.D. Powell, and J.K.Reid published a seminal paper on the estimation of J... more In 1974 A.R. Curtis, M.J.D. Powell, and J.K.Reid published a seminal paper on the estimation of Jacobian matrices which was later coined as the CPR method. Central to the CPR method is the effective utilization of a priori known sparsity information. It is only recently that the optimal CPR method in its general form is characterized and the theoretical underpinning for the optimality is shown. In this short note we provide an overview of the development of computational techniques and software tools for the estimation of Jacobian matrices.

Deteriorating Convergence for Asynchronous

A block iterative method is used for solving linear least squares problems. The subproblems are s... more A block iterative method is used for solving linear least squares problems. The subproblems are solved asynchronously on a distributed memory multiprocessor. It is observed that an increased number of processors results in deteriorating rate of convergence. This deteriorating convergence is illustrated by numerical experiments. The deterioration of the convergence can be explained by contamination of the residual. Our purpose is to show that the residual is contaminated by old information. The issues investigated here are the effect of the number of processors, the role of essential neighbors, and synchronization. The characterization of old information remains an open problem.

Deteriorating convergence for asynchronous methods on linear least squares problems

Lecture Notes in Computer Science, 1997

... 154-156 (1991) 311-330 7. George, JA, Heath, MT: Solution of Sparse Linear Least Squares Prob... more

Are Three Parameters Enough to Represent a Parallel Computer Architecture?

ABSTRACT The purpose of the paper is to build an analytical model to predict the performance of a... more ABSTRACT The purpose of the paper is to build an analytical model to predict the performance of an iterative linear least squares solver on a class of architectures using master-slave strategy. The model is used to analyze the behavior of the algorithm on varying number of processors, problem sizes and different architectures. Though it is based on only three architecture parameters, it gives fair predictions of execution time. Experimental results on SGI Origin 2000 and Cray T3E are included. 1 Introduction In the existence of different algorithms to solve any given problem, a tool to estimate the comparative performances of these algorithms can make the final decision to choose one of them easier. Before coding the actual program a performance estimation tool can be used to predict the behavior of the different algorithms for various problem sizes, architectures and number of available processors. Performance estimation tools are based on &quot;analytical models&quot;. The chosen analytical model need...

Optimal direct determination of sparse Jacobian matrices

It is well known that a sparse Jacobian matrix can be determined with fewer function evaluations ... more It is well known that a sparse Jacobian matrix can be determined with fewer function evaluations using finite differencing or forward automatic differentiation (AD) passes than the number of independent variables of the underlying function. In this paper, we show that by grouping together rows into blocks and partitioning the resulting column segments, one can reduce this number further. We suggest a graph colouring technique for row partitioned Jacobian matrices to efficiently determine the nonzero entries using direct determination. We characterize optimal direct determination and derive results on the optimality of any direct determination technique based on column computation. Previously published computational results by the authors [S. Hossain and T. Steihaug, Graph colouring in the estimation of sparse derivative matrices: Instances and applications, Discrete Appl. Math. 156(2) (2008), pp. 280–288; S. Hossain, CsegGraph: A graph colouring instance generator, Int. J. Comput. Math. 86(10–11) (2009), pp. 1956–1967] have demonstrated that the row partitioned direct determination can yield considerable savings in function evaluations or AD passes over methods based on the Curtis, Powell, and Reid technique.

Graph Coloring and the Estimation of Sparce Jacobian Matrices Using Row and Column Partitioning

Asynchronous methods and least squares: An example of deteriorating convergence

ABSTRACT We use block iterative methods for solving linear least squares problems. The subproblem... more ABSTRACT We use block iterative methods for solving linear least squares problems. The subproblems are solved asynchronously on a distributed memory multiprocessor. It is observed that increased number of processors results in deteriorating convergence. We illustrate the deteriorating convergence by some numerical experiments. The deterioration of the convergence can be explained by contamination of the residual. The residual is contaminated by old information. Our purpose is to reduce the effect of old information. The issues investigated here are the effect of the number of processors, the role of essential neighbors [8] and heterogeneous processors. We include two heuristics to identify the information to be discarded and reduce the effect of old information: a relaxation factor and synchronization. The characterization of old information remains as an open problem. 1 Introduction In this paper we use block iterative method for solving sparse linear least squares problems. A general framewo...

A Ferris-Mangasarian technique applied to linear least squares problems

This note specializes to linear least squares problems an approach suggested by Ferris and Mangas... more This note specializes to linear least squares problems an approach suggested by Ferris and Mangasarian 4] for solving constrained optimization problems on parallel computers. It will be shown here that this specialization leads to an algorithm which is mathematically equivalent to an acceleration and convergence forcing modi cation of the block Jacobi iteration applied to the normal equations. The resulting algorithm is a promising way to speed up a parallel multisplitting algorithm of Renaut 9] for linear least squares. Renaut's algorithm is related to a specialization of part of the Ferris and Mangasarian approach.

Practical Application of Super-computing in Black-Oil Reservoir Simulation

North Sea Oil and Gas Reservoirs—II, 1990

Are Three Parameters Enough to Represent a Parallel Computer Architecture

by Yasemin Yalcinkaya and Trond Steihaug

Sixth Meeting of the Nordic Section of the Mathematical Programming Society. Proceedings. Erik Dotzauer, Mattias Björkman, Kenneth Holmström (Eds.) Opuscula, Mälardalen University, Västerås, Sweden. ISSN 1400 – 5468, 1999

The purpose of the paper is to build an analytical model to predict the performance of an iterati... more The purpose of the paper is to build an analytical model to predict the performance of an iterative linear least squares solver on a class of architectures using master-slave strategy. The model is used to analyze the behavior of the algorithm on varying number of processors, problem sizes and different architectures. Though it is based on only three architecture parameters, it gives fair predictions of execution time. Experimental results on SGI Origin 2000 and Cray T3E are included.

An Analytical Model for a Class of Architectures under Master-Slave Paradigm

by Yasemin Yalcinkaya and Trond Steihaug

We build an analytical model for an application utilizing master-slave paradigm. In the model, on... more We build an analytical model for an application utilizing master-slave paradigm. In the model, only three architecture parameters are used: latency, bandwidth and flop rate. Instead of using the vendor supplied or experimentally determined values, these parameters are estimated using the analytical model itself. Experimental results on Cray T3E and SGI Origin 2000 indicate that this simple model can give fair predictions.

Reducing the Deteriorating Effect of Old History in Asynchronous Iterations

by Yasemin Yalcinkaya and Trond Steihaug

Lecture Notes in Computer Science Volume 3149, 2004, pp 698-707, 2004

The deteriorating effect of old history in asynchronous iterations is investigated on an applicat... more The deteriorating effect of old history in asynchronous iterations is investigated on an application based on the specialization of parallel variable distribution approach of Ferris and Mangasarian [4] to linear least squares problem. A partially asynchronous algorithm is developed which employs a combination of synchronization, a relaxation parameter and a certain form of overlap between subproblems. It is shown by numerical experiments that this combined effort to decrease the effect of old history is more effective than the single attempts considered in [9,11].

Aggregating Variables for Asynchronous Iterations

by Yasemin Yalcinkaya and Trond Steihaug

The deteriorating effect of old history in asynchronous implementations of Jacobi-type iterative ... more The deteriorating effect of old history in asynchronous implementations of Jacobi-type iterative methods applied to linear least squares problem is well documented. A partially asynchronous algorithm is developed which employs a combination of synchronization, a relaxation parameter and an aggregation of variables. It is shown by numerical experiments that this combined effort to decrease the effect of old history is effective.

Asynchronous Methods and Least Squares: An Example of Deteriorating Convergence

by Yasemin Yalcinkaya and Trond Steihaug

Achim Sydow (ed.): "15th IMACS World Congress on Scientific Computation, Modeling and Applied Mathematics, Berlin, August 1997, Volume 1 Computational Mathematics", Wissenschaft & Technik Verlag, pp. 535-540, Aug 1997

We use a block iterative method for solving linear least squares problems. The subproblems are so... more We use a block iterative method for solving linear least squares problems. The subproblems are solved asynchronously on a distributed memory multiprocessor. It is observed that increased number of processors results in deteriorating convergence. We illustrate the deteriorating convergence by some numerical experiments. The deterioration of the convergence can be explained by contamination of the residual by old information. Our purpose is to reduce the effect of old information. The issues investigated here are the effect of the number of processors, the role of "essential neighbors" [7] and heterogeneous processors. We include two heuristics to identify the information to be discarded and reduce the effect of old information: a relaxation factor and synchronization. The characterization of old information remains as an open problem.

Deteriorating Convergence for Asynchronous Methods on Linear Least Squares Problems

by Yasemin Yalcinkaya and Trond Steihaug

Christian Lengauer, Martin Griebl, and Sergei Gorlatch (eds.): "Euro-Par'97 Parallel Processing", LNCS Vol. 1300, Springer Verlag, pp. 750-759, 1997

A block iterative method is used for solving linear least squares problems. The subproblems are s... more A block iterative method is used for solving linear least squares problems. The subproblems are solved asynchronously on a distributed memory multiprocessor. It is observed that an increased number of processors results in deteriorating rate of convergence. This deteriorating convergence is illustrated by numerical experiments. The deterioration of the convergence can be explained by contamination of the residual. Our purpose is to show that the residual is contaminated by old information. The issues investigated here are the effect of the number of processors, the role of essential neighbors, and synchronization. The characterization of old information remains an open problem.

On computing with general sparse third derivatives in unconstrained optimization

In this paper it is shown that when the sparsity structure of the problem is utilized higher orde... more In this paper it is shown that when the sparsity structure of the problem is utilized higher order methods are competitive to second order methods (Newton), for solving large unconstrained optimization problems when the objective function is three times continuously differentiable. It is also shown how to arrange the computations of general sparse third derivatives.

On the final steps of Newton and higher order methods

by Sara Suleiman and Trond Steihaug

Optimization Letters, 2015

An Analytical Model for a Class of Architectures under Master-Slave Paradigm

Lecture Notes in Computer Science, 2000

We build an analytical model for an application utilizing master-slave paradigm. In the model, on... more We build an analytical model for an application utilizing master-slave paradigm. In the model, only three architecture parameters are used: latency, bandwidth and flop rate. Instead of using the vendor supplied or experimentally determined values, these parameters are estimated using the analytical model itself. Experimental results on Cray T3E and SGI Origin 2000 indicate that this simple model can give fair predictions.

Pattern Graphs for Sparse Matrices1

The excellent research in the area of sparse derivative matrix determination over the last three ... more The excellent research in the area of sparse derivative matrix determination over the last three decades culminated into some highly innovative and efficient determination techniques. Graph theory has been employed as an important tool to study the complexity of the problems and to design "good" heuristics for solving them. Consequently, different graph models that have been proposed are rather dependent on specific properties e.g., symmetry, that the methods try to exploit. We introduce a new graph model that is a natural representation of the zero-nonzero structure of a sparse matrix . This new representation is also economical with respect to the storage space compared with the "Element Isolation (EI)" graph . Furthermore, the unidirectional (see ) and the bidirectional determination problems (see ) can now be stated using a single graph in a natural way and the connection between the two problems can be clearly stated as compared with the bipartite graph .

Graph coloring and the estimation of sparse Jacobian matrices with segmented columns

It is well known that a sparse Jacobian matrix can be estimated by fewer function evaluations tha... more It is well known that a sparse Jacobian matrix can be estimated by fewer function evaluations than the number of columns by using the CPR technique. An example shows that if the rows of the matrix are partitioned into two blocks then fewer function evaluations are needed. In this paper we show the relationship between estimating the Jacobian matrix by grouping together both rows and columns and the graph coloring problem. We give an easy implementation of the element isolation principle.

The CPR Method and Beyond : Prologue

In 1974 A.R. Curtis, M.J.D. Powell, and J.K.Reid published a seminal paper on the estimation of J... more In 1974 A.R. Curtis, M.J.D. Powell, and J.K.Reid published a seminal paper on the estimation of Jacobian matrices which was later coined as the CPR method. Central to the CPR method is the effective utilization of a priori known sparsity information. It is only recently that the optimal CPR method in its general form is characterized and the theoretical underpinning for the optimality is shown. In this short note we provide an overview of the development of computational techniques and software tools for the estimation of Jacobian matrices.

Deteriorating Convergence for Asynchronous

A block iterative method is used for solving linear least squares problems. The subproblems are s... more A block iterative method is used for solving linear least squares problems. The subproblems are solved asynchronously on a distributed memory multiprocessor. It is observed that an increased number of processors results in deteriorating rate of convergence. This deteriorating convergence is illustrated by numerical experiments. The deterioration of the convergence can be explained by contamination of the residual. Our purpose is to show that the residual is contaminated by old information. The issues investigated here are the effect of the number of processors, the role of essential neighbors, and synchronization. The characterization of old information remains an open problem.

Deteriorating convergence for asynchronous methods on linear least squares problems

Lecture Notes in Computer Science, 1997

... 154-156 (1991) 311-330 7. George, JA, Heath, MT: Solution of Sparse Linear Least Squares Prob... more

Are Three Parameters Enough to Represent a Parallel Computer Architecture?

ABSTRACT The purpose of the paper is to build an analytical model to predict the performance of a... more ABSTRACT The purpose of the paper is to build an analytical model to predict the performance of an iterative linear least squares solver on a class of architectures using master-slave strategy. The model is used to analyze the behavior of the algorithm on varying number of processors, problem sizes and different architectures. Though it is based on only three architecture parameters, it gives fair predictions of execution time. Experimental results on SGI Origin 2000 and Cray T3E are included. 1 Introduction In the existence of different algorithms to solve any given problem, a tool to estimate the comparative performances of these algorithms can make the final decision to choose one of them easier. Before coding the actual program a performance estimation tool can be used to predict the behavior of the different algorithms for various problem sizes, architectures and number of available processors. Performance estimation tools are based on &quot;analytical models&quot;. The chosen analytical model need...

Optimal direct determination of sparse Jacobian matrices

It is well known that a sparse Jacobian matrix can be determined with fewer function evaluations ... more It is well known that a sparse Jacobian matrix can be determined with fewer function evaluations using finite differencing or forward automatic differentiation (AD) passes than the number of independent variables of the underlying function. In this paper, we show that by grouping together rows into blocks and partitioning the resulting column segments, one can reduce this number further. We suggest a graph colouring technique for row partitioned Jacobian matrices to efficiently determine the nonzero entries using direct determination. We characterize optimal direct determination and derive results on the optimality of any direct determination technique based on column computation. Previously published computational results by the authors [S. Hossain and T. Steihaug, Graph colouring in the estimation of sparse derivative matrices: Instances and applications, Discrete Appl. Math. 156(2) (2008), pp. 280–288; S. Hossain, CsegGraph: A graph colouring instance generator, Int. J. Comput. Math. 86(10–11) (2009), pp. 1956–1967] have demonstrated that the row partitioned direct determination can yield considerable savings in function evaluations or AD passes over methods based on the Curtis, Powell, and Reid technique.

Graph Coloring and the Estimation of Sparce Jacobian Matrices Using Row and Column Partitioning

Asynchronous methods and least squares: An example of deteriorating convergence

ABSTRACT We use block iterative methods for solving linear least squares problems. The subproblem... more ABSTRACT We use block iterative methods for solving linear least squares problems. The subproblems are solved asynchronously on a distributed memory multiprocessor. It is observed that increased number of processors results in deteriorating convergence. We illustrate the deteriorating convergence by some numerical experiments. The deterioration of the convergence can be explained by contamination of the residual. The residual is contaminated by old information. Our purpose is to reduce the effect of old information. The issues investigated here are the effect of the number of processors, the role of essential neighbors [8] and heterogeneous processors. We include two heuristics to identify the information to be discarded and reduce the effect of old information: a relaxation factor and synchronization. The characterization of old information remains as an open problem. 1 Introduction In this paper we use block iterative method for solving sparse linear least squares problems. A general framewo...

A Ferris-Mangasarian technique applied to linear least squares problems

This note specializes to linear least squares problems an approach suggested by Ferris and Mangas... more This note specializes to linear least squares problems an approach suggested by Ferris and Mangasarian 4] for solving constrained optimization problems on parallel computers. It will be shown here that this specialization leads to an algorithm which is mathematically equivalent to an acceleration and convergence forcing modi cation of the block Jacobi iteration applied to the normal equations. The resulting algorithm is a promising way to speed up a parallel multisplitting algorithm of Renaut 9] for linear least squares. Renaut's algorithm is related to a specialization of part of the Ferris and Mangasarian approach.

Practical Application of Super-computing in Black-Oil Reservoir Simulation

North Sea Oil and Gas Reservoirs—II, 1990