Automatic Parallelization

Estimating the execution time of nested loops or the volume of data transferred between processors is necessary to make appropriate processor or data allocation. To achieve this goal one need to estimate the execution time of the body and thus the number of nested loop iterations. This work could be a preprocessing step in an automatic parallelizing compilers to enhance the performance of the resulting parallel program. A bounded convex polyhedron can be associated with each loop nest. The number of its integer points corresponds to the iteration space size. In this paper, we present an algorithm that approximates this number. The algorithm is not restricted to a fixed dimension. The worst case complexity of the algorithm is infrequently reached in our context where the nesting level is rather small and the loop bound expressions are not very complex

Adaptive Mesh Re®nement (AMR) calculations carried out on structured meshes play an exceedingly important role in several areas of science and engineering. This is so not just because AMR techniques allow us to carry out calculations very eciently but also because they model very precisely the multi-scale fashion in which nature itself works. Many AMR applications are also amongst the most computationally intensive calculations undertaken making it necessary to use parallel supercomputers for their solution. While class library-based approaches are being attempted for parallel AMR we point out here that recent advances in the Fortran 90/95 standard and the OpenMP standard now make it possible to carry out highly parallel AMR calculations using language-based approaches. The language-based approaches oer several advantages over library-based approaches, the two principal ones being portability across parallel platforms and the best possible utilization of Distributed Shared Memory (DSM) hardware on machines that have such hardware. They also free up the applications scientist from being constrained by the static features of a class library. The choice of Fortran also ensures maximal reuse of pre-existing Fortran 77 applications and full Fortran 77-based processing eciency on each computational node. Our implementation of the ideas presented here in the ®rst author's RIEMANN framework essentially permits any serial, uniform grid, stencil-based Fortran code to be turned into a parallel AMR code. In this paper we ®rst describe our strategy for using Fortran 90 in an object-oriented fashion. This permits AMR applications to be expressed in terms of familiar abstractions that are natural to the www.elsevier.com/locate/parco Parallel Computing 27 (2001) 37±70 (D.S. Balsara), [email protected] (C.D. Norton). 0167-8191/01/$ -see front matter Ó 2001 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 7 -8 1 9 1 ( 0 0 ) 0 0 0 8 8 -0

Distributed-memory multicomputers such as the the Intel Paragon, the IBM SP-2, and the Thinking Machines CM-5 o er signi cant advantages over shared-memory multiprocessors in terms of cost and scalability. Unfortunately, extracting all the computational power from these machines requires users to write e cient software for them, which is a laborious process. The PARADIGM compiler project provides an automated means to parallelize programs, written using a serial programming model, for e cient execution on distributed-memory multicomputers. In addition to performing traditional compiler optimizations, PARADIGM is unique in that it addresses many other issues within a uni ed platform: automatic data distribution, communication optimizations, support for irregular computations, exploitation of functional and data parallelism, and multithreaded execution. This paper describes the techniques used and provides experimental evidence of their e ectiveness on the Intel Paragon, the IBM SP-1, and the Thinking Machines CM-5.

This paper extends the algorithms which were developed in Part I to cases in which there is no a ne schedule, i.e. to problems whose parallel complexity is polynomial but not linear. The natural generalization is to multidimensional schedules with lexicographic ordering as temporal succession. Multidimensional a ne schedules, are, in a sense, equivalent to polynomial schedules, and are much easier to handle automatically. Furthermore, there is a strong connection between multidimensional schedules and loop nests, which allows one to prove that a static control program always has a multidimensional schedule. Roughly, a larger dimension indicates less parallelism. In the algorithm which is presented here, this dimension is computed dynamically, and is just su cient for scheduling the source program. The algorithm lends itself to a \divide and conquer" strategy. The paper gives some experimental evidence for the applicability, performances and limitations of the algorithm.

This paper extends the algorithms which were developed in Part I to cases in which there is no a ne schedule, i.e. to problems whose parallel complexity is polynomial but not linear. The natural generalization is to multidimensional schedules with lexicographic ordering as temporal succession. Multidimensional a ne schedules, are, in a sense, equivalent to polynomial schedules, and are much easier to handle automatically. Furthermore, there is a strong connection between multidimensional schedules and loop nests, which allows one to prove that a static control program always has a multidimensional schedule. Roughly, a larger dimension indicates less parallelism. In the algorithm which is presented here, this dimension is computed dynamically, and is just su cient for scheduling the source program. The algorithm lends itself to a \divide and conquer" strategy. The paper gives some experimental evidence for the applicability, performances and limitations of the algorithm.

Static scheduling of a program represented by a directed task graph on a multiprocessor system to minimize the program completion time is a well-known problem in parallel processing. Since finding an optimal schedule is an NPcomplete problem in general, researchers have resorted to devising efficient heuristics. A plethora of heuristics have been proposed based on a wide spectrum of techniques, including branch-and-bound, integer-programming, searching, graphtheory, randomization, genetic algorithms, and evolutionary methods. The objective of this survey is to describe various scheduling algorithms and their functionalities in a contrasting fashion as well as examine their relative merits in terms of performance and time-complexity. Since these algorithms are based on diverse assumptions, they differ in their functionalities, and hence are difficult to describe in a unified context. We propose a taxonomy that classifies these algorithms into different categories. We consider 27 scheduling algorithms, with each algorithm explained through an easy-to-understand description followed by an illustrative example to demonstrate its operation. We also outline some of the novel and promising optimization approaches and current research trends in the area. Finally, we give an overview of the software tools that provide scheduling/mapping functionalities

an NSF Graduate Research Fellowship and NSF and Darpa grants to the Fugu and Raw projects. While provided a vital support network. Most of all, I have relied on my wife, Kathleen Shannon, and my children, Karissa and Anya. Their love has carried me through this period. Without them none of this would have been possible, or worth doing.

Existing theories of multiple object tracking (MOT) offer different predictions concerning the role of higher level cognitive processes, individual differences, effortful attention and parallel processing in MOT. Pylyshyn's model (1989) argues for an automatic parallel processing mechanism separate from other cognition, whereas alternative models (e.g., or spotlight models) are based on higher level cognition such as spatial short-term memory and/or effortful attention switching. These predictions were examined in Experiment 1 where identical objects and in Experiment 2 where visually and semantically distinct objects were tracked. Both experiments demonstrated a substantial individual variation in the estimated tracking capacity. Tasks measuring visuospatial short-term memory and attention switching proved to be significant predictors of MOT. In addition, tracking performance deteriorated as a function of tracking time and set size. Our results are in contrast to Pylyshyn's model. A mechanism with both parallel and serial processing and temporary spatial memory is outlined to accommodate the observed pattern of results. The completion of this study was made possible by a grant provided by Finnish Scientific Advisory Board of Defence to the first author. The second author acknowledges the support of Suomen Akatemia (the Academy of Finland). We thank Marja-Leena Haavisto, Krista Oinonen and Juhani Lehto, who participated in the planning of the test battery, especially in constructing the visuospatial complex span task. We also thank Krista Oinonen and Maija Seppa Ènen for collecting the data. Juhani Sinivuo supported the work in many ways. Finally, we are grateful to Glyn Humphreys and two anonymous reviewers for their constructive and thoughtful criticism on an earlier version of this paper, and to Jari Laarni and Mika Koivisto for their valuable comments.

GPUs are a class of specialized parallel architectures with tremendous computational power. The new Compute Unified Device Architecture (CUDA) programming model from NVIDIA facilitates programming of general purpose applications on their GPUs. However, manual development of high-performance parallel code for GPUs is still very challenging. In this paper, a number of issues are addressed towards the goal of developing a compiler framework for automatic parallelization and performance optimization of affine loop nests on GPGPUs: 1) approach to program transformation for efficient data access from GPU global memory, using a polyhedral compiler model of data dependence abstraction and program transformation; 2) determination of optimal padding factors for conflict-minimal data access from GPU shared memory; and 3) model-driven empirical search to determine optimal parameters for unrolling and tiling. Experimental results on a number of kernels demonstrate the effectiveness of the compiler optimization approaches developed.

per. Software package build upon proposed algorithm is described. Several practical examples of mesh generation on multiprocessor computational systems are given. It is shown that developed parallel algorithm enables us to reduce mesh generation time significantly (dozens of times). Moreover, it easily produces meshes with number of elements of order 5 · 10 7 , construction of those on a single CPU is problematic. Questions of time consumption, efficiency of computations and quality of generated meshes are also considered. Keywords: Generalized LBE, free-surface phenomena, interface boundary conditions, filling processes, Bingham viscoplastic model, regularized models (22 pages, 2001)

We introduce StarFlow, a script-centric environment for data analysis. StarFlow has four main features: (1) extraction of control and data-flow dependencies through a novel combination of static analysis, dynamic runtime analysis, and user annotations, (2) command-line tools for exploring and propagating changes through the resulting dependency network, (3) support for workflow abstractions enabling robust parallel executions of complex analysis pipelines, and (4) a seamless interface with the Python scripting language. We describe real applications of StarFlow, including automatic parallelization of complex workflows in the cloud.

A robot-controlled wafer bonding machine was developed for the bonding of different sizes of wafers ranging up to 8 inches diameter. The features of this equipment are such that: (1) After the automatic parallel adjustment for 8-inch wafers to a margin of error within ±1 μm, the X, Y, and θ axis alignments are performed, allowing a margin of error within ±0.5 μm in bonding accuracy; and (2) Room-temperature bonding is enabled using the surface activated (SAB) bonding concept. 8-inch diameter silicon wafers ware successfully bonded by the SAB process at room temperature for the first time. Preliminary investigations across the interface using an Infrared camera show that no bubbles are visibly present in the bonding region

The Support Vector Machine (SVM) is a supervised learning algorithm used for recognizing patterns in data. It is a very popular technique in Machine Learning and has been successfully used in applications such as image classification, protein classification, and handwriting recognition. However, the computational complexity of the kernelized version of the algorithm grows quadratically with the number of training examples. To tackle this high computational complexity we have developed a directive-based approach that converts a gradient-ascent based training algorithm for the CPU to an efficient GPU implementation. We compare our GPU-based SVM training algorithm to the standard LibSVM CPU implementation, a highly-optimized GPU-LIBSVM implementation, as well as to a directive-based OpenACC implementation. The results on different handwritten digit classification datasets demonstrate an important speed-up for the current approach when compared to the CPU and OpenACC versions. Furthermore, our solution is almost as fast and sometimes even faster than the highly optimized CUBLAS-based GPU-LIBSVM implementation, without sacrificing the algorithm's accuracy. 2 VALERIU CODREANU ET AL.

The Set-Sharing domain has been widely used to infer at compiletime interesting properties of logic programs such as occurs-check reduction, automatic parallelization, and finite-tree analysis. However, performing abstract unification in this domain requires a closure operation that increases the number of sharing groups exponentially. Much attention has been given to mitigating this key inefficiency in this otherwise very useful domain. In this paper we present a novel approach to Set-Sharing: we define a new representation that leverages the complement (or negative) sharing relationships of the original sharing set, without loss of accuracy. Intuitively, given an abstract state shV over the finite set of variables of interest V, its negative representation is ℘(V) \ shV. Using this encoding during analysis dramatically reduces the number of elements that need to be represented in the abstract states and during abstract unification as the cardinality of the original set grows toward 2 |V|. To further compress the number of elements, we express the set-sharing relationships through a set of ternary strings that compacts the representation by eliminating redundancies among the sharing sets. Our experiments show that our approach can compress the number of relationships, reducing significantly the memory usage and running time of all abstract operations, including abstract unification.

Parallelizing compilers have traditionally focussed mainly on parallelizing loops. This paper presents a new framework for automatically parallelizing recursive procedures that typically appear in divide-and-conquer algorithms. We present compile-time analysis, using powerful, symbolic array section analysis, to detect the independence of multiple recursive calls in a procedure. This allows exploitation of a scalable form of nested parallelism, where each parallel task can further spawn off parallel work in subsequent recursive calls.

With the advent of digitization and growing abundance of graphic and image processing tools, use cases for clipping using circular windows have grown considerably. This paper presents an efficient clipping algorithm for line segments using geometrical features of circle and vector calculus. Building upon the research with rectangular windows, this method is proposed with the belief that computations are more expensive (heavy) than other computations. Execution time can be drastically improved if we replace expensive computations with cheaper computations. The cheaper computations can be computed even more efficiently using parallelization thus improving time complexity.

Data-oriented workflows are often used in scientific applications for executing a set of dependent tasks across multiple computers. We discuss how these can be modeled using lambda calculus, and how ideas from functional programming are applicable in the design of workflows. Such an approach avoids the restrictions often found in workflow languages, permitting the implementation of complex application logic and data manipulation.

Abstract This paper explains why lambda calculus is an appropriate model for workflow representation, and how a suitably efficient implementation can provide a wide range of capabilities to developers. The presented approach also permits high-level workflow features to be implemented at user level, in terms of a small set of low-level primitives provided by the language implementation.

MapReduce is a programming model and an associated implementation for processing and generating large datasets that is amenable to a broad variety of real-world tasks. Users specify the computation in terms of a map and a reduce function, and the underlying runtime system automatically parallelizes the computation across large-scale clusters of machines, handles machine failures, and schedules inter-machine communication to make efficient use of the network and disks. Programmers find the system easy to use: more than ten thousand distinct MapReduce programs have been implemented internally at Google over the past four years, and an average of one hundred thousand MapReduce jobs are executed on Google's clusters every day, processing a total of more than twenty petabytes of data per day.

Recent advances in polyhedral compilation technology have made it feasible to automatically transform affine sequential loop nests for tiled parallel execution on multi-core processors. However, for multi-statement input programs with statements of different dimensionalities, such as Cholesky or LU decomposition, the parallel tiled code generated by existing automatic parallelization approaches may suffer from significant load imbalance, resulting in poor scalability on multi-core systems. In this paper, we develop a completely automatic parallelization approach for transforming input affine sequential codes into efficient parallel codes that can be executed on a multi-core system in a load-balanced manner. In our approach, we employ a compile-time technique that enables dynamic extraction of inter-tile dependences at run-time, and dynamic scheduling of the parallel tiles on the processor cores for improved scalable execution. Our approach obviates the need for programmer intervention and re-writing of existing algorithms for efficient parallel execution on multi-cores. We demonstrate the usefulness of our approach through comparisons using linear algebra computations: LU and Cholesky decomposition.

Previous literature in alphabetic languages suggests that the occipital-temporal region (the ventral pathway) is specialized for automatic parallel word recognition, whereas the parietal region (the dorsal pathway) is specialized for serial letter-by-letter reading . However, few studies have directly examined the role of the ventral and dorsal pathways in Chinese reading compared to English reading. To investigate this issue, we adopted the degraded word processing paradigm used by and compared brain regions involved in the processing of degraded Chinese characters and English words during lexical decision, using functional magnetic resonance imaging (fMRI). The degraded characters/words were created by inserting blank spaces between radicals of Chinese characters or syllables of English polysyllabic words. Generally, the current study replicated the effects of , showing that in Chinese -like in alphabetic languages -character spacing modulates both ventral (bilateral cuneus, left middle occipital gyrus) and dorsal (left superior parietal lobule and middle frontal gyrus) pathways. In addition, the current study showed greater activation in bilateral cuneus and right lingual gyrus for Chinese versus English when comparing spaced to normal stimuli, suggesting that Chinese character recognition relies more on ventral visual-spatial processing than English word recognition. Interestingly, bilateral cuneus showed monotonic patterns in response to increasing spacing, while the rest of the regions of interest showed non-monotonic patterns, indicating different profiles for these regions in visual-spatial processing.

We describe pHPF, an research prototype HPF compiler for the IBM SP series parallel machines. The compiler accepts as input Fortran 90 and Fortran 77 programs, augmented with HPF directives; sequential loops are automatically parallelized. The compiler supports symbolic analysis of expressions. This allows parameters such as the number of processors to be unknown at compile-time without signi cantly a ecting performance. Communication schedules and computation guards are generated in a parameterized form at compile-time. Several novel optimizations and improved versions of well-known optimizations have been implemented in pHPF to exploit parallelism and reduce communication costs. These optimizations include elimination of redundant communication using data-availability analysis; using collective communication; new techniques for mapping scalar variables; coarse-grain wavefronting; and communication reduction in multi-dimensional shift communications. We present experimental results for some well-known benchmark routines. The results show the e ectiveness of the compiler in generating e cient code for HPF programs.

This paper develops and experimentally demonstrates a robust automatic parallel parking algorithm for parking in tight spaces. Novel fuzzy logic controllers are designed for each step of the maneuvering process. The controllers are first demonstrated by simulation using the kinematic model of a skid steering autonomous ground vehicle (AGV). They are then demonstrated experimentally on a skid steering AGV and it is shown that the developed algorithm has the ability to parallel park AGVs in tight spaces under both vehicle localization errors and parking space detection errors. This paper also presents a genetic fuzzy system which uses a genetic algorithm's learning ability to determine effective parameters for the developed fuzzy logic controllers. The genetic fuzzy system is used to tune the fuzzy logic algorithm for both a skid steering AGV and a front-wheel steering AGV.

The problem of writing software for multicore processors is greatly simplified if we could automatically parallelize sequential programs. Although auto-parallelization has been studied for many decades, it has succeeded only in a few application areas such as dense matrix computations. In particular, auto-parallelization of irregular programs, which are organized around large, pointer-based data struc- tures like graphs, has seemed intractable.

Recent advances in polyhedral compilation technology have made it feasible to automatically transform affine sequential loop nests for tiled parallel execution on multi-core processors. However, for multi-statement input programs with statements of different dimensionalities, such as Cholesky or LU decomposition, the parallel tiled code generated by existing automatic parallelization approaches may suffer from significant load imbalance, resulting in poor scalability on multi-core systems. In this paper, we develop a completely automatic parallelization approach for transforming input affine sequential codes into efficient parallel codes that can be executed on a multi-core system in a load-balanced manner. In our approach, we employ a compile-time technique that enables dynamic extraction of inter-tile dependences at run-time, and dynamic scheduling of the parallel tiles on the processor cores for improved scalable execution. Our approach obviates the need for programmer interventi...

We report on a detailed study of the application and e ectiveness of program analysis based on abstract interpretation to automatic program parallelization. We study the case of parallelizing logic programs using the notion of strict independence. We rst propose and prove correct a methodology for the application in the parallelization task of the information inferred by abstract interpretation, using a parametric domain. The methodology is generic in the sense of allowing the use of di erent analysis domains. A number of well-known approximation domains are then studied and the transformation into the parametric domain de ned. The transformation directly illustrates the relevance and applicability of each abstract domain for the application. Both local and global analyzers are then built using these domains and embedded in a complete parallelizing compiler. Then, the performance of the domains in this context is assessed through a number of experiments. A comparatively wide range of aspects is studied, from the resources needed by the analyzers in terms of time and memory to the actual bene ts obtained from the information inferred. Such bene ts are evaluated both in terms of the characteristics of the parallelized code and of the actual speedups obtained from it. The results show that data ow analysis plays an important role in achieving e cient parallelizations, and that the cost of such analysis can be reasonable even for quite sophisticated abstract domains. Furthermore, the results also o er signi cant insight into the characteristics of the domains, the demands of the application, and the trade-o s involved.

Current approaches to parallelizing compilation perform a purely structural analysis of the sequential code. Conversely, a semantic analysis performing concept assignment for code sections, can support the recognition of the algorithms that the code implements. This can considerably help the parallelization process, by allowing the introduction of heuristics and an extensive pruning of the search space, and thus enabling the application of more aggressive code transformations. It can play an important role in overcoming the current limitations to Automatic Parallelization. In this paper we discuss the applicability of concept comprehension to the parallelization process, and we present a novel technique for automatic algorithmic recognition we have designed and implemented. We are currently developing a reverse engineering tool supporting the translation of sequential Fortran code into HPF, which is based on the recognition technique we have developed. Its working criteria are illustrated and discussed.

Automatic parallelization is a promising strategy to improve application performance in the multicore era. However, common programming practices such as the reuse of data structures introduce artificial constraints that obstruct automatic parallelization. Privatization relieves these constraints by replicating data structures, thus enabling scalable parallelization. Prior privatization schemes are limited to arrays and scalar variables because they are sensitive to the layout of dynamic data structures. This work presents Privateer, the first fully automatic privatization system to handle dynamic and recursive data structures, even in languages with unrestricted pointers. To reduce sensitivity to memory layout, Privateer speculatively separates memory objects. Privateer's lightweight runtime system validates speculative separation and speculative privatization to ensure correct parallel execution. Privateer enables automatic parallelization of general-purpose C/C++ applications, yielding a geomean whole-program speedup of 11.4× over best sequential execution on 24 cores, while non-speculative parallelization yields only 0.93×.

HEXAR, a new software product developed at Cray Research, Inc., automatically generates good quality meshes directly from surface data produced by computeraided design (CAD) packages. The HEXAR automatic mesh generator is based on a proprietary and parallel algorithm that relies on pattern recognition, local mesh refinement and coarsening, and variational mesh smoothing techniques to create all-hexahedral volume meshes. HEXAR generates grids two to three orders of magnitude faster than current manual approaches. Although approximate by design, the resulting meshes have qualities acceptable by many commercial structural and CFD (computational fluid dynamics) software. HEXAR turns mesh generation into an automatic process .for most commercial engineerin 9 applications.

We discuss the parallelization and object-oriented implementation of Monte Carlo simulations for physical problems. We present a C++ Monte Carlo class library for the automatic parallelization of Monte Carlo simulations. Besides discussing the advantages of object-oriented design in the development of this library, we show examples how C++ template techniques have allowed very generic but still optimal algorithms to be implemented for wide classes of problems. These parallel and object-oriented codes have allowed us to perform the largest quantum Monte Carlo simulations ever done in condensed matter physics.

We describe and evaluate a novel approach for the automatic parallelization of programs that use pointer-based dynamic data structures, written in Java. The approach exploits parallelism among methods by creating an asynchronous thread of execution for each method invocation in a program.

The evolution of high performance computers is progressing toward increasingly heterogeneous systems. These new architectures pose new challenges, particularly in the field of programming languages. New tools and languages are needed if we want to make a full use of the advantages offered by these new architectures. llc is a language with a C-like syntax where parallelism is expressed using compiler directives. In this work we focus our attention on the new backend of our prototype compiler for llc which generates CUDA code. We evaluate the performance of the target code using three different applications. The preliminary results that we present make us believe that our approach is worth to be explored more deeply.

automatically generates good quality meshes directly from surface data produced by computeraided design (CAD) packages. The HEXAR automatic mesh generator is based on a proprietary and parallel algorithm that relies on pattern recognition, local mesh refinement and coarsening, and variational mesh smoothing techniques to create all-hexahedral volume meshes. HEXAR generates grids two to three orders of magnitude faster than current manual approaches. Although approximate by design, the resulting meshes have qualities acceptable by many commercial structural and CFD (computational fluid dynamics) software. HEXAR turns mesh generation into an automatic process .for most commercial engineerin 9 applications.

Two key steps in the compilation of strict functional languages are the conversion of higher-order functions to data structures (closures) and the transformation to tail-recursive style. We show how to perform both steps at once by applying first-order offline partial evaluation to a suitable interpreter. The resulting code is easy to transliterate to low-level C or native code. We have implemented the compilation to C; it yields a performance comparable to that of other modern Scheme-to-C compilers. In addition, we have integrated various optimizations such as constant propagation, higherorder removal, and arity raising simply by modifying the underlying interpreter. Purely first-order methods suffice to achieve the transformations. Our approach is an instance of semantics-directed compiler generation.

This paper introduces an analysis technique, commutativity analysis, for automatically parallelizing computations that manipulate dynamic, pointer-based data structures. Commutativity analysis views computations as composed of operations on objects. It then analyzes the program to discover when operations commute, i.e. leave the objects in the same state regardless of the order in which they execute. If all of the operations required to perform a given computation commute, the compiler can automatically generate parallel code. Commutativity analysis eliminates many of the limitations that have prevented existing compilers, which use data dependence analysis, from successfully parallelizing pointer-based applications. It enables compilers to parallelize computations that manipulate graphs and eliminates the need to analyze the data structure construction code to extract global properties of the data structure topology. This paper shows how to use symbolic execution and expression manipulation to statically determine that operations commute and how to exploit the extracted commutativity information to generate parallel code. It also presents performance results that demonstrate that commutativity analysis can be used to successfully parallelize the Barnes-Hut hierarchical N-body solver, an important scientific application that manipulates a complex pointer-based data structure.

Atomic operations are a key primitive in parallel computing systems. The standard implementation mechanism for atomic operations uses mutual exclusion locks. In an object-based programming system, the natural granularity is to give each object its own lock. Each operation can then make its execution atomic by acquiring and releasing the lock for the object that it accesses. But this fine lock granularity may have high synchronization overhead because it maximizes the number of executed acquire and release constructs. To achieve good performance it may be necessary to reduce the overhead by coarsening the granularity at which the computation locks objects.

Tree contraction algorithms, whose idea was first proposed by Miller and Reif, are important parallel algorithms to implement efficient parallel programs manipulating trees. Despite their efficiency, the tree contraction algorithms have not been widely used due to the difficulties in deriving the tree contracting operations. In particular, the derivation of the tree contracting operations is much difficult when multiple values are referred and updated in each step of the contractions. Such computations often appear in dynamic programming problems on trees.

Divide-and-conquer algorithms are suitable for modern parallel machines, tending to have large amounts of inherent parallelism and working well with caches and deep memory hierarchies. Among others, list homomorphisms are a class of recursive functions on lists, which match very well with the divide-and-conquer paradigm. However, direct programming with list homomorphisms is a challenge for many programmers. In this paper, we propose and implement a novel system that can automatically derive costoptimal list homomorphisms from a pair of sequential programs, based on the third homomorphism theorem. Our idea is to reduce extraction of list homomorphisms to derivation of weak right inverses. We show that a weak right inverse always exists and can be automatically generated from a wide class of sequential programs. We demonstrate our system with several nontrivial examples, including the maximum prefix sum problem, the prefix sum computation, the maximum segment sum problem, and the line-ofsight problem. The experimental results show practical efficiency of our automatic parallelization algorithm and good speedups of the generated parallel programs.