We propose to use the heuristic static loadbalancing (HSLB) algorithm for solving load-balancing ... more We propose to use the heuristic static loadbalancing (HSLB) algorithm for solving load-balancing problems in the Community Earth System Model (CESM), a climate model, using fitted benchmark data as an alternative to the current manual approach. The problem of allocating the optimal number of CPU cores to CESM components is formulated as a mixed-integer nonlinear optimization problem which is solved by using an optimization branch-and-bound solver implemented in the MINLP package MINOTAUR. The key feature of the branch-and-bound method is that it guarantees to provide an optimal solution or show that none exists. Our algorithm was tested for the 1 • and 1/8 • resolution simulations on 32,768 nodes (131,072 cores) of IBM Blue Gene/P where we consistently achieved well load-balanced results. This work is a part of a broader effort to eliminate the need for manual tuning of the code for each platform and simulation type, improve the performance and scalability of CESM, and develop automated tools to achieve these goals.
ABSTRACT Load balancing scientific codes on massively parallel architectures is becoming an incre... more ABSTRACT Load balancing scientific codes on massively parallel architectures is becoming an increasingly challenging task. In this paper, we focus on the Community Earth System Model, a widely used climate modeling code. It comprises six components each of which exhibits different scalability patterns. Previously, an analytical performance model has been used to find optimal load-balancing parameter configurations for each component. Nevertheless, for the Community Ice Code component, the analytical performance model is too restrictive to capture its scalability patterns. We therefore developed machine-learning-based loadbalancing algorithm. It involves fitting a surrogate model to a small number of load-balancing configurations and their corresponding runtimes. This model is then used to find high-quality parameter configurations. Compared with the current practice of expert-knowledge-based enumeration over feasible configurations, the machine-learning-based load-balancing algorithm requires six times fewer evaluations to find the optimal configuration.
Coupled climate models are multiphysics models comprising multi-ple separately developed codes th... more Coupled climate models are multiphysics models comprising multi-ple separately developed codes that are combined into a single physical system. This composition of codes is amenable to a scripting solution, and Python is a language that offers many desirable properties for this task. We have prototyped a version of the Community Climate System Model (CCSM) with coupling infrastructure written in Python. Our objective was to improve dramatically CCSM's already flexible cou-pling infrastructure to enable research uses of the model not currently supported. Here we report the progress in the first steps in this effort: the construction of Python bindings for the Model Coupling Toolkit, a key piece of third-party coupling middleware used in CCSM, and a Python-based CCSM coupler application. We find that the choice of Python over the original Fortran implementation in the coupler imposes minimal visible performance impact to the overall coupled system. We believe our results augur wel...
Coupled climate models are multiphysics models comprising multi-ple separately developed codes th... more Coupled climate models are multiphysics models comprising multi-ple separately developed codes that are combined into a single physical system. This composition of codes is amenable to a scripting solution, and Python is a language that offers many desirable properties for this task. We have prototyped a Python coupling and control infrastruc-ture for version 3.0 of the Community Climate System Model (ccsm3). Our objective was to improve dramatically ccsm3's already flexible coupling facilities to enable research uses of the model not currently supported. We report the progress in the first steps in this effort: the construction of Python bindings for the Model Coupling Toolkit, a key piece of third-party coupling middleware used in ccsm3, and a Python-based ccsm3 coupler (pypcl) application. We report prelim-inary performance results for this new system, which we call pyccsm. Contents C1113 and explain how pypcl's performance may be improved to support production runs. We b...
According to Milankovitch's theory of the ice ages, astronomically-forced high-latitude summe... more According to Milankovitch's theory of the ice ages, astronomically-forced high-latitude summer insolation variations controlled the initiation and growth of Northern Hemisphere ice sheets. Fluctuations in Earth's precession and obliquity both substantially influence high-latitude insolation, and should be preserved as variability in the ice-volume record. Yet, while Pleistocene records of benthic delta18O possess large spectral power in the obliquity band, they show only feeble power in the precessional band. To explain the missing precessional signal, we have conducted a series of coupled ocean-atmosphere GCM experiments that test the high-latitude climate sensitivity to orbital parameters. In agreement with the paleoclimate record, our model demonstrates a much larger response to obliquity variations than to precessional variations. For example, the mean-annual Northern Hemisphere sea-ice extent and snowfall rate response to a reduction in axial tilt is three times greater...
Biofuels are a key component of renewable energy mix proposed as a substitute to fossil fuels. Bi... more Biofuels are a key component of renewable energy mix proposed as a substitute to fossil fuels. Biofuels are suggested as both economical and having potential for reducing atmospheric emissions of carbon from the transportation sector, by building up soil carbon levels when planted on lands where these levels have been reduced by intensive tillage. The purpose of this research is to develop a carbon-nitrogen based crop module (CNC) for the community land model (CLM) and to improve the characterization of the below and above ground carbon sequestration for bioenergy crops. The CNC simulates planting, growing, maturing and harvesting stages for three major crops: maize, soybean and wheat. In addition, dynamic root module is implemented to simulate fine root distribution and development based on relative availability of soil water and nitrogen in the root zone. Coupled CLM-CNC models is used to study crop yields, geographic locations for bioenergy crop production and soil carbon changes...
Bioenergy is generating tremendous interest as an alternative energy source that is both environm... more Bioenergy is generating tremendous interest as an alternative energy source that is both environmentally friendly and economically competitive. The amount of land designated for agriculture is expected to expand, including changes in the current distribution of crops, as demand for biofuels increases as a carbon neutral alternative fuel source. However, the influence of agriculture on the carbon cycle is complex, and varies depending on land use change and management practices. The purpose of this research is to integrate agriculture in the carbon-nitrogen based Community Land Model (CLM) to evaluate the above and below ground carbon storage for corn, soybean, and wheat crop lands. The new model, CLM-Crop simulates carbon allocation during four growth stages, a soybean nitrogen fixation scheme, fertilizer, and harvest practices. We present results from this model simulation, which includes the impact of a new dynamic roots module to simulate the changing root structure and depth wit...
Climate models are continuing to increase both in their resolution and the number of variables us... more Climate models are continuing to increase both in their resolution and the number of variables used, resulting in multi-terabyte model outputs. This large volume of data overwhelms the series of processing steps used to derive climate averages and produce visualizations. Since many of the tasks in the post- processing sequence are independent, we have applied task-parallel scripting to speed up the post-processing. We have re-written portions of the complex shell script that processes output from the Community Atmosphere Model in Swift, a high-level implicitly-parallel scripting language that uses data dependencies to automatically parallelize a workflow. This has resulted in valuable speedups in model analysis for this heavily-used procedure. We describe the structure, usage, performance, and our experiences with the resulting script.
2005 IEEE International Conference on Cluster Computing, 2005
Summary form only given. Commodity clusters have enabled ambitious multiphysics or coupled modeli... more Summary form only given. Commodity clusters have enabled ambitious multiphysics or coupled modeling of complex, mutually interacting, computationally intensive systems in science and engineering. Each individual sub-system is represented as a component with its own parallel processor layout and requirements for temporal advance. A central challenge in developing such systems is the parallel coupling problem, which involves overall system architecture
The Fast Ocean Atmosphere Model (FOAM) is a climate system model intended for application to clim... more The Fast Ocean Atmosphere Model (FOAM) is a climate system model intended for application to climate science questions that require long simulations. FOAM is a distributed-memory parallel climate model consisting of parallel general circulation models of the atmosphere and ocean with complete physics paramaterizations as well as sea-ice, land surface, and river transport models. FOAM's coupling strategy was chosen for high throughput (simulated years per day). A new coupler was written for FOAM and some ...
We propose to use the heuristic static loadbalancing (HSLB) algorithm for solving load-balancing ... more We propose to use the heuristic static loadbalancing (HSLB) algorithm for solving load-balancing problems in the Community Earth System Model (CESM), a climate model, using fitted benchmark data as an alternative to the current manual approach. The problem of allocating the optimal number of CPU cores to CESM components is formulated as a mixed-integer nonlinear optimization problem which is solved by using an optimization branch-and-bound solver implemented in the MINLP package MINOTAUR. The key feature of the branch-and-bound method is that it guarantees to provide an optimal solution or show that none exists. Our algorithm was tested for the 1 • and 1/8 • resolution simulations on 32,768 nodes (131,072 cores) of IBM Blue Gene/P where we consistently achieved well load-balanced results. This work is a part of a broader effort to eliminate the need for manual tuning of the code for each platform and simulation type, improve the performance and scalability of CESM, and develop automated tools to achieve these goals.
ABSTRACT Load balancing scientific codes on massively parallel architectures is becoming an incre... more ABSTRACT Load balancing scientific codes on massively parallel architectures is becoming an increasingly challenging task. In this paper, we focus on the Community Earth System Model, a widely used climate modeling code. It comprises six components each of which exhibits different scalability patterns. Previously, an analytical performance model has been used to find optimal load-balancing parameter configurations for each component. Nevertheless, for the Community Ice Code component, the analytical performance model is too restrictive to capture its scalability patterns. We therefore developed machine-learning-based loadbalancing algorithm. It involves fitting a surrogate model to a small number of load-balancing configurations and their corresponding runtimes. This model is then used to find high-quality parameter configurations. Compared with the current practice of expert-knowledge-based enumeration over feasible configurations, the machine-learning-based load-balancing algorithm requires six times fewer evaluations to find the optimal configuration.
Coupled climate models are multiphysics models comprising multi-ple separately developed codes th... more Coupled climate models are multiphysics models comprising multi-ple separately developed codes that are combined into a single physical system. This composition of codes is amenable to a scripting solution, and Python is a language that offers many desirable properties for this task. We have prototyped a version of the Community Climate System Model (CCSM) with coupling infrastructure written in Python. Our objective was to improve dramatically CCSM's already flexible cou-pling infrastructure to enable research uses of the model not currently supported. Here we report the progress in the first steps in this effort: the construction of Python bindings for the Model Coupling Toolkit, a key piece of third-party coupling middleware used in CCSM, and a Python-based CCSM coupler application. We find that the choice of Python over the original Fortran implementation in the coupler imposes minimal visible performance impact to the overall coupled system. We believe our results augur wel...
Coupled climate models are multiphysics models comprising multi-ple separately developed codes th... more Coupled climate models are multiphysics models comprising multi-ple separately developed codes that are combined into a single physical system. This composition of codes is amenable to a scripting solution, and Python is a language that offers many desirable properties for this task. We have prototyped a Python coupling and control infrastruc-ture for version 3.0 of the Community Climate System Model (ccsm3). Our objective was to improve dramatically ccsm3's already flexible coupling facilities to enable research uses of the model not currently supported. We report the progress in the first steps in this effort: the construction of Python bindings for the Model Coupling Toolkit, a key piece of third-party coupling middleware used in ccsm3, and a Python-based ccsm3 coupler (pypcl) application. We report prelim-inary performance results for this new system, which we call pyccsm. Contents C1113 and explain how pypcl's performance may be improved to support production runs. We b...
According to Milankovitch's theory of the ice ages, astronomically-forced high-latitude summe... more According to Milankovitch's theory of the ice ages, astronomically-forced high-latitude summer insolation variations controlled the initiation and growth of Northern Hemisphere ice sheets. Fluctuations in Earth's precession and obliquity both substantially influence high-latitude insolation, and should be preserved as variability in the ice-volume record. Yet, while Pleistocene records of benthic delta18O possess large spectral power in the obliquity band, they show only feeble power in the precessional band. To explain the missing precessional signal, we have conducted a series of coupled ocean-atmosphere GCM experiments that test the high-latitude climate sensitivity to orbital parameters. In agreement with the paleoclimate record, our model demonstrates a much larger response to obliquity variations than to precessional variations. For example, the mean-annual Northern Hemisphere sea-ice extent and snowfall rate response to a reduction in axial tilt is three times greater...
Biofuels are a key component of renewable energy mix proposed as a substitute to fossil fuels. Bi... more Biofuels are a key component of renewable energy mix proposed as a substitute to fossil fuels. Biofuels are suggested as both economical and having potential for reducing atmospheric emissions of carbon from the transportation sector, by building up soil carbon levels when planted on lands where these levels have been reduced by intensive tillage. The purpose of this research is to develop a carbon-nitrogen based crop module (CNC) for the community land model (CLM) and to improve the characterization of the below and above ground carbon sequestration for bioenergy crops. The CNC simulates planting, growing, maturing and harvesting stages for three major crops: maize, soybean and wheat. In addition, dynamic root module is implemented to simulate fine root distribution and development based on relative availability of soil water and nitrogen in the root zone. Coupled CLM-CNC models is used to study crop yields, geographic locations for bioenergy crop production and soil carbon changes...
Bioenergy is generating tremendous interest as an alternative energy source that is both environm... more Bioenergy is generating tremendous interest as an alternative energy source that is both environmentally friendly and economically competitive. The amount of land designated for agriculture is expected to expand, including changes in the current distribution of crops, as demand for biofuels increases as a carbon neutral alternative fuel source. However, the influence of agriculture on the carbon cycle is complex, and varies depending on land use change and management practices. The purpose of this research is to integrate agriculture in the carbon-nitrogen based Community Land Model (CLM) to evaluate the above and below ground carbon storage for corn, soybean, and wheat crop lands. The new model, CLM-Crop simulates carbon allocation during four growth stages, a soybean nitrogen fixation scheme, fertilizer, and harvest practices. We present results from this model simulation, which includes the impact of a new dynamic roots module to simulate the changing root structure and depth wit...
Climate models are continuing to increase both in their resolution and the number of variables us... more Climate models are continuing to increase both in their resolution and the number of variables used, resulting in multi-terabyte model outputs. This large volume of data overwhelms the series of processing steps used to derive climate averages and produce visualizations. Since many of the tasks in the post- processing sequence are independent, we have applied task-parallel scripting to speed up the post-processing. We have re-written portions of the complex shell script that processes output from the Community Atmosphere Model in Swift, a high-level implicitly-parallel scripting language that uses data dependencies to automatically parallelize a workflow. This has resulted in valuable speedups in model analysis for this heavily-used procedure. We describe the structure, usage, performance, and our experiences with the resulting script.
2005 IEEE International Conference on Cluster Computing, 2005
Summary form only given. Commodity clusters have enabled ambitious multiphysics or coupled modeli... more Summary form only given. Commodity clusters have enabled ambitious multiphysics or coupled modeling of complex, mutually interacting, computationally intensive systems in science and engineering. Each individual sub-system is represented as a component with its own parallel processor layout and requirements for temporal advance. A central challenge in developing such systems is the parallel coupling problem, which involves overall system architecture
The Fast Ocean Atmosphere Model (FOAM) is a climate system model intended for application to clim... more The Fast Ocean Atmosphere Model (FOAM) is a climate system model intended for application to climate science questions that require long simulations. FOAM is a distributed-memory parallel climate model consisting of parallel general circulation models of the atmosphere and ocean with complete physics paramaterizations as well as sea-ice, land surface, and river transport models. FOAM's coupling strategy was chosen for high throughput (simulated years per day). A new coupler was written for FOAM and some ...
Uploads
Papers by Robert Jacob