Rion Dooley

Followers

Following

Public Views

Don Ross

University College Cork

Alejandra B Osorio

Wellesley College

Steffen Boehm

University of Exeter

Armando Marques-Guedes

UNL - New University of Lisbon

Giulia Sissa

Ucla

Florin Curta

University of Florida

Harry Cleaver

The University of Texas at Austin

J.-C. Spender

Kozminski University

Shabib H . Syed

University of the Punjab, Lahore, Pakistan

ELCHİN SULEYMANOV

Azerbaijan National Academy of Sciences

Interests

Uploads

Papers by Rion Dooley

Adapting Legacy Monolithic Applications into API Services for Web, Mobile and Other Clients

Download

A distributed quadtree dictionary approach to multi-resolution compression

We have developed a distributed quadtree dictionary (DQTD) algorithm, which allows lossless, mult... more We have developed a distributed quadtree dictionary (DQTD) algorithm, which allows lossless, multiresolution compression of Single-Crystal Diffractometer (SCD) datasets from the Argonne National Laboratory in Chicago, IL. This is of prime importance to high-energy physicists who need to manipulate and visualize SCD datasets, but cannot due to their overwhelming memory requirements. Distributing a quadtree dictionary necessarily introduces redundancy to what was previously a minimal QTD. We have developed a method to reduce the tree redundancy in the QTD, thereby providing a tighter upper bound on the size of our QTD. We compare the DQTD algorithm with a distributed square wavelet transform (SWT). Experimental results on three sample 1GB SCD datasets show that, on a level-bylevel basis, our algorithm performs no worse than SWT in terms of energy conservation and adjusted energy conservation, while providing 59:1 overall compression in the average case.

Download

Recipes 2.0: building for today and tomorrow

Concurrency and Computation: Practice and Experience, Apr 29, 2014

The history of science gateway development has, in many ways, been a story of the "Haves" vs. the... more The history of science gateway development has, in many ways, been a story of the "Haves" vs. the "Have-nots." Large infrastructure projects led the way, building thick client portals to provide coherent interfaces to an incoherent environment. Contrast this with the way the modern web is designed using light, front end components and outsourcing much of the heavy lifting to a mash-up of REST APIs, and it is easy to see why modern web applications can be prototyped and refined into stable products in the time it previously took thick client portals to do an initial release. This paper argues that a "build for today" philosophy can lead to the rapid development of science gateways to serve the "Have-nots." Also presented is a set of responsive front end components built on top of the iPlant Foundation API that provide the boilerplate for rapid development of lightweight science gateways using only HTML, JavaScript, and CSS. Using these components, developers can easily stand up new gateways or quickly add new functionality to existing ones.

Download

Trusted CI Webinar: Best Practices for Academic Cloud Service Providers with Rion Dooley

A distributed Quadtree Dictionary approach to multi-resolution visualization of scattered neutron data

As of this writing, there are over three-dozen major Grid projects underway [Foster, 2002]. Despi... more As of this writing, there are over three-dozen major Grid projects underway [Foster, 2002]. Despite these efforts, Grid computing is still in its infancy. In this section we will examine the need for Grid computing, the underlying architecture that makes up the Grid, the state of Grid computing today, and conclude with a look at some current Grid applications, including LSU's own Access Grid. The following material borrows heavily from [Foster, et al., 2001][Foster and Kesselman, 1999] and [Foster, et al., 2002]. 1.1.1 The Need for Grid Computing Distributed computing has existed for decades. Dating back to ARPANET in the 1970's, researchers have been looking for ways to pool their resources [Leiner, et al., 2000]. The ways in which they use these resources are as numerous as the people using them. Environmental, materials, and flight simulations, radio, telescope, and Magnetic Resonance Imaging (MRI) data analysis, weather forecasting, and visualization of large data sets are all ways in which scientists and researchers use distributed computing. Despite these applications, the power of distributed computing however is still grossly underutilized. Due in large part to the initial investment required, only a small group of people ever embrace distributed technology. The people who could use it the most: contractors, industrialists, lawmakers, and engineers, miss the opportunity to leverage great pools of computing power to designing better houses, factories, roadways, and bridges [Foster and Kesselman, 1999]. computing power. fightAIDSathome is very similar to seti@home. It uses the volunteer's extra CPU time to examine possible AIDS drugs. Results of individual drug simulations are returned to the central node and examined by experts in the field. The Department of Computer Science at the University of Wisconson sponsors the Condor project. Condor arose out of the search for distributed resource management systems. Researchers were looking for a way to provide distributed allocation and preemptive job submission. From that work sprung research on load balancing, which lead to the Condor project of today; "Distributively owned computing environments where owners have full control over the resources they own [Livny, 2002]." Users submit jobs to the Condor scheduler where a pool of volunteers runs them in a stable environment. On-demand applications use Grid capabilities to meet short-term requirements for resources that cannot be cost-effectively or conveniently located locally [Foster and Kesselman, 1999]." Applications with high computational needs for a short period of time, such as MRI, Monte Carlo market analysis software, and batch processing of daily transactions for financial institutions are all representatives of the on-demand category. Data intensive applications utilize data from several remote repositories to make new discoveries. Analyzing shopping trends of Wal-Mart customers who purchase diapers by accessing data stored in several data warehouses is an example of a data intensive application. Collaborative applications use the Grid to coordinate resources between coworkers and enhance human-to-human interactions [Foster and Kesselman, 1999] [Kikuchi, et al., 2002]. An example is a research group where each researcher has access to the same data, instruments, and computing power in the weekly meetings. If a discovery is made, each researcher can view the results of the experiment at the same time and simultaneously view the results with his colleagues-all while teleconferencing from their respective institutions. Louisiana State University has recently become a contributing member of the Access Grid (AG). The AG is a collection of, "Integrated whole room scale visual environments. These environments consist of group work rooms that have been augmented with multiple displays including: large-format whole wall displays (e.g. ActiveMural [ANL's] high-resolution rear projected tiled display), driven by PC clusters, or multi-processor visualization engines, semiimmersive or immersive displays (Workbenches, ImmersaDesks, CAVEs), multiple desktop devices, and multiple front projection systems [Stevens, 2000]." The AG allows researchers at LSU to communicate with researchers at any other nodes of the AG in a real-time visual environment using PowerPoint presentations, movies, images, etc. thus enabling greater collaboration in disparate locations. As stated above, there are several categories of Grid network users. Some of the larger Grid efforts going on today are listed in Table 1.2 with brief descriptions of their focus. Table 1.2: Major Grid initiatives in the world today. Project Sponsor/Website Description Globus toolkit Department of Energy (DoE) http://www.globus.org Community-based, open-architecture, opensource set of services and software libraries that support Grids and Grid applications [Foster, et al., 2002].

Download

A quadtree dictionary approach to multi-resolution compression

As we attempt to harness the power of tomorrow's mobile devices, intelligent information delivery... more As we attempt to harness the power of tomorrow's mobile devices, intelligent information delivery becomes a primary concern. Getting the appropriate data to the resources that need it is referred to as the needs mismatch problem. In this paper we present a lossless multi-resolution compression technique based on a quadtree dictionary (QTD) that achieves 8:1 compression in the worst case. Test results show our method improves upon probabilistic, wavelet, and tree reduction techniques while providing 54:1 compression in the average case.

Download

Proceedings of the 2011 ACM workshop on Gateway computing environments

IEEE International Conference on High Performance Computing, Data, and Analytics, Nov 18, 2011

It is our great pleasure to welcome you to the 2011 ACM Workshop on Gateway Computing Environment... more It is our great pleasure to welcome you to the 2011 ACM Workshop on Gateway Computing Environments -- GCE'11. This year's workshop, the sixth in the annual series, continues to serve as a premier forum for the latest research and developments in building science gateways and related cyberinfrastructure. The call for papers attracted 16 submissions, of which 9 were accepted. The program committee chose Nancy Wilkins-Diehr's retrospective paper on the TeraGrid Science Gateway program as the keynote presentation. Other submissions cover gateway development aspects such as distributed security based on modern Web protocols, open community software development for gateways, modern gateway component development techniques, new underlying technologies to support real time processing, computing cloud-based gateway infrastructure, and applicationn gateways for bioinformatics and astrophysics.

Securing Science Gateways

Zenodo (CERN European Organization for Nuclear Research), Sep 17, 2022

Gateways provide access to computational codes as a service to their users. One popular design pa... more Gateways provide access to computational codes as a service to their users. One popular design pattern is to use a single user account on an HPC system to run all workloads for the gateway's user community. This introduces several security concerns for the owner of the user account, the machine, and the integrity of the workloads run by the gateway. In this paper we present preliminary work on pieshell, a limited, secure Linux shell that runs in user space. We begin by discussing the use case pieshell addresses. Next we discuss how it fills a gap in the existing secure shell landscape. We then describe pieshell's design and usage within a gateway context before concluding with areas of future work.

Download

From Proposal to Production: Lessons Learned Developing the Computational Chemistry Grid Cyberinfrastructure

Journal of Grid Computing, Jun 1, 2006

The Computational Chemistry Grid (CCG) is a 3-year, National Middleware Initiative (NMI) program ... more The Computational Chemistry Grid (CCG) is a 3-year, National Middleware Initiative (NMI) program to develop cyberinfrastructure for the chemistry community. CCG is led by the University of Kentucky, and involves collaborating sites at Louisiana State University, Ohio Supercomputing Center, Texas Advanced Computing Center, and the National Center for Supercomputing Applications. In this paper we discuss our experience developing the CCG cyberinfrastructure in the first year of the project. We pay special attention to sociological as well as technical issues we faced, and look forward to challenges we foresee in the remaining two years. Computational chemistry algorithms and software are now widely used across a broad range of disciplines, including nanotechnology, biotechnology, and material science. Around the world users of both commercial and academic chemistry software packages, such as Gaussian, GAMESS, MolPro and NWChem, are major users of both high and middle end compute resources. CCG will leverage existing established Grid middleware to provide an easy-to-use integrated computing environment for these and other chemistry applications, including allocations on computing resources. CCG is led by the University of Kentucky (UKy), and involves collaborating sites at Louisiana State University (LSU), Ohio Supercomputing Center (OSC), Texas Advanced Computing Center (TACC), and the National Center for Supercomputing Applications (NCSA). In this paper we discuss initial experiences developing the CCG cyberinfrastructure through the first year of the project and look ahead to challenges in the remaining two years. This paper discusses the important issues faced, both technological and sociological, as well as the chosen solutions.

Download

Dynamically Provisioning Portable Gateway Infrastructure Using Docker and Agave

ABSTRACT The iPlant Agave Developer APIs are a Science-as-a-Service platform for developing moder... more ABSTRACT The iPlant Agave Developer APIs are a Science-as-a-Service platform for developing modern science gateways. One trend we see emerging from our users is the aggregation of many different, distributed compute and storage systems. The rise in popularity in IaaS, PaaS, and container technologies has made the rapid deployment of elastic gateway infrastructure a reality. In this talk we will introduce Docker and the Agave Developer APIs then demonstrate how to use them to provision applications and infrastructure that are portable across any Linux hosting environment. We will conclude by using our lightweight gateway technology, GatewayDNA, to run an application and move data across multiple systems simultaneously.

Distributed Systems of Microservices Using Docker and Serfnode

We review container technology and the challenge of service discovery in microservice architectur... more We review container technology and the challenge of service discovery in microservice architectures and introduce Serfnode, a fully decentralized open source solution to the service discovery problem, based on the Serf project. Serfnode is a nonintrusive Docker image that composes one or more arbitrary Docker containers. The new images can be deployed into a cluster of Serfnodes, where it advertises itself and provides service discovery mechanisms, monitoring, and self-healing. The resulting cluster is a homogeneous and complete graph, with no master node. We survey existing solutions to the service discovery problem and compare them to Serfnode. As an example of the extensibility of Serfnode, we show the construction of a file system synchronization solution between Docker containers using Git.

Download

Endofday: A Container Workflow Engine for Scalable, Reproducible Computation

Container technologies such as Docker [1] are transforming the way distributed systems are deploy... more Container technologies such as Docker [1] are transforming the way distributed systems are deployed onto cloud platforms by providing a simple mechanism for packaging and isolating an application and its dependencies from the host machine on which it is running. The same ideas and technologies can be applied to computational science applications to obtain exceptional ease of installation and reproducibility of results. In this paper, we introduce endofday [2], a workflow engine that orchestrates a directed acyclic graph (DAG) of computational science apps where the nodes of the DAG are Docker containers. The endofday engine enables users to execute entire workflows of science applications without actually installing any of the applications themselves. As an example, we present the Validate [3] system, a suite of software applications for testing the accuracy and precision of Genome Wide Association methods, and illustrate how it can be run using endofday with zero installation. We also show how endofday integrates with the Agave [4] platform's application catalog and compare running Docker containers on cloud systems to running traditional applications on systems like Stampede.

Download

Experiences Migrating the Agave Platform to A Kubernetes Native System on the Jetstream Cloud

Practice and Experience in Advanced Research Computing, 2021

The Agave Platform is a Science-as-a-Service (ScaaS) platform for reproducible science. The curre... more The Agave Platform is a Science-as-a-Service (ScaaS) platform for reproducible science. The current production deployments of the platform are deployed and managed using Ansible and Docker Compose. While capable, this has historically led to operational complexity for those adopting the platform. Over the last year, we have worked to migrate the platform to a Kubernetes native deployment. In this paper we discuss our experiences evolving the platform, its architecture, and getting the most out of the Jetstream cloud.

From Zero to Science-as-a-Service with the Agave Platform, Helm, and Kubernetes

Securing Science Gateways

Proceedings of the 2011 ACM workshop on Gateway computing environments - GCE '11

From Zero to Science-as-a-Service with the Agave Platform, Helm, and Kubernetes

OSF, Oct 6, 2020

Lightning Talk - The Agave Platform: An Open Science-As-A-Service Cloud Platform for Reproducible Science

The Agave Platform (Agave) is an open source, standards-based platform delivering Science-as-a-Se... more The Agave Platform (Agave) is an open source, standards-based platform delivering Science-as-a-Service to the open science community. Agave uses standards-based technologies and community promoted best practices to enable users to run code, manage data, collaborate meaningfully, and integrate anywhere.Chances are, Agave already works with the academic and commercial infrastructure you are already using, in the languages and frameworks you love, with zero installation required.

Tacc/Abaco: Zenodo Release

Actor Based Co(mputing)ntainers

Reactive, Interactive, and High Throughput Computation in the Agave Platform

The Agave Platform is an open Science-as-a-Service platform that empowers users to run code, mana... more The Agave Platform is an open Science-as-a-Service platform that empowers users to run code, manage data, collaborate meaningfully, and integrate easily with the world around them. In this extended abstract, we examine how new use cases have helped evolve the ways in which Agave supports code execution. We start out by identifying three new code execution paradigms identified in the past 2 years: Interactive, Reactive, and High Throughput computing. We then briefly describe the primary use cases driving each paradigm and how the platform evolved in response to them. Finally, we highlight areas of future work in the platform related to code execution.

Download

Adapting Legacy Monolithic Applications into API Services for Web, Mobile and Other Clients

Download

A distributed quadtree dictionary approach to multi-resolution compression

Download

Recipes 2.0: building for today and tomorrow

Concurrency and Computation: Practice and Experience, Apr 29, 2014

Download

Trusted CI Webinar: Best Practices for Academic Cloud Service Providers with Rion Dooley