Projects

I am currently working on the following projects:

In 2006 co-chaired the NSF-funded Workshop on Challenges of Scientific Workflows, http://vtcpc.isi.edu/wiki

Past projects:

Pegasus and LIGO
National Science Foundation
February 2007-September 2007
Ewa Deelman, PI
Supporting Laser Gravitational-Wave Observatory applications on the Open Science Grid. More about this effort can be found at http://pegasus.isi.edu and http://pegasus.isi.edu/applications.php

Windward
Air Force Research Laboratory (AFRL).
Grant number FA8750-06-C-0210. September 2006 - September 2010.
Yolanda Gil (PI). ISI co-PIs: Paul Cohen and Ewa Deelman.

Distributed workflows are emerging as a key technology to conduct large-scale and large-scope scientific applications in earthquake science, physics, astronomy, and many other sciences.  In this new project, we will investigate the use of workflow technologies for Artificial Intelligence applications with a particular focus on data analysis and knowledge discovery tasks. 

Based on the data to be analyzed, an initial workflow template is formed by selecting from a library of  known-to-work compositions of general-purpose machine learning algorithms.  The workflow template is specialized through knowledge-based selection and configuration of algorithms.  Finally, the workflow is mapped to available resources and restructured to improve execution time.  Data analysis and knowledge discovery applications will benefit from the automation, scale, and distributed data and resource integration supported by distributed workflow systems. 

We will also conduct new research in important aspects of workflow systems.  To what extent can we represent complex algorithms and their subtle differences so that they can be automatically selected and configured to satisfy the stated application requirements?  Can we develop learning techniques that improve the performance of the workflow system by exploiting an episodic memory of prior workflow executions?  What mechanisms will be needed to support autonomous and robust execution of concurrent workflows over continuously changing data?

Intelligent Optimization of Parallel and Distributed Applications
National Science Foundation (NSF).
Grant number CSR-0615412. August 2006 ­ September 2009.
Principal Investigators: Mary Hall (PI), Kristina Lerman (co-PI), Ewa Deelman (co-PI), Aichiro Nakano (co-PI), Joel Saltz (co-PI). ISI co-PIs: Yolanda Gil.

This project will develop a domain-specific programming system supporting Petascale application optimization of molecular dynamics simulation, in which applications will be viewed as workflows consisting of composable components to be mapped to a diversity of machine resources. The application components will be viewed as dynamically adaptive algorithms for which there exist a set of variants and parameters that can be chosen to develop an optimized implementation. A variant describes a distinct implementation of a code segment, perhaps even a different algorithm. A paramater is an unbound variable that affects application performance. By encoding an application in this way, we can capture a large set of possible application mappings with a very compact representation. Because the space of mappings is prohibitively large, the system captures and utilizes domain knowledge from the domain scientists and designers of the compiler, run-time and performance models to prune most of the possible implementation. Knowledge representation and machine learning techniques utilize this domain knowledge and past experience to navigate the search space efficiently. Incorporating cognitive search techniques and taking advantage of parallel resources, these alternative implementations are searched automatically by tools to find a high-quality implementation.
 

Cyberinfrastructure in Support of Research: A New Imperative (NCSA)
National Science Foundation
Grant Number OCI-0438712, September 2006-June 2008

As part of this work, I extended and improved metadata services, in particular the Metadata Catalog Service (MCS).

Southern California Earthquake Center
Community Modeling Environment

This project focuses in part on bringing workflow technologies to earthquake science applications. One of the most important applications is CyberShake. More information about CyberShake can be found at http://pegasus.isi.edu/applications.php and relevant papers can be found at: www.isi.edu/~deelman/papers.htm

NSF Workshop on Challenges of Scientific Workflows
National Science Foundation (NSF).  
Grant number IIS-0629361. May 2006 - October 2007.
Yolanda Gil (PI) and Ewa Deelman (co-PI).

In recent years, workflows have emerged as a paradigm for conducting large-scale scientific analyses. The structure of a workflow specifies what analysis routines need to be executed, the data flow amongst them, and relevant execution details. Workflows provide a systematic way to capture scientific methodology and provide provenance information for their results. Robust and flexible workflow creation, mapping, and execution are largely open research problems. Under this project, Ewa Deelman and Yolanda Gil chaired an invitation-only workshop on "Challenges of Scientific Workflows" at the National Science Foundation. The aim of this workshop was to bring together IT researchers and practitioners working on a variety of aspects of workflow management as well as domain scientists that use workflows for day-to-day data analysis and simulation. The National Science Foundation expects a final report with recommendations to the community regarding the challenges of scientific workflows and their role in cyber infrastructure planning for 21st century science and engineering research and education.

Towards Cognitive Grids : Knowledge-Rich Grid Services for Autonomous Workflow Refinement and Robust Execution.
National Science Foundation (NSF) Shared Cyberinfrastructure program.
Grant number SCI-0455361. December 2004 - November 2006.
Ewa Deelman (PI), Yolanda Gil (co-PI).

This research combines Artificial Intelligence and Distributed Computing techniques to create knowledge-rich workflow services that can support the execution of large-scale scientific workflows. The main foundation will be provided by expressive formal representations of the application workflow and of the execution environment. These representations will support resource selection that will enhance application performance, resource reservation based on anticipated workflow needs, workflow repair capabilities in case of failures or in case of new resources coming on line.

NVO

As part of this work, I worked with the Montage scientists to fully parallelize the application using the workflow paradigm and today Pegasus is used to generate science-grade mosaics of the sky. This work resulted in publication in both computer science and astronomy venues. Publications can be found at www.isi.edu/~deelman/papers.htm

GriPhyN

In GriPhyN I lead the effort to develop the methodology and tools to support the “virtual data” paradigm, where the scientist would ask for a particular data product and obtain the desired results. If the data was not yet instantiated, it would produce the data on the fly. The Pegasus software was developed as part of this effort and used by LIGO for gravitational-wave analyses. 

Many papers can be found at www.isi.edu/~deelman/papers.htm.

" The GriPhyN (Grid Physics Network) collaboration is a team of experimental physicists and information technology (IT) researchers who plan to implement the first Petabyte-scale computational environments for data intensive science in the 21st century. Driving the project are unprecedented requirements for geographically dispersed extraction of complex scientific information from very large collections of measured data. To meet these requirements, which arise initially from the four physics experiments involved in this project but will also be fundamental to science and commerce in the 21st century, GriPhyN will deploy computational environments called Petascale Virtual Data Grids (PVDGs) that meet the data-intensive computational needs of a diverse community of thousands of scientists spread across the globe.

Our team is composed of IT research groups and members of four NSF-funded frontier physics experiments. Our integrated research effort provides the coordination and tight feedback from prototypes and tests that will enable both communities to meet their goals. The four physics experiments are about to enter a new era of exploration of the fundamental forces of nature and the structure of the universe. The CMS and ATLAS experiments at the Large Hadron Collider (LHC) at CERN will search for the origins of mass and probe matter at the smallest length scales; LIGO (Laser Interferometer Gravitational-wave Observatory) will detect the gravitational waves of pulsars, supernovae and in-spiraling binary stars; and SDSS (Sloan Digital Sky Survey) will carry out an automated sky survey enabling systematic studies of stars, galaxies, nebula, and large-scale structure.

The data analysis for these experiments presents enormous IT challenges. Communities of thousands of scientists, distributed globally and served by networks of varying bandwidths, need to extract small signals from enormous backgrounds via computationally demanding analyses of datasets that will grow from the 100 Terabyte to the 100 Petabyte scale over the next decade. The computing and storage resources required will be distributed, for both technical and strategic reasons, across national centers, regional centers, university computing centers, and individual desktops. The scale of this task, far outpaces our current ability to manage and process data in a distributed environment. The GriPhyN collaboration proposes to carry out the necessary computer science and validate the concepts through a series of staged deployments, ultimately resulting in a set of production Data Grids. "

iVDGL

"The iVDGL is a global Data Grid that will serve forefront experiments in physics and astronomy. Its computing, storage and networking resources in the U.S., Europe, Asia and South America provide a unique laboratory that will test and validate Grid technologies at international and global scales. Sites in Europe and the U.S. will be linked by a multi-gigabit per second transatlantic link "

As part of iVDGL Pegasus was then transitioned into other NSF-funded projects and is regularly released and supported as part of iVDGL’s Virtual Data Toolkit (VDT).

CRCNS:  Assembling Visible Neurons for Simulation
NIH-funded, grant number 5 R01 NS046068-03

This project continues to advance efforts to enhance and accelerate the process of very large-field 3D laser-scanning light microscopy to increase the throughput of generating multi-resolution “visible” cells for computational neuroscience. For this workflow process, key data, computation, time, and labor intensive tasks are being identified and solutions for increasing end-to-end performance, accelerating computation, enhancing visualization, and interfacing federated databases are being explored.

Globus

The Globus project is developing fundamental technologies needed to build computational grids. Grids are persistent environments that enable software applications to integrate instruments, displays, computational and information resources that are managed by diverse organizations in widespread locations.

Gridlab

NSF Next Generations Systems Project

This project will extend and adapt POEMS technology from model-based analysis of adaptive parallel and distributed systems to model-based control of adaptive parallel and distributed systems. This one-year project will involve a manually executed exploration and feasibility demonstration of model-based control of a distributed/parallel implementation of an adaptive application. This feasibility demonstration will bring to the surface further research problems and technology requirements for model-based control. It will explore the conditions under which basing control on more comprehensive system models will have benefit. Model-based control can address issues of variability of resource availability as well as issues of variability in resource requirements engendered by adaptive algorithms. Variability of resource availability as well as variability of resource demand will be studied in this project.

POEMS: Performance Modeling of Parallel Systems

The POEMS project will create and demonstrate a capability for prediction of the end-to-end performance of parallel/distributed implementations of large scale adaptive applications. POEMS modeling capability will span applications, operating systems including parallel I/O, and architecture. Effort will focus on the areas where there is little convention wisdom such execution behaviors of adaptive algorithms on multi-level memory hierarchies and parallel I/O operations.

SESAME: System Software Measurement and Evaluation

The goal of this research is to identify major performance bottlenecks in supercomputer system software. This will be achieved by a coordinated measurement, modeling, simulation, architectural evaluation, and experimental alteration effort, taking a global view of the overall systems software environment.

Parsec: Parallel Simulation Language

Parsec is a C-based simulation language, developed by the Parallel Computing Laboratory at UCLA, for sequential and parallel execution of discrete-event simulation models. It can also be used as a parallel programming language.