Workflow INstance Generation and Selection











WINGS was initially developed to create large-scale workflows for physics-based simulations to construct seismic hazard maps for the Southern California Earthquake Center. It was used in combination with Pegasus, with WINGS providing automatic workflow validation and generation and Pegasus assigning resources and managing the execution.

The picture below shows the high-level workflow template created for this application, which represents very compactly the parallel processing of data collections.

Rectangles indicate datasets, ovals indicate computations (or workflow components). Light blue signifies single datasets, darker blue and dark green indicate collections of data. Ovals with double red outline mean collections of computations of the same kind performed over a data collection, which are to be expanded by WINGS when the workflow template is applied to selected datasets.

For that application, WINGS generated a workflow with more than eight thousand computations that were mapped by Pegasus into more than twenty four thousand jobs. WINGS also generated more than 2M triples of metadata attributes for 100,000 new workflow data products.

WINGS (Workflow INstance Generation and Specialization) uses artificial intelligence planning and semantic reasoners to assist users in creating workflows while validating that the workflows comply with the requirements of the software components and datasets. WINGS can reason about the constraints of the components and the characteristics of the data and propagate them through the workflow structure.

Once the workflow components and datasets are selected, Pegasus automatically maps the workflow to the available computing resources and manages its execution on local resources as well as in distributed and high-performance cyberinfrastructure resources such as the Open Science Grid and the TeraGrid as well as clouds. Detailed records of how new data products were generated by the system are captured in a provenance catalog, which enables easy reproducibility of results.

The WINGS/Pegasus workflow system is built on open web standards from the World Wide Web Consortium (W3C) such as the Web Ontology Language (OWL) and the Semantic Wed Rule Language (SWRL) as well as NSF National Middleware Infrastructure (NMI) software such as Condor and Globus.

<< Back to IKCAP