Abstract

As the scientific community prepares for extreme-scale computing, Big Data analytics are becoming an essential part of the scientific process enabling new insights and discoveries. This poster describes how we utilized workflow ensembles to model the next generation of computational workflows, where a long-running simulation job periodically generates data that needs to be analyzed concurrently on high-performance computing resources. In this work, we developed extensions to the open source Pegasus Ensemble Manager service to enable support for event-based triggering that can be used to add workflows to an existing running ensemble.

Date

January 1, 1970

Authors

Suraj Pandey, Karan Vahi, Rafael Ferreira da Silva, Ewa Deelman, Ming Jiang, Cyrus Harrison, Al Chu, Henri Casanova

Journal

HPC Asia