Abstract

In this paper, we describe how to leverage cloud resources to generate large-scale mosaics of the galactic plane in multiple wavelengths. Our goal is to generate a 16-wavelength infrared Atlas of the Galactic Plane at a common spatial sampling of 1 arcsec, processed so that they appear to have been measured with a single instrument. This will be achieved by using the Montage image mosaic engine process observations from the 2MASS, GLIMPSE, MIPSGAL, MSX and WISE datasets, over a wavelength range of 1/m to 24/m, and by using the Pegasus Workflow Management System for managing the workload. When complete, the Atlas will be made available to the community as a data product.
We are generating images that cover±180◦ in Galactic longitude and±20◦ in Galactic latitude, to the extent permitted by the spatial coverage of each dataset. Each image will be 5◦ x5◦ in size (including an overlap of 1◦ with neighboring tiles), resulting in an atlas of 1,001 images. The final size will be about 50 TBs. This paper will mainly focus on the computational challenges, solutions and lessons learned in producing the Atlas. To manage the computation we are using the Pegasus Workflow Management System, a mature, highly fault-tolerant system now in release 4.2. 2 that has found wide applicability across many science disciplines. A scientific workflow describes the dependencies between the tasks and in most cases the workflow is described as a directed acyclic graph, where the nodes are tasks and the edges denote the task dependencies. A defining property for a scientific workflow is that it manages data flow between tasks. Applied to the …

Date

2013

Authors

Mats Rynge, Gideon Juve, Jamie Kinney, John Good, Bruce Berriman, Ann Merrihew, Ewa Deelman

Journal

23rd Annual Astronomical Data Analysis Software and Systems, ADASS, Conference