Information Sciences Institute

The beginning of 2020 was the hottest January in recorded history, according to the National Oceanic and Atmospheric Administration (NOAA).

Public views on climate change continue to lean toward calls to action, as new research about the damages of climate change persists and people continue to live through severe weather events around the globe. In some cases, weather scientists are dealing with highly unusual weather events that are extremely difficult to forecast.

So how can weather scientists better inform the public about strange or dangerous weather? Computer scientists from around the U.S. have just the solution.

In a joint collaboration between the USC Information Sciences Institute (ISI), the Renaissance Computing Institute (RENCI) at the University of North Carolina (UNC) at Chapel Hill, the University of Massachusetts (UMass) Amherst, and Rutgers University, researchers developed the Dynamic Network-Centric Multi-Cloud Platform, or DyNamo, a cyberinfrastructure platform that can support real-time weather forecasting workflows to help weather scientists improve their predictions.

The team participated in SCinet’s inaugural Technology Challenge this past November, which focused on “demonstrating that both networking and high performance computing resources are essential elements of the cyber infrastructure required to advance modern data-driven scientific applications.” After presenting a live demonstration of the platform, the team was awarded prizes for Most Diverse Resources Set and Most Original Technical Approach, pointing to the broad utility of the platform.

“As climate change continues to intensify, we’ll have more severe weather events, so the ability to predict them in a timely manner is critical from the point of view of public safety,” said Ewa Deelman, research professor of computer science, principal scientist and lead of the Pegasus project at ISI, and co-PI of the DyNamo project. “DyNamo allows you to dynamically provision cloud resources and networks to support time-critical workflows, such as those that model tornadoes and other severe weather events.”

The system provides scientists with the tools they need to move fast in analyzing developing and rapidly changing weather patterns. “The DyNamo platform makes it easier for weather scientists to leverage advanced networking and computing capabilities to run their weather prediction models and to move weather data,” explained Anirban Mandal, PI of the DyNamo project and assistant director of Network Research & Infrastructure at RENCI. “It helps them expeditiously provision the required resources to analyze the weather data in real-time for timely response to quickly developing weather events.”

The platform also incorporated aspects from Panorama 360, a separate project of the Pegasus group at ISI and RENCI, which is funded by the U.S. Department of Energy. “Panorama’s goal is to provide a way to systematically collect data from workflow execution and provide intuition to the users on their workflows performance,” said George Papadimitriou, a graduate research assistant at ISI. “In DyNamo, we incorporated modules developed under the Panorama Project to provide real-time workflow monitoring capabilities to DyNamo.”

Rounding out the DyNamo team are  co-PIs Michael Zink, associate professor in the Department of Electrical and Computer Engineering at UMass Amherst and deputy director for technical integration and distributing thrust leader at the Collaborative Adaptive Sensing of the Atmosphere (CASA), and Ivan Rodero, associate director and associate research professor at the Rutgers Discovery Informatics Institute.

Together, the team developed the DyNamo platform, which enables weather-related data to flow across a group of cloud resources. The platform uses advanced infrastructures to analyze live weather data coming in from radars to help weather scientists develop a timely response to weather events, improving the performance and efficiency of their forecasting workflows.

The DyNamo platform can provide substantial assistance to organizations such as CASA, which focuses on observing, understanding, predicting and responding to weather events. The platform helps enhance weather forecasting computations but isn’t limited in its reach. “DyNamo can be used for a range of applications, any application that has dynamic resource needs and that accesses large amounts of data from remote locations,” Deelman noted.

At SCinet’s Technology Challenge, the DyNamo platform was demonstrated live for the audience. “Participating in the SCinet tech challenge was a very demanding and stressing experience. Although Dynamo is used in production by CASA, when you are trying to showcase its latest features live, things can go wrong,” remarked Papadimitriou. “However, the outcome was extremely rewarding!”

Thanks to the effectiveness of the platform, the DyNamo team won the Most Diverse Resources Set award, “for the highest diversity of resource types and geographic distribution, their utilization, and degree of orchestration and automation.” The team also received the Most Original Technical Approach recognition “for excellence in innovation and originality of the technical demonstration.”

“The Tech Challenge award for using the most diverse resource set was a recognition and affirmation of our approach to federation and dynamic integration of a varied set of resources for supporting distributed workflows,” said Mandal. “The award for the most innovative approach has inspired us to develop several new approaches to workflow provisioning and resource management in support of some novel use cases during the current year.”

The SCinet Tech Challenge awards helped emphasize the advantages and utility of the DyNamo platform, and the research team is now looking ahead to enhance the system even more. “The two awards that DyNamo received definitely highlight the benefits and the impact that this project can have to the scientific community, looking to better the way we provision resources and incorporate distributed datasets into science workflows,” said Papadimitriou. “Now we look forward to improve and extend the DyNamo framework, including more cloud computing providers and introduce more advanced capabilities to the system.”

This comprises incorporating features to assist researchers in other scientific fields, as well as developing capabilities to work with data streamed from analytical scientific instruments. “Our future plans include supporting scientists from other observational science areas, such as ocean sciences, and applying our technologies to more use cases that deal with streaming data from scientific instruments,” Mandal said. “We also plan to expand the capabilities of the DyNamo platform for better usability, additional resource integration and dynamic monitoring and control of the infrastructure.”

Published on March 23rd, 2020

Last updated on July 1st, 2021