A Scalable, Open Source Platform for Data Processing, Archiving and Dissemination


End users have a lot of data, but do not have the expertise needed to analyze it. The goal of this project is to empower end users to analyze big data by demonstrating that: 1) data analytics experts can use open source software to quickly assemble workflows, 2) end users can easily run these expert-grade workflows and get useful views on their data. Our work combines semantic workflow capabilities of the WINGS workflow system with scalable data systems and workflow execution infrastructure available in the OODT framework. Our work includes a release of the integrated system as open source software within the Apache OODT project.



An important need for users of big data systems is to get an answer within a specified timebound. A current limitation of workflow systems is that they do not take into account user deadlines: they run workflows selected by the user, but take their time to do so. This is impractical when large datasets are at stake, since users often prefer to see an answer faster even if it has lower precision or quality. We recently developed WOOT, an extension to workflow systems that enables them to take into account user deadlines by automatically generating alternative workflow candidates and ranking them according to performance estimates. The WOOT system makes these estimates based on workflow performance models created from workflow executions, and uses semantic technologies to reason about workflow options. Possible workflow candidates are presented to the user in a compact manner, and are ranked according to their runtime estimates. WOOT combines and extends capabilities from the WINGS semantic workflow system and the Apache OODT Object Oriented Data Technology and workflow execution system.



More -->


More publications -->


See a demo -->


Group Members:

More -->


  • US Air Force Office of Scientific Research (AFOSR)

More -->


More links -->

<< Back to IKCAP