University of Southern California

Pedro Szekely



Current Projects


Publishing Structured Sources to the Semantic Web

Karma (info, sw) can import data from a variety of structured souces (databases, spreadsheets, CSV, XML, JSON, KML and soon RDF), map it to an ontology or vocabulary of the user's choice, and then publish the data as RDF according to the chosen ontology. Karma significantly reduces the work to define the required mappings by automatically inferring the type of the data and the relationships among the different data elements. Users can easily refine the automatically generated mapping using menus to resolve ambiguities that require user judgement or to correct incorrectly inferred types and relationships. Karma learns from these interactions to improve its ability to generate correct mappings for new data sets containing similar information. Evaluations show that less than one user action per column is needed to produce correct mappings. Papers: ISWC Linked Science Workshop'2011, SSO'2011. Students: Aman Goel, Mohsen Taheriyan. Collaborators: Craig Knoblock, Jose Luis Ambite. More info.


Data Preparation for Semantic Workflows

Scientific metadata containing semantic descriptions of scientific data is expensive to capture and is typically not used across entire data analytic processes. We present an approach based on Karma (info, sw) and WINGS where semantic metadata is generated as scientific data is being prepared, and then subsequently used to configure models and to customize them to the data. The metadata captured includes sensor descriptions, data characteristics, data types, and process documentation. This metadata is then used in a workflow system to select analytic models dynamically and to set up model parameters automatically. In addition, all aspects of data processing are documented, and the system is able to generate extensive provenance records for new data products based on the metadata. Papers: ISWC'2011, Poster: AGU'2011. Collaborators: Craig Knoblock, Yolanda Gil, Tom Harmon.


Enriching Sensor Data with Context Information for Use in 3D Game Environments

We are extending Karma (info, sw) to enrich sensor data with information that enables users to understand the context in which the sensor data was collected. With Karma, users first extract from open sources a variety of information for the region of interest: business information from online phone books and directories, weather, news, events, and road vectors from raster maps. Once information is extracted, Karma helps users geolocate it, and then integrate it and associate it with sensors and buildings. Karma then exports all the information in KML layers, so that the Cosmopolis game engine can load it and show it in the 3D environment. Collaborators: Craig Knoblock, Mike Zyda.


Geospatial Analysis of Motion-Based Intelligence and Tracking: GAMBIT

Movement data can be combined with geospatial information and transformed into probabilistic graphical models that represent both social and temporal relationships between objects in the observed area. We then apply machine-learning techniques to cluster patterns in these graphical models to assist human users in performing strategic level analysis such as behavior prediction and anomaly detection. We are extending Karma (info, sw) to enrich the track data with information that provides the features that the analysis algorithms need to detect patterns of life in the track data and enables users to understand the context of the data. Collaborators: Craig Knoblock, Yu-Han Chang, Rajiv Maheswaran.


Data Normalization and Cleaning

Data normalization involves transforming data sets to make more regular and compatible with other data sets. Examples of such transformations include changing the format of dates, removing extraneous HTML fragments, transposing names, decomposing addresses, etc. We are investigating a by-example approach where users provide one or two examples of transformed data and the system infers a general procedure for transforming the whole data set. Student: Bo Wu. Collaborator: Craig Knoblock.


Mashup Construction: Karma

Karma uses a programming-by-demonstration approach that enables users to perform complex information integration tasks without requiring any knowledge of programming or widgets. Users perform information integration tasks by simply demonstrating them on one or more concrete examples through its user-interface. Karma is now available as open source at GitHub. More information about the older Java version of Karma is available at the project web site, Movies: earthquake relief example, bioinformatics example, Papers: TWEB'2011, IUI'2008, IUI'2007. First student: Rattapoom Tuchninda (graduated). Karma spawned several research threads being pursued by other students. These threads include Data Normalization and Cleaning, Enriching Sensor Data, Publishing Structured Sources in the Semantic Web, and are listed as separate entries in this page. The capabilities enabled in these threads are being integrated into the open source Karma system. Collaborators: Craig Knoblock, Jose Luis Ambite.


Plan Analysis using Stochastic Simulation: COMPASS

COMPASS is an interactive real-time tool that analyzes schedule uncertainty for a stochastic task network. An important feature is that it concurrently calculates stochastic critical paths and critical tasks. COMPASS visualizes this information on top of a traditional Gantt view, giving users insight into how delays caused by uncertain durations propagate down the schedule. Users use sliders to adjust the distribution of the duration of any set of activities and see in real-time the effects on the start and end times of activities, and the critical paths they give rise to. Evaluations with 10 users show that users can use \compass to answer a variety of questions about the possible evolutions of a schedule (e.g., what is the likelihood that all activities will complete before a given date?). Movie: COMPASS features, Paper: IUI'2012. Students: Yan Wang, Huihui Chen, Karan Singh. Collaborators: Rajiv Maheswaran, Yu-Han Chang


Completed Projects

Execution View
Agent View
Histogram View

Visualization Tools: VizScript

VizScript is a generic framework that expedites the process of creating visualizations to debug and under- stand complex multi-agent systems. VizScript combines a generic application instrumentation, a knowledge-base, and simple scene definitions primitives with a reasoning system, to produce an easy to use visualization platform. Using VizScript, users are able to recreate the visualizations for a complex multi- agent system with an order-of-magnitude less effort than was required in a Java implementation. Papers: IUI'2008, IUI'2008. Student: Jin Jing (graduated), Collaborators: Rajiv Maheswaran, Romeo Sanchez.


Visual Analytics: VizPattern

VizPattern is an interactive visual query environment that uses a comic strip metaphor to enable users to easily and quickly define and locate complex temporal patterns. Evalu- ations provide evidence that VizPattern is applicable in many domains, and that it enables a wide variety of users to answer questions about temporal data faster and with fewer errors than existing state-of-the-art visual analysis systems. Movie, Papers: VL/HCC'2009, VAST'2010. Student: Jin Jing (graduated).

Living Classroom

K-12 Education: Living Classroom

The Living Classroom project seeks to offer children unique, differentiated learning experiences that reflect their specific needs – and to do so with the affordability and ease of use that brings premiere teaching within the reach of every American student.  Its goal: to provide teachers with the comprehensive, integrated information they need to customize each child's experiences readily every school day. Project web site

CSC Screenshot

Multi-Agent Systems: Criticality-Sensitive Coodination (CSC)

The objective is to create distributed intelligent software systems that will help fielded units adapt their mission plans as the situation around them changes and impacts their plans. Intelligent software Coordinators do this by reasoning about the tasks assigned to a given unit, the task timings, how the tasks interact with those of other units, and by evaluating possible changes such as changing task timings, task assignments, or selecting from pre-planned contingencies. Movies: AAMAS/ICAPS 2010 demo, system, Papers: ICAPS'2005 (wkshp), AAAI SS'2006, AAMAS'2006, AAMAS MSDM'2006, AAMAS LSMAS'2006, AAAI'2007, AAMAS'2008, ICAPS'2008, AAMAS'2009, ICAPS PSUU'2010, AAMAS OPTMAS'2010, AAMAS'2010 (best demo), AAMAS CHACIE'2010, AAMAS ARDE'2010, PRIMA'2011


Distributed Control Algorithms: LANdroids

The objective is to develop intelligent autonomous radio relay nodes that exploit movement to establish and manage mesh networks in urban settings using small, inexpensive, smart robotic radio relay nodes. As the situation changes, the nodes will adapt the network, self-healing if nodes are destroyed, stretching if soldiers move. Through movement and density, the LANdroids will enable effective communications in complex non-line-of-sight environments. Our bio-inspired, distributed control algorithm called TENTACLES directs robots' exploration to grow tentacles starting from soldiers of the gateway, establishing links when tentacles meet. Tentacles are disolved when they fail to meet other tentacles. Simulation movies 1, 2, 3. Paper: IROS'2009

Commander's Coordinator Master Schedule

Human in the Loop Planning and Scheduling: Commander's Coordinator

The Commander's Coordinator retrieves current status from the units in the field and assembles and integrated plan overview that shows Commanders who is doing what and where. It monitors plan execution predicting possible failures, presents options for Commanders to choose from, intelligently alerts affected units about possible ripple effects, and distributes plan modifications to repair plans. Movie

SNAP Main Screen

Planning and Scheduling: Schedules Negotiated by Agent-based Planners (SNAP)

The objective was to build a human-in-the-loop scheduling system that enables operators to state scheduling goals and that helps them refine their goals and the schedule to produce schedules that balance many conflicting goals. The system was implemented in the context of US Marines aviation, and was used to help operators produce weekly and daily flight schedules. The system was deployed for testing in the US Marines air base in Yuma Arizona and onboard ships operating in the Middle East. More details available from the Final Report to DARPA, ANTS ebook and the project web site.


End-User Programming: DEALMAKER

The objective was to build a system to enable end-users to specify rules for specifying the best contracts for fullfilling orders in a purchasing system. The system receives requisitions for items in electronic form, queries a database of preferred vendors to determine possible sources of supply, and applies rules to filter and rank these sources according to policies established by contract managers. The system was developed and tested in the context of applications for the Defense Logistics Agency (DLA). Project final documentation.



Project Leader
USC Information Sciences Institute

Research Assistant Professor
USC Computer Science Department

+1 310.448.8641
pszekely at

4676 Admiralty Way
Marina del Rey
CA 90292

See when I am free for meetings: Calendar

December 2012: Published press release about our Smithsonian project in USC.

September 2012: Karma paper on modeling services accepted for ISWC in Boston.

August 2012: Presented work on applying Karma to populate VIVO datasets at the VIVO conference in Miami, Fl.

August 2012: Karma now has its own web page.

January 2012: Teaching at USC the Information Integration on the Web (CSCI548) with Craig Knoblock and Jose Luis Ambite.

November 2011: attended PRIMA in Australia to present our COORDINATORS paper related to the field exercise.

October 2011: attended ISWC in Bonn to present our Karma paper.

May 2011: gave invited talk at the 40th anniversary of the computer science department in the Universidad de los Andes in Bogota, Colombia slides.

March 2011: gave talk at CMU about our COORDINATORS work.

March 2011: co-organized AAAI Spring Symposium titled
Help Me Help You: Bridging the Gaps in Human-Agent Collaboration.

Sept 2010: Rajiv Maheswaran and I win award on DARPA OBTW program.

May 2010: our CSC demo wins best demonstration award at AAMAS 2010 in Toronto.

April 2010: my student Jing Jin wins award for most creative Ph.D. dissertation in the USC Viterbi School of Engineering.

April 2010: transferred within ISI to the AI division working in Craig Knoblock's group.