Instructor: Craig Knoblock
Meeting Time: Tuesdays 3.30 - 6:20pm
Location: THH 212
Office Hours:
    Tuesdays 6.20pm (immediately after class in the lobby of GFS)
    Or by appointment (ISI 922 or by phone)
    Wednesdays 3-4pm (ISI 922)
Teaching Assistant:

•  Anon Plangprasopchok
• Rattapoom Tuchinda (Pipe)

This course will focus on the basic foundations and techniques in Information Extraction and Integration. There has been a great deal of interest and research over the last few years on this topic and the course will cover the research and tools for addressing the technical problems. The topics covered will include structured data querying (Xquery), view integration techniques, machine learning techniques for turning web sites into structured data sources, high-performance query execution systems based on streaming dataflow, constraint-based integration systems, and approaches to resolving naming inconsistencies across sites. We will also cover the applications of information integration to geospatial data integration.

The class will be run as a lecture course with lots student participation and hands-on experience. As an integral part of the course each student will develop and build an integrated Web application using the research and tools covered in the class.

Course Syllabus and Schedule

• Lecture 1 (January 15)
• Topic: Introduction
• Introduction
• Overview of the course
• Course project
• Xquery (Dan Goldberg)
• Required Readings
• What is XQuery ?
  Per Bothner.
• XQuery: A Guided Tour
  Chapter 1 of XQuery from the Experts: A Guide to the W3C XML Query Language
  Howard Katz (ed), Addison-Wesley, San Francisco, CA, 2004
• Recommended Readings
• XQuery 1.0: An XML Query Language
• Slides
• Introduction
• Xquery


• Lecture 2 (January 22)
• Topic: Information Extraction
• Information Extraction Systems
• IE Tools (Matthew Michelson)
• Homework
• Assignment 1 Due (January 25 at midnight)
• Required Reading:
• Information Extraction: Distilling Structured Data from Unstructured Text,
  Andrew McCallum.
  ACM Queue, volume 3, Number 9, November 2005.
Semantic annotation of unstructured and ungrammatical text,
Matthew Michelson and Craig A. Knoblock.
In Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI-2005), Edinburgh, Scotland, 2005.
• Accurate Information Extraction from Research Papers using Conditional Random Fields.
  Fuchun Peng and Andrew McCallum.
  In Proceedings of Human Language Technology Conference and North American Chapter of the Association for Computational Linguistics (HLT-NAACL), 2004.
• Adaptive Information Extraction from Text by Rule Induction and Generalisation,
  Fabio Ciravegna,
  In Proceedings of 17th International Joint Conference on Artificial Intelligence (IJCAI 2001), Seattle, August 2001.
• Recommended Readings:
• Conditional Random Fields: An Introduction,
  Hanna M. Wallach,
  Technical Report MS-CIS-04-21. Dept. of Computer and Information Science, University of Pennsylvania, 2004
• Slides
  • Information Extraction
  • MALLET and CRFs


• Lecture 3 (January 29)
• Topic: Wrapper Generation
• Automatic Wrapper Generation (Prof. Kristina Lerman)
• Automatic wrapper tools (Rattapoom Tuchinda)
• Homework
• Assignment 2 Due (February 1 at midnight)
• Data Integration
• Required Readings:
• Using the Structure of Web Sites for Automatic Segmentation of Tables
  Kristina Lerman, Lise Getoor, Steven Minton and Craig Knoblock,
  in Proc. of SIGMOD-2004, Paris, France,2004.
• RoadRunner: Towards Automatic Data Extraction from Large Web Sites
  W. Crescenzi, G. Mecca, P. Merialdo,
  The VLDB Journal, 109-118, 2001.
•  Overview of AutoFeed: An Unsupervised Learning System for Generating Webfeeds,
  B. Cenk Gazen and Steven Minton,
  In Proceedings of AAAI, 2006
• Recommended Readings:
• Automatic information extraction from large websites.
  Crescenzi, V. and Mecca, G.
  J. ACM 51, 5 (Sep. 2004), 731-779, 2004.
• Piggy Bank:Experience the Semantic Web Inside Your Web Browser
  David Huynh, Stefano Mazzocchi, David Karger.
• Slides
• Wrapper Generation
• Extracting and Integrating Data


• Lecture 4 (February 5)
• Topic: Wrapper Learning
• Wrapper Learning and Maintenance
• Wrapper Learning Tools (Martin Michalowski)
• Homework
• Assignment 3 Due (February 8 at midnight)
• Required Readings
• Wrapper Maintenance,
  Kristina Lerman and Craig A. Knoblock,
  To appear in the Encylopedia of Database Systems. Springer, Leipzig, Germany 2008.
• A Hierarchical Approach to Wrapper Induction,
  Ion Muslea, Steve Minton, and Craig A. Knoblock.
  In Proceedings of the 3rd International Conference on Autonomous Agents 1999, Seattle, WA, 1999.
• Wrapper Induction for Information Extraction,
  Kushmerick, N., Weld, D., & Doorenbos, B.
  In Proceedings of IJCAI, 1997.
• Recommended Readings:
• Hierarchical Wrapper Induction for Semistructured Information Sources,
  I. Muslea, S. Minton, C. Knoblock
  Journal of Autonomous Agents and Multi-Agent Systems, 4:93-114, 2001.
• Wrapper induction: Efficiency and expressiveness,
  Nicholas Kushmerick
  Artificial Intelligence J.(special issue on Intelligent Internet Systems),(1-2):15-68,2000.
• Wrapper maintenance: A machine learning approach,
  Kristina Lerman, Steven Minton, and Craig A. Knoblock
  Journal of Artificial Intelligence Research, 2003
• Slides
• Wrapper Learning
• Wrapper Learning Tools


• Lecture 5 (February 12)
• Topic: Data Integration
• Data Integration
• Data Integration Tools (Prof. Jose Luis Ambite)
• Homework
• Assignment 4 Due (February 15 at midnight)
• Required Readings:
• Beauty and the Beast: The Theory and Practice of Information Integration,
  Laura Haas.
  In Proceedings of 11th International Conference on Database Theory, (ICDT 2007), Barcelona, Spain. January 10-12, 2007.
• Logic-Based Techniques in Data Integration,
  A. Levy
  in J. Minker (ed.): "Logic-based Artificial Intelligence".
• Recommended Readings:
• Planning for Information Gathering: A Tutorial Survey,
  E. Lambrecht and S. Kambhampati,
  ASU CSE Tech Report 96-017, May 1997.
• A view integration approach to dynamic composition of web services,
  Snehal Thakkar, Jose-Luis Ambite, and Craig A. Knoblock.
  In Proceedings of 2003 ICAPS Workshop on Planning for Web Services, Trento, Italy, 2003.
• Slides
• Data Integration
• Data Intergration Tool (part 1)
• Data Intergration Tool (part 2)


• Lecture 6 (February 19)
• Topic: Schema Mapping / Source Modeling
• Schema Mapping/Source Modeling
• Mapping/Modeling Tools (Anon Plangprasopchok)
• Homework
• Assignment 5 Due (February 22 at midnight)
• Required Readings:
• iMAP: Discovering Complex Semantic Matches between Database Schemas,
  R. Dhamanka, Y. Lee, A. Doan, A. Halevy, and P. Domingos.
  In Proceedings of SIGMOD-04, 2004.
• Automatically Labeling the Inputs and Outputs of Web Services ,
  Kristina Lerman, Anon Plangrasopchok and Craig Knoblock.
  In Proceedings of AAAI-2006, Boston, MA, 2006.
• Learning Semantic Definitions of Online Information Sources.,
  Mark James Carman and Craig A. Knoblock.
  Journal of Artificial Intelligence Research (JAIR) Volume 30, pages 1-50, 2007.
• Slides
• Schema Matching
• Source Modeling
• Schema Matching Tools


• Lecture 7 (February 26)
• Topic: Semantic Web
• Semantic Web (Prof. Jose Luis Ambite)
• Semantic Web Tools (Anon Plangprasopchok)
• Homework
• Assignment 6 Due (February 29 at midnight)
• Required Readings:
• Semantic Web
  By Tim Berners-Lee, James Hendler and Ora Lassila
  Scientific American, May 2001.
• The Semantic Web: A Network of Content for the Digital City
  Aaron Swartz and James Hendler
  In Proc. Second Annual Digital Cities Workshop, 2001.
• The Semantic Web: An Introduction
  Sean B. Palmer, 2001.
• Recommended Readings:
• See the web site http://www.disobey.com/detergent/2002/sw123/ for a great list of suggested readings on the semantic web.
• Slides
• Semantic Web Overview
• Semantic Web Tools
• OWL

• Lecture 8 (March 4)
  • Topics: Record Linkage 
    • Record Linkage
    • Blocking / Matching Tools (Matt Michelson)
  • Homework
    • Assignment 7 Due (March 7 at midnight)
  • Required Readings:
    • Learning Blocking Schemes for Record Linkage,
      Matthew Michelson and Craig A. Knoblock,
      In Proceedings of the 21st National Conference on Artificial Intelligence (AAAI-2006), Boston, MA, 2006.
    • Adaptive Duplicate Detection Using Learnable String Similarity Measures.
      Mikhail Bilenko and Raymond J. Mooney.
      In Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2003), pp.39-48, Washington, DC, August 2003.
    • EntityBases: Compiling, Organizing and Querying Massive Entity Repositories  Craig A. Knoblock, José Luis Ambite, Kavita Ganesan, Maria Muslea, Steven Minton, Greg Barish, Evan Gamble, Claude Nanjo, Kane See, Cyrus Shahabi, Ching-Chien Chen:  International Conference on Artificial Intelligence, 2007
  • Optional Readings:
    • Rohan Baxter, Peter Christen, Tim Churches, A Comparison of Fast Blocking Methods for Record Linkage, In Proceedings of the ACM SIGKDD international conference on Knowledge Discovery and Data Mining (KDD-03), 2003
    • Andrew McCallum, Kamal Nigam and Lyle H. Ungar, Efficient clustering of high-dimensional data sets with application to reference matching, In Proceedings of the ACM SIGKDD international conference on Knowledge Discovery and Data Mining (KDD-00), 2000
  • Slides
    • Record Linkage
    • Blocking

• Lecture 9 (March 11)
  • Topic: Dataflow Execution
    • Streaming Dataflow Execution
    • Optimizing Streaming Dataflow (Dr. Greg Barish)
  • Homework
    • Assignment 8 Due (March 14 at midnight)
  • Required Readings:
    • An efficient and expressive language for information gathering on the web, Greg Barish and Craig A. Knoblock.
      In Proceedings of the {AIPS-2002} Workshop on Is there life after operator sequencing? -- Exploring real world planning, pages 5--12, Tolouse, France, 2002.
    • Integrating Network-Bound XML Data. Zachary G. Ives, Alon Y. Halevy, Daniel S. Weld.   IEEE Data Engineering Bulletin, June 2001.
    • Speculative Execution for Information Gathering Plans,
      Greg Barish and Craig A. Knoblock,
      Proceedings of the Sixth International Conference on AI Planning and Scheduling (AIPS-2002), Toulouse, France. April 2002.
    • Learning value predictors for the speculative execution of information gathering plans,
      Greg Barish and Craig A. Knoblock.
      In Proceedings of the 18th International Joint Conference on Artificial Intelligence (IJCAI-2003), Acapulco, Mexico, 2003.
  • Optional Readings:
    • An XML Query Engine for Network-Bound Data
      Zachary G. Ives, Alon Y. Halevy, Daniel S. Weld
      VLDB Journal,11(4),2002.
    • An expressive language and efficient execution system for software agents,
      Greg Barish and Craig A. Knoblock. Journal of Artificial Intelligence Research, 23:625--666, 2005.
  • Slides
    • Dataflow Execution
    • Optimizing Dataflow Execution

• Lecture 10 (March 25)
  • Topic: Constraint Integration / Social Networking
    • Constraint Integration 
    • Social Networking (Professor Kristina Lerman)
  • Required Readings:
    • Heracles II: Conditional Constraint Networks for Interleaved Planning and Information Gathering,
      Jose Luis Ambite, Craig A. Knoblock, Maria Muslea, and Steve Minton
      IEEE Intelligence Systems, 2005
    • Smart Clients: Constraint Satisfaction as a Paradigm for Scaleable Intelligent Information Systems
      Marc Torrens, Boi Faltings and Pearl Pu.
      Constraints, Special issue on Constraints and Agents, pages. 49-69, International Journal of Constraints, Volume 7, Issue 1. Kluwer Academic Publishers. January 2002.
    •  A constraint satisfaction approach to geospatial reasoning,  Martin Michalowski and Craig A. Knoblock. In Proceedings of The Twentieth National Conference on Artificial Intelligence (AAAI-05), Pittsburgh PA, July 2005, pp. 423-429.
  • Optional Readings:
    • Constraint Programming: In Pursuit of the Holy Grail
      Barták, R., in Proceedings of the Week of Doctoral Students (WDS99), Part IV, MatFyzPress, Prague, June 1999, pp. 555-564.

• Slides
  • Constraint Integration Slides
  • Model Refinement for Constraint Satisfaction Problems

• Lecture 11 (April 1)
  • Topics: Mashup Construction / Geospatial Reasoning
    • Mashup Construction
    • Constraint satisfaction for geospatial reasoning (Martin Michalowski)
  • Required Readings:
    • Making Mashups with Marmite: Towards End-User Programming for the Web
    • Intel Mashmaker http://portal.acm.org/citation.cfm?id=1247480.1247626
    • Rattapoom Tuchinda, Pedro Szekely, and Craig A. Knoblock, Building Mashups by Example, In Proceedings of the International Conference on Intelligent User Interface 2008, tuchinda08-iui.pdf
  • Video Demonstrations:
    • http://www.cs.cmu.edu/~jasonh/projects/marmite/Marmite-Screencast.mov
    • http://softwarecommunity.intel.com/articles/eng/1505.htm
  • Optional Readings:
    • Kristina Lerman (2007), Social Information Processing in Social News Aggregation, Extended version of the paper in IEEE Internet Computing special issue on Social Search 11(6), pp.16-28, 2007. (cs.CY/0703087) [pdf] [bibtex]
  • Slides
    • Demo.mp4
    • Mashups.pdf
    • SocialWeb-lecture.pdf

• Lecture 12 (April 8)
  • Topic: Geospatial Data Integration
    • Integrating Text and Imagery 
    • Map Search and Extraction (Yao-Yi Chaing)
  • Required Readings:
    • Automatically and accurately conflating orthoimagery and street maps,
      Ching-Chien Chen, Craig A. Knoblock, Cyrus Shahabi, Snehal Thakkar, and Yao-Yi Chiang.
      In Proceedings of the 12th ACM International Symposium on Advances in Geographic Information Systems (ACM-GIS'04), 2004.
    • Automatic extraction of road intersections from raster maps,
      Yao-Yi Chiang, Craig A. Knoblock, and Ching-Chien Chen.
      In The 13th ACM International Symposium on Advances in Geographic Information Systems (ACM-GIS'05), Bremen, Germany, November, 2005.
    • Integrated text and line-art extraction from a topographic map, , Luyang Li, George Nagy, Ashok Samal, Sharad Seth, Yihong Xu, International Journal on Document Analysis and Recognition, Volume 2, Number 4 / June, 2000
  • Slides
    • Automatically Exploiting Raster Maps to Integrate and Utilize Other Geospatial Sources

• Lecture 13 (April 15)
  • Topic: Intellectual Property / Geospatial Data Integration 
    • Intellectual Property
    • Vector, Map, and Image Registration and Conflation (Dr. Jason Chen)
  • Required Readings:
    • Scrape It, Scrub It and Show It: The Battle Over Data Aggregation
    • Automatically conflating road vector data with orthoimagery,
      Ching-Chien Chen, Craig A. Knoblock, and Cyrus Shahabi.
      Geoinformatica, 10(4):495--530, December 2006.
  • Slides
    • Intellectual Property
    • Geospatial Integration

• Location: THH 212