Instructor: Craig Knoblock and Kristina Lerman
Meeting Time: Monday and Wednesday 3.30 - 4:50pm
Location: THH 212
Office Hours:
    Monday and Wednesday 4.50pm (immediately after class in the lobby of GFS)
    Or by appointment (ISI 922 or by phone)
Teaching Assistant:

•  Yao-Yi Chang

This course will focus on the basic foundations and techniques in Information Extraction and Integration. There has been a great deal of interest and research over the last few years on this topic and the course will cover the research and tools for addressing the technical problems. The topics covered will include structured data querying (Xquery), view integration techniques, machine learning techniques for turning web sites into structured data sources, high-performance query execution systems based on streaming dataflow, constraint-based integration systems, and approaches to resolving naming inconsistencies across sites. We will also cover the applications of information integration to geospatial data integration.

The class will be run as a lecture course with lots student participation and hands-on experience. As an integral part of the course each student will develop and build an integrated Web application using the research and tools covered in the class.

Course Syllabus and Schedule

• Lecture 1 (January 12)
• Topic: Introduction
• Introduction
• Overview of the course
• Course project
• Required Readings
• What is XQuery ?
  Per Bothner.
• XQuery: A Guided Tour
  Chapter 1 of XQuery from the Experts: A Guide to the W3C XML Query Language
  Howard Katz (ed), Addison-Wesley, San Francisco, CA, 2004
• Recommended Readings
• XQuery 1.0: An XML Query Language
• Slides
• Introduction


• Lecture 2 & 3 (January 14 & 21 )
• Topic: Wrapper Learning
• Wrapper Learning and Maintenance
• Wrapper Learning Tools (Martin Michalowski)
• Homework
• Assignment 1 Due (January 23 at midnight)
• Required Readings
• A Hierarchical Approach to Wrapper Induction,
  Ion Muslea, Steve Minton, and Craig A. Knoblock.
  In Proceedings of the 3rd International Conference on Autonomous Agents 1999, Seattle, WA, 1999.
• Wrapper Induction for Information Extraction,
  Kushmerick, N., Weld, D., & Doorenbos, B.
  In Proceedings of IJCAI, 1997.
• Recommended Readings:
• Hierarchical Wrapper Induction for Semistructured Information Sources,
  I. Muslea, S. Minton, C. Knoblock
  Journal of Autonomous Agents and Multi-Agent Systems, 4:93-114, 2001.
• Wrapper induction: Efficiency and expressiveness,
  Nicholas Kushmerick
  Artificial Intelligence J.(special issue on Intelligent Internet Systems),(1-2):15-68,2000.
• Wrapper maintenance: A machine learning approach,
  Kristina Lerman, Steven Minton, and Craig A. Knoblock
  Journal of Artificial Intelligence Research, 2003
• Active Learning with Multiple Views,
  Ion Muslea,
  Journal of Artificial Intelligence Research 27, pp. 202-233
• Slides
• Wrapper Learning
• Wrapper Learning Tools


• Lecture 4 & 5 (January 26 & 28)
• Topic: Information Extraction
• Information Extraction Systems
• IE Tools
• Homework
• Assignment 2 Due (January 30 at midnight)
• Required Reading (Lecture 4):
• Information Extraction: Distilling Structured Data from Unstructured Text,
  Andrew McCallum.
  ACM Queue, volume 3, Number 9, November 2005.
Semantic annotation of unstructured and ungrammatical text,
Matthew Michelson and Craig A. Knoblock.
In Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI-2005), Edinburgh, Scotland, 2005.
• Required Reading (Lecture 5):
• Accurate Information Extraction from Research Papers using Conditional Random Fields.
  Fuchun Peng and Andrew McCallum.
  In Proceedings of Human Language Technology Conference and North American Chapter of the Association for Computational Linguistics (HLT-NAACL), 2004.
• Adaptive Information Extraction from Text by Rule Induction and Generalisation,
  Fabio Ciravegna,
  In Proceedings of 17th International Joint Conference on Artificial Intelligence (IJCAI 2001), Seattle, August 2001.
• Optional Reading:
• Conditional Random Fields: An Introduction,
  Hanna M. Wallach,
  Technical Report MS-CIS-04-21. Dept. of Computer and Information Science, University of Pennsylvania, 2004
• Slides
  • Information Extraction
  • Information Extraction II


• Lecture 6 & 7 (February 2 & 4)
• Topic: Data Integration
• Data Integration
• Data Integration Tools
• Homework
• Assignment 3 Due (February 7 at midnight)
• Required Readings:
• Beauty and the Beast: The Theory and Practice of Information Integration,
  Laura Haas.
  In Proceedings of 11th International Conference on Database Theory, (ICDT 2007), Barcelona, Spain. January 10-12, 2007.
• Logic-Based Techniques in Data Integration,
  A. Levy
  in J. Minker (ed.): "Logic-based Artificial Intelligence".
• Recommended Readings:
• Planning for Information Gathering: A Tutorial Survey,
  E. Lambrecht and S. Kambhampati,
  ASU CSE Tech Report 96-017, May 1997.
• A view integration approach to dynamic composition of web services,
  Snehal Thakkar, Jose-Luis Ambite, and Craig A. Knoblock.
  In Proceedings of 2003 ICAPS Workshop on Planning for Web Services, Trento, Italy, 2003.
• Slides
• Data Integration


• Lecture 8 & 9 (February 9 & 11)
• Topic: Wrapper Generation
• Automatic Wrapper Generation (Prof. Kristina Lerman)
• Automatic wrapper tools
• Homework
• Assignment 4 Due (February 14 at midnight)
• Required Readings (Lecture 8):
• RoadRunner: Towards Automatic Data Extraction from Large Web Sites
  W. Crescenzi, G. Mecca, P. Merialdo,
  The VLDB Journal, 109-118, 2001.
•  Overview of AutoFeed: An Unsupervised Learning System for Generating Webfeeds,
  B. Cenk Gazen and Steven Minton,
  In Proceedings of AAAI, 2006
• Required Readings (Lecture 9):
• Using the Structure of Web Sites for Automatic Segmentation of Tables
  Kristina Lerman, Lise Getoor, Steven Minton and Craig Knoblock,
  in Proc. of SIGMOD-2004, Paris, France,2004.
• Methods for Domain-Independent Information Extraction from the Web: An Experimental Comparison
  Etzioni, O., Cafarella, M., Downey, D., Popescu, A., Shakel, T., Soderland, S., Weld, D., and Yates, A.   2004 in Proceedings of AAAI.
• Recommended Readings:
• Automatic information extraction from large websites.
  Crescenzi, V. and Mecca, G.
  J. ACM 51, 5 (Sep. 2004), 731-779, 2004.
• Slides
• Wrapper Generation I
• Wrapper Generation II


• Lecture 10 (February 18)
• Topic: Mashups
• Building Mashups
• Mapping/Modeling Tools
• Required Readings:
• Making Mashups with Marmite: Towards End-User Programming for the Web.
• Intel Mashmaker http://portal.acm.org/citation.cfm?id=1247480.1247626.
• Rattapoom Tuchinda, Pedro Szekely, and Craig A. Knoblock, Building Mashups by Example, In Proceedings of the International Conference on Intelligent User Interface 2008, tuchinda08-iui.pdf.
• Video Demonstrations:
• http://www.cs.cmu.edu/~jasonh/projects/marmite/Marmite-Screencast.mov
• Intel Mashmaker http://softwarecommunity.intel.com/articles/eng/1505.htm
• Slides
• Building Mashups


• Lecture 11 (February 23)
• Topic: Schema Matching
• Schema Matching
• Homework
• Assignment 5 Due (February 21 at midnight)
• Required Readings:
• iMAP: Discovering Complex Semantic Matches between Database Schemas,
  R. Dhamanka, Y. Lee, A. Doan, A. Halevy, and P. Domingos.
  In Proceedings of SIGMOD-04, 2004.
• A Survey of Approaches to Automatic Schema Matching,
  E. Rahm and P. Bernstein,
  The VLDB Journal 10:334-550, 2001.
• Recommended Readings:
• Automatically Labeling the Inputs and Outputs of Web Services ,
  Kristina Lerman, Anon Plangrasopchok and Craig Knoblock.
  In Proc. WebDB workshop, 2000.
• Learning source descriptions for data integration,
  AH Doan, P Domingos and A Levy,
  In Proceedings of AAAI-2006, Boston, MA, 2006.
• Slides
• Schema Matching


• Lecture 12 (February 25)
• Topic: Source Modeling
• Source Modeling
• Homework
• Assignment 6 Due (February 28 at midnight)
• Required Readings:
• Learning Semantic Definitions of Online Information Sources.,
  Mark James Carman and Craig A. Knoblock.
  Journal of Artificial Intelligence Research (JAIR) Volume 30, pages 1-50, 2007.
• Automatically Labeling the Inputs and Outputs of Web Services ,
  Kristina Lerman, Anon Plangrasopchok and Craig Knoblock.
  In Proc. WebDB workshop, 2000.
• Slides
• Source Modeling


• Lecture 13 & 14 (March 2 & 4)
• Topics: Record Linkage 
• Record Linkage
• Homework
• Assignment 7 Due (March 7 at midnight)
• Required Readings:
  • Learning Blocking Schemes for Record Linkage,
    Matthew Michelson and Craig A. Knoblock,
    In Proceedings of the 21st National Conference on Artificial Intelligence (AAAI-2006), Boston, MA, 2006.
  • Adaptive Duplicate Detection Using Learnable String Similarity Measures.
    Mikhail Bilenko and Raymond J. Mooney.
    In Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2003), pp.39-48,Washington, DC, August 2003.
  • EntityBases: Compiling, Organizing and Querying Massive Entity Repositories  Craig A. Knoblock, José Luis Ambite, Kavita Ganesan, Maria Muslea, Steven Minton, Greg Barish, Evan Gamble, Claude Nanjo, Kane See, Cyrus Shahabi, Ching-Chien Chen:  International Conference on Artificial Intelligence, 2007
• Optional Readings:
• Rohan Baxter, Peter Christen, Tim Churches, A Comparison of Fast Blocking Methods for Record Linkage, In Proceedings of the ACM SIGKDD international conference on Knowledge Discovery and Data Mining (KDD-03), 2003
• Andrew McCallum, Kamal Nigam and Lyle H. Ungar, Efficient clustering of high-dimensional data sets with application to reference matching, In Proceedings of the ACM SIGKDD international conference on Knowledge Discovery and Data Mining (KDD-00), 2000
• Slides
• Record Linkage


• Lecture 15 & 16 (March 9 & 11)
• Topic: Semantic Web
• Semantic Web
• Semantic Web Tools
• Homework
• Assignment 8 Due (March 14 at midnight)
• Required Readings:
• Semantic Web
  By Tim Berners-Lee, James Hendler and Ora Lassila
  Scientific American, May 2001.
• The Semantic Web: A Network of Content for the Digital City
  Aaron Swartz and James Hendler
  In Proc. Second Annual Digital Cities Workshop, 2001.
• The Semantic Web: An Introduction
  Sean B. Palmer, 2001.
• Recommended Readings:
• See the web site http://www.disobey.com/detergent/2002/sw123/ for a great list of suggested readings on the semantic web.
• Slides
• Semantic Web Tools
• Semantic Web Overview


• Lecture 17 & 18 (March 23 & 25)
  • Topic: Dataflow Execution/ Optimization
  • Dataflow Execution
  • Dataflow Optimization
  • Required Readings (Lecture 17):
    • An efficient and expressive language for information gathering on the web,Greg Barish and Craig A. Knoblock.
      In Proceedings of the {AIPS-2002} Workshop on Is there life after operator sequencing? -- Exploring real world planning, pages 5--12, Tolouse, France, 2002.
    • Integrating Network-Bound XML Data. Zachary G. Ives, Alon Y. Halevy, Daniel S. Weld.   IEEE Data Engineering Bulletin, June 2001.
  • Recommended Readings (Lecture 18):
  • Speculative Execution for Information Gathering Plans,
    Greg Barish and Craig A. Knoblock,
    Proceedings of the Sixth International Conference on AI Planning and Scheduling (AIPS-2002), Toulouse, France. April 2002.
  • Learning value predictors for the speculative execution of information gathering plans,
    Greg Barish and Craig A. Knoblock.
    In Proceedings of the 18th International Joint Conference on Artificial Intelligence (IJCAI-2003), Acapulco, Mexico , 2003.
  • Optional Readings:
  • An XML Query Engine for Network-Bound Data
    Zachary G. Ives, Alon Y. Halevy, Daniel S. Weld
    VLDB Journal,11(4),2002.
  • An expressive language and efficient execution system for software agents,
    Greg Barish and Craig A. Knoblock. Journal of Artificial Intelligence Research, 23:625--666, 2005.
  • Slides
  • Dataflow Execution
  • Dataflow Optimization
  
  
• Lecture 19 & 20 (March 30 & April 1)
  • Topic: Social Web
  • Social Tagging
  • Folksonomies
  • Required Readings (Lecture 19):
    • The Structure of Collaborative Tagging Systems,
      Golder, S. and Huberman, B. ,2005
    • Exploiting Social Annotation for Automatic Resource Discovery
      Plangprasopchok, A. and Lerman, K
      In Proc. of AAAI Workshop on Information Integration on the Web, 2007.
  • Recommended Readings (Lecture 20):
  • Harvesting Social Knowledge from Folksonomies,
    Wu, H., Zubair, M. and Maly, K.
    In Proc. of Conf. on Hypertext and Hypermedia, 2006.
  • Learning value predictors for the speculative execution of information gathering plans,
    Plangprasopchok, A. and Lerman, K.
    In Proc. of International World Wide Web Conference, 2009.
  • Optional Readings:
  • Latent Dirichlet Allocation     (Lecture 19)
    Blei, D., Ng, A., and Jordan, M.
    2003. Journal of Machine Learning Research 3, pp.993-1022.
  • Inducing Ontologies from Flickr Tags
    Schmitz, P.
    InProc. of WWW Collaborative Web Tagging workshop, 2006.
  • Slides
  • Social Tagging
  • Folksonomies
  
  
• Lecture 21 (April 6)
• Topic: Geospatial Data Integration
• Geospatial Data Integration
• Required Readings:
• Automatically conflating road vector data with orthoimagery,
  Ching-Chien Chen, Craig A. Knoblock, and Cyrus Shahabi.
  Geoinformatica, 10(4):495--530, December 2006.
• Automatically and accurately conflating orthoimagery and street maps,
  Ching-Chien Chen, Craig A. Knoblock, Cyrus Shahabi, Snehal Thakkar, and Yao-Yi Chiang.
  In Proceedings of the 12th ACM International Symposium on Advances in Geographic Information Systems (ACM-GIS'04), 2004.
• Slides
• Geospatial Integration


• Lecture 22 (April 8)
• Topic: Map and Imagery Fusion
• Map and Imagery Fusion
• Required Readings:
• Accurate and Automatic Extraction of Road Intersections From Raster Maps,
  Yao-Yi Chang, Ching-Chien Chen, Craig A. Knoblock, and Cyrus Shahabi.
  Geoinformatica, 13(2):121--157, 2008.
• Integrated text and line-art extraction from a topographic map,
  Luyang Li, George Nagy, Ashok Samal, Sharad Seth, Yihong Xu .
  International Journal on Document Analysis and Recognition, Volume 2, Number 4 / June, 2000
• Slides
• Map Fusion


• Lecture 22 (April 13)
  • Topic: Constraint Integration
  • Constraint Integration
  • Required Readings:
    • Heracles II: Conditional Constraint Networks for Interleaved Planning and Information Gathering,
      Jose Luis Ambite, Craig A. Knoblock, Maria Muslea, and Steve Minton
      IEEE Intelligence Systems, 2005
    • Smart Clients: Constraint Satisfaction as a Paradigm for Scaleable Intelligent Information Systems
      Marc Torrens, Boi Faltings and Pearl Pu.
      Constraints, Special issue on Constraints and Agents, pages. 49-69, International Journal of Constraints, Volume 7, Issue 1. Kluwer Academic Publishers. January 2002.
  • Optional Readings:
  • Constraint Programming: In Pursuit of the Holy Grail
    Barták, R., in Proceedings of the Week of Doctoral Students (WDS99), Part IV, MatFyzPress, Prague, June 1999, pp. 555-564.
  • A constraint satisfaction approach to geospatial reasoning,
    Martin Michalowski and Craig A. Knoblock.
    In Proceedings of The Twentieth National Conference on Artificial Intelligence (AAAI-05), Pittsburgh, PA, July 2005, pp. 423-429.
  • Slides
  • Constraint Integration Slides
  
  
• Lecture 23 (April 15)
• Topic: Intellectual Property
• Intellectual Property and Web
• Required Readings:
• Scrape It, Scrub It and Show It: The Battle Over Data Aggregation
• Slides
• Intellectual Property and Web


• Lecture 24 (April 20)
  • Topic: Geocoding
  • Geocoding
  • Required Readings:
    • A Flexible Addressing System for Approximate Geocoding,
      Davis
    • A comparison of address point, parcel and street geocoding techniques
      Paul A. Zandbergen
      
  • Slides
  • Geocoding Research
  • Geocoding Talk
  
  
• Project Presentations 1 (April 22)


• Project Presentations 2 (April 27)


• Project Presentations 3 (April 29)


• Final Exam (Thursday, May 7, 2-4pm)
• Location: THH 212