CSCI 503 - Parallel Programming

Spring 2008

Course Description:

This course is a comprehensive exploration of parallel programming paradigms, examining challenges introduced by the growing diversity of modern parallel computing architectures, surveying the set of contemporary research ideas in the area, evaluating historical examples and their impact, and providing context with parallel algorithms. In the last few years, this area has been the subject of renewed interest as the advent of multi-core microprocessors will soon make parallel computing available to the masses. Students in the course will learn how to develop scalable programs for the parallel architectural continuum from systems-on-a chip to high-end platforms with tens of thousands of processors. As this course combines hands-on programming and a discussion of research in the area, it is suitable for advanced Masters students and PhD students who wish to learn how to write parallel applications or are engaged in research in related areas.

Prerequisites:

Basic knowledge of: programming in C (CS101L, CS455X, or equivalent); data structures (CS102L or equivalent); and computer architecture or game hardware architecture (EE457, EE452, or equivalent).

Textbook:

  • Grama, A. Gupta, G. Karypis, and V. Kumar, Introduction to Parallel Computing, 2nd Ed. (Addison-Wesley, 2003).
  • Additional Readings, connected with lectures


    • Instructor: Class Schedule:   Basis for grades:
      Assignments:


    Syllabus


    Lecture Notes:

  • Lecture 1: Introduction

  • Lecture 2

  • Lecture 3: Memory Hierarchies (1perpage) (4perpage)

  • Lecture 4: SIMD / Multimedia Extensions (1perpage) (4perpage)

  • Lecture 5: Shared Memory Programming (1perpage) (4perpage)

  • Lecture 6: Parallel Algorithms (1perpage) (4perpage)

  • Lecture 7: Dense Matrix Algorithms (1perpage) (4perpage)
  • Supplement: The Future of Compilers for Parallel Architectures, from Systems-on-a-Chip to Supercomputers

  • Lecture 8: Graph Algorithms (1perpage) (4perpage)
  • VTUNE, Paul Petersen, Intel

  • Lecture 9: Data Parallel Programming Models (1perpage) (4perpage)

  • Lecture 10: CUDA (1perpage)

  • Lecture 11: HPCView (1perpage) (4perpage)

  • Lecture 12: Global Address Space Languages (1perpage) (4perpage)
  • UPC, Kathy Yelick
  • Chapel, Brad Chamberlain

  • Lecture 13: Message Passing (1perpage) (4perpage)

  • Lecture 14: MapReduce (1perpage) (4perpage)