CO471 - Parallel Programming, Jan-May, 2016

Welcome to the C0471 - Parallel Programming course page. All the necessary course related information can be found here.

Course Objectives

  • Provide systematic and comprehensive treatment of present day concepts in shared and distributed parallel programming.

Course Syllabus

  • Programming Paradigms: OpenMP, MPI, CUDA, OpenCL.
  • Important publications from literature.

Suggested list of papers that can be discussed in the class. If you have any of your choice, meet up and discuss.

  • Yoo, et. al., "A Scalable Distributed Parallel Breadth-First Search Algorithm on BlueGene/L", Supercomputing Conference, 2005. Paper Link.
  • Abhisek Pan and Vijay Pai, "Runtime-Driven Shared Last-Level Cache Management for Task-Parallel Programs", SC 2015. Paper Link.
  • Yu, et. al., "Performance Optimization for the K Nearest-Neighbor Kernel on x86 Architectures", SC 2015. Paper Link.
  • Li, et. al., "Adaptive and Transparent Cache Bypassing for GPUs", SC 2015. Paper Link
  • Flick and Aluru, "Parallel distributed memory construction of suffix and longest common prefix arrays", SC 2015. Paper Link.
  • Grossman, et. al., "Filtering, reductions and synchronization in the Anton 2 network", IPDPS 2015. Paper Link
  • Sethia, et. al., APOGEE: Adaptive Prefetching On GPUs for Energy Efficiency. PACT 2015. Paper Link.
  • Kim, et. al., Memory-centric system interconnect design with hybrid memory cubes, PACT 2015. Paper Link.
  • Volkov and Demmel, "Benchmarking GPUs to Tune Dense Linear Algebra", SC2008. Paper Link.
  • Ropars et. al., Efficient Process Replication for MPI Applications: Sharing Work Between Replicas, IPDPS 2015. Paper Link
  • Georganas, merAligner: A Fully Parallel Sequence Aligner, IPDPS 2015. Paper Link
  • Meyerhenke, et. al., "Parallel Graph Partitioning for Complex Networks", IPDPS 2015. Paper Link.
  • Chatarasi, et. al., Polyhedral Transformations of Explicitly Parallel Programs, PACT 2015. Paper Link.
  • Kyrola, et. al., "GraphChi: Large-Scale Graph Computation on Just a PC", Operating System Design and Implementation (OSDI), 2012. Usenix Link.

Reference Materials

Reference Books/Textbooks:

  • Grama, et. al. Introduction to Parallel Computing. Pearson Ed.
  • Culler, Singh and Gupta. Parallel Computer Architecture. MK.
  • Herlihy and Shavit, The Art of Multiprocessor Programming. MK.

Apart from the above material, each programming assignment PDF lists relevant references.

Programming Assignments

Submit input, code, screenshots, in an archive. Email to

Title Documents/Demo ExamplesSubmission Date
1 OpenMP A1 - OpenMP Programming Assignment., Molecular Dynamics Serial code., CDAC's OpenMP demo examples.. Jan 8, 5PM.
2 MPI A2 - MPI Programming Assignment.. Jan 20, 5PM.
3 CUDA A3 - CUDA Programming Assignment. Feb 01, Midnight.
4 OpenCl A4 - OpenCl Programming Assignment.Feb 12, Midnight.
5 Course Projects Information.
M1 Project Proposal Deadline. Feb 28, Midnight.
M2 Project Mid-progress Report Deadline. March 24, Midnight.
M3 Project Final Report Deadline. April 17, Midnight.

Class Schedule

Date/Week Class/Paper details Shepherds
Dec 28 Class Zero
Programming assignments - A1 (OpenMP), A2 (MPI) out.
Jan 8 OpenMP Assignment - A1. Deadline - 5PM.
Jan 13 Khorasani, et. al., "Stadium Hashing: Scalable and Flexible Hashing on GPUs", PACT 2015.. Pooja M Kumar
Jan 15 OpenMP Programming Test.
Programming assignments - A3 (CUDA), A4 (OpenCl) out.
Jan 20 Pregel: a system for large-scale graph processing, ACM SIGMOD '10Pranav T Bhat, Rohit Varkey Thankachan
Jan 20 MPI Assignment - A2. Deadline - 11:59PM.
Jan 22 MPI Programming Test.
Jan 27 Tang, et. al., Efficient Selection Algorithm for Fast k-NN Search on GPUs", Parallel and Distributed Processing Symposium (IPDPS), 2015.Sumukha R M
Feb 01 CUDA Assignment - A3. Deadline - Midnight.
Feb 03 Harshvardhan, et. al., An Algorithmic Approach to Communication Reduction in Parallel Graph Algorithms, PACT 2015. Nimisha Sharath, Chintan Doshi
Feb 05 CUDA Programming Test.
Feb 08 Course Projects Document out.
Feb 10 Ramapantulu, et. al., An Approach for Energy Efficient Execution of Hybrid Parallel Programs, IPDPS 2015.Akshay Atul Deshmukh
Feb 12 OpenCl Assignment - A4. Deadline - Midnight.
Feb 18. 330PM - 5PM Midsem Exam
Midterm Syllabus 1. Chapter 2 and Section 3.1 from the Grama, et. al. Introduction to Parallel Computing. Pearson Ed. book.
2. Papers presented in class. (Prints of papers exams allowed in the exam.)
Feb 27. 930AM - 12PM Expert Lecture: Parallel Graph Algorithms (Abstract) Dr. Rupesh Nasre, CSE, IIT Madras.
Feb 28 Project Proposal Due. Deadline: Midnight.
March 2 Liu and Wu, "ScaAnalyzer: A Tool to Identify Memory Scalability Bottlenecks in Parallel Programs", SC 2015. Paper Link.Yash Kedia, Sanket Gawande
March 12. 830AM - 1230PM Expert Lecture: Deep Neural Networks on Heterogeneous Parallel Processors (Abstract) Dr. Prakash Raghavendra, HP, Bangalore.