CSE 6220: Intro to High-Performance Computing

Course Creator and Instructor

Rich Vuduc
 
Rich Vuduc
Creator, Instructor
 

Course Developers

Catherine Gamboa

Catherine Gamboa
Course Developer

 

Amanda Deisler

Amanda Deisler
Video Producer

 

Overview

This course is a graduate-level introduction to scalable parallel algorithms. “Scale” really refers to two things: efficient as the problem size grows, and efficient as the system size (measured in numbers of cores or compute nodes) grows. To really scale your algorithm in both of these senses, you need to be smart about reducing asymptotic complexity the way you’ve done for sequential algorithms since CS 101; but you also need to think about reducing communication and data movement. This course is about the basic algorithmic techniques you’ll need to do so.   The techniques you’ll encounter covers the main algorithm design and analysis ideas for three major classes of machines: for multicore and manycore shared memory machines, via the work-span model; for distributed memory machines like clusters and supercomputers, via network models; and for sequential or parallel machines with deep memory hierarchies (e.g., caches). You will see these techniques applied to fundamental problems, like sorting, search on trees and graphs, and linear algebra, among others. The practical aspect of this course is implementing the algorithms and techniques you’ll learn to run on real parallel and distributed systems, so you can check whether what appears to work well in theory also translates into practice. (Programming models you’ll use include Cilk Plus, OpenMP, and MPI, and possibly others.)

Prerequisites

Please review the course readiness survey for CSE 6220. If you are unable to answer any of them you may want to refresh your knowledge of the area prior to taking this course.

  • A “second course” in algorithms and data structures, a la Georgia Tech’s CS 3510-B or Udacity’s Intro to Algorithms
  • For the programming assignments, programming experience in a “low- level” “high-level” language like C or C++
  • Experience using command line interfaces in *nix environments (e.g., Unix, Linux)

Course Preview

Evaluation

Students will be evaluated via programming assignments, a midterm, a final exam, and class participation via quizzes.

  • Exams - Exams will be proctored by ProctorU or a comparable service (to be determined).
  • Programming assignments (mini-projects) - Students will implements a number of parallel and distributed computations, which we will grade by testing their performance and scalability on a cluster or cloud-based HPC resource that we will provide. We estimate that there will be 4-5 such assignments.
  • Class participation - Class participation will be determined by your com- pletion of Udacity quizzes. Note: For the Udacity quizzes, you are not required to get the correct answer on the first try. You should feel free to submit answers even if you are not 100% sure that you are correct. If you do not get the correct answer after several attempts, try watching the solution video, and then come back to the quiz. Only your last submission will be checked, so if you get it right, then change you answer later, you will not get credit for that quiz.

Grading

  • The assignments will be weighted as follows:
  • Midterm - 15%
  • Final exam - 20%
  • Programming assignments - 50%
  • Class participation - 5%

Required Course Readings

  • There will be a mix of readings from published papers, instructor’s notes, and material from the course textbook (see below). All of this material will be available electronically via Georgia Tech’s Library Proxy, including the course textbook.
  • Course textbook: Grama et. al., Introduction to Parallel Computing

Minimum Technical Requirements

  • Browser and connection speed: We strongly recommend an up-to-date version of Chrome or Firefox. We also support Internet Explorer 9 and the desktop versions of Internet Explorer 10 and above (not the metro versions). 2+ Mbps recommended; at minimum 0.768 Mbps download speed/p>
  • Operating System: The following are the recommended operating systems for the course. We may also elect to provide virtual machines with a standardized environment, though most of the assignments can be completed by directly logging into the HPC resource we will provide via secure shell (ssh).

  • PC: Windows XP or higher with latest updates installed

  • Mac: OS X 10.6 or higher with latest updates installed
  • Linux - Any recent distribution that has the supported browsers installed.

Other Info

  • All assignments are due no later than 23:59 anywhere on earth (11:59 p.m. AOE) on the posted due date

  • No late assignments will be accepted

  • You should complete the GT OMS Orientation before the first class

Academic Honesty 

All Georgia Tech students are expected to uphold the Georgia Tech Academic Honor Code.