Using Multiple Paradigms for Teaching High Performance Computing

Jawwad A Shamsi, Nouman M Durrani, Nadeem Kafi Khan

{jawwad.shamsi, muhammad.nouman, nadeem.kafi}@nu.edu.pk

FAST NUCES, Karachi

A.  Introduction

Teaching High Performance Computing (HPC) involves special considerations and challenges. The task includes explanation of huge computational problems and understanding of supported platforms that can exploit massive computing resources. The students must be given practical exposure and hands on development experience in order to strengthen their development and analytical skills. Further, advancements and enormous growth in parallel computing, such as Big data or data-intensive computing and GPGPU (General Purpose Graphic Processing Unit) highlight the significance of effectively teaching this course.

There are multiple issues involved in teaching such a course at the undergraduate level. The foremost challenge is that HPC include computing using multiple platforms such as MPI [2], Open MP [3], Hadoop/MapReduce [4], Intel TBB and GPGPU [5] - with each having unique characteristics. In order to understand the important concepts of HPC, it is pertinent that the curriculum should consist of development and understanding of these platforms so that comprehension and practical exposure could be strengthened. Recent advancements [4] suggests that such systems are being increasingly utilized for many research, industrial, and commercial projects. These include Internet Search and Social Networking. In addition to above mentioned considerations, incorporating HPC curriculum for undergraduate students requires special efforts for practical exposure and hands on development.

B.  Course Goals

The overall goal for offering the course was to increase student comprehension. In addition, the course aimed to cover topics related to both CPU-intensive and data-intensive platforms. Using multiple platforms is expected to enhance student learning. Table 1 lists a set of important pedagogical goals of the course.

C.  Course Contents

The course contents were specifically selected to give breadth of knowledge to students. Initially, many important concepts such as architectural support for HPC, task division, interaction, and localization were discussed. This was followed by introducing students to programming paradigms of HPC. These include MPI, OpenMP, Intel TBB, Hadoop, and GPGPU. For each of these paradigms, the focus is to effectively communicate the problem solving techniques to students. In addition, each paradigm is taught with a strong focus on practical applicability and matching recommended PDC topics. Table 2 lists the weekly plan of the course. For many topics, programming examples were demonstrated through live execution on the cluster.

Table 1: Pedagogical Goal

G1

/

Understand basic concepts of High Performance computing

G2

/

Provide practical and hands on experience on CPU Inetsnive Computing

G3

/

Provide practical and hands on experience on data Inetsnive Computing

G4

/

Motivate students for advanced topics and learning

Table 2 – Weekly Plan

Topic / Alignment with Goal
Introduction to important concepts of HPC / G1
Task Division, interaction, and Solving Techniques. HPC Clusters. Introduction to MPI / G1, G2, G3, and G4
MPI Communication, Message Passing. (A1) / G2 and G4
Process creation in MPI. File access in MPI. Quiz 1 / G2 and G4
Shared Memory Clusters. Open MP Programming Assignment on Open MP (A2) / G2 and G4
First Hourly. Hybrid Clusters using MPI / G2 and G4
Project Proposals. Introduction to Object Oriented programming in HPC. Concepts of Intel TBB. / G2 and G4
Quiz 2. Task Stealing and Auto Partitioning . Programming Assignment on Intel TBB (A3) / G2 and G4
Data Intensive Computing. Introduction to MapReduce / G3 and G4
Hadoop – Open Source Platform for MapReduce / G3 and G4
Architecture of Hadoop. Assignment on Hadoop (A4) / G3 and G4
Hadoop usage in data intensive applications / G3 and G4
Hadoop and MapReduce : Applications / G3 and G4
GPU/GPGPU Programming. Architecture concepts in GPU. / G2 and G4
Grid, thread, and block concepts. Device to host communication. Programming Assignment on GPU (A5) / G2 and G4
Project Presentations / G2 and G4

Table 3 – Description of Programming Assignments

Assignment Description
A1 / MPI program to scatter computational task to worker nodes in the cluster and gather the results back at the root node.
A2 / For a large computational problem, identify opportunities for parallelism. Use OpenMP to solve the computational problem.
A3 / Group Assignment. Using hashing develop a sentence-level plagiarism detector using Intel Thread Building Block (Intel TBB)
A4 / Solve the movie-ratings problem from Netflix using Hadoop and MapReduce.
A5 / Multiplication of large matrixes using GPUs

This ensures higher level of comprehension for students. Table 3 shows description of five programming assignments in the course.

D.  Evaluations and Feedback:

A comprehensive evaluation is necessary in order to assess the performance of the students of the course. Student evaluations were conducted through two hourly-exams, programming assignments and projects, quizzes, and final examination. This allowed observation of students’ performance and identification of weak students throughout the course. Moreover, a thorough effort was made to assess the teaching methodology and students’ performance. The student’s feedback and result of evaluations is shown in Table 4.

E.  Conclusions Lessons Learned and future work

This paper presents outline and methodology to teach TCPP recommended curriculum through programming based approach and using multiple HPC platforms. The course is further aimed to enhance students learning by teaching advanced topics and increase their interest in research material. The paper also presents a strong concentration on student feedback. The approaches and the course contents delivered are well acknowledged by the students. Following are the important lessons learned from the course.

-  Programming provides an effective method for learning.

-  Using multiple HPC platforms provided an effective way of learning.


Table 4 – Description of Programming Assignments

Question / Results
Multiple HPC platforms enhanced Learning / Strongly Agree (63.63%),
Agree (31.81%), Neutral (4.54%), Disagree(0%), Strongly Disagree(0%)
Programming Helped In Learning / Strongly Agree (77.27%), Agree (13.63 %), Neutral (9.09%), Disagree(0%), Strongly Disagree(0%)
Group Discussion and Interactive Style helped in Learning / Strongly Agree (63.63%),
Agree (18.18%), Neutral (13.63%), Disagree (4.54%), Strongly Disagree(0%)

-  Both data and CPU intensive computing are needed to be covered in parallel computing course.

-  TCPP recommended curriculum is well designed. Multiple courses are needed in order to comprehensively cover the course.

-  Cutting edge topics such as GPUGP, Hadoop, and Cloud computing are very popular among students

-  Interactive learning, peer discussion, and group discussions are effective in teaching.

-  Students’ feedback should be incorporated for elective courses.

In future, a hybrid programming approach between MPI and OpenMP could be useful in further increasing students’ understanding. In addition, we plan to expose students on clusters with heterogeneous configuration. This will enable them to learn cost of parallelism on different platforms.

References

[1]  NSF TCPP Curriculum Initiative

http://www.cs.gsu.edu/~tcpp/curriculum/?q=home

[2]  MPI. http://www.mcs.anl.gov/research/projects/mpi/

[3]  OpenMP. http://openmp.org/wp/

[4]  Apache Hadoop Project. http://hadoop.apache.org/

[5]  GPGPU. General Purpose Computing on Graphic hardware. http://gpgpu.org/

[6]  High Performance Computing course at Florida State University

http://www.cs.fsu.edu/~engelen/courses/HPC/

[7]  North Carolina State University

http://www.ncsu.edu/itd/hpc/Courses/Courses.php

[8]  High Performance computing at Georgia Tech University http://www.cs.nyu.edu/courses/fall10/G22.2945-001/syllabus.html

[9]  High Performance Computing at UNC- Charlotte

http://coitweb.uncc.edu/~abw/ITCS4145S12/

[10]  GPU programming course at UNC- Charlotte

http://coitweb.uncc.edu/~abw/ITCS6010S11/index.html

[11]  MapReduce Courses http://code.google.com/edu/parallel/

[12]  Hadoop Course at Stanford http://www.stanford.edu/class/cs246/

[13]  HPC at NIST http://www.nist.gov/itl/math/hpcvg/hpc.cfm