Introduction to Parallel and Distributed Programming

Overview

This repository contains materials, notes, and practical implementations for the course "Introduction to Parallel and Distributed Programming". The course covers fundamental and advanced topics in parallel and distributed computing, focusing on data partitioning, parallel computation, communication models, fault tolerance, synchronization, and performance analysis.

Module 1: Introduction (Coming Soon)

Updates will be added soon.

Lab 1

For practical exercises related to Module 1, refer to: Lab 1, Bloack Diagram

---

Module 2: Distributed Processing with MPI

This module focuses on parallel computation across multiple nodes, efficient communication using OpenMPI, handling faults, and optimizing performance.

Lab 2

For practical exercises related to Module 2, refer to: Lab 2

---

Introduction to MPI

• Notes
• Video
• Use Case: Solving Numerical Integration using the Trapezoidal Rule in MPI

Introduction Communication in MPI

• Notes

Introduction Data Distribution

• Notes
• Blog

Handling Input and Output in MPI

• Notes

Collective Communications

• Notes

Collective vs. Point-to-Point Communication in MPI

• Notes

MPI Scatter, Gather, and Allgather

• Notes
• Examples

Derived datatypes: MPI_Datatype

• Notes
• Video

Performance

• Notes

---

Module 3: Multi-threading for single node performance

Lab 3

For practical exercises related to Module 3, refer to: Lab 3,

Processes, threads

• Notes

Pthreads

• Notes

Matrix-vector multiplication using Pthreads

• Notes

Critical Section

• Notes

Busy-Waiting

• Notes

Mutex

• Notes

MPI, Pthreads, and OpenMP

• Notes

First Program in OpenMP

• Notes

Mutext in OpenMP

• Notes

---

How to Contribute

• Feel free to submit issues and improvements via GitHub.
• Share your resources

---

Resources

• Intro to Parallel and Distributed Programming - Complete Concepts
• MPI and OpenMP
• OpenMP