Question 1

What is High Performance Computing (HPC)?

Accepted Answer

**HPC** is the practice of aggregating computing power in a way that delivers much higher performance than one could get out of a typical desktop computer or workstation to solve large problems in science, engineering, or business.

### Key Components:
1.  **Parallel Processing**: Running programs on many processors simultaneously.
2.  **Cluster**: A set of computers connected together to work as a single unit.
3.  **High-Speed Network**: Low latency interconnects (like InfiniBand) to allow fast communication between nodes.

Question 2

What is NVIDIA DGX and why is it special?

Accepted Answer

**NVIDIA DGX** is a line of workstations and servers specifically purpose-built for AI and Deep Learning workloads.

### Key Features:
*   **NVLink**: A high-speed interconnect that essentially allows multiple GPUs to share memory and act as one giant GPU.
*   **Tensor Cores**: Specialized hardware for matrix multiplication (essential for AI).
*   **Software Stack**: Comes pre-installed with optimized AI frameworks (TensorFlow, PyTorch) via NVIDIA NGC.

Question 3

MPI vs OpenMP - What is the standard use case for each?

Accepted Answer

These are the two dominant parallel programming models in HPC.

### MPI (Message Passing Interface)
*   **Model**: Distributed Memory. Processors have their own local memory and communicate by sending messages over a network.
*   **Scope**: Runs across **multiple nodes** (inter-node).
*   *Analogy*: Sending emails between different offices.

### OpenMP (Open Multi-Processing)
*   **Model**: Shared Memory. Multiple threads share the same memory space.
*   **Scope**: Runs within a **single node** (intra-node).
*   *Analogy*: Talking to colleagues in the same conference room.

Question 4

Explain Amdahl's Law.

Accepted Answer

**Amdahl's Law** is a formula used to find the maximum improvement possible by improving a particular part of a system.

In parallel computing, it states that the theoretical speedup is limited by the **serial part** of the program.
*   If 95% of the program is parallelizable, but 5% must run sequentially, the maximum speedup is 20x, no matter how many processors you add.

*Lesson*: Optimize the serial portion to break scalability bottlenecks.

Question 5

What is a Compute Cluster? (Head Node vs Compute Node)

Accepted Answer

A **Cluster** is a group of linked computers working together.

### Head Node (Login/Master Node)
*   The gateway to the system. Users log in here.
*   Used for compiling code, submitting jobs, and file management.
*   **Do NOT** run heavy calculations here.

### Compute Nodes
*   Where the actual work happens.
*   They execute the jobs scheduled by the workload manager.
*   Usually accessed only via the job scheduler, not directly.

Question 6

What is Slurm?

Accepted Answer

**Slurm** (Simple Linux Utility for Resource Management) is a highly scalable job scheduler widely used on Supercomputers.

### Functions:
1.  **Resource Allocation**: Allocates exclusive/non-exclusive access to resources (compute nodes) for a duration.
2.  **Job Execution**: Provides a framework for starting, executing, and monitoring work.
3.  **Queue Management**: Arbitrates contention for resources by managing a queue of pending work.

Question 7

What are FLOPS?

Accepted Answer

**FLOPS** (Floating Point Operations Per Second) is a measure of computer performance.

### Scale:
*   **TeraFLOPS**: $10^{12}$ (Trillion). High-end gaming PC.
*   **PetaFLOPS**: $10^{15}$ (Quadrillion). Modern Supercomputers.
*   **ExaFLOPS**: $10^{18}$ (Quintillion). The "Exascale" era barrier (recently broken by Frontier).

Question 8

InfiniBand vs Ethernet in HPC?

Accepted Answer

### Ethernet
*   Standard networking technology (TCP/IP).
*   Good for general purpose, but has higher **latency** and CPU overhead.

### InfiniBand
*   Specialized high-speed interconnect.
*   **Low Latency**: Microsecond-level delays.
*   **High Throughput**: 200Gbps/400Gbps+.
*   **RDMA (Remote Direct Memory Access)**: Allows data transfer directly from memory of one computer to another without involving the CPU. Crucial for HPC.

Question 9

What is CUDA?

Accepted Answer

**CUDA** (Compute Unified Device Architecture) is a parallel computing platform and programming model created by **NVIDIA**.

*   It allows software developers to use a "CUDA-enabled graphics processing unit (GPU)" for general purpose processing (**GPGPU**).
*   Developers write C/C++ code with special extensions to execute thousands of threads in parallel on the GPU.

Question 10

Why use Containers (Apptainer/Singularity) in HPC?

Accepted Answer

Standard Docker containers often require **root** privileges, which is a major security risk on shared supercomputers.

**Apptainer (formerly Singularity)** is designed for HPC:
*   **Security**: Runs as the user, not root.
*   **Mobility**: Encapsulates the entire environment (OS, libraries, code) in a single file (.sif).
*   **Reproducibility**: Ensures scientific results are reproducible regardless of the host system.

HPC & DGX

1What is High Performance Computing (HPC)?

2What is NVIDIA DGX and why is it special?

3MPI vs OpenMP - What is the standard use case for each?

4Explain Amdahl's Law.

5What is a Compute Cluster? (Head Node vs Compute Node)

6What is Slurm?

7What are FLOPS?

8InfiniBand vs Ethernet in HPC?

9What is CUDA?

10Why use Containers (Apptainer/Singularity) in HPC?