Academic Compute Partitions Hardware Specs
CCR core 10G Arista networking equipment
The UB-HPC cluster contains several partitions available only to academic users. These partitions are comprised of various Linux "nodes" (a.k.a. servers) with differing hardware specs and manufactured by several different vendors. The hardware is similar enough that when networked together, users can run complex problems across many nodes to complete their problems faster. This is known as a "Beowulf cluster."
For information about the nodes in the "Industry" partition of this cluster, please see this page.
On this page:
Fun Fact!
In the history of the English language, Beowulf is the earliest surviving epic poem written in English. It is a story about a hero with the strength of many men who defeated a fearsome monster called Grendel. In computing, a Beowulf class cluster computer is a multicomputer architecture used for parallel computations, i.e., it uses many computers together so that it has the brute force to defeat fearsome number crunching problems.
Disk Layout
- /user, User $HOME directories, NFS mounted from the CCR SAN to the compute nodes and front-ends.
- /scratch, Primary high-performance scratch space, located on each compute node (see above for what is available on each /scratch, as it varies for each type of node).
- Accessible through SLURM, which will automatically create a unique scratch directory in /scratch for each new batch job
- All scratch space will be scrubbed automatically at the end of each batch job. Files that need to be stored long term should be kept elsewhere.
- /panasas/scratch, globally accessible high-performance parallel scratch space for staging/preserving data between runs. Being removed from service 2024
- /vscratch, globally accessible high-performance parallel scratch space for staging/preserving data between runs. New for users in 2024
Ordered from oldest to newest:
These servers are for interactive use, job submissions, and debugging code. CPU time limit of 30 minutes in effect to prevent users from running software on the login servers
- Pool hostname = vortex.ccr.buffalo.edu --> use this name to be put on one of the front end servers. This helps to distribute the load. Logging into vortex will put you on one of these two servers (more may be added in the future):
- Hostname = vortex1.ccr.buffalo.edu and vortex2.ccr.buffalo.edu
- Vendor = Dell
- Number of Processor Cores = 32
- Processor Description:
- Intel Xeon Gold 6130 CPU @ 2.10GHz Processor
- 2 sockets, 16 cores per socket
- Main memory size: 192 GB
Accessible only through the SLURM scheduler
- Dell PowerEdge R640 - dual socket CPUs, 16 cores per socket
- Number of nodes = 16
- Vendor = Dell
- Number of CPU cores = 16
- Number of threads = 32
- Processor Description:
- 32 x 2.10GHz Intel Xeon Gold 6130
- Main memory size: 768 GB
- Instruction cache size: 32 Kbytes
- Data cache size: 32 Kbytes
- Secondary unified instruction/data cache size: 1024 Kbytes
- 32 x 2.10GHz Intel Xeon Gold 6130
- Omni-Path HFI Silicon 100 Series Network Card
- Local scratch disk space is approximately 3.5TB
- Number of nodes = 16
Accessible only through the SLURM scheduler
- PowerEdge R740 - dual socket CPUs, 16 cores per socket
- Number of nodes = 16
- Vendor = Dell
- Number of CPU cores = 16
- Number of threads = 32
- Processor Description:
- 32 x 2.10GHz Intel Xeon Gold 6130 CPU
- Main memory size: 192 GB
- Clock speed: 1810MHz
- Instruction cache (L1) size: 32 Kbytes
- Data cache (L1) size: 32 Kbytes
- Secondary unified instruction/data cache (L2) size: 1024 Kbytes
- 32 x 2.10GHz Intel Xeon Gold 6130 CPU
- GPU Description:
- Number of GPUs: 2
- NVIDIA Volta Tesla V100 PCIe GPUs
- 16GB HBM2 Memory in each card
- Memory bandwidth 900 GB/sec
- Double-precision performance: 7 TFLOPS
- Single-precision performance: 14 TFLOPS
- Omni-Path HFI Silicon 100 Series Network Card
- Local scratch is approximately 827GB
Accessible only through the SLURM scheduler: --constraint=mri
- PowerEdge R440 - dual socket, 16 cores per socket
- Number of nodes = 86
- Vendor = Dell
- Number of CPU cores = 16
- Number of threads = 32
- Processor Description:
- 32 x 2.10GHz Intel Xeon Gold 6130 CPU
- Main memory size: 192 GB
- Clock speed: 1810MHz
- Instruction cache (L1) size: 32 Kbytes
- Data cache (L1) size: 32 Kbytes
- Secondary unified instruction/data cache (L2) size: 1024 Kbytes
- 32 x 2.10GHz Intel Xeon Gold 6130 CPU
- Omni-Path HFI Silicon 100 Series Network Card
- Local scratch disk space is approximately 827GB
Accessible only through the SLURM scheduler: --constraint=nih
- PowerEdge R440 - dual socket, 20 cores per socket
- Number of nodes = 96
- Vendor = Dell
- Number of CPU cores = 20
- Number of threads = 40
- Processor Description:
- 40 x 2.10GHz Intel Xeon Gold 6230 CPU
- Main memory size: 192 GB
- Memory type: DDR4-2933
- Maximum memory speed: 2933MHz
- Instruction cache (L1) size: 32 Kbytes
- Data cache (L1) size: 1 Mbytes
- Shared cache (L3) size: 27.5 Mbytes
- 40 x 2.10GHz Intel Xeon Gold 6230 CPU
- Number of nodes = 96
- Infiniband Network Card
- Local scratch disk space is approximately 835GB
Accessible only through the SLURM scheduler: --constraint=nih
- PowerEdge C6420 - dual socket, 20 cores per socket
- Number of nodes = 24
- Vendor = Dell
- Number of CPU cores = 20
- Number of threads = 40
- Processor Description:
- 40 x 2.10GHz Intel Xeon Gold 6230 CPU
- Main memory size: 754GB
- Memory type: DDR4-2933
- Maximum memory speed: 2933MHz
- Instruction cache (L1) size: 32 Kbytes
- Data cache (L1) size: 1 Mbytes
- Shared cache (L3) size: 27.5 Mbytes
- 40 x 2.10GHz Intel Xeon Gold 6230 CPU
- Number of nodes = 24
- Infiniband Network Card
- Local scratch disk space is approximately 3.5TB
Accessible only through the SLURM scheduler: --constraint=nih
- PowerEdge R740 - dual socket, 20 cores per socket
- Number of nodes = 8
- Vendor = Dell
- Number of CPU cores = 20
- Number of threads = 40
- Processor Description:
- 40 x 2.10GHz Intel Xeon Gold 6230 CPU
- Main memory size: 192 GB
- Memory type: DDR4-2933
- Maximum memory speed: 2933MHz
- Instruction cache (L1) size: 32 Kbytes
- Data cache (L1) size: 1 Mbytes
- Shared cache (L3) size: 27.5 Mbytes
- 40 x 2.10GHz Intel Xeon Gold 6230 CPU
- Number of nodes = 8
- Infiniband Network Card
- Local scratch disk space is approximately 835GB
GPU Description:
- Number of GPUs: 2
- NVIDIA Volta Tesla V100 PCIe GPUs
- 32GB HBM2 Memory in each card
- Memory bandwidth 900 GB/sec
- Double-precision performance: 7 TFLOPS
- Single-precision performance: 14 TFLOPS
Accessible only through the SLURM scheduler
- PowerEdge R440 - dual socket, 28 cores per socket
- Number of nodes = varies - shared with industry cluster
- Vendor = Dell
- Number of CPU cores = 2
- Number of threads = 28
- Processor Description:
- 28 x 2.10GHz Intel Xeon Gold 6330 CPU
- Main memory size: varies from 512 GB - 1TB
- Memory type: DDR4-2933
- Maximum memory speed: 2933MHz
- Instruction cache (L1) size: 42 Kbytes
- Data cache (L1) size: 1 Mbytes
- Shared cache (L3) size: 27.5 Mbytes
- 28 x 2.10GHz Intel Xeon Gold 6330 CPU
- Number of nodes = varies - shared with industry cluster
- HDR Infiniband Network Card
- Local scratch disk space is approximately 960GB
- PowerEdge R440 - dual socket, 28 cores per socket
- Number of nodes = varies - shared with industry cluster
- Vendor = Dell
- Number of CPU cores = 2
- Number of threads = 28
- Processor Description:
- 28 x 2.10GHz Intel Xeon Gold 6330 CPU
- Main memory size: 512 GB
- Memory type: DDR4-2933
- Maximum memory speed: 2933MHz
- Instruction cache (L1) size: 42 Kbytes
- Data cache (L1) size: 1 Mbytes
- Shared cache (L3) size: 27.5 Mbytes
- 28 x 2.10GHz Intel Xeon Gold 6330 CPU
- Number of nodes = varies - shared with industry cluster
- HDR Infiniband Network Card
- Local scratch disk space is approximately 960GB
GPU Description:
- Number of GPUs: 2
- NVIDIA Volta Tesla A100 PCIe GPUs
- 40GB HBM2e Memory in each card
- Memory bandwidth 1,556 GB/sec
- Double-precision performance: 9.746 TFLOPS
- Single-precision performance: 19.49 TFLOPS