# compute resources

- In general, the offers are limited to members of I-MATH & DM3L and their students.
- Other UZH members are welcome to contact support@math.uz.ch .
- First time user: contact support@math.uzh.ch first.

### Reservations

Make reservations on [https://made-fb1.math.uzh.ch/](https://made-fb1.math.uzh.ch/)

- Short (< 1 day) computations don't need a reservation,
- Longer (> 1 day) are recommended to be done with a reservation.
- With reservation: In case of maintenance need, we typically schedule the maintenance according to the needs of the user.
- Without reservation: We try to contact the user and wait several hours for an answer.
- General: if we see running computations, we always contact the user first (independent of whether or not a reservation exists). If the user doesn't answer (weekend, holiday, ...) we decide on our own what to do.

### Offers

- For small / ad hoc computations, just start the compute software via the ThinLinc environment.
- For medium computations, please contact support@math.uzh.ch to ask for the best fitting possibility and to reserve nodes.
- For large computations and professional support please check [http://www.s3it.uzh.ch/.](http://www.s3it.uzh.ch/.)

### Resources at I-MATH & DM3L

<table id="bkmrk-hostpurposecpugpuram" style="width: 100%; border-collapse: collapse;"><thead><tr><th scope="col">**Host**</th><th scope="col">**Purpose**</th><th scope="col">**CPU**</th><th scope="col">**GPU**</th><th scope="col">**RAM**</th><th scope="col">**Local disks**</th><th scope="col">**Purchase**</th><th scope="col">**OS**</th></tr></thead><tbody><tr><td>Model</td><td>Speed</td><td>Cores</td><td>Model</td><td>VRAM</td></tr><tr><td>aroca</td><td>research</td><td>[AMD EPYC 9334](https://www.amd.com/de/products/processors/server/epyc/4th-generation-9004-and-8004-series/amd-epyc-9334.html)^1^ x2</td><td>2.70 GHz</td><td>64 (HT)</td><td>[NVIDIA RTX 6000 Ada](https://www.nvidia.com/en-us/products/workstations/rtx-6000/)</td><td>48 GB</td><td>512 GB</td><td>2x 240 GB, 8TB NVMe</td><td>12/2024</td><td>Ubuntu 24.04</td></tr><tr><td>bellow</td><td>research</td><td>[AMD EPYC 9334](https://www.amd.com/de/products/processors/server/epyc/4th-generation-9004-and-8004-series/amd-epyc-9334.html)^1^ x2</td><td>2.70 GHz</td><td>64 (HT)</td><td>[NVIDIA L40S](https://www.nvidia.com/de-de/data-center/l40s/) x4</td><td>48 GB</td><td>512 GB</td><td>2x 240 GB, 8TB NVMe</td><td>12/2024</td><td>Ubuntu 24.04</td></tr><tr><td>sulem</td><td>research</td><td>[AMD EPYC 9334](https://www.amd.com/de/products/processors/server/epyc/4th-generation-9004-and-8004-series/amd-epyc-9334.html)^1^ x2</td><td>2.70 GHz</td><td>64 (HT)</td><td>[NVIDIA A40](https://www.nvidia.com/de-de/data-center/a40/) x2 (stacked)</td><td>96 GB</td><td>512 GB</td><td>2x 240 GB, 8TB NVMe</td><td>12/2024</td><td>Ubuntu 24.04</td></tr><tr><td>zahedi</td><td>research</td><td>[AMD EPYC 9334](https://www.amd.com/de/products/processors/server/epyc/4th-generation-9004-and-8004-series/amd-epyc-9334.html)^1^ x2</td><td>2.70 GHz</td><td>64 (HT)</td><td>[NVIDIA L40S](https://www.nvidia.com/de-de/data-center/l40s/)</td><td>48 GB</td><td>512 GB</td><td>2x 240 GB, 8TB NVMe</td><td>12/2024</td><td>Ubuntu 24.04</td></tr><tr><td>ribeiro</td><td>research</td><td>[AMD EPYC 7763](https://www.amd.com/de/products/processors/server/epyc/7003-series/amd-epyc-7763.html)^2^</td><td>2,45 GHz</td><td>64 (HT)</td><td>[NVIDIA L40S](https://www.nvidia.com/de-de/data-center/l40s/)</td><td>48 GB</td><td>512 GB</td><td>2x240 GB, 8TB NVMe</td><td>12/2024</td><td>Ubuntu 24.04</td></tr><tr><td>sushko</td><td>research</td><td>[AMD EPYC 7763](https://www.amd.com/de/products/processors/server/epyc/7003-series/amd-epyc-7763.html)^2^</td><td>2,45 GHz</td><td>64 (HT)</td><td>[NVIDIA L40S](https://www.nvidia.com/de-de/data-center/l40s/)</td><td>48 GB</td><td>512 GB</td><td>2x240 GB, 8TB NVMe</td><td>12/2024</td><td>Ubuntu 24.04</td></tr><tr><td>swart</td><td>research</td><td>[AMD EPYC 7763](https://www.amd.com/de/products/processors/server/epyc/7003-series/amd-epyc-7763.html)^2^</td><td>2,45 GHz</td><td>64 (HT)</td><td>[NVIDIA L40S](https://www.nvidia.com/de-de/data-center/l40s/)</td><td>48 GB</td><td>512 GB</td><td>2x240 GB, 8TB NVMe</td><td>12/2024</td><td>Ubuntu 24.04</td></tr><tr><td>vernon</td><td>research</td><td>[AMD EPYC 7763](https://www.amd.com/de/products/processors/server/epyc/7003-series/amd-epyc-7763.html)^2^</td><td>2,45 GHz</td><td>64 (HT)</td><td>[NVIDIA L40S](https://www.nvidia.com/de-de/data-center/l40s/)</td><td>48 GB</td><td>512 GB</td><td>2x240 GB, 8TB NVMe</td><td>12/2024</td><td>Ubuntu 24.04</td></tr><tr><td>vinzant</td><td>research</td><td>[AMD EPYC 7763](https://www.amd.com/de/products/processors/server/epyc/7003-series/amd-epyc-7763.html)^2^</td><td>2,45 GHz</td><td>64 (HT)</td><td>[NVIDIA L40S](https://www.nvidia.com/de-de/data-center/l40s/)</td><td>48 GB</td><td>512 GB</td><td>2x240 GB, 8TB NVMe</td><td>12/2024</td><td>Ubuntu 24.04</td></tr><tr><td>wheeler</td><td>research</td><td>[AMD EPYC 7763](https://www.amd.com/de/products/processors/server/epyc/7003-series/amd-epyc-7763.html)^2^</td><td>2,45 GHz</td><td>64 (HT)</td><td>[NVIDIA L40S](https://www.nvidia.com/de-de/data-center/l40s/)</td><td>48 GB</td><td>512 GB</td><td>2x240 GB, 8TB NVMe</td><td>12/2024</td><td>Ubuntu 24.04</td></tr><tr><td>willbanks</td><td>research</td><td>[AMD EPYC 7763](https://www.amd.com/de/products/processors/server/epyc/7003-series/amd-epyc-7763.html)^2^</td><td>2,45 GHz</td><td>64 (HT)</td><td>---</td><td>512 GB</td><td>2x 240 GB, 8TB NVMe</td><td>12/2024</td><td>Ubuntu 24.04</td></tr><tr><td>terracini</td><td>research</td><td>[AMD EPYC 7763](https://www.amd.com/de/products/processors/server/epyc/7003-series/amd-epyc-7763.html)^2^</td><td>2,45 GHz</td><td>64 (HT)</td><td>---</td><td>512 GB</td><td>2x 240 GB, 8TB NVMe</td><td>12/2024</td><td>Ubuntu 24.04</td></tr><tr><td>rambo</td><td>research</td><td>[AMD-EPYC-7742](https://www.amd.com/de/products/cpu/amd-epyc-7742)</td><td>2,25 GHz</td><td>64 (HT)</td><td>---</td><td>512 GB</td><td>240GB, 2TB</td><td>08/2021</td><td>Ubuntu 24.04</td></tr><tr><td>olive</td><td>research</td><td>[AMD-EPYC-7502P](https://www.amd.com/de/products/cpu/amd-epyc-7502p)</td><td>2,50 GHz</td><td>32 (HT)</td><td>---</td><td>512 GB</td><td>240GB</td><td>04/2022</td><td>Ubuntu 24.04</td></tr><tr><td>oja</td><td>research</td><td>[AMD-EPYC-7502P](https://www.amd.com/de/products/cpu/amd-epyc-7502p)</td><td>2,50 GHz</td><td>32 (HT)</td><td>---</td><td>512 GB</td><td>240GB</td><td>04/2022</td><td>Ubuntu 20.04</td></tr><tr><td>stokes</td><td>research</td><td>[AMD-EPYC-7502P](https://www.amd.com/de/products/cpu/amd-epyc-7502p)</td><td>2,50 GHz</td><td>32 (HT)</td><td>---</td><td>512 GB</td><td>240GB</td><td>04/2022</td><td>Ubuntu 24.04</td></tr><tr><td>walker</td><td>student</td><td>[AMD-EPYC-7502P](https://www.amd.com/de/products/cpu/amd-epyc-7502p)</td><td>2,50 GHz</td><td>32 (HT)</td><td>---</td><td>512 GB</td><td>240GB</td><td>04/2022</td><td>Ubuntu 24.04</td></tr><tr><td>Old servers</td></tr><tr><td>brady</td><td>research</td><td>[Intel Core 6C i7-8700K](https://www.intel.com/content/www/us/en/products/processors/core/i7-processors/i7-8700k.html)</td><td>3,70 GHz</td><td>12 (HT)</td><td>[GigaByte GeForce GTX 1070 G1](https://www.gigabyte.com/ch/Graphics-Card/GV-N1070G1-GAMING-8GD-rev-10)</td><td>8 GB</td><td>64 GB</td><td>80GB, 2TB</td><td>02/2018</td><td>Ubuntu 24.04</td></tr><tr><td>david</td><td>research</td><td>[Intel Xeon 10C E7-2850](https://www.intel.com/content/www/us/en/products/sku/53573/intel-xeon-processor-e72850-24m-cache-2-00-ghz-6-40-gts-intel-qpi/specifications.html) x 8</td><td>2,00 GHz</td><td>80 (no HT)</td><td>[GigaByte GeForce GTX 1070 G1](https://www.gigabyte.com/ch/Graphics-Card/GV-N1070G1-GAMING-8GD-rev-10)</td><td>8 GB</td><td>512 GB</td><td>240GB</td><td>05/2013</td><td>Ubuntu 24.04</td></tr><tr><td>estonia0</td><td>research</td><td>[Intel Xeon 6C E5-2640](https://www.intel.com/content/www/us/en/products/sku/64591/intel-xeon-processor-e52640-15m-cache-2-50-ghz-7-20-gts-intel-qpi/specifications.html)</td><td>2.50 GHz</td><td>12 (no HT)</td><td>---</td><td>256 GB</td><td>2 TB</td><td>05/2012</td><td>Ubuntu 24.04</td></tr><tr><td>estonia1</td><td>research</td><td>[Intel Xeon 6C E5-2640](https://www.intel.com/content/www/us/en/products/sku/64591/intel-xeon-processor-e52640-15m-cache-2-50-ghz-7-20-gts-intel-qpi/specifications.html)</td><td>2.50 GHz</td><td>12 (no HT)</td><td>---</td><td>256 GB</td><td>2 TB</td><td>05/2012</td><td>Ubuntu 24.04</td></tr><tr><td>estonia2</td><td>research</td><td>[Intel Xeon 6C E5-2640](https://www.intel.com/content/www/us/en/products/sku/64591/intel-xeon-processor-e52640-15m-cache-2-50-ghz-7-20-gts-intel-qpi/specifications.html)</td><td>2.50 GHz</td><td>12 (no HT)</td><td>---</td><td>256 GB</td><td>2 TB</td><td>05/2012</td><td>Ubuntu 24.04</td></tr><tr><td>estonia3</td><td>research</td><td>[Intel Xeon 6C E5-2640](https://www.intel.com/content/www/us/en/products/sku/64591/intel-xeon-processor-e52640-15m-cache-2-50-ghz-7-20-gts-intel-qpi/specifications.html)</td><td>2.50 GHz</td><td>12 (no HT)</td><td>---</td><td>256 GB</td><td>2 TB</td><td>05/2012</td><td>Ubuntu 24.04</td></tr><tr><td>georgia0</td><td>research</td><td>[Intel Xeon 6C E5-2640](https://www.intel.com/content/www/us/en/products/sku/64591/intel-xeon-processor-e52640-15m-cache-2-50-ghz-7-20-gts-intel-qpi/specifications.html)</td><td>2.50 GHz</td><td>12 (no HT)</td><td>---</td><td>256 GB</td><td>2 TB</td><td>05/2012</td><td>Ubuntu 24.04</td></tr><tr><td>georgia1</td><td>research</td><td>[Intel Xeon 6C E5-2640](https://www.intel.com/content/www/us/en/products/sku/64591/intel-xeon-processor-e52640-15m-cache-2-50-ghz-7-20-gts-intel-qpi/specifications.html)</td><td>2.50 GHz</td><td>12 (no HT)</td><td>---</td><td>256 GB</td><td>2 TB</td><td>05/2012</td><td>Ubuntu 24.04</td></tr><tr><td>georgia2</td><td>research</td><td>[Intel Xeon 6C E5-2640](https://www.intel.com/content/www/us/en/products/sku/64591/intel-xeon-processor-e52640-15m-cache-2-50-ghz-7-20-gts-intel-qpi/specifications.html)</td><td>2.50 GHz</td><td>12 (no HT)</td><td>---</td><td>256 GB</td><td>2 TB</td><td>05/2012</td><td>Ubuntu 24.04</td></tr><tr><td>georgia3</td><td>research</td><td>[Intel Xeon 6C E5-2640](https://www.intel.com/content/www/us/en/products/sku/64591/intel-xeon-processor-e52640-15m-cache-2-50-ghz-7-20-gts-intel-qpi/specifications.html)</td><td>2.50 GHz</td><td>12 (no HT)</td><td>---</td><td>256 GB</td><td>2 TB</td><td>05/2012</td><td>Ubuntu 24.04</td></tr><tr><td>iran0</td><td>research</td><td>[Intel Xeon 6C E5-2640](https://www.intel.com/content/www/us/en/products/sku/64591/intel-xeon-processor-e52640-15m-cache-2-50-ghz-7-20-gts-intel-qpi/specifications.html)</td><td>2.50 GHz</td><td>12 (no HT)</td><td>---</td><td>256 GB</td><td>2 TB</td><td>05/2012</td><td>Ubuntu 24.04</td></tr><tr><td>iran1</td><td>research</td><td>[Intel Xeon 6C E5-2640](https://www.intel.com/content/www/us/en/products/sku/64591/intel-xeon-processor-e52640-15m-cache-2-50-ghz-7-20-gts-intel-qpi/specifications.html)</td><td>2.50 GHz</td><td>12 (no HT)</td><td>---</td><td>256 GB</td><td>2 TB</td><td>05/2012</td><td>Ubuntu 24.04</td></tr><tr><td>iran2</td><td>research</td><td>[Intel Xeon 6C E5-2640](https://www.intel.com/content/www/us/en/products/sku/64591/intel-xeon-processor-e52640-15m-cache-2-50-ghz-7-20-gts-intel-qpi/specifications.html)</td><td>2.50 GHz</td><td>12 (no HT)</td><td>---</td><td>256 GB</td><td>2 TB</td><td>05/2012</td><td>Ubuntu 24.04</td></tr><tr><td>iran3</td><td>research</td><td>[Intel Xeon 6C E5-2640](https://www.intel.com/content/www/us/en/products/sku/64591/intel-xeon-processor-e52640-15m-cache-2-50-ghz-7-20-gts-intel-qpi/specifications.html)</td><td>2.50 GHz</td><td>12 (no HT)</td><td>---</td><td>256 GB</td><td>2 TB</td><td>05/2012</td><td>Ubuntu 24.04</td></tr><tr><td>jordan0</td><td>professors &amp; assistants: matlab, mathematica, maple, R</td><td>[Intel Xeon 6C E5-2640](https://www.intel.com/content/www/us/en/products/sku/64591/intel-xeon-processor-e52640-15m-cache-2-50-ghz-7-20-gts-intel-qpi/specifications.html)</td><td>2.50 GHz</td><td>12 (no HT)</td><td>---</td><td>256 GB</td><td>2 TB</td><td>05/2012</td><td>Ubuntu 24.04</td></tr><tr><td>jordan1</td><td>research</td><td>[Intel Xeon 6C E5-2640](https://www.intel.com/content/www/us/en/products/sku/64591/intel-xeon-processor-e52640-15m-cache-2-50-ghz-7-20-gts-intel-qpi/specifications.html)</td><td>2.50 GHz</td><td>12 (no HT)</td><td>---</td><td>256 GB</td><td>2 TB</td><td>05/2012</td><td>Ubuntu 24.04</td></tr><tr><td>jordan2</td><td>courses</td><td>[Intel Xeon 6C E5-2640](https://www.intel.com/content/www/us/en/products/sku/64591/intel-xeon-processor-e52640-15m-cache-2-50-ghz-7-20-gts-intel-qpi/specifications.html)</td><td>2.50 GHz</td><td>12 (no HT)</td><td>---</td><td>256 GB</td><td>2 TB</td><td>05/2012</td><td>Ubuntu 24.04</td></tr><tr><td>jordan3</td><td>courses</td><td>[Intel Xeon 6C E5-2640](https://www.intel.com/content/www/us/en/products/sku/64591/intel-xeon-processor-e52640-15m-cache-2-50-ghz-7-20-gts-intel-qpi/specifications.html)</td><td>2.50 GHz</td><td>12 (no HT)</td><td>---</td><td>256 GB</td><td>2 TB</td><td>05/2012</td><td>Ubuntu 24.04</td></tr></tbody></table>

Mainboard:

- 1 [ASUS RS720A-E12-RS12](https://servers.asus.com/products/servers/rack-servers/RS720A-E12-RS12#Specifications)
- 2 [Supermicro H12SSL-CT](https://www.supermicro.com/en/products/motherboard/H12SSL-CT)

## General

### Access

- The compute nodes can be accessed **by SSH** **only** and only from **inside** I-MATH/DM3L.
- Option A: Use **ThinLinc** as starting point 
    - MatLab, Mathematica, Maple, R, RStudio: start the application via menu 'Applications &gt; Science &gt; ...' - you will be redirected automatically to the appropriate compute server.
    - For **non standard compute software** (e.g. python programs) please log on the compute node via `ssh` (no jumphost required), then start the program.
- Option B: **SSH -** sign into `ssh.math.uzh.ch` (=jumphost) and jump to the compute node again via `ssh`.

#### *Best practice:* Combinated SSH setup for compute user

<p class="callout info">The following described setup is a multi step and complicated setup - use it, if you frequently have to log on and/or to make the log in as easy as possible. Invest 20mins to do the setup, if you fail: contact our support ... the benefits are worth.</p>

- Benefit: no password required, only **one** command to get a **running ssh session**.
- Cost: setup has to be done on every computer you want to use for the ssh session.

##### 1) Initial SSH login on ssh.math.uzh.ch

To use 'ssh', you need a ssh-client program:

- MS Windows: [install/activate](https://learn.microsoft.com/en-us/windows/wsl/install) WSL (suggested Distro: Ubuntu).
- Linux / MacOS X: ssh is already installed.

Open a terminal (WIndows: CMD) and type `ssh ssh.math.uzh.ch`:

```
$ ssh ssh.math.uzh.ch
Welcome to Ubuntu 22.04.5 LTS (GNU/Linux 5.15.0-186-generic x86_64)
...
```

- If you do this the first time, you have to accept the host Key.
- Without private/public key setup (next step), you will be asked for your password (same as thinlinc) and [MFA token](https://wiki.math.uzh.ch/public/books/public-wiki/page/mfa-i-math-dm3l).
- Keep this windows open during the next step.

##### 2) SSH: private / public key setup

Optional: [more detailed](https://wiki.math.uzh.ch/public/books/public-wiki/page/ssh#bkmrk-working-with-ssh-key) description.

- Open a second terminal and type `ssh-keygen -t ed25519`:

```
[your computer]
$ ssh-keygen -t ed25519
Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/.../.ssh/id_ed25519): 
Enter passphrase (empty for no passphrase):
...
Your identification has been saved in /home/...
Your public key has been saved in /home/...
...

$ cat ~/.ssh/id_ed25519.pub
ssh-ed25519 AAA....
```

- Show the new created public key and copy the content (on line 12: ssh-ed25519...) to the clipboard.
- Go back to the first terminal where you are still logged in by ssh on host ssh.math.uzh.ch. Replace everything between the double quotes with your newly created public key (it's in your clipboard).   
    Hint: after typing `echo "` use the right mouse button and choose *paste*, than type the rest `" >> ~/.ssh/authhorized_keys` and presse `ENTER`.

```
$ echo "ssh-ed25519 AAA..." >> ~/.ssh/authhorized_keys
```

- Final check: open a third terminal and type again `ssh ssh.math.uzh.ch` - you should be logged in without providing a password or MFA token.

##### 3) Automated login on final compute host 

##### 4) SSH session resumeable

### Operating System

- Ubuntu LTS (Linux) - all compute servers.
- Very very limited software might be installed on some Windows virtual machines.

### Storage

- If you need more disk space, please contact support@math.uzh.ch .
- Remote (NFS) 
    - Personal Home directory
    - `/compute/`
- Local 
    - `/export/user/`

### Programs / Software

- All compute servers use the same applications and versions as installed on the ThinLinc terminals.
- Please report missing or outdated software or wishes to support@math.uzh.ch .
- There is no Intel C or Intel Fortran Compiler.
- If you need other program versions than the default one: open a terminal, type the program name followed by two 'TAB' presses: this will show all available installed versions of the specified program.

### Parallel computing

- Cluster software 
    - Not installed / offered at I-MATH/DM3L.
    - Various MPI packages and libs are installed. Compiling and preparation to run programs later on a distributed compute cluster are possible.
- Some programs, like `matlab`, offer limited built-in auto parallelization.
- All of the compute nodes are shared memory machines. It's reasonable to start as many programs in parallel as long the `load` is less or equal the number of cores. 
    - Determine the load by using the command `uptime` via a terminal. The last three numbers are the average number of jobs (=load) in the run queue over the last 1, 5 and 15 minutes.

```bash
$ uptime
 15:55:20 up 22 days, 24 min,  1 user,  load average: 11.97, 11.73, 11.62
```

#### Monitoring

- List GPUs

```bash
nvidia-smi
```

- Show GPU-Usage (Load, Temperature, Watts-Usage etc.)

```bash
nvtop -d 2
```

- Fan status (Rounds per Minute etc.)

```bash
fanstatus
```

### Limitations

- Memory 
    - Soft: 64GB
    - Hard: unlimited
- Number of processes per user 
    - Soft: 2048
    - Hard: 4096
- Change limitations

```bash
$ ulimit -m unlimited
$ ulimit -u 4096
```

- Make sure to increase a hard limit only one time. If you try to increase it a second time, you'll get an error. You have to log out and log in again to get a new chance.