# TTT4HPC - Resources episode

###### tags: `Training` `TTT4HPC`

Day 1 of TTT4HPC - Tue 16/April/2024: 10-12, 13-14.30 EEST

## Instructors

Richard Darst, Simo Tuomisto, Diana Iusan, Dhanya Pushpadas, Radovan Bast, Jarno Rantaharju

## Learning outcomes

- Can check what resources their HPC job uses and request appropriate resources
- Understands the meaning of different resource types
- Can anticipate load on the file system and store files in an appropriate format

## Course page

- who will create it?

## Time table

- Section 1: Live stream on TwitchTV 2h

- 1.1 Benchmarking & choosing bjob parameters (50 min)
- DI, RB

- 1.2 I/O best practices (50 min)
- JR, ST

- Lunch break

- Section 2: Zoom exercises 1.5h
- DP
- 2-3 exercises from both sections, prepared by Jarno and Diana

## Sections

### 1.1 Benchmarking & choosing job parameters

50 min

- Diana + Radovan

- How to create a test case from the actual code
- Example code that does, say 50 iterations and takes "forever" -> show how to reduce the run time without compromising the profiling
- How to find out what resources the code uses

- material from Norway that can be used as a starting point
- https://documentation.sigma2.no/jobs/choosing-memory-settings.html
- https://documentation.sigma2.no/jobs/choosing-number-of-cores.html
- typically we present it as demo where we go through some of the steps
* Aalto has this document about how to determine the program size / execution length: https://scicomp.aalto.fi/triton/usage/program-size/#how-big-is-my-program

### 1.2 Monitoring and improving I/O

* Three points
* Filesystem can be the slowest part of many jobs
* networked filesystems tend to be best at large files, bad at many small
* This point mainly applies to file reading (writing commonly is independent)
* Also ( at least to my knowledge, they deal well with sequential reads, not necessarily large files, so if you try to randomly access your large file, the IO speed is not better than many small ones)
* Tools to run to see the performance
* Motivating example/exercise:
* we demonstrate a tool that is accessible to all which demonstrates this
* https://github.com/simo-tuomisto/data-format-tests
* https://users.aalto.fi/~tuomiss1/data-format-tests/talk/io-presentation.html
* You can check single command's IO calls with: `strace -c -e trace=%file,read,write ./command`