or
or
By clicking below, you agree to our terms of service.
New to HackMD? Sign up
Syntax | Example | Reference | |
---|---|---|---|
# Header | Header | 基本排版 | |
- Unordered List |
|
||
1. Ordered List |
|
||
- [ ] Todo List |
|
||
> Blockquote | Blockquote |
||
**Bold font** | Bold font | ||
*Italics font* | Italics font | ||
~~Strikethrough~~ | |||
19^th^ | 19th | ||
H~2~O | H2O | ||
++Inserted text++ | Inserted text | ||
==Marked text== | Marked text | ||
[link text](https:// "title") | Link | ||
 | Image | ||
`Code` | Code |
在筆記中貼入程式碼 | |
```javascript var i = 0; ``` |
|
||
:smile: | ![]() |
Emoji list | |
{%youtube youtube_id %} | Externals | ||
$L^aT_eX$ | LaTeX | ||
:::info This is a alert area. ::: |
This is a alert area. |
On a scale of 0-10, how likely is it that you would recommend HackMD to your friends, family or business associates?
Please give us some advice and help us improve HackMD.
Do you want to remove this version name and description?
Syncing
xxxxxxxxxx
Using Python in an HPC environment
Welcome!
Link to this document: https://hackmd.io/@bclaremar/HPC-python
Link to first part of Q/A: https://hackmd.io/IoseeprATrWH0ZI7z6wJ-w?view
Lecture Zoom link: https://umu.zoom.us/j/68897489502?pwd=TXlraXI5UkFvNWxIV0xGcWZ4M1lQdz09
Separate UPPMAX session Zoom: https://uu-se.zoom.us/j/68160935750
Password: 298853
Separate HPC2N session Zoom: https://umu.zoom.us/j/64721465122?pwd=NGtZM2h0bmJBTU4xNThjT0t5eFZndz09
Course material: https://uppmax.github.io/HPC-python/index.html
MPI session
ctypes
is a low-level good start to interface already existing C code without automatic "intent" guessing, also#include <Python.h>
is a slightly harder way to write a python module in C. I have written some examples for my students here--reservation=snic2022-22-641_1
as slurm flag in bash script or in command line forsbatch
orinteractive
Q: Why keeps this hackmd deleting what I write? Should I setup something?
Q: Cores vs. processes vs. threads, what is the difference?
A: Core: part of a multicore CPU, processor. Hardware. Think of it like a small CPU, I guess, inside the CPU. Usually cores share memory with other cores in the same socket on the CPU. Process: it's a task or instance of a program that is active. Thread: software. A process can do stuff concurrently by having several threads. Shares memory
A: so with MPI
is copied to several instances (tasks)
they work independently from each-other but has to communicate to exchange data and to wait for some calculations to be finished before continuing.
A: https://davescomputertips.com/cpu-cores-versus-threads-explained/
A: https://www.geeksforgeeks.org/difference-between-process-and-thread/
A: https://smileipic.github.io/Smilei/parallelization.html
A: something that really improved my low-level optimization skills was understanding of CPU/GPU memory cache levels and how to chunk memory between communication and calculations (this seems to explain levels pretty OK: https://www.makeuseof.com/tag/what-is-cpu-cache/ )
A: A node is the physical box containing a computer. An hpc cluster usually has several nodes.
A: At UPPMAX our clusters has multi threading turned off so 1core=1thread. The same at HPC2N.
Q: In the GPU example you login to snowy with
interactive
but in the snowy user guide they write that you must usesalloc
. So can I useinteractive
for snowy?interactive
is a wrapper aroundsalloc
, it works for Snowy too.Machine Learning session
UPPMAX session - Conda and Bianca
Q: Once I got an error "the environment is inconsistent, please check the package plan carefully" when I used conda before. How to solve it?
conda install
andpip install
for the same environment. Many answers seem to suggest thatconda update --all
could solve the issue.Q:
Q:
HPC2N session: Machine learning (continued)
Q/A on demand
Storage basics UPPMAX and similar to HPC2N
All nodes can access:
If you’re reading/writing a file once, use a directory on Crex or Castor
If you’re reading/writing a file many times:
Copy the file to ”scratch”, the node local disk:
cp myFile
$SNIC_TMP
Exercise help
Please write the number of the breakout room here if you need help:
Archive Q&A from the earlier sessions: https://hackmd.io/@dianai/HPC-Python-2022