# MPI learning(用法)
[TOC]
## Init
example code
```c=
#include <stdio.h>
#include <stdlib.h>
#include <mpi.h>
int main(int argc, char *argv[])
{
MPI_Init(&arvc, &argv); //no mpi calls before this line
int world_size, process_rank;
//MPI_Comm_size: number of processes
MPI_Comm_size(MPI_COMM_WORLD, &world_size);
//MPI_Comm_rank: this process's rank(mpi裡的id)
MPI_Comm_rank(MPI_COMM_WORLD, &process_rank);
printf("Hello, world from rank %d/%d\n", process_rank, world_size);
MPI_Finalize(); //no mpi calls after this line
return;
}
```
result in two processes
```
Hello, world from rank 1/2
Hello, world from rank 0/2
```
Compile and execute
```
mpiicc -o filename filename.c
mpiexec -n 8 ./filename
```
## Communicators
Communicators: process that can sendsend messages to each other
MPI_Init 會產生兩個communicators MPI_COMM_WORLD和MPI_COMM_SELF
* WORLD: 所有參與運算的CPU
* SELF: 自己的CPU
## Sending and Recieving
**point to point communication:**
屬於相同Communicator的CPU之間互相傳送。
```c
MPI_SEND (MSG,MSG_SIZE,DATA_TYPE,IDEST,ITAG,MPI_COMM)
```
* MSG: 訊息本人
* MSG_SIZE: 大小，if>1 data即為陣列
* DATA_TYPE: MPI_datatype
* IDEST: 目的CPU id
* ITAG: 送出的資料標籤
* MPI_COMM: communicator
```c
MPI_RECV(MSG,MSG_SIZE,DATA_TYPE,ISRC,ITAG,MPI_COMM,&status)
```
* status: 有source、tag、error
receiver可以在不知道data_size、sender或tag的情況下收件。
```c
MPI_GET_COUNT(ISTATUS,DATA_TYPE,ISIZE)
```
### example
假設要由process0收資料
```c=
#include <stdio.h>
#include <mpi.h>
int main(int argc, char *argv[])
{
MPI_Init(&argc, &argv); //no mpi calls before this line
int world_size, process_rank;
//MPI_Comm_size: number of processes
MPI_Comm_size(MPI_COMM_WORLD, &world_size);
//MPI_Comm_rank: this process's rank(mpi裡的id)
MPI_Comm_rank(MPI_COMM_WORLD, &process_rank);
int recv_data;
int total = 0;
if (process_rank == 0)
{
MPI_Status status;
int i;
for (i = 1; i < world_size; i++)
{
//傳輸data要寫address
MPI_Recv(&recv_data, 1, MPI_INT, i, 1, MPI_COMM_WORLD, &status);
total = total + recv_data;
}
}
else
{
MPI_Send(&process_rank, 1, MPI_INT, 0, 1, MPI_COMM_WORLD);
}
printf("I am %d/%d,total = %d ,finished\n", process_rank, world_size, total);
MPI_Finalize(); //no mpi calls after this line
return 0;
}
```
result
* 可知for迴圈不會有等待data的問題，且不一定是0最後完成
## I/O
只有process0才可以access stdin，所有procees都是透過process0做讀寫(傳資料給其他process或其他process送資料來)
## Reduce
把大家的運算結果蒐集起來到一個process
(recvbuf會蒐集所有人的結果，大家的DEST都一樣，會收在同一個process，recv_buf只有DEST process可以使用)
```c
MPI_Reduce(sendbuf,recvbuf,DATA_SIZE,DATA_TYPE,operator,DEST,MPI_COMM)
```
* reduce的matching是以在process中的呼叫順序(如:process0的第一個配process1的第一個，如果順序改了match到別人)
### collective vs point to point
* 同communicator中的所有process要使用相同collective function(同時使用MPI_Recv和MPI_Reduce很有可能出現錯誤)
* 不可以有些process reduce到0有些reduce到1(同一條reduce線)
* reduce不須使用tag
### Allreduce
如果所有process都需要同一個result時(不須指定DEST)
```c
MPI_Allreduce(sendbuf,recvbuf,DATA_SIZE,DATA_TYPE,operator,MPI_COMM)
```
## Broadcast
某個資料只有一條process擁有，但是所有process都需要
(ex:process0把input給大家)
```c
MPI_Bcast(share_msg, msg_size, type, source_process, MPI_COMM_WORLD);
```
使用範例
```c=
//這裡寫寫看pointer為msg的方式
int *input, tmp;
tmp = 0;
input = &tmp;
if (process_rank == 0)
{
printf("I am process 0\n");
//scan入ptr不需要&
scanf("%d", input);
}
MPI_Bcast(input, 1, MPI_INT, 0, MPI_COMM_WORLD);
```
## 分享vector(Scatter and Gather)
當process0擁有整個vector(array)要分給其他process運算後收回
ex:12345678分給4個process會有4組如(1,2)(3,4)(5,6)(7,8)
### MPI_Scatter
```c
MPI_Scatter(array,send_size,send_type,local_array,recv_size,type,src,MPI_COMM_WORLD);
```
### MPI_Gather
```c
MPI_Gather(array,send_size,send_type,local_array,recv_size,type,dest,MPI_COMM_WORLD);
```
src和recv端的scatter和gather是寫一樣的
使用範例
```c=
//scatter
if (id == 0)
{
for (i = 0; i < n; i++)
{
a[i] = i;
}
MPI_Scatter(a, localn, MPI_INT, locala, localn, MPI_INT, 0, MPI_COMM_WORLD);
}
else
{
MPI_Scatter(a, localn, MPI_INT, locala, localn, MPI_INT, 0, MPI_COMM_WORLD);
}
for (i = 0; i < localn; i++)
{
printf("%d ", locala[i]);
}
printf("\nthis is process %d\n\n", id);
//gather
for (i = 0; i < localn; i++)
{
locala[i] = 7;
}
int *b = NULL;
if (id == 0)
{
b = malloc(n * sizeof(int));
MPI_Gather(locala, localn, MPI_INT, b, localn, MPI_INT, 0, MPI_COMM_WORLD);
printf("I am result\n");
for (i = 0; i < n; i++)
{
printf("%d ", b[i]);
}
printf("\n\n");
}
else
{
MPI_Gather(locala, localn, MPI_INT, b, localn, MPI_INT, 0, MPI_COMM_WORLD);
}
```
原本是123456789的順序送出，後變為777777傳回
但由結果得，就算在print也會有互相競爭
* 4 processes
### Allgather
把大家的結果合併在一起並分享給所有process
把Gather中dest拿掉即可
### Gatherv and Scatterv
當需要指定位置的時候使用
```c=
//scatterv example
int array[8];
//每個process會接收到的陣列大小
int rc_n[] = {2, 2, 2, 2};
//用來接收的陣列
int recv_arr[2];
//照disp位置分給大家
int disp[] = {0, 6, 2, 4};
int i;
if (id == 0)
{
for (i = 0; i < 8; i++)
{
array[i] = i;
}
MPI_Scatterv(array, rc_n, disp, MPI_INT, recv_arr, 2, MPI_INT, 0, MPI_COMM_WORLD);
}
else
MPI_Scatterv(array, rc_n, disp, MPI_INT, recv_arr, 2, MPI_INT, 0, MPI_COMM_WORLD);
for (i = 0; i < 2; i++)
{
printf("%d ", recv_arr[i]);
}
printf("\nthis is process %d\n\n", id);
```
* 4 process result
Sign in with Wallet
Connect another wallet