---
# System prepended metadata

title: MPI learning(用法)

---

# MPI learning(用法)
[TOC]

## Init
example code
```c=
#include <stdio.h>
#include <stdlib.h>
#include <mpi.h>
int main(int argc, char *argv[])
{
    MPI_Init(&arvc, &argv); //no mpi calls before this line

    int world_size, process_rank;
    //MPI_Comm_size: number of processes
    MPI_Comm_size(MPI_COMM_WORLD, &world_size);
    //MPI_Comm_rank: this process's rank(mpi裡的id)
    MPI_Comm_rank(MPI_COMM_WORLD, &process_rank);

    printf("Hello, world from rank %d/%d\n", process_rank, world_size);
    
    MPI_Finalize(); //no mpi calls after this line

    return;
}
```
result in two processes
```
Hello, world from rank 1/2
Hello, world from rank 0/2
```
Compile and execute
```
mpiicc -o filename filename.c
mpiexec -n 8 ./filename
```
## Communicators
Communicators: process that can sendsend messages to each other

MPI_Init 會產生兩個communicators MPI_COMM_WORLD和MPI_COMM_SELF
* WORLD: 所有參與運算的CPU
* SELF: 自己的CPU
## Sending and Recieving
**point to point communication:** 
屬於相同Communicator的CPU之間互相傳送。
```c
MPI_SEND (MSG,MSG_SIZE,DATA_TYPE,IDEST,ITAG,MPI_COMM)
```
* MSG: 訊息本人
* MSG_SIZE: 大小，if>1 data即為陣列
* DATA_TYPE: MPI_datatype
* IDEST: 目的CPU id
* ITAG: 送出的資料標籤
* MPI_COMM: communicator

```c
MPI_RECV(MSG,MSG_SIZE,DATA_TYPE,ISRC,ITAG,MPI_COMM,&status)
```
* status: 有source、tag、error

receiver可以在不知道data_size、sender或tag的情況下收件。
```c
MPI_GET_COUNT(ISTATUS,DATA_TYPE,ISIZE)
```
### example
假設要由process0收資料
```c=
#include <stdio.h>
#include <mpi.h>
int main(int argc, char *argv[])
{
    MPI_Init(&argc, &argv); //no mpi calls before this line

    int world_size, process_rank;
    //MPI_Comm_size: number of processes
    MPI_Comm_size(MPI_COMM_WORLD, &world_size);
    //MPI_Comm_rank: this process's rank(mpi裡的id)
    MPI_Comm_rank(MPI_COMM_WORLD, &process_rank);
    int recv_data;
    int total = 0;
    if (process_rank == 0)
    {
        MPI_Status status;
        int i;
        for (i = 1; i < world_size; i++)
        {
            //傳輸data要寫address
            MPI_Recv(&recv_data, 1, MPI_INT, i, 1, MPI_COMM_WORLD, &status);
            total = total + recv_data;
        }
    }
    else
    {
        MPI_Send(&process_rank, 1, MPI_INT, 0, 1, MPI_COMM_WORLD);
    }
    printf("I am %d/%d,total = %d ,finished\n", process_rank, world_size, total);

    MPI_Finalize(); //no mpi calls after this line

    return 0;
}
```
result
![](https://i.imgur.com/03FsgMx.png)

* 可知for迴圈不會有等待data的問題，且不一定是0最後完成
## I/O
只有process0才可以access stdin，所有procees都是透過process0做讀寫(傳資料給其他process或其他process送資料來)
## Reduce
把大家的運算結果蒐集起來到一個process
(recvbuf會蒐集所有人的結果，大家的DEST都一樣，會收在同一個process，recv_buf只有DEST process可以使用)
```c
MPI_Reduce(sendbuf,recvbuf,DATA_SIZE,DATA_TYPE,operator,DEST,MPI_COMM)
```
* reduce的matching是以在process中的呼叫順序(如:process0的第一個配process1的第一個，如果順序改了match到別人)
### collective vs point to point
* 同communicator中的所有process要使用相同collective function(同時使用MPI_Recv和MPI_Reduce很有可能出現錯誤)
* 不可以有些process reduce到0有些reduce到1(同一條reduce線)
* reduce不須使用tag
### Allreduce
如果所有process都需要同一個result時(不須指定DEST)
```c
MPI_Allreduce(sendbuf,recvbuf,DATA_SIZE,DATA_TYPE,operator,MPI_COMM)
```
## Broadcast
某個資料只有一條process擁有，但是所有process都需要
(ex:process0把input給大家)
```c
MPI_Bcast(share_msg, msg_size, type, source_process, MPI_COMM_WORLD);
```

使用範例
```c=
//這裡寫寫看pointer為msg的方式
int *input, tmp;
tmp = 0;
input = &tmp;
if (process_rank == 0)
{
    printf("I am process 0\n");
    //scan入ptr不需要&
    scanf("%d", input);
}
MPI_Bcast(input, 1, MPI_INT, 0, MPI_COMM_WORLD);
```
## 分享vector(Scatter and Gather)
當process0擁有整個vector(array)要分給其他process運算後收回
ex:12345678分給4個process會有4組如(1,2)(3,4)(5,6)(7,8)
### MPI_Scatter
```c
MPI_Scatter(array,send_size,send_type,local_array,recv_size,type,src,MPI_COMM_WORLD);
```
### MPI_Gather
```c
MPI_Gather(array,send_size,send_type,local_array,recv_size,type,dest,MPI_COMM_WORLD);
```
src和recv端的scatter和gather是寫一樣的
使用範例
```c=
//scatter
    if (id == 0)
    {
        for (i = 0; i < n; i++)
        {
            a[i] = i;
        }
        MPI_Scatter(a, localn, MPI_INT, locala, localn, MPI_INT, 0, MPI_COMM_WORLD);
    }
    else
    {
        MPI_Scatter(a, localn, MPI_INT, locala, localn, MPI_INT, 0, MPI_COMM_WORLD);
    }
    for (i = 0; i < localn; i++)
    {
        printf("%d ", locala[i]);
    }
    printf("\nthis is process %d\n\n", id);

    //gather
    for (i = 0; i < localn; i++)
    {
        locala[i] = 7;
    }
    int *b = NULL;
    if (id == 0)
    {
        b = malloc(n * sizeof(int));
        MPI_Gather(locala, localn, MPI_INT, b, localn, MPI_INT, 0, MPI_COMM_WORLD);
        printf("I am result\n");
        for (i = 0; i < n; i++)
        {
            printf("%d ", b[i]);
        }
        printf("\n\n");
    }
    else
    {
        MPI_Gather(locala, localn, MPI_INT, b, localn, MPI_INT, 0, MPI_COMM_WORLD);
    }
```
原本是123456789的順序送出，後變為777777傳回
但由結果得，就算在print也會有互相競爭
* 4 processes
![](https://i.imgur.com/bWDAlzl.png)
### Allgather
把大家的結果合併在一起並分享給所有process
把Gather中dest拿掉即可
### Gatherv and Scatterv
當需要指定位置的時候使用
```c=
//scatterv example
int array[8];
    //每個process會接收到的陣列大小
    int rc_n[] = {2, 2, 2, 2};
    //用來接收的陣列
    int recv_arr[2];
    //照disp位置分給大家
    int disp[] = {0, 6, 2, 4};
    int i;
    if (id == 0)
    {
        for (i = 0; i < 8; i++)
        {
            array[i] = i;
        }
        MPI_Scatterv(array, rc_n, disp, MPI_INT, recv_arr, 2, MPI_INT, 0, MPI_COMM_WORLD);
    }
    else
        MPI_Scatterv(array, rc_n, disp, MPI_INT, recv_arr, 2, MPI_INT, 0, MPI_COMM_WORLD);
    for (i = 0; i < 2; i++)
    {
        printf("%d ", recv_arr[i]);
    }
    printf("\nthis is process %d\n\n", id);
```
* 4 process result
![](https://i.imgur.com/jInoolb.png)

