Try   HackMD

CCurl Note

產出: iota-pow-in-c

Intorduction

ccurl (C port of the Curl library) 為 IOTA POW liarary.

Installation

$ git clone https://github.com/iotaledger/ccurl
$ cd ccurl
$ git submodule update --init --recursive
$ mkdir build && cd build && cmake .. && cmake --build . && cd ..

Usage

  • example:
./build/bin/ccurl-cli 14 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999QDPOHWNPSATIPICDZX9KAVIYPGUNXJETREPEAFASEFPYUFMHYZUNGMGDCXPGQJQDYCAYQLAQJTWCQHLCB999999999999999999999999999G99999999999999999999999999O9DAKXD99999999999999999999JLSCQZXU9XDFYWGUIHDUFFUIWVBFHPEKJCJLKEKTRQYUIFLYGOBFRLINNWLZLBXKDSDUNKIFVKQBRRXDB999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999
  • output: (比較 input 的內容，你會發現多了 nonce 了！)
Getting platforms... 
Cannot get the number of OpenCL platforms available. E:-1
Thread Hashing...
I: Starting search threads.
I: Found threads. Returning.
999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999QDPOHWNPSATIPICDZX9KAVIYPGUNXJETREPEAFASEFPYUFMHYZUNGMGDCXPGQJQDYCAYQLAQJTWCQHLCB999999999999999999999999999G99999999999999999999999999O9DAKXD99999999999999999999JLSCQZXU9XDFYWGUIHDUFFUIWVBFHPEKJCJLKEKTRQYUIFLYGOBFRLINNWLZLBXKDSDUNKIFVKQBRRXDB9999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999D99999999CV9B9999999999999

上述的 output 其實沒有辦法檢驗正確性，我們必須把它再 Hash 一次，再檢查字尾是否有 "9999"
所以可以更改 src/cli/ccurl.c 裡面的內容













if (out != NULL) {
    char digest[HASH_LENGTH];
    char *trits, *trytes;
    curl_t curl;
    
    init_curl(&curl);
    absorb(&curl, trits, 2673 * 3);
    squeeze(&curl, trytes, HASH_LENGTH);
    trytes = trytes_from_trits(digest, 0, HASH_LENGTH);
    
    fputs(trytes, stdout);
}

得到之 Output 為 JQS9WMZEWIVRCJHTDFIQAVKTAQJXGGKJLJYPXSEDBZVPRUC9O9LPFYPWBUMTPOEZTSEAGNXETPFXA9999，驗證正確

Discussion

  • ccurl 主要計算 POW 的地方在於 ccurl_pow
  • 在計算完 POW 之後，你可以看到函數利用一段 memcpy 把 nonce 寫進去 input trytes 中了，此時 POW 完成。
  • 目前支援的實作為 Pure C & OpenCL

有關 PoW 的程式架構

  • 直接使用 OpenCL 的實作
  • 進入點為 ccurl_pow
    • 可以 concurrently 地被呼叫
  • host 會開啟多條 thread （pthread)，一條 thread 會負責啟動其中一個 Device 的 Command Queue
    • 在 pearlcldiver.c 裡可以看到 pearl_search() 宣告了與 Device 數量相同的 thread 數
  • 一個 pdcl_node 代表著一個 src，裡面的結構 (PearCLDiver) 預先宣告了所有 Device 數量的 Context, Command Queue, …etc 給不同的 Device 使用
    • 見 claccess/clcontext.h

程式碼分析

  • 一個 pdcl_node 代表著一道 Pow Task
    • 不同的 Pow Task，其 pdcl_node 會用 linked-list 連起來

但是該 linked-list 並沒有上鎖的機制預防 race condition

​​​​* defined in ```src/lib/exports_cl.c```







typedef struct pdcl_node {
  PearCLDiver* pdcl;
  int initialized;
  int cl_available;
  struct pdcl_node* next;
} pdcl_node_t;
  • 其中的 PearCLDiver
    • defined in src/lib/pearcldiver.h






typedef struct {
  CLContext cl;
  PearlDiver pd;
  size_t num_groups;
  size_t loop_count;
} PearCLDiver;
  • 其中的 CLContext
    • defined in src/lib/claccess/clcontext.h
    • 為每一個 Device 都保留一份 cl_context, … 等













typedef struct {
  cl_uint num_devices;
  cl_device_id device[CLCONTEXT_MAX_DEVICES];
  cl_command_queue clcmdq[CLCONTEXT_MAX_DEVICES];
  cl_mem buffers[CLCONTEXT_MAX_DEVICES][MAX_BUFFERS];
  cl_kernel clkernel[CLCONTEXT_MAX_DEVICES][MAX_KERNELS];
  cl_program programs[CLCONTEXT_MAX_DEVICES];
  cl_context clctx[CLCONTEXT_MAX_DEVICES];
  cl_uint num_cores[CLCONTEXT_MAX_DEVICES];
  size_t num_multiple[CLCONTEXT_MAX_DEVICES];
  cl_ulong max_memory[CLCONTEXT_MAX_DEVICES];
  KernelInfo kernel;
} CLContext;
  • 當 user 呼叫 ccurl_pow，便會新增一個 pdcl_node，並接上 linked-list；或是找到一個已經搜尋完的 pdcl_node 直接使用，免去初始化的 overhead
    • defined in src/lib/exports_cl.c
    • 其中的 pearcl_search 便是 Pow 的起點















char* ccurl_pow(char* trytes, int minWeightMagnitude) 
{
  pdcl_node_t* pd_node = &base;

  ccurl_pow_node_init(pd_node);

  while (pd_node->pdcl->pd.status == PD_SEARCHING) {
    if (pd_node->next != NULL) {
      pd_node = pd_node->next;
    }
  }
  if (ccurl_pow_node_init(pd_node) == 0) {
    pearcl_search(pd_node->pdcl, &curl, offset, minWeightMagnitude);
  }
}
  • 在執行 pearcl_search 之前，有關於 cl_context 都已初始化，kernel 也已經填好，而 pearcl_search 便負責 create threads 分別 launch 不同 Device 上的 command queue
    • 一條 thread 負責 launch 一個 Device 的 command queue!
    • defined in src/lib/pearcldiver.c












PDCLThread* pdthreads = (PDCLThread*)malloc(numberOfThreads * sizeof(PDCLThread));
while (numberOfThreads-- > 0) {
    pdthreads[numberOfThreads] =
        (PDCLThread){.states = states,
                     .curl = curl,
                     .min_weight_magnitude = min_weight_magnitude,
                     .index = numberOfThreads,
                     .offset = offset,
                     .pdcl = pdcl};
    pthread_create(&tid[numberOfThreads], NULL, &pearcl_find,
                   (void*)&(pdthreads[numberOfThreads]));
}
  • 而 pearcl_find 會負責將資料寫入 OpenCL memory buffer，並分別 launch 三個 Kernel

TODO

  • 把剩下的 code 看完。
  • 驗證 multi-thread POW！
Last changed by 
 
黃俊毓
0
1403
Published on HackMD