# CCurl Note 產出: [iota-pow-in-c](https://github.com/chenweiii/iota-pow-in-c) ## Intorduction [ccurl (C port of the Curl library)](https://github.com/iotaledger/ccurl) 為 IOTA POW liarary. ## Installation ``` $ git clone https://github.com/iotaledger/ccurl $ cd ccurl $ git submodule update --init --recursive $ mkdir build && cd build && cmake .. && cmake --build . && cd .. ``` ## Usage * example: ``` ./build/bin/ccurl-cli 14 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999QDPOHWNPSATIPICDZX9KAVIYPGUNXJETREPEAFASEFPYUFMHYZUNGMGDCXPGQJQDYCAYQLAQJTWCQHLCB999999999999999999999999999G99999999999999999999999999O9DAKXD99999999999999999999JLSCQZXU9XDFYWGUIHDUFFUIWVBFHPEKJCJLKEKTRQYUIFLYGOBFRLINNWLZLBXKDSDUNKIFVKQBRRXDB999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 ``` * output: (比較 input 的內容，你會發現多了 nonce 了！) ``` Getting platforms... Cannot get the number of OpenCL platforms available. E:-1 Thread Hashing... I: Starting search threads. I: Found threads. Returning. 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999QDPOHWNPSATIPICDZX9KAVIYPGUNXJETREPEAFASEFPYUFMHYZUNGMGDCXPGQJQDYCAYQLAQJTWCQHLCB999999999999999999999999999G99999999999999999999999999O9DAKXD99999999999999999999JLSCQZXU9XDFYWGUIHDUFFUIWVBFHPEKJCJLKEKTRQYUIFLYGOBFRLINNWLZLBXKDSDUNKIFVKQBRRXDB9999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999D99999999CV9B9999999999999 ``` 上述的 output 其實沒有辦法檢驗正確性，我們必須把它再 Hash 一次，再檢查字尾是否有 "9999" 所以可以更改 ```src/cli/ccurl.c``` 裡面的內容 ```clike= if (out != NULL) { char digest[HASH_LENGTH]; char *trits, *trytes; curl_t curl; init_curl(&curl); absorb(&curl, trits, 2673 * 3); squeeze(&curl, trytes, HASH_LENGTH); trytes = trytes_from_trits(digest, 0, HASH_LENGTH); fputs(trytes, stdout); } ``` 得到之 Output 為 ```JQS9WMZEWIVRCJHTDFIQAVKTAQJXGGKJLJYPXSEDBZVPRUC9O9LPFYPWBUMTPOEZTSEAGNXETPFXA9999```，驗證正確 ## Discussion * ccurl 主要計算 POW 的地方在於 [ccurl_pow](https://github.com/iotaledger/ccurl/blob/73f6346f78b21bc0c040a891bcd1c769bfba312d/src/lib/exports_cl.c#L69) * 在計算完 POW 之後，你可以看到函數利用一段 [memcpy](https://github.com/iotaledger/ccurl/blob/117e19cbfbdc37000d27452cfdbaf829608f9ee4/src/lib/exports_cl.c#L105) 把 nonce 寫進去 input trytes 中了，此時 POW 完成。 * 目前支援的實作為 Pure C & OpenCL ## 有關 PoW 的程式架構 * 直接使用 OpenCL 的實作 * 進入點為 ```ccurl_pow``` * 可以 concurrently 地被呼叫 * host 會開啟多條 thread （pthread)，一條 thread 會負責啟動其中一個 Device 的 Command Queue * 在 ```pearlcldiver.c``` 裡可以看到 ```pearl_search()``` 宣告了與 Device 數量相同的 thread 數 * 一個 pdcl_node 代表著一個 src，裡面的結構 (PearCLDiver) 預先宣告了所有 Device 數量的 Context, Command Queue, ...etc 給不同的 Device 使用 * 見 ```claccess/clcontext.h``` ## 程式碼分析 * 一個 pdcl_node 代表著一道 Pow Task * 不同的 Pow Task，其 pdcl_node 會用 linked-list 連起來 :::info 但是該 linked-list 並沒有上鎖的機制預防 race condition ::: * defined in ```src/lib/exports_cl.c``` ```clike= typedef struct pdcl_node { PearCLDiver* pdcl; int initialized; int cl_available; struct pdcl_node* next; } pdcl_node_t; ``` * 其中的 PearCLDiver * defined in ```src/lib/pearcldiver.h``` ```clike= typedef struct { CLContext cl; PearlDiver pd; size_t num_groups; size_t loop_count; } PearCLDiver; ``` * 其中的 CLContext * defined in ```src/lib/claccess/clcontext.h``` * 為每一個 Device 都保留一份 cl_context, ... 等 ```clike= typedef struct { cl_uint num_devices; cl_device_id device[CLCONTEXT_MAX_DEVICES]; cl_command_queue clcmdq[CLCONTEXT_MAX_DEVICES]; cl_mem buffers[CLCONTEXT_MAX_DEVICES][MAX_BUFFERS]; cl_kernel clkernel[CLCONTEXT_MAX_DEVICES][MAX_KERNELS]; cl_program programs[CLCONTEXT_MAX_DEVICES]; cl_context clctx[CLCONTEXT_MAX_DEVICES]; cl_uint num_cores[CLCONTEXT_MAX_DEVICES]; size_t num_multiple[CLCONTEXT_MAX_DEVICES]; cl_ulong max_memory[CLCONTEXT_MAX_DEVICES]; KernelInfo kernel; } CLContext; ``` * 當 user 呼叫 ccurl_pow，便會新增一個 pdcl_node，並接上 linked-list；或是找到一個已經搜尋完的 pdcl_node 直接使用，免去初始化的 overhead * defined in ```src/lib/exports_cl.c``` * 其中的 ```pearcl_search``` 便是 Pow 的起點 ```clike= char* ccurl_pow(char* trytes, int minWeightMagnitude) { pdcl_node_t* pd_node = &base; ccurl_pow_node_init(pd_node); while (pd_node->pdcl->pd.status == PD_SEARCHING) { if (pd_node->next != NULL) { pd_node = pd_node->next; } } if (ccurl_pow_node_init(pd_node) == 0) { pearcl_search(pd_node->pdcl, &curl, offset, minWeightMagnitude); } } ``` * 在執行 ```pearcl_search``` 之前，有關於 cl_context 都已初始化，kernel 也已經填好，而 ```pearcl_search``` 便負責 create threads 分別 launch 不同 Device 上的 command queue * 一條 thread 負責 launch 一個 Device 的 command queue! * defined in ```src/lib/pearcldiver.c``` ```clike= PDCLThread* pdthreads = (PDCLThread*)malloc(numberOfThreads * sizeof(PDCLThread)); while (numberOfThreads-- > 0) { pdthreads[numberOfThreads] = (PDCLThread){.states = states, .curl = curl, .min_weight_magnitude = min_weight_magnitude, .index = numberOfThreads, .offset = offset, .pdcl = pdcl}; pthread_create(&tid[numberOfThreads], NULL, &pearcl_find, (void*)&(pdthreads[numberOfThreads])); } ``` * 而 ```pearcl_find``` 會負責將資料寫入 OpenCL memory buffer，並分別 launch 三個 Kernel ## TODO - [ ] 把剩下的 code 看完。 - [ ] 驗證 multi-thread POW！
Last changed by  
 
黃俊毓
1305
Published on HackMD