PoW remote server archives

non-blocked RPC

One AVX-accelerated remote worker
Two FPGA-accelerated remote worker - not workable

$ make BUILD_REMOTE=1 BUILD_DEBUG=1 check
  CC	build/curl.o
  CC	build/constants.o
  CC	build/trinary.o
  CC	build/dcurl.o
  CC	build/implcontext.o
  CC	build/common.o
  CC	build/pow_avx.o
  CC	build/remote_common.o
  CC	build/remote_interface.o
  CC	build/test-trinary.o
  LD	build/test-trinary
*** Validating build/test-trinary ***
	[ Verified ]
  CC	build/test-curl.o
  LD	build/test-curl
*** Validating build/test-curl ***
	[ Verified ]
  CC	build/test-dcurl.o
  LD	build/test-dcurl
*** Validating build/test-dcurl ***
[dcurl] Implementation CPU (Intel AVX) is initialized successfully
[dcurl-remote] Implementation Remote interface is initialized successfully
[dcurl-remote] callback queue amq.gen-eYJ-mmq7a2lDTloUifSENg 
RPC timeout: request timed out
[dcurl] Implementation CPU (Intel AVX) is initialized successfully
[dcurl-remote] Implementation Remote interface is initialized successfully
[dcurl-remote] callback queue amq.gen-BRRuF7yG2W_Qo0y7b1FGFg 
[dcurl-remote] Frame type: 1 channel: 1
[dcurl-remote] Method: AMQP_BASIC_DELIVER_METHOD
[dcurl-remote] Delivery: 1 exchange:  routingkey: amq.gen-BRRuF7yG2W_Qo0y7b1FGFg
[dcurl-remote] Content-type: text/plain
---
[dcurl-remote] PoW result: 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999A9RGRKVGWMWMKOLVMDFWJUHNUNYWZTJADGGPZGXNLERLXYWJE9WQHWWBMCPZMVVMJUMWWBLZLNMLDCGDJ999999999999999999999999999999999999999999999999999999YGYQIVD99999999999999999999TXEFLKNPJRBYZPORHZU9CEMFIFVVQBUSTDGSJCZMBTZCDTTJVUFPTCCVHHORPMGCURKTH9VGJIXUQJVHK999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999LGSA999999F9999999999999999
[dcurl] Implementation CPU (Intel AVX) is initialized successfully
[dcurl-remote] Implementation Remote interface is initialized successfully
[dcurl-remote] callback queue amq.gen-fzt4N3enjux4LRTl8slPLg 
RPC timeout: request timed out
[dcurl] Implementation CPU (Intel AVX) is initialized successfully
[dcurl-remote] Implementation Remote interface is initialized successfully
[dcurl-remote] callback queue amq.gen-4UHebpJjd2GdRlqLRcx2XA 
RPC timeout: request timed out
[dcurl] Implementation CPU (Intel AVX) is initialized successfully
[dcurl-remote] Implementation Remote interface is initialized successfully
[dcurl-remote] callback queue amq.gen-BeklR2QFuug8PQngfXXz2A 
[dcurl-remote] Frame type: 1 channel: 1
[dcurl-remote] Method: AMQP_BASIC_DELIVER_METHOD
[dcurl-remote] Delivery: 1 exchange:  routingkey: amq.gen-BeklR2QFuug8PQngfXXz2A
[dcurl-remote] Content-type: text/plain
---
[dcurl-remote] PoW result: 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999A9RGRKVGWMWMKOLVMDFWJUHNUNYWZTJADGGPZGXNLERLXYWJE9WQHWWBMCPZMVVMJUMWWBLZLNMLDCGDJ999999999999999999999999999999999999999999999999999999YGYQIVD99999999999999999999TXEFLKNPJRBYZPORHZU9CEMFIFVVQBUSTDGSJCZMBTZCDTTJVUFPTCCVHHORPMGCURKTH9VGJIXUQJVHK999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999WI99999999L9999999999999999
	[ Verified ]
  CC	build/test-pow.o
  LD	build/test-pow
*** Validating build/test-pow ***
CPU - AVX
[dcurl] Implementation CPU (Intel AVX) is initialized successfully
PoW execution times: 1 times.
Success.
	[ Verified ]
  CC	build/test-multi-pow.o
  LD	build/test-multi-pow
*** Validating build/test-multi-pow ***
[dcurl] Implementation CPU (Intel AVX) is initialized successfully
[dcurl-remote] Implementation Remote interface is initialized successfully
[dcurl-remote] callback queue amq.gen-Kg7awRc5RCQENPIcAaNx7Q 
[dcurl-remote] callback queue amq.gen-890FiaCY2ZxmYoMVAHZ2Dw 
[dcurl-remote] callback queue amq.gen-TPDsw4kZRZ_Nx0Cb4nTsYg 
[dcurl-remote] callback queue amq.gen-s8bQZNmHqWahOllhZlby6w 
[dcurl-remote] callback queue amq.gen-hVrt7XQecqGjeGutzgasKg 
[dcurl-remote] callback queue amq.gen-eRU5Hmmg6aJ2pXMAJpm7Dg 
[dcurl-remote] callback queue amq.gen-0k214C17xOprGP0wZSHiyA 
[dcurl-remote] callback queue amq.gen-rsw6sGC-uxBXwuIxWnoKQQ 
[dcurl-remote] callback queue amq.gen-21M1MyczZ_zhH-zYBNvc-Q 
[dcurl-remote] callback queue amq.gen-XmPCAEVocbLIioDZs0hjug 
[dcurl-remote] Frame type: 1 channel: 1
[dcurl-remote] Method: AMQP_BASIC_DELIVER_METHOD
[dcurl-remote] Delivery: 1 exchange:  routingkey: amq.gen-TPDsw4kZRZ_Nx0Cb4nTsYg
[dcurl-remote] Content-type: text/plain
---
[dcurl-remote] PoW result: 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999A9RGRKVGWMWMKOLVMDFWJUHNUNYWZTJADGGPZGXNLERLXYWJE9WQHWWBMCPZMVVMJUMWWBLZLNMLDCGDJ999999999999999999999999999999999999999999999999999999YGYQIVD99999999999999999999TXEFLKNPJRBYZPORHZU9CEMFIFVVQBUSTDGSJCZMBTZCDTTJVUFPTCCVHHORPMGCURKTH9VGJIXUQJVHK999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999HIG9999999RVA99999999999999
[dcurl-remote] Frame type: 1 channel: 1
[dcurl-remote] Method: AMQP_BASIC_DELIVER_METHOD
[dcurl-remote] Delivery: 1 exchange:  routingkey: amq.gen-0k214C17xOprGP0wZSHiyA
[dcurl-remote] Content-type: text/plain
---
[dcurl-remote] PoW result: 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999A9RGRKVGWMWMKOLVMDFWJUHNUNYWZTJADGGPZGXNLERLXYWJE9WQHWWBMCPZMVVMJUMWWBLZLNMLDCGDJ999999999999999999999999999999999999999999999999999999YGYQIVD99999999999999999999TXEFLKNPJRBYZPORHZU9CEMFIFVVQBUSTDGSJCZMBTZCDTTJVUFPTCCVHHORPMGCURKTH9VGJIXUQJVHK999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999HIG9999999RVA99999999999999
[dcurl-remote] Frame type: 1 channel: 1
[dcurl-remote] Method: AMQP_BASIC_DELIVER_METHOD
[dcurl-remote] Delivery: 1 exchange:  routingkey: amq.gen-XmPCAEVocbLIioDZs0hjug
[dcurl-remote] Content-type: text/plain
---
[dcurl-remote] PoW result: 999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999A9RGRKVGWMWMKOLVMDFWJUHNUNYWZTJADGGPZGXNLERLXYWJE9WQHWWBMCPZMVVMJUMWWBLZLNMLDCGDJ999999999999999999999999999999999999999999999999999999YGYQIVD99999999999999999999TXEFLKNPJRBYZPORHZU9CEMFIFVVQBUSTDGSJCZMBTZCDTTJVUFPTCCVHHORPMGCURKTH9VGJIXUQJVHK999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999HIG9999999RVA99999999999999
RPC timeout: request timed out
RPC timeout: request timed out
RPC timeout: request timed out
RPC timeout: request timed out
RPC timeout: request timed out
RPC timeout: request timed out
RPC timeout: request timed out
	[ Verified ]
rm build/test-trinary.o build/test-curl.o build/test-pow.o build/test-multi-pow.o build/test-dcurl.o

~~## FPGA problem~~
$ dmesg

[  571.483734] Initializing Curl POW Driver module
[  571.488328] Curl POW Driver Probe enter!
[  571.493392] Curl POW Driver Probe exit success
[  571.498064] Curl POW Driver module successfully initialized!
[  583.989081] curl_ctrl_open
[  583.989102] curl_idata_dev_open
[  583.989115] curl_odata_dev_open
[  609.258014] curl_idata_dev_lseek begin
[  609.258021] curl_idata_dev_lseek end
[  609.258027] curl_odata_dev_lseek begin
[  609.258029] curl_odata_dev_lseek end
[  609.258036] curl_idata_dev_write begin
[  609.258045] curl_idata_dev_write end
[  609.258050] curl_ctrl_dev_write begin
[  609.258053] curl_ctrl_dev_write end
[  609.258060] curl_ctrl_read begin
[  609.258423] Get interrupt
[  609.269920] hash_cnt = 8344
[  609.269931] tick_cnt = 24735
[  609.269941] curl_ctrl_read end
[  609.269948] curl_odata_dev_read begin
[  609.269953] curl_odata_dev_read end
[  609.415246] curl_idata_dev_lseek begin
[  609.415251] curl_idata_dev_lseek end
[  609.415258] curl_odata_dev_lseek begin
[  609.415260] curl_odata_dev_lseek end
[  609.415264] curl_idata_dev_write begin
[  609.415272] curl_idata_dev_write end
[  609.415276] curl_ctrl_dev_write begin
[  609.415279] curl_ctrl_dev_write end
[  609.415284] curl_ctrl_read begin
[  609.415645] Get interrupt
[  609.427227] hash_cnt = 8344
[  609.427236] tick_cnt = 24735
[  609.427246] curl_ctrl_read end
[  609.427253] curl_odata_dev_read begin
[  609.427257] curl_odata_dev_read end
[  610.263500] curl_idata_dev_lseek begin
[  610.263506] curl_idata_dev_lseek end
[  610.263512] curl_odata_dev_lseek begin
[  610.263515] curl_odata_dev_lseek end
[  610.263519] curl_idata_dev_write begin
[  610.263528] curl_idata_dev_write end
[  610.263533] curl_ctrl_dev_write begin
[  610.263536] curl_ctrl_dev_write end
[  610.263541] curl_ctrl_read begin
[  610.425484] Get interrupt
[  610.428116] hash_cnt = 5459216
[  610.428120] tick_cnt = 16182677
[  610.428123] curl_ctrl_read end
[  610.428130] curl_odata_dev_read begin
[  610.428134] curl_odata_dev_read end
[  610.428826] curl_idata_dev_lseek begin
[  610.428831] curl_idata_dev_lseek end
[  610.428837] curl_odata_dev_lseek begin
[  610.428840] curl_odata_dev_lseek end
[  610.428844] curl_idata_dev_write begin
[  610.428852] curl_idata_dev_write end
[  610.428856] curl_ctrl_dev_write begin
[  610.428860] curl_ctrl_dev_write end
[  610.428864] curl_ctrl_read begin
[  610.473018] Get interrupt
[  610.475648] hash_cnt = 1485680
[  610.475652] tick_cnt = 4403981
[  610.475654] curl_ctrl_read end
[  610.475659] curl_odata_dev_read begin
[  610.475663] curl_odata_dev_read end
[  610.476333] curl_idata_dev_lseek begin
[  610.476338] curl_idata_dev_lseek end
[  610.476343] curl_odata_dev_lseek begin
[  610.476346] curl_odata_dev_lseek end
[  610.476350] curl_idata_dev_write begin
[  610.476358] curl_idata_dev_write end
[  610.476362] curl_ctrl_dev_write begin
[  610.476366] curl_ctrl_dev_write end
[  610.476369] curl_ctrl_read begin
[  610.528291] Get interrupt
[  610.530941] hash_cnt = 1747732
[  610.530948] tick_cnt = 5180778
[  610.530950] curl_ctrl_read end
[  610.530956] curl_odata_dev_read begin
[  610.530960] curl_odata_dev_read end
[  610.531568] curl_idata_dev_lseek begin
[  610.531572] curl_idata_dev_lseek end
[  610.531578] curl_odata_dev_lseek begin
[  610.531580] curl_odata_dev_lseek end
[  610.531584] curl_idata_dev_write begin
[  610.531592] curl_idata_dev_write end
[  610.531596] curl_ctrl_dev_write begin
[  610.531599] curl_ctrl_dev_write end
[  610.531603] curl_ctrl_read begin
[  610.583524] Get interrupt
[  610.586154] hash_cnt = 1747732
[  610.586158] tick_cnt = 5180778
[  610.586160] curl_ctrl_read end
[  610.586165] curl_odata_dev_read begin
[  610.586169] curl_odata_dev_read end
[  610.586756] curl_idata_dev_lseek begin
[  610.586759] curl_idata_dev_lseek end
[  610.586765] curl_odata_dev_lseek begin
[  610.586767] curl_odata_dev_lseek end
[  610.586771] curl_idata_dev_write begin
[  610.586779] curl_idata_dev_write end
[  610.586783] curl_ctrl_dev_write begin
[  610.586787] curl_ctrl_dev_write end
[  610.586791] curl_ctrl_read begin
[  610.638712] Get interrupt
[  610.641342] hash_cnt = 1747732
[  610.641345] tick_cnt = 5180778
[  610.641348] curl_ctrl_read end
[  610.641353] curl_odata_dev_read begin
[  610.641357] curl_odata_dev_read end
[ 1101.222621] systemd-logind[2083]: Failed to start user service, ignoring: Unknown unit: user@0.service
[ 1101.247524] systemd-logind[2083]: New session c1 of user root.
[71033.093436] curl_idata_dev_lseek begin
[71033.093443] curl_idata_dev_lseek end
[71033.093449] curl_odata_dev_lseek begin
[71033.093452] curl_odata_dev_lseek end
[71033.093456] curl_idata_dev_write begin
[71033.093464] curl_idata_dev_write end
[71033.093468] curl_ctrl_dev_write begin
[71033.093471] curl_ctrl_dev_write end
[71033.093476] curl_ctrl_read begin

Rabbitmq managment

https://pulse.mozilla.org/api/

AMQP_STATUS_TIMEOUT
AMQP_STATUS_TIMER_FAILURE
AMQP_STATUS_HEARTBEAT_TIMEOUT

dcurl API interface初步討論結果

先確定整合方式，再討論dcurl interface介面的修改。

IRI's AttachToTangle的實作

AttachToTangle為sync
input為array of transaction

我們分成兩個狀況討論

不修改 IRI
修改 IRI
- patch IRI's AttachToTangle

如果我們不改IRI，加上remote PoW的話，先不討論dcurl介面問題，而整合結果會是remote PoW並沒有幫助加速。理由如下:

理由:

由於一次只會處理一筆transaction，message queue也只會存放一筆資料給remote dcurl worker做運算。
就算是local dcurl來說，因為一次只執行一個transaction來做PoW，如果dcurl的加速運算綁定CPU core 1，那麼其他硬體資源就會是idle。

勢必要patch IRI的AttachToTangle，分成AttachToTangle是sync和async來討論:

[錯誤規劃，因為Bundle裡的transactions不能平行做PoW] 對於AttachToTangle Sync，dcurl介面有兩種情況
1. 在AttachToTangle裡做multithreads，每一個thread呼叫自己的dcurl，dcurl介面不用動
2. AttachToTangle呼叫dcurl，在dcurl裡做multithreads，dcurl介面要修改成傳array of transactions
而對於AttachToTangle Sync整合remote PoW，可以直接使用rabbitmq的RPC pattern，雖然每個thread得到的結果時間不同(失序狀況)，但由於AttachToTangle是Sync，我們只要保證thread拿到的結果正確，存至正確PoW結果的array位置就可以。

https://www.rabbitmq.com/tutorials/tutorial-six-python.html
AttachToTangle Aync
不管是IRI或者tangle accelerator都沒有AttachToTangle的async實作，所以如果要採取此方案，對應修改
1. IRI與tangle accelerator修改成async處理邏輯，範例是PoWbox的client: curl-remote
2. 使用Design asynchronous N-N remote PoW architecture

client interface

callback(null, taskResult.response.trytes)
https://github.com/iotaledger/curl-remote/blob/master/index.js#L47

呼叫範例
https://github.com/iotaledger/iota.js/blob/788464db6ccacc8321ce2bd8670468e1033ac6b4/packages/core/test/integration/attachToTangle.test.ts#L84

Callback:
https://github.com/iotaledger/iota.js/blob/175898187eaea955fb8ccc244ec7638e02b0f2ac/packages/types.ts#L277

overrideAttachToTangle(api) - howto attach? #17
https://github.com/iotaledger/curl.lib.js/issues/17

iota.api.attachToTangle = localAttachToTangle
https://github.com/pRizz/iota.transactionSpammer.js/blob/9ad59895a846849adf344de9d0d52b0ff83b8e2c/src/transactionSpammer.js#L224
https://github.com/pRizz/iota.transactionSpammer.js/blob/9ad59895a846849adf344de9d0d52b0ff83b8e2c/src/transactionSpammer.js#L118

https://github.com/iotaledger/iota.js/blob/e2c4f132bb98c5a8e86a28a513ca03cd106a1e2e/api_reference.md#module_core.attachToTangle

Poison messages 處理

TransactionValidator.java
- 驗證PoW results
- …
protocol errors
- AMQP_STATUS_BAD_AMQP_DATA
- AMQP_STATUS_UNKNOWN_CLASS
- AMQP_STATUS_UNKNOWN_METHOD

重送3次錯，就使用原來dcurl。
此recording buffer大小?
- sendTransfer, https://github.com/iotaledger/iri/blob/b4e6ee4b6309d91636cd2c7f437a0487844d422f/src/test/java/com/iota/iri/service/APIIntegrationTests.java#L433

Multi-workers

amqp_consume_message
event driven

找到在de10-nano上，bug Deliver 2 nwm = 999的問題。

remote_worker.c#L54


已確定 envelope.message.body.bytes 沒問題

memcpy(trytes, envelope.message.body.bytes, TRANSACTION_TRYTES_LENGTH);

printf(buf) 
round1: 9, 
round2: 999y, -> buf還沒執行下面memcpy，buf就變成999y
round3: 9, 
round4: 9

memcpy(buf, envelope.message.body.bytes + TRANSACTION_TRYTES_LENGTH, 4);

printf(buf) 
round1: 9, 
round2: 999y, 
round3: 9, 
round4: 9

sscanf(buf, "%d", &mwm);

printf(buf) 
round1: 9, 
round2: 999y, 
round3: 9, 
round4: 9

[dcurl-remote] Content-type: text/plain
[dcurl-remote] Doing PoW with mwm = 14...
[dcurl-remote] PoW is done
[dcurl-remote] Sending an ack is done
[dcurl-remote] Publishing PoW result to callback queue is done
[dcurl-remote] ---
[dcurl-remote] Delivery 63, exchange , routingkey incoming_queue, callback queue: amq.gen-fqTuJBsuhSStntdr4flWjg
[dcurl-remote] Content-type: text/plain
[dcurl-remote] Doing PoW with mwm = 94...

workaround:


char trytes[TRANSACTION_TRYTES_LENGTH];
char buf[4];
int mwm;

for()
{
   

}


for()
{

char trytes[TRANSACTION_TRYTES_LENGTH];
char buf[4];
int mwm;
   

}

test-multi-pow THREAD_MAX 1000
FPGA X2

time ./build/test-multi-pow
real    0m38.044s
user    0m7.592s
sys    0m0.354s

FPGA X1

$ time ./build/test-multi-pow
real    1m6.340s
user    0m7.547s
sys    0m0.403s

real	0m25.015s
user	0m7.732s
sys	0m0.258s

bug Deliver 2 nwm = 999

https://github.com/ajblane/dcurl/blob/remotedcurl/remotedcurl/remotedcurl.c#L85

How to install Latest RabbitMQ Server on Ubuntu 18.04 LTS
https://computingforgeeks.com/how-to-install-latest-rabbitmq-server-on-ubuntu-18-04-lts/

sudo docker-compose run -d -p 5672:5672 rabbitmq
/dcurl$ ./build/test-remotepow-new-task
Message formate: trasnsaction | mwm
/dcurl$ ./build/remotepow

IRI
例外處理
遠端dcurl失敗，就用local dcurl。

例外處理

version: "3"
services:

  api:
    build:
      context: .
      dockerfile: Dockerfile.api
    ports:
      - 3000:3000
    volumes:
      - .:/app
    ```
	
	```links:
	
      - rabbitmq:rabbitmq-service
      - mongodb:mongodb-service
    environment:
      - AMQP_APP_ID=sandbox
      - BROKER_URL=amqp://guest:guest@rabbitmq-service:5672
      - INCOMING_QUEUE=attach-to-tangle
      - COMPLETED_QUEUE=attach-complete
      - UPDATE_QUEUE=attach-progress
      - MONGO_CONN=mongodb://localhost:27017/sandbox
      - IRI_HOST=https://nodes.iota.cafe
      - IRI_PORT=443
    depends_on:
      - rabbitmq
      - mongodb

  worker:
    build:
      context: .
      dockerfile: Dockerfile.worker
    volumes:
      - .:/app
    links:
      - rabbitmq:rabbitmq-service
      - mongodb:mongodb-service
    environment:
      - BROKER_URL=amqp://guest:guest@rabbitmq-service:5672
      - INCOMING_QUEUE=attach-to-tangle
      - COMPLETED_QUEUE=attach-complete
      - UPDATE_QUEUE=attach-progress
      - CCURL_PATH=/opt
    depends_on:
      - api

  rabbitmq:
    image: rabbitmq
    ports:
      - 5672:5672

  mongodb:
    image: mongo
    ports:
      - 27017:27017

server/commands.js, https://github.com/iotaledger/powbox/blob/develop/src/server/commands.js#L66
JSON
RabbitMQ C client, https://github.com/alanxz/rabbitmq-c
Docker Official Images: rabbitmq,
- RabbitMQ is an open source multi-protocol messaging broker, https://hub.docker.com/_/rabbitmq
- API 和 WORK
  - BROKER_URL=amqp://guest:guest@rabbitmq-service:5672, https://github.com/alexkvak/powbox/blob/develop/docker-compose.yml#L17
  - amqp[s]://[user:password@]hostname[:port][/vhost]

broker (左) PoW (右)

https://github.com/DLTcollab/dcurl/issues/91

PoWbox可能無法放入Cyclone V FPGA board

https://thetangle.org/transaction/AVVCGZSDQJFGQLGLIORSAEUXW9EMODBLKMTWCYNDXLJGPVFGWZPEUPVSPUPZTPAWUIEHAUPRPEWZZ9999

curl http://localhost:3000/api/v1/commands \
  -X POST \
  -H 'Content-Type: application/json' \
  -H 'X-IOTA-API-Version: 1' \
  -d '{"command": "attachToTangle", "trunkTransaction": "MVVOWQWDKEWOCPABQ9AXXHUFCVPIPIYJQHDTNRKUWX9TYIFZGYECIWCKSQQOGQDJFXUAMVLPDSXGA9999", "branchTransaction": "YSEMPQQYXMLIBYSKWWYOVABETYLTFDXIFYPYOLZ9DTPFVDCRMC9HQ9GFKBYZTQOPZVXNYOEJHDEPA9999", "minWeightMagnitude": 14, "trytes": ["999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999TANGLEBEAT9PROMOTE9999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999CNGLE9BEAT9999999999999999YYNXA9D99999999999999999999WAGEPNOHCCLAINBAQADCCWFUXCZBCIUWZBB9YXORVLEDPBHOHPICFFVBEKFLVP9OFGLRYFDNXUQEVLLVYMVVOWQWDKEWOCPABQ9AXXHUFCVPIPIYJQHDTNRKUWX9TYIFZGYECIWCKSQQOGQDJFXUAMVLPDSXGA9999YSEMPQQYXMLIBYSKWWYOVABETYLTFDXIFYPYOLZ9DTPFVDCRMC9HQ9GFKBYZTQOPZVXNYOEJHDEPA9999TANGLE9BEAT9999999999999999PLUZGEEME999999999K99999999POWSRVIO9MD99999FTEPKPGMMMM"]}'

架構

TA
- curl-remote 轉 c or c++, https://github.com/iotaledger/curl-remote/blob/master/index.js#L28
  - timer
  - ansy fetch

TA
- 同上
worker
- amqplib + ccurl 轉 RabbitMQ C AMQP client library + dcurl (FPGA) https://github.com/iotaledger/powbox/blob/develop/src/worker/index.js#L166
  - https://github.com/alanxz/rabbitmq-c

TA
- const attachToTangle = async (req, callback) -> 轉 c https://github.com/iotaledger/powbox/blob/ae003df500e47f4a8f59650f7056d48277a2ec69/src/server/commands.js#L15
worker
- 同上。

PoWbox - make-it-work

$ sudo docker-compose up
Creating volume "powbox_app-node-modules" with default driver
Creating volume "powbox_worker-node-modules" with default driver
Starting powbox_mongodb_1  ... done
Starting powbox_rabbitmq_1 ... done
Recreating powbox_api_1    ... done
Recreating powbox_worker_1 ... done
Attaching to powbox_mongodb_1, powbox_rabbitmq_1, powbox_api_1, powbox_worker_1
mongodb_1   | 2019-02-11T14:43:34.552+0000 I CONTROL  [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
mongodb_1   | 2019-02-11T14:43:34.885+0000 I CONTROL  [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=413e2d46cfd1
mongodb_1   | 2019-02-11T14:43:34.885+0000 I CONTROL  [initandlisten] db version v4.0.6
mongodb_1   | 2019-02-11T14:43:34.885+0000 I CONTROL  [initandlisten] git version: caa42a1f75a56c7643d0b68d3880444375ec42e3
mongodb_1   | 2019-02-11T14:43:34.885+0000 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016
mongodb_1   | 2019-02-11T14:43:34.885+0000 I CONTROL  [initandlisten] allocator: tcmalloc
mongodb_1   | 2019-02-11T14:43:34.885+0000 I CONTROL  [initandlisten] modules: none
mongodb_1   | 2019-02-11T14:43:34.885+0000 I CONTROL  [initandlisten] build environment:
mongodb_1   | 2019-02-11T14:43:34.885+0000 I CONTROL  [initandlisten]     distmod: ubuntu1604
mongodb_1   | 2019-02-11T14:43:34.885+0000 I CONTROL  [initandlisten]     distarch: x86_64
mongodb_1   | 2019-02-11T14:43:34.885+0000 I CONTROL  [initandlisten]     target_arch: x86_64
mongodb_1   | 2019-02-11T14:43:34.885+0000 I CONTROL  [initandlisten] options: { net: { bindIpAll: true } }
mongodb_1   | 2019-02-11T14:43:34.902+0000 I STORAGE  [initandlisten] Detected data files in /data/db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
mongodb_1   | 2019-02-11T14:43:34.902+0000 I STORAGE  [initandlisten] 
mongodb_1   | 2019-02-11T14:43:34.902+0000 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
mongodb_1   | 2019-02-11T14:43:34.902+0000 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
mongodb_1   | 2019-02-11T14:43:34.902+0000 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=3464M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),
mongodb_1   | 2019-02-11T14:43:50.944+0000 I STORAGE  [initandlisten] WiredTiger message [1549896230:944056][1:0x7f6a9b911a40], txn-recover: Main recovery loop: starting at 1/24576 to 2/256
mongodb_1   | 2019-02-11T14:43:51.591+0000 I STORAGE  [initandlisten] WiredTiger message [1549896231:591494][1:0x7f6a9b911a40], txn-recover: Recovering log 1 through 2
mongodb_1   | 2019-02-11T14:43:53.307+0000 I STORAGE  [initandlisten] WiredTiger message [1549896233:307123][1:0x7f6a9b911a40], txn-recover: Recovering log 2 through 2
mongodb_1   | 2019-02-11T14:43:53.484+0000 I STORAGE  [initandlisten] WiredTiger message [1549896233:484197][1:0x7f6a9b911a40], txn-recover: Set global recovery timestamp: 0
mongodb_1   | 2019-02-11T14:43:53.546+0000 I RECOVERY [initandlisten] WiredTiger recoveryTimestamp. Ts: Timestamp(0, 0)
mongodb_1   | 2019-02-11T14:43:53.645+0000 I CONTROL  [initandlisten] 
mongodb_1   | 2019-02-11T14:43:53.645+0000 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
mongodb_1   | 2019-02-11T14:43:53.646+0000 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
mongodb_1   | 2019-02-11T14:43:53.646+0000 I CONTROL  [initandlisten] 
api_1       | 
api_1       | > sandbox.js@1.0.0 start:api /app
api_1       | > node src/server/index.js
api_1       | 
mongodb_1   | 2019-02-11T14:43:54.044+0000 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/data/db/diagnostic.data'
api_1       | Listening on port 3000...
mongodb_1   | 2019-02-11T14:43:54.090+0000 I NETWORK  [initandlisten] waiting for connections on port 27017
mongodb_1   | 2019-02-11T14:44:00.213+0000 I NETWORK  [listener] connection accepted from 172.19.0.4:54194 #1 (1 connection now open)
api_1       | Failed to connect rabbit mq, retry in 5 seconds
api_1       | mongoose connected to mongodb://mongodb:27017/sandbox
mongodb_1   | 2019-02-11T14:44:00.219+0000 I NETWORK  [conn1] received client metadata from 172.19.0.4:54194 conn1: { driver: { name: "nodejs", version: "2.2.34" }, os: { type: "Linux", name: "linux", architecture: "x64", version: "4.15.0-13-generic" }, platform: "Node.js v8.15.0, LE, mongodb-core: 2.1.18" }
api_1       | Failed to connect rabbit mq, retry in 5 seconds
mongodb_1   | 2019-02-11T14:44:00.242+0000 I STORAGE  [conn1] createCollection: sandbox.sessions with generated UUID: 7a3af4a9-0d1c-4ee4-b861-e7bb1cfe5b3a
api_1       | Failed to connect rabbit mq, retry in 5 seconds
mongodb_1   | 2019-02-11T14:44:00.769+0000 I INDEX    [conn1] build index on: sandbox.sessions properties: { v: 2, key: { expires: 1 }, name: "expires_1", ns: "sandbox.sessions", expireAfterSeconds: 0 }
api_1       | Failed to connect rabbit mq, retry in 5 seconds
mongodb_1   | 2019-02-11T14:44:00.770+0000 I INDEX    [conn1] 	 building index using bulk method; build may temporarily use up to 500 megabytes of RAM
mongodb_1   | 2019-02-11T14:44:00.799+0000 I INDEX    [conn1] build index done.  scanned 0 total records. 0 secs
mongodb_1   | 2019-02-11T14:44:00.855+0000 I COMMAND  [conn1] command sandbox.$cmd command: createIndexes { createIndexes: "sessions", indexes: [ { name: "expires_1", key: { expires: 1 }, expireAfterSeconds: 0 } ], $db: "sandbox" } numYields:0 reslen:129 locks:{ Global: { acquireCount: { r: 2, w: 2 } }, Database: { acquireCount: { w: 2, W: 1 } }, Collection: { acquireCount: { w: 2 } } } protocol:op_query 556ms
worker_1    | 
worker_1    | > sandbox.js@1.0.0 start:worker /app
worker_1    | > node src/worker/index.js
worker_1    | 
worker_1    | Failed to connect rabbit mq, retry in 5 seconds
api_1       | Failed to connect rabbit mq, retry in 5 seconds
rabbitmq_1  | 2019-02-11 14:44:20.917 [info] <0.236.0> 
rabbitmq_1  |  Starting RabbitMQ 3.7.11 on Erlang 21.2.5
rabbitmq_1  |  Copyright (C) 2007-2019 Pivotal Software, Inc.
rabbitmq_1  |  Licensed under the MPL.  See http://www.rabbitmq.com/
rabbitmq_1  | 
rabbitmq_1  |   ##  ##
rabbitmq_1  |   ##  ##      RabbitMQ 3.7.11. Copyright (C) 2007-2019 Pivotal Software, Inc.
rabbitmq_1  |   ##########  Licensed under the MPL.  See http://www.rabbitmq.com/
rabbitmq_1  |   ######  ##
rabbitmq_1  |   ##########  Logs: <stdout>
rabbitmq_1  | 
rabbitmq_1  |               Starting broker...
rabbitmq_1  | 2019-02-11 14:44:20.920 [info] <0.236.0> 
rabbitmq_1  |  node           : rabbit@7a735f848b21
rabbitmq_1  |  home dir       : /var/lib/rabbitmq
rabbitmq_1  |  config file(s) : /etc/rabbitmq/rabbitmq.conf
rabbitmq_1  |  cookie hash    : UNdn0n6e+CiAH49uAo/06Q==
rabbitmq_1  |  log(s)         : <stdout>
rabbitmq_1  |  database dir   : /var/lib/rabbitmq/mnesia/rabbit@7a735f848b21
rabbitmq_1  | 2019-02-11 14:44:21.027 [info] <0.252.0> Memory high watermark set to 3181 MiB (3335974092 bytes) of 7953 MiB (8339935232 bytes) total
rabbitmq_1  | 2019-02-11 14:44:21.170 [info] <0.273.0> Enabling free disk space monitoring
rabbitmq_1  | 2019-02-11 14:44:21.171 [info] <0.273.0> Disk free limit set to 50MB
rabbitmq_1  | 2019-02-11 14:44:21.179 [info] <0.292.0> Limiting to approx 1048476 file handles (943626 sockets)
rabbitmq_1  | 2019-02-11 14:44:21.180 [info] <0.293.0> FHC read buffering:  OFF
rabbitmq_1  | 2019-02-11 14:44:21.180 [info] <0.293.0> FHC write buffering: ON
rabbitmq_1  | 2019-02-11 14:44:21.196 [info] <0.236.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
rabbitmq_1  | 2019-02-11 14:44:21.266 [info] <0.236.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
rabbitmq_1  | 2019-02-11 14:44:21.267 [info] <0.236.0> Peer discovery backend rabbit_peer_discovery_classic_config does not support registration, skipping registration.
rabbitmq_1  | 2019-02-11 14:44:21.270 [info] <0.236.0> Priority queues enabled, real BQ is rabbit_variable_queue
rabbitmq_1  | 2019-02-11 14:44:21.297 [info] <0.320.0> Starting rabbit_node_monitor
rabbitmq_1  | 2019-02-11 14:44:21.347 [info] <0.347.0> Making sure data directory '/var/lib/rabbitmq/mnesia/rabbit@7a735f848b21/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists
rabbitmq_1  | 2019-02-11 14:44:21.354 [info] <0.347.0> Starting message stores for vhost '/'
rabbitmq_1  | 2019-02-11 14:44:21.354 [info] <0.351.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index
rabbitmq_1  | 2019-02-11 14:44:21.361 [info] <0.347.0> Started message store of type transient for vhost '/'
rabbitmq_1  | 2019-02-11 14:44:21.361 [info] <0.354.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index
rabbitmq_1  | 2019-02-11 14:44:21.394 [info] <0.347.0> Started message store of type persistent for vhost '/'
rabbitmq_1  | 2019-02-11 14:44:21.400 [warning] <0.375.0> Setting Ranch options together with socket options is deprecated. Please use the new map syntax that allows specifying socket options separately from other options.
rabbitmq_1  | 2019-02-11 14:44:21.403 [info] <0.389.0> started TCP listener on [::]:5672
rabbitmq_1  | 2019-02-11 14:44:21.408 [info] <0.236.0> Setting up a table for connection tracking on this node: tracked_connection_on_node_rabbit@7a735f848b21
rabbitmq_1  | 2019-02-11 14:44:21.408 [info] <0.236.0> Setting up a table for per-vhost connection counting on this node: tracked_connection_per_vhost_on_node_rabbit@7a735f848b21
rabbitmq_1  | 2019-02-11 14:44:21.751 [info] <0.8.0> Server startup complete; 0 plugins started.
rabbitmq_1  |  completed with 0 plugins.
rabbitmq_1  | 2019-02-11 14:44:22.750 [info] <0.395.0> accepting AMQP connection <0.395.0> (172.19.0.5:37700 -> 172.19.0.3:5672)
rabbitmq_1  | 2019-02-11 14:44:22.853 [info] <0.395.0> connection <0.395.0> (172.19.0.5:37700 -> 172.19.0.3:5672): user 'guest' authenticated and granted access to vhost '/'
worker_1    | {"queue":"attach-to-tangle","name":"listening-for-mesages","message":"Timeout set to 300 seconds"}
rabbitmq_1  | 2019-02-11 14:44:25.334 [info] <0.413.0> accepting AMQP connection <0.413.0> (172.19.0.4:39384 -> 172.19.0.3:5672)
rabbitmq_1  | 2019-02-11 14:44:25.389 [info] <0.413.0> connection <0.413.0> (172.19.0.4:39384 -> 172.19.0.3:5672): user 'guest' authenticated and granted access to vhost '/'
api_1       | [api] Rabbit connection ready
api_1       | {"name":"listening-for-mesages"}
api_1       | {"queue":"attach-complete","name":"listening-for-mesages"}

Compose file version 3 reference
https://docs.docker.com/compose/compose-file/

Build PoWbox

Get started with Docker Compose, https://docs.docker.com/compose/gettingstarted/

Prerequisites
- docker
- docker compose

docker-compose build

docker-compose.yml
- services
  - Get Started, Part 3: Services, https://docs.docker.com/get-started/part3/
  - Services are really just “containers in production.” A service only runs one image
- Dockerfile.api
- Dockerfile.worker-alpine
  - 無GPU
  - npm install -> package.json
- Dockerfile.worker-debian
  - OpenCL on nvidia
  - npm install

https://github.com/iotaledger/powbox/blob/ae003df500e47f4a8f59650f7056d48277a2ec69/Dockerfile.api#L15

CMD npm run start:api

https://github.com/iotaledger/powbox/blob/ae003df500e47f4a8f59650f7056d48277a2ec69/package.json#L12

"scripts": {
        "build": "webpack -p --config webpack.prod.js",
        "build:dev": "webpack --config webpack.dev.js",
        "precommit": "lint-staged",
        "start:api": "node src/server/index.js",
        "start:worker": "node src/worker/index.js",
        "test": "eslint --ext=js,jsx src"

src/server/index.js

Overview of Docker Compose
https://docs.docker.com/compose/overview/

"Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services."
Using Compose is basically a three-step process:
- Define your app’s environment with a Dockerfile so it can be reproduced anywhere.
- Define the services that make up your app in docker-compose.yml so they can be run together in an isolated environment.
- Run docker-compose up and Compose starts and runs your entire app

powbox/docker-compose.yml
https://github.com/iotaledger/powbox/blob/develop/docker-compose.yml

What is the difference between docker-compose build and docker build? , https://stackoverflow.com/questions/50230399/what-is-the-difference-between-docker-compose-build-and-docker-build

"basically docker-compose build will read your docker-compose.yml, look for all services containing the build: statement and run a docker build for each one"

iotaledger/powbox

Jason

curl-remote

fetch(${sandbox}/api/v1/commands, params)
https://github.com/iotaledger/powbox/blob/ae003df500e47f4a8f59650f7056d48277a2ec69/src/server/index.js#L40
https://github.com/iotaledger/powbox/blob/ae003df500e47f4a8f59650f7056d48277a2ec69/src/server/commands.js#L15
- const { channel } = req.rabbit;
- await channel.assertQueue(INCOMING_QUEUE);

-> BROKER_URL(amqp://guest:guest@rabbitmq-service:5672) -> minikube -> the attach-to-tangle worker (AMQP)

Kubernetes
- 分配nodes資源
curl-remote
- monkeyPatchIOTA
AMQP (Advanced Message Queuing Protocol)
- amqplib
- Asyn

The IOTA PoWbox (Proof of Work box) is a service provided by the IOTA Foundation that enables developers to offload PoW to an optimized remote service, thus speeding up their development workflow — think of it as PoWaaS

The PoWbox now leverages Kubernetes autoscaling across our dedicated GPU farm anda separate cloud provider. PoW should take less than a second in most cases — about one-tenth of a second (1/10 s) at MWM=9.
Kubernetes
The PoWbox uses a simple shim to integrate with iota.lib.js. We have provided the package @iota/curl-remote that monkeypatches the attachToTangle command.
minikube
- Running Kubernetes Locally via Minikube
- dockers
Example Usage:

monkeyPatchIOTA，https://github.com/iotaledger/curl-remote/blob/313990b946fbb6a7e96998611b6ea9f10625f401/index.js#L7
- sandboxATT(..,iotaInstance.sandboxUrl,..)
  - https://github.com/iotaledger/curl-remote/blob/313990b946fbb6a7e96998611b6ea9f10625f401/index.js#L63
- sandboxCheckResult
  - jobCompletionChecker = async ()
    - await sandboxCheckResult
      - https://github.com/iotaledger/curl-remote/blob/313990b946fbb6a7e96998611b6ea9f10625f401/index.js#L63
  - setTimeoutsetTimeout(
    jobCompletionChecker,…)
This is the entry point for the attach-to-tangle worker
- AMQP (Advanced Message Queuing Protocol)
  - amqplib
    https://github.com/iotaledger/powbox/blob/ae003df500e47f4a8f59650f7056d48277a2ec69/src/worker/index.js#L166

muXxer/diverDriver

A client/server interface for hardware POW for IOTA
go語言
https://github.com/muXxer/diverDriver/blob/master/client/client_test.go#L43
- diverClient.PowFunc(data, MWM)
IPC
https://github.com/muXxer/diverDriver/blob/91f4ae2b6dffd009e60e1ec040e38d8bb8f6d9b2/server/ipc/powFunc.go#L211
powMutex.Lock()
- https://github.com/muXxer/diverDriver/blob/91f4ae2b6dffd009e60e1ec040e38d8bb8f6d9b2/server/ipc/server.go#L24

protobuf-c/protobuf-c

Error of PoWbox (https://github.com/iotaledger/powbox/tree/ae003df500e47f4a8f59650f7056d48277a2ec69)

x$ sudo docker-compose up
powbox_mongodb_1 is up-to-date
powbox_rabbitmq_1 is up-to-date
Creating powbox_api_1 ... done
Creating powbox_worker_1 ... done
Attaching to powbox_mongodb_1, powbox_rabbitmq_1, powbox_api_1, powbox_worker_1
api_1       | 
api_1       | > sandbox.js@1.0.0 start:api /app
api_1       | > node src/server/index.js
api_1       | 
api_1       | module.js:550
api_1       |     throw err;
api_1       |     ^
api_1       | 
api_1       | Error: Cannot find module 'dotenv'
api_1       |     at Function.Module._resolveFilename (module.js:548:15)
api_1       |     at Function.Module._load (module.js:475:25)
api_1       |     at Module.require (module.js:597:17)
api_1       |     at require (internal/module.js:11:18)
api_1       |     at Object.<anonymous> (/app/src/common/env.js:6:1)
api_1       |     at Module._compile (module.js:653:30)
api_1       |     at Object.Module._extensions..js (module.js:664:10)
api_1       |     at Module.load (module.js:566:32)
api_1       |     at tryModuleLoad (module.js:506:12)
api_1       |     at Function.Module._load (module.js:498:3)
api_1       | npm ERR! code ELIFECYCLE
api_1       | npm ERR! errno 1
api_1       | npm ERR! sandbox.js@1.0.0 start:api: `node src/server/index.js`
api_1       | npm ERR! Exit status 1
api_1       | npm ERR! 
api_1       | npm ERR! Failed at the sandbox.js@1.0.0 start:api script.
api_1       | npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
api_1       | npm WARN Local package.json exists, but node_modules missing, did you mean to install?
api_1       | 
api_1       | npm ERR! A complete log of this run can be found in:
api_1       | npm ERR!     /root/.npm/_logs/2019-02-11T05_48_55_670Z-debug.log
mongodb_1   | 2019-02-11T05:30:56.853+0000 I CONTROL  [main] Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'
mongodb_1   | 2019-02-11T05:30:56.858+0000 I CONTROL  [initandlisten] MongoDB starting : pid=1 port=27017 dbpath=/data/db 64-bit host=413e2d46cfd1
mongodb_1   | 2019-02-11T05:30:56.858+0000 I CONTROL  [initandlisten] db version v4.0.6
mongodb_1   | 2019-02-11T05:30:56.858+0000 I CONTROL  [initandlisten] git version: caa42a1f75a56c7643d0b68d3880444375ec42e3
mongodb_1   | 2019-02-11T05:30:56.858+0000 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.2g  1 Mar 2016
mongodb_1   | 2019-02-11T05:30:56.859+0000 I CONTROL  [initandlisten] allocator: tcmalloc
mongodb_1   | 2019-02-11T05:30:56.859+0000 I CONTROL  [initandlisten] modules: none
mongodb_1   | 2019-02-11T05:30:56.859+0000 I CONTROL  [initandlisten] build environment:
mongodb_1   | 2019-02-11T05:30:56.859+0000 I CONTROL  [initandlisten]     distmod: ubuntu1604
mongodb_1   | 2019-02-11T05:30:56.859+0000 I CONTROL  [initandlisten]     distarch: x86_64
mongodb_1   | 2019-02-11T05:30:56.859+0000 I CONTROL  [initandlisten]     target_arch: x86_64
mongodb_1   | 2019-02-11T05:30:56.859+0000 I CONTROL  [initandlisten] options: { net: { bindIpAll: true } }
mongodb_1   | 2019-02-11T05:30:56.859+0000 I STORAGE  [initandlisten] 
mongodb_1   | 2019-02-11T05:30:56.859+0000 I STORAGE  [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
mongodb_1   | 2019-02-11T05:30:56.859+0000 I STORAGE  [initandlisten] **          See http://dochub.mongodb.org/core/prodnotes-filesystem
mongodb_1   | 2019-02-11T05:30:56.859+0000 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=3464M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),
mongodb_1   | 2019-02-11T05:30:58.616+0000 I STORAGE  [initandlisten] WiredTiger message [1549863058:616330][1:0x7fa837061a40], txn-recover: Set global recovery timestamp: 0
mongodb_1   | 2019-02-11T05:30:58.666+0000 I RECOVERY [initandlisten] WiredTiger recoveryTimestamp. Ts: Timestamp(0, 0)
mongodb_1   | 2019-02-11T05:30:58.781+0000 I CONTROL  [initandlisten] 
rabbitmq_1  | 
rabbitmq_1  |   ##  ##
rabbitmq_1  |   ##  ##      RabbitMQ 3.7.11. Copyright (C) 2007-2019 Pivotal Software, Inc.
rabbitmq_1  |   ##########  Licensed under the MPL.  See http://www.rabbitmq.com/
rabbitmq_1  |   ######  ##
rabbitmq_1  |   ##########  Logs: <stdout>
rabbitmq_1  | 
rabbitmq_1  |               Starting broker...
rabbitmq_1  | 2019-02-11 05:31:09.279 [info] <0.208.0> 
rabbitmq_1  |  Starting RabbitMQ 3.7.11 on Erlang 21.2.5
rabbitmq_1  |  Copyright (C) 2007-2019 Pivotal Software, Inc.
rabbitmq_1  |  Licensed under the MPL.  See http://www.rabbitmq.com/
rabbitmq_1  | 2019-02-11 05:31:09.289 [info] <0.208.0> 
rabbitmq_1  |  node           : rabbit@7a735f848b21
rabbitmq_1  |  home dir       : /var/lib/rabbitmq
rabbitmq_1  |  config file(s) : /etc/rabbitmq/rabbitmq.conf
rabbitmq_1  |  cookie hash    : UNdn0n6e+CiAH49uAo/06Q==
rabbitmq_1  |  log(s)         : <stdout>
rabbitmq_1  |  database dir   : /var/lib/rabbitmq/mnesia/rabbit@7a735f848b21
rabbitmq_1  | 2019-02-11 05:31:11.875 [info] <0.216.0> Memory high watermark set to 3181 MiB (3335977369 bytes) of 7953 MiB (8339943424 bytes) total
rabbitmq_1  | 2019-02-11 05:31:11.882 [info] <0.218.0> Enabling free disk space monitoring
rabbitmq_1  | 2019-02-11 05:31:11.882 [info] <0.218.0> Disk free limit set to 50MB
rabbitmq_1  | 2019-02-11 05:31:11.888 [info] <0.221.0> Limiting to approx 1048476 file handles (943626 sockets)
rabbitmq_1  | 2019-02-11 05:31:11.888 [info] <0.222.0> FHC read buffering:  OFF
rabbitmq_1  | 2019-02-11 05:31:11.888 [info] <0.222.0> FHC write buffering: ON
rabbitmq_1  | 2019-02-11 05:31:11.889 [info] <0.208.0> Node database directory at /var/lib/rabbitmq/mnesia/rabbit@7a735f848b21 is empty. Assuming we need to join an existing cluster or initialise from scratch...
rabbitmq_1  | 2019-02-11 05:31:11.889 [info] <0.208.0> Configured peer discovery backend: rabbit_peer_discovery_classic_config
rabbitmq_1  | 2019-02-11 05:31:11.889 [info] <0.208.0> Will try to lock with peer discovery backend rabbit_peer_discovery_classic_config
rabbitmq_1  | 2019-02-11 05:31:11.889 [info] <0.208.0> Peer discovery backend does not support locking, falling back to randomized delay
rabbitmq_1  | 2019-02-11 05:31:11.890 [info] <0.208.0> Peer discovery backend rabbit_peer_discovery_classic_config does not support registration, skipping randomized startup delay.
rabbitmq_1  | 2019-02-11 05:31:11.890 [info] <0.208.0> All discovered existing cluster peers: 
mongodb_1   | 2019-02-11T05:30:58.781+0000 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
mongodb_1   | 2019-02-11T05:30:58.781+0000 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
mongodb_1   | 2019-02-11T05:30:58.781+0000 I CONTROL  [initandlisten] 
mongodb_1   | 2019-02-11T05:30:58.782+0000 I STORAGE  [initandlisten] createCollection: admin.system.version with provided UUID: 5bee79a9-1e52-4b2d-a01d-e88358a50e44
mongodb_1   | 2019-02-11T05:30:58.913+0000 I COMMAND  [initandlisten] setting featureCompatibilityVersion to 4.0
mongodb_1   | 2019-02-11T05:30:58.957+0000 I STORAGE  [initandlisten] createCollection: local.startup_log with generated UUID: 1d0af8c1-4f73-438b-a52c-71b00bce8c22
mongodb_1   | 2019-02-11T05:30:59.108+0000 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/data/db/diagnostic.data'
mongodb_1   | 2019-02-11T05:30:59.112+0000 I NETWORK  [initandlisten] waiting for connections on port 27017
mongodb_1   | 2019-02-11T05:30:59.113+0000 I STORAGE  [LogicalSessionCacheRefresh] createCollection: config.system.sessions with generated UUID: f059aa16-f3e3-4884-b30a-689ca29be939
mongodb_1   | 2019-02-11T05:30:59.200+0000 I INDEX    [LogicalSessionCacheRefresh] build index on: config.system.sessions properties: { v: 2, key: { lastUse: 1 }, name: "lsidTTLIndex", ns: "config.system.sessions", expireAfterSeconds: 1800 }
mongodb_1   | 2019-02-11T05:30:59.200+0000 I INDEX    [LogicalSessionCacheRefresh] 	 building index using bulk method; build may temporarily use up to 500 megabytes of RAM
mongodb_1   | 2019-02-11T05:30:59.203+0000 I INDEX    [LogicalSessionCacheRefresh] build index done.  scanned 0 total records. 0 secs
rabbitmq_1  | 2019-02-11 05:31:11.890 [info] <0.208.0> Discovered no peer nodes to cluster with
rabbitmq_1  | 2019-02-11 05:31:11.901 [info] <0.43.0> Application mnesia exited with reason: stopped
rabbitmq_1  | 2019-02-11 05:31:12.264 [info] <0.208.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
rabbitmq_1  | 2019-02-11 05:31:12.308 [info] <0.208.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
rabbitmq_1  | 2019-02-11 05:31:12.394 [info] <0.208.0> Waiting for Mnesia tables for 30000 ms, 9 retries left
rabbitmq_1  | 2019-02-11 05:31:12.394 [info] <0.208.0> Peer discovery backend rabbit_peer_discovery_classic_config does not support registration, skipping registration.
rabbitmq_1  | 2019-02-11 05:31:12.397 [info] <0.208.0> Priority queues enabled, real BQ is rabbit_variable_queue
rabbitmq_1  | 2019-02-11 05:31:12.405 [info] <0.392.0> Starting rabbit_node_monitor
rabbitmq_1  | 2019-02-11 05:31:12.450 [info] <0.208.0> message_store upgrades: 1 to apply
rabbitmq_1  | 2019-02-11 05:31:12.450 [info] <0.208.0> message_store upgrades: Applying rabbit_variable_queue:move_messages_to_vhost_store
rabbitmq_1  | 2019-02-11 05:31:12.451 [info] <0.208.0> message_store upgrades: No durable queues found. Skipping message store migration
rabbitmq_1  | 2019-02-11 05:31:12.451 [info] <0.208.0> message_store upgrades: Removing the old message store data
rabbitmq_1  | 2019-02-11 05:31:12.452 [info] <0.208.0> message_store upgrades: All upgrades applied successfully
rabbitmq_1  | 2019-02-11 05:31:12.502 [info] <0.208.0> Adding vhost '/'
rabbitmq_1  | 2019-02-11 05:31:12.592 [info] <0.427.0> Making sure data directory '/var/lib/rabbitmq/mnesia/rabbit@7a735f848b21/msg_stores/vhosts/628WB79CIFDYO9LJI6DKMI09L' for vhost '/' exists
rabbitmq_1  | 2019-02-11 05:31:12.602 [info] <0.427.0> Starting message stores for vhost '/'
rabbitmq_1  | 2019-02-11 05:31:12.602 [info] <0.431.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_transient": using rabbit_msg_store_ets_index to provide index
rabbitmq_1  | 2019-02-11 05:31:12.604 [info] <0.427.0> Started message store of type transient for vhost '/'
rabbitmq_1  | 2019-02-11 05:31:12.605 [info] <0.434.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": using rabbit_msg_store_ets_index to provide index
rabbitmq_1  | 2019-02-11 05:31:12.606 [warning] <0.434.0> Message store "628WB79CIFDYO9LJI6DKMI09L/msg_store_persistent": rebuilding indices from scratch
rabbitmq_1  | 2019-02-11 05:31:12.608 [info] <0.427.0> Started message store of type persistent for vhost '/'
rabbitmq_1  | 2019-02-11 05:31:12.610 [info] <0.208.0> Creating user 'guest'
rabbitmq_1  | 2019-02-11 05:31:12.626 [info] <0.208.0> Setting user tags for user 'guest' to [administrator]
rabbitmq_1  | 2019-02-11 05:31:12.637 [info] <0.208.0> Setting permissions for 'guest' in '/' to '.*', '.*', '.*'
rabbitmq_1  | 2019-02-11 05:31:12.666 [warning] <0.458.0> Setting Ranch options together with socket options is deprecated. Please use the new map syntax that allows specifying socket options separately from other options.
rabbitmq_1  | 2019-02-11 05:31:12.667 [info] <0.472.0> started TCP listener on [::]:5672
rabbitmq_1  | 2019-02-11 05:31:12.677 [info] <0.208.0> Setting up a table for connection tracking on this node: tracked_connection_on_node_rabbit@7a735f848b21
rabbitmq_1  | 2019-02-11 05:31:12.704 [info] <0.208.0> Setting up a table for per-vhost connection counting on this node: tracked_connection_per_vhost_on_node_rabbit@7a735f848b21
rabbitmq_1  | 2019-02-11 05:31:13.052 [info] <0.8.0> Server startup complete; 0 plugins started.
rabbitmq_1  |  completed with 0 plugins.
worker_1    | module.js:550
worker_1    |     throw err;
worker_1    |     ^
worker_1    | 
worker_1    | Error: Cannot find module 'dotenv'
worker_1    |     at Function.Module._resolveFilename (module.js:548:15)
worker_1    |     at Function.Module._load (module.js:475:25)
worker_1    |     at Module.require (module.js:597:17)
worker_1    |     at require (internal/module.js:11:18)
worker_1    |     at Object.<anonymous> (/app/src/common/env.js:6:1)
worker_1    |     at Module._compile (module.js:653:30)
worker_1    |     at Object.Module._extensions..js (module.js:664:10)
worker_1    |     at Module.load (module.js:566:32)
worker_1    |     at tryModuleLoad (module.js:506:12)
worker_1    |     at Function.Module._load (module.js:498:3)
powbox_worker_1 exited with code 1
powbox_api_1 exited with code 1

Docker Official Images: rabbitmq

RabbitMQ is an open source multi-protocol messaging broker, https://hub.docker.com/_/rabbitmq

amqplib - AMQP 0-9-1 library and client for Node.JS
https://www.npmjs.com/package/amqplib

https://www.npmjs.com/package/amqp

Options can also be passed in a single URL of the form

amqp[s]://[user:password@]hostname[:port][/vhost]

BROKER_URL=amqp://guest:guest@rabbitmq-service:5672

If One public host is dev.rabbitmq.com:

URL=amqp://dev.rabbitmq.com npm test

AMQP 0-9-1 Model Explained

https://www.rabbitmq.com/tutorials/amqp-concepts.html

Documentation: Table of Contents

Properties

Name
Durable (the queue will survive a broker restart)
Exclusive (used by only one connection and the queue will be deleted when that connection closes)
Auto-delete (queue that has had at least one consumer is deleted when last consumer unsubscribes)
Arguments (optional; used by plugins and broker-specific features such as message TTL, queue length limit, etc)

write requests大小設定
https://github.com/alanxz/rabbitmq-c/blob/257d2918271e9fa3bf32170dc0d8a49ac323392f/librabbitmq/amqp.h#L1748

#define AMQP_DEFAULT_FRAME_SIZE 131072 (128kb)

https://github.com/alanxz/rabbitmq-c/blob/a65c64c0efd883f3e200bd8831ad3ca066ea523c/librabbitmq/amqp_framing.h#L50

#define AMQP_PROTOCOL_PORT 5672      /**< Default AMQP Port */
#define AMQP_FRAME_METHOD 1          /**< Constant: FRAME-METHOD */
#define AMQP_FRAME_HEADER 2          /**< Constant: FRAME-HEADER */
#define AMQP_FRAME_BODY 3            /**< Constant: FRAME-BODY */
#define AMQP_FRAME_HEARTBEAT 8       /**< Constant: FRAME-HEARTBEAT */
#define AMQP_FRAME_MIN_SIZE 4096     /**< Constant: FRAME-MIN-SIZE */
#define AMQP_FRAME_END 206           /**< Constant: FRAME-END */
#define AMQP_REPLY_SUCCESS 200       /**< Constant: REPLY-SUCCESS */
#define AMQP_CONTENT_TOO_LARGE 311   /**< Constant: CONTENT-TOO-LARGE */
#define AMQP_NO_ROUTE 312            /**< Constant: NO-ROUTE */
#define AMQP_NO_CONSUMERS 313        /**< Constant: NO-CONSUMERS */
#define AMQP_ACCESS_REFUSED 403      /**< Constant: ACCESS-REFUSED */
#define AMQP_NOT_FOUND 404           /**< Constant: NOT-FOUND */
#define AMQP_RESOURCE_LOCKED 405     /**< Constant: RESOURCE-LOCKED */
#define AMQP_PRECONDITION_FAILED 406 /**< Constant: PRECONDITION-FAILED */
#define AMQP_CONNECTION_FORCED 320   /**< Constant: CONNECTION-FORCED */
#define AMQP_INVALID_PATH 402        /**< Constant: INVALID-PATH */
#define AMQP_FRAME_ERROR 501         /**< Constant: FRAME-ERROR */
#define AMQP_SYNTAX_ERROR 502        /**< Constant: SYNTAX-ERROR */
#define AMQP_COMMAND_INVALID 503     /**< Constant: COMMAND-INVALID */
#define AMQP_CHANNEL_ERROR 504       /**< Constant: CHANNEL-ERROR */
#define AMQP_UNEXPECTED_FRAME 505    /**< Constant: UNEXPECTED-FRAME */
#define AMQP_RESOURCE_ERROR 506      /**< Constant: RESOURCE-ERROR */
#define AMQP_NOT_ALLOWED 530         /**< Constant: NOT-ALLOWED */
#define AMQP_NOT_IMPLEMENTED 540     /**< Constant: NOT-IMPLEMENTED */
#define AMQP_INTERNAL_ERROR 541      /**< Constant: INTERNAL-ERROR */

r.reply_type

amqp_queue_*

amqp_queue_

amqp_table_t，https://github.com/alanxz/rabbitmq-c/blob/257d2918271e9fa3bf32170dc0d8a49ac323392f/librabbitmq/amqp.h#L428
Optional queue arguments, also know as "x-arguments" because of their field name in the AMQP 0-9-1 protocol

Optional arguments can be provided in two ways:

To groups of queues using policies (recommended)
On a per-queue basis when a queue is declared by a client

Message Ordering

Queues in RabbitMQ are ordered collections of messages. Messages are enqueued and dequeued (consumed) in the FIFO manner, although priority queues, sharded queues and other features may affect this.

rabbitmq-sharding
https://github.com/rabbitmq/rabbitmq-sharding/

Determining Queue Length

With AMQP 0-9-1, using a property on the queue.declare method response (queue.declare-ok). The field name is message_count. How it is accessed varies from client library to client library.
Using RabbitMQ HTTP API.
Using the rabbitmqctl list_queues command.

Message States
Enqueued messages therefore can be in one of two states:

Ready for delivery
Delivered but not yet acknowledged by consumer

AMQP 0-9-1 (Advanced Message Queuing Protocol)

AMQP is a Programmable Protocol (可以設定裡面broker元件)

AMQP 0-9-1 is a programmable protocol in the sense that AMQP 0-9-1 entities and routing schemes are primarily defined by applications themselves, not a broker administrator. Accordingly, provision is made for protocol operations that declare queues and exchanges, define bindings between them, subscribe to queues and so on.

Exchanges and Exchange Types
Exchanges are AMQP 0-9-1 entities where messages are sent. Exchanges take a message and route it into zero or more queues. The routing algorithm used depends on the exchange type and rules called bindings

Default Exchange

every queue that is created is automatically bound to it with a routing key which is the same as the queue name

Consumers

Have messages delivered to them ("push API")
- register a consumer/ simply put/ subscribe to a queue
Fetch messages as needed ("pull API")

Message Attributes and Payload

Content type
Content encoding
Routing key
Delivery mode (persistent or not)
Message priority
Message publishing timestamp
Expiration period
Publisher application id (可知這個message是哪一個client?)

Payload

It is possible for messages to contain only attributes and no payload. It is common to use serialisation formats like JSON, Thrift, Protocol Buffers and MessagePack to serialize structured data in order to publish it as the message payload
- content-type
- content-encoding

Message Acknowledgements

Since networks are unreliable and applications fail
If an application crashes (the AMQP broker notices this when the connection is closed),
if an acknowledgement for a message was expected but not received by the AMQP broker,
the message is re-queued (and possibly immediately delivered to another consumer, if any exists).

Channels

lightweight connections that share a single TCP connection

An API layer

prefetch_count=1
This tells RabbitMQ not to give more than one message to a worker at a time. Or, in other words, don't dispatch a new message to a worker until it has processed and acknowledged the previous one. Instead, it will dispatch it to the next worker that is not still busy

Reference

IOTA Sandbox Guide，https://hackmd.io/s/ByWQUqfh-
node-amqp, https://www.npmjs.com/package/amqp
attachToTangle, https://iota.readme.io/reference#gettransactionstoapprove
Get started with Docker Compose, https://docs.docker.com/compose/gettingstarted/
iotaledger/powbox, https://github.com/iotaledger/powbox
About the PoWbox, https://blog.iota.org/relaunching-the-powbox-d392236b6939
On setting up highly available Kubernetes clusters, https://elastisys.com/2018/01/25/setting-highly-available-kubernetes-clusters/
Running Kubernetes Locally via Minikube, https://kubernetes.io/docs/setup/minikube/
@iota/curl-remote, https://www.npmjs.com/package/@iota/curl-remote
https://github.com/iotaledger/curl-remote
muXxer/diverDriver, https://github.com/muXxer/diverDriver
RabbitMQ, https://github.com/rabbitmq
iotaledger/ccurl.interface.js, https://github.com/iotaledger/ccurl.interface.js
protobuf-c/protobuf-c, https://github.com/protobuf-c/protobuf-c
protobuf-c-rpc, https://github.com/protobuf-c/protobuf-c-rpc
protobuf-c RPC Example, https://github.com/protobuf-c/protobuf-c/wiki/RPC-Example
RabbitMQ C client
, https://github.com/alanxz/rabbitmq-c

Syntax	Example	Reference
# Header	Header	基本排版
- Unordered List	Unordered List
1. Ordered List	Ordered List
- [ ] Todo List	Todo List
> Blockquote	Blockquote
Bold font	Bold font
Italics font	Italics font
~~Strikethrough~~	~~Strikethrough~~
19^th^	19^th
H~2~O	H₂O
++Inserted text++	Inserted text
==Marked text==	Marked text
[link text](https:// "title")	Link
![image alt](https:// "title")	Image
`Code`	`Code`	在筆記中貼入程式碼
```javascript var i = 0; ```	`var i = 0;`	在筆記中貼入程式碼
:smile:		Emoji list
{%youtube youtube_id %}	Externals
$L^aT_eX$	L^aT_eX
:::info This is a alert area. :::	This is a alert area.