# Signature-based off-chain data/computing

* tangle-accelerator aarch64 crossbuild
* aarch64-linux-gnu-
* https://github.com/dthaler/OTrP/blob/fc4a6e5a1ad3c8ab8cc2feebe7f84fff90571a55/OTrPCommonTALib/optee/linux_gcc.mak#L13
* tensorflow-aarch64-crossbuild
https://github.com/xifengcun/tensorflow-aarch64-crossbuild

$ bazel build //accelerator --cpu=aarch64 --crosstool_top=//tools/aarch64_compiler:toolchain --verbose_failures

INFO: Invocation ID: 8adbeb7f-2e1c-49ac-8230-e4708000ce86
ERROR: /home/ajubuntu/Desktop/tangle-accelerator/tools/aarch64_compiler/BUILD:3:1: in cc_toolchain_suite rule //tools/aarch64_compiler:toolchain: cc_toolchain_suite '//tools/aarch64_compiler:toolchain' does not contain a toolchain for cpu 'aarch64'
ERROR: Analysis of target '//accelerator:accelerator' failed; build aborted: Analysis of target '//tools/aarch64_compiler:toolchain' failed; build aborted
INFO: Elapsed time: 1.287s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded, 3 targets configured)
    Fetching @boost; fetching

## Open trusted protocol - CA


* TrP/DeviceHost/DeviceHost.c，https://github.com/dthaler/OTrP/blob/71351ad925a1cceacfe2926e33b5dc86ce9ac415/DeviceHost/DeviceHost.c#L6

#define DEFAULT_TAM_URI "http://localhost:54321/OTrP"

* OTrP/OTrPTamBrokerLib/HttpServer.h，https://github.com/dthaler/OTrP/blob/71351ad925a1cceacfe2926e33b5dc86ce9ac415/OTrPTamBrokerLib/HttpServer.h#L4

#define OTRP_URI L"http://localhost:54321/OTrP"

https://github.com/DLTcollab/general/issues/122

https://github.com/torvalds/linux/blob/master/Documentation/tee.txt
TEE subsys, linux/drivers/tee/tee_core.c, https://github.com/torvalds/linux/blob/master/drivers/tee/tee_core.c
* EXPORT_SYMBOL_GPL
* tee_device_alloc
* tee_device_register
* tee_device_unregister
* tee_get_drvdata
* tee_client_open_context
* tee_client_close_context
* tee_client_get_version
* tee_client_open_session
* tee_client_close_session
* tee_client_invoke_func
* Example:
* smaf/optee interface driver
<https://github.com/etienne-lms/linux/commit/89e66efb27d927ba1f6f54f54a7274f5759e5b13>

## 發起交易

### Secure data path issue


:::danger
如果收集的資料至normal world，在做加密，會有一段時間有機會被惡意程式竊取。
解法:
要把抓資料的driver porting至optee的driver層，並提供optee API給TA使用。如果要做這步需要不少時間。
* 安裝secure interrupt
* serure optee drvier
* 提供 optee API給 tasklet用
:::

### Send a transaction flow

* send transactio flow，參考: https://hackmd.io/s/H10c9WcyN#

### Trustzone

* 假設一組seed和private key對應一個IOT device。
* 在TA中產生一組且唯一的seed與pk，存至secure storage中
* seed
* Creating a new Seed: You must generate a random 81 character seed using only A-Z and the number 9, https://iotasupport.com/gui-newseed.shtml
* optee_examples/random/ta/random_example_ta.c, https://github.com/linaro-swg/optee_examples/blob/1c5d96fb2f6abab232bc06705fe557bc4f76964b/random/ta/random_example_ta.c
* UUID 32個16進位數字
* TEE_GenerateRandom
* byte random generator
* byte % (26 + 1)
* 0 -> A
* 1 -> B
* ...
* 26 -> 9
* private key

* 在qemu+optee上撰寫TA負責generate address，並驗證此TA產生正確。
* 在qemu+optee上撰寫TA負責generatre signature，並驗證此TA執行正確。
* https://github.com/ajblane/IOTA_SEED_RELATED_API
* https://github.com/iotaledger/entangled/blob/4e1434d0fbf5926f23c8c4f0fb1385971e47b45c/common/helpers/tests/test_sign.cpp
* 在qemu+optee上傳撰寫TA負責加解密Message，並驗證此TA執行正確。
* Cryptographic operations functions
* TEE_AsymmetricEncrypt()
* Decrypt()
* SignDigest()
* VerifyDigest()
* TEE_GenerateKey()
* optee_examples/acipher/ta/acipher_ta.c, https://github.com/linaro-swg/optee_examples/blob/9396d279b0c112a0bb2b83cc53ee95fac6ef626c/acipher/ta/acipher_ta.c

* 在硬體上建置 tangle-accelerator(dcurl)
* https://github.com/DLTcollab/tangle-accelerator/tree/9f9ce5b9288b3fe1f950170efae008d3c9d61111

* How to Configure Pi with Static IP Address?, https://www.raspberrypi.org/forums/viewtopic.php?t=221060

	sudo nano /etc/dhcpcd.conf
	#Config for static IP on eth0
	interface eth0 
	static ip_address=192.168.1.121/24 
    static routers=192.168.1.1 
	static domain_name_servers=192.168.1.1

* How to use your Raspberry Pi as a wireless access point, https://thepi.io/how-to-use-your-raspberry-pi-as-a-wireless-access-point/
* raspberry-pi-turnkey/config/, https://github.com/schollz/raspberry-pi-turnkey/tree/master/config
* Setting up Raspberry Pi WiFi with Static IP on Raspbian Stretch Lite, https://electrondust.com/2017/11/25/setting-raspberry-pi-wifi-static-ip-raspbian-stretch-lite/
* [教學] 啟用 Pi 3B+的WiFi, https://www.raspberrypi.com.tw/tag/wpa_supplicant-conf/
* https://www.op-tee.org/docs/rpi3/
* rpi 3
* 新硬體
* tangle-accelerator與seed-related TA-implemented IOTA API做整合並驗證執行正確。
* 無加入Queue驗證，先給一組資料驗證整合seed-related TA-implemented IOTA API至tangle-accelerator是否執行正確。
* 加入抓資料的API與Queue整合至tangle-accelerator並可以產生transaction

* 實作tangle-accelerator + seed-related TA-implemented IOTA API 的run time cost分析
* https://github.com/linaro-swg/optee_benchmark
* Profiling user Trusted Applications with gprof，https://github.com/OP-TEE/optee_os/blob/master/documentation/gprof.md
* 實作seed-related TA-implemented IOTA API的footprint
:::danger
關於Seed-Related TA-implemented IOTA API (iota_sign_address_gen/ iota_sign_signature_gen)加速的想法，目前未定案。

參考結案報告，需做Keccak運算，可能需要做不使用FPGA的加速(FPGA資源是做POW)，來增加產生交易速度。或者也可以做normal world的muti-thread設計來與ta合作取得address或者signature，參考:[ta/concurrent](https://github.com/OP-TEE/optee_test/tree/master/ta/concurrent)
:::
* Performance problem: Normal world vs Trusted Application，https://github.com/OP-TEE/optee_os/issues/1566

### How to verify secure seed implementation


* 預計使用metasploit拿到reverse shell。
* 先假設惡意程式可執行。

* download file
* https://github.com/yillkid/iota-swarm-node/blob/80be3fa4084155588d7cfa2bfb56e2ae18bca34f/config.py

* string
* https://github.com/DLTcollab/tangle-accelerator/blob/3508fbdcaf12f2c9e9dafb2ae821b36f6b661e47/config.h

### FPGA

參考[FPGA-accelerated TrustZone-enabled IOTA Swarm Nodes](https://hackmd.io/s/Hk-5i41bm)，由於在新的FPGA板子，如果不是Cyclone V，需做以下工作:
* 修改verilog，整合FPGA與CPU溝通的介面
* 重新合成
* 需修改linux driver，包含與FPGA溝通介面或者linux版本不同所需的修改。

## Bazel

* Installing Bazel on Ubuntu, https://docs.bazel.build/versions/master/install-ubuntu.html
* Configuring bazel to cross compile for the Raspberry Pi 3, https://jpieper.com/2018/10/28/configuring-bazel-to-cross-compile-for-the-raspberry-pi-3/
* mjbots/rpi_bazel, https://github.com/mjbots/rpi_bazel
* Detailed examples demonstrating cross-platform building with Bazel, https://github.com/mobileink/bazel-crosscompile
* Building with a custom toolchain, https://github.com/bazelbuild/bazel/wiki/Building-with-a-custom-toolchain
* Building TensorFlow for Raspberry Pi: a Step-By-Step Guide, https://github.com/samjabrahams/tensorflow-on-raspberry-pi/blob/master/GUIDE.md#3-build-bazel
* Raspberry Pi binaries, https://github.com/ochafik/rpi-raspbian-bazel#raspberry-pi-binaries

cc_library(
    name = "api",
    srcs = [
        "iota_client_core_api.c",
        "iota_client_extended_api.c",
    ],
    hdrs = [
        "iota_client_core_api.h",
        "iota_client_extended_api.h",
    ],
    visibility = ["//visibility:public"],
    deps = [
        ":service",
        "//cclient/http",
        "//common/helpers:checksum",
        "//common/helpers:digest",
        "//common/helpers:pow",
        "//common/helpers:sign",
        "//utils:time",
        "//utils/containers/hash:hash243_queue",
    ],
)
cc_library(
    name = "shared",
    hdrs = ["http.h"],
    visibility = ["//visibility:public"],
    deps = ["//cclient:service"],
)

cc_library(
    name = "http",
    srcs = [
        "http.c",
    ],
    visibility = ["//visibility:public"],
    deps = [
        ":shared",
        "//network:sockets",
        "@http_parser",
    ],
)
cc_binary(
    name = "libcclient.so",
    linkshared = True,
    deps = [":api"],
)

cc_library(
    name = "service",
    srcs = [
        "service.c",
    ],
    hdrs = ["service.h"],
    visibility = ["//visibility:public"],
    deps = [
        "//cclient/request:requests",
        "//cclient/response:responses",
        "//cclient/serialization:serializer_json",
    ],
)

## CPU比較
* RPi3: BCM2837 quad-core ARM Cortex A53 (ARMv8) https://www.raspberrypi.org/documentation/hardware/raspberrypi/bcm2837/README.md
* 1GB RAM
* Intel Cyclone V: dual-core ARM Cortex-A9 MPCore* processor
* 1GB DDR3 SDRAM

## FPGA與Truszone整合描述

* 整合方案一: RPi 3透過區域網路方式連接FPGA board，**但對外連接需要hub或者開啟無線網路。**

* 整合方案二:
* rpi3與FPGA板子透過GPIO溝通或者其他介面。

LampaLab/iota_fpga
* DE10-nano board, https://github.com/LampaLab/iota_fpga
https://www.terasic.com.tw/cgi-bin/page/archive.pl?Language=Taiwan&No=1047


891*4.6 = 4098

Hikey 960, https://www.96boards.org/documentation/consumer/hikey/hikey960/getting-started/

* 整合方案三: 重現pidriver專案，但需要買cyclone 10 lp (10cl025) 支援SPI的板子，作者自己客製化一個PCB版 with a modern FPGA Cyclone 10 LP。
https://microengineer.eu/2018/04/25/iota-pearl-diver-fpga/

## 查詢交易 (解交易)，尚未規畫完整

* 需找出相關使用seed與PK的IOTA API
* 撰寫TA
* 提供secure API給normal world使用
* 放交易queue設計

## ta examples

* Demo Trusted Application using Comcast Crypto API and OP-TEE, https://github.com/jbech-linaro/comcast_cryptoapi_ta_example
* TrustZone-backed Bitcoin Wallet, https://fenix.tecnico.ulisboa.pt/downloadFile/1689244997256880/Thesis.pdf, https://github.com/Miraje/TrustZone-backed-Bitcoin-Wallet/blob/master/ta/wallet_ta.c

## taobao
https://world.taobao.com/dianpu/cat-35646333-810067291.htm


https://item.taobao.com/item.htm?spm=a21wu.12321156-tw.0.0.2558b750ADxZIS&id=557802312480
40000
https://www.mouser.tw/ProductDetail/Intel-Altera/5CSEMA4U23C6N?qs=TN6zvtaX%252baNEGVGZbjwaqA==
Intel Cyclone® V SE 5CSEBA6U23I7NDK
5382


## pidiver

* Raspberry-Pi-Mode
* PiDiver doing USBDiving

https://microengineer.eu/2018/04/25/iota-pearl-diver-fpga/

https://microengineer.eu/2018/04/24/old-iota-pow-hardware-accelerator-fpga-for-altera-de1/

Cyclone 10 LP SPI Slave to Avalon Master Bridge Design Example
https://fpgacloud.intel.com/devstore/platform/17.0.0/Standard/cyclone-10-lp-spi-slave-to-avalon-master-bridge-design-example/

INTEL® CYCLONE® 10 LP FPGA APPLICATIONS
https://www.intel.com.tw/content/www/tw/zh/products/programmable/fpga/cyclone-10/lp/applications.html

Design Store
https://fpgacloud.intel.com/devstore/platform/?acds_version=any&family=cyclone-10-lp

How to enable SPI on HiKey 960
https://discuss.96boards.org/t/how-to-enable-spi-on-hikey-960/4750/7

## Shadow-Box
https://github.com/kkamagui/shadow-box-for-arm

## for general

hook

## PoW remote server
https://hackmd.io/RkjqvkH8Scel53V7-XgHOA
## issues

* Enhance data security design for IOT applications based on IOTA, https://github.com/DLTcollab/general/issues/118
* transaction cloud
* fpga cloud

## Trace optee 編譯流程

https://hackmd.io/n_wwCngqSI65f1SBUVR8ww

## Trace tangle accelerator

https://hackmd.io/kSQfXM-OTQ-KdziaVS_msA?both

## tangle accelerator and rabbitmq c client

https://hackmd.io/kSQfXM-OTQ-KdziaVS_msA?both

## Cloud design pattern

Competing Consumers pattern
https://docs.microsoft.com/en-us/azure/architecture/patterns/competing-consumers

RabbitMQ Work Queues: Avoiding Data Inconsistency with Rebalanser, https://jack-vanlightly.com/blog/2018/7/25/rabbitmq-work-queues-avoiding-data-inconsistency-with-rebalanser

What is an idempotent operation?
https://stackoverflow.com/questions/1077412/what-is-an-idempotent-operation
* f(f(x)) = f(x)
* abs(abs(x)) = abs(x)

Asynchronous Messaging Primer
https://docs.microsoft.com/en-us/previous-versions/msp-n-p/dn589781(v=pandp.10)

Request/response messaging

Broadcast messaging

Cloud Design Patterns: Prescriptive Architecture Guidance for Cloud Applications
https://docs.microsoft.com/en-us/previous-versions/msp-n-p/dn568099(v%3dpandp.10)

Microservices: Asynchronous Request Response Pattern
https://medium.com/@pulkitswarup/microservices-asynchronous-request-response-pattern-6d00ab78abb6

Asynchronous communication with message queue
https://codeblog.dotsandbrackets.com/asynchronous-communication-with-message-queue/

* Callback queue
* reply_to
* correlation_id
* RPC workers

RPC calls to multiple consumers
https://stackoverflow.com/questions/12407485/rpc-calls-to-multiple-consumers

Is the RabbitMQ RPC a kind of “Work queue” with response?
https://stackoverflow.com/questions/45695071/is-the-rabbitmq-rpc-a-kind-of-work-queue-with-response

RabbitMQ in Depth
https://manning-content.s3.amazonaws.com/download/b/5ceeb08-0e8d-4f94-8d06-0300996d9427/sample_ch01_Roy_Rabbit-MQ-in-Depth_September11.pdf

https://manning-content.s3.amazonaws.com/download/c/8a042e9-9293-43aa-bea1-b42ee0008c09/sample_ch02_Roy_Rabbit-MQ-in-Depth_September11.pdf

A Survey of Distributed Message Broker Queues
https://arxiv.org/pdf/1704.00411.pdf

RabbitMQ Essentials
http://aad.tpu.ru/1955/Java%20books/RabbitMQ%20Essentials.pdf


MESSAGE BROKERS AND RABBITMQ IN ACTION
http://aad.tpu.ru/1955/Java%20books/RabbitMQ%20Essentials.pdf

Getting Started with RabbitMQ and CloudAMQP
http://193.40.244.77/idu0080/wp-content/uploads/2016/02/Getting_Started_with_RabbitMQ_and_CloudAMQP-1.pdf

Asynchronous Joins Using RabbitMQ
https://www.compose.com/articles/asynchronous-joins-using-rabbitmq/

Decoupling Microservices using Message-based RPC IPC, with Spring, RabbitMQ, and AMPQ
https://programmaticponderings.com/2017/05/08/decoupling-microservices-using-message-based-rpc-ipc-with-spring-rabbitmq-and-ampq/
* Direct reply-to, https://www.rabbitmq.com/direct-reply-to.html

RabbitMQ Tutorials
http://www.rabbitmq.com/getstarted.html

rabbitmq direct reply-to 在springAMQP和python之间的使用
https://segmentfault.com/a/1190000010418978
:::info
1 理解传统的RPC模式运行流程
传统模式下 Client端向一个指定的队列里推送消息，并声明一个一次性排他队列，然后将发送消息头部的reply-to属性的值设置为队列的名字，correlation_id属性设置为一个随机生成的值用于消息鉴定然后发送消息。在发送后Client端监听声明的排他队列，当收到消息后比对correaltiion_id,正确则处理消息断开监听连接，然后此队列被系统自动回收。 在Server端收到消息后处理消息然后将消息返回，返回的消息的routing-key设置为reply-to的值，properties中设置correlation_id为收到的correlation_id值。这样就完成一次RPC交互模式。
要解决今天这个问题我们还要知道几个知识点：

1当消息发送到exchange后如果没有队列接收此消息，那么此消息就会丢失。
2 一次性的排他队列在Client不在监听此队列就会自动被rabbitmq删除
:::

Real-world RPC with RabbitMQ and Node.JS
https://facundoolano.wordpress.com/2016/06/26/real-world-rpc-with-rabbitmq-and-node-js/
* Direct reply-to uses a pseudo-queue instead
* suggested not only using a single channel but registering a single consumer (i.e. a single callback function to handle all RPC responses). This meant introducing some structure to be able to route responses back to the promise that was expecting it.

Using direct reply-to for RPC calls #259
https://github.com/squaremo/amqp.node/issues/259

const REPLY_QUEUE = 'amq.rabbitmq.reply-to';
 
const createClient = (settings) => amqp.connect(settings.url, settings.socketOptions)
  .then((conn) => conn.createChannel())
  .then((channel) => {
    // create an event emitter where rpc responses will be published by correlationId
    channel.responseEmitter = new EventEmitter();
    channel.responseEmitter.setMaxListeners(0);
    channel.consume(REPLY_QUEUE,
      (msg) => channel.responseEmitter.emit(msg.properties.correlationId, msg.content),
      {noAck: true});
 
    return channel;
  });
 
const sendRPCMessage = (channel, message, rpcQueue) => new Promise((resolve) => {
  const correlationId = uuid.v4();
  // listen for the content emitted on the correlationId event
  channel.responseEmitter.once(correlationId, resolve);
  channel.sendToQueue(rpcQueue, new Buffer(message), { correlationId, replyTo: REPLY_QUEUE })
});

* correlationId 接收時用來比對是否是送出的message

Creating a queue per remote method when using RabbitMQ?
https://stackoverflow.com/questions/31687652/creating-a-queue-per-remote-method-when-using-rabbitmq/31705146#31705146
* exclusive queue
* used by only one connection and the queue will be deleted when that connection close

RabbitMQ vs Kafka Part 2 - RabbitMQ Messaging Patterns
https://jack-vanlightly.com/blog/2017/12/5/rabbitmq-vs-kafka-part-2-rabbitmq-messaging-patterns-and-topologies

RabbitMQ vs Kafka Part 4 - Message Delivery Semantics and Guarantees
https://jack-vanlightly.com/blog/2017/12/15/rabbitmq-vs-kafka-part-4-message-delivery-semantics-and-guarantees

How to Make Your Messaging System Reliable and Keep Your Support Engineers Happy
https://jack-vanlightly.com/blog/2017/6/11/improving-reliability-and-incident-response-via-a-message-lifecycle

RabbitMQ: direct reply-to?
https://stackoverflow.com/questions/38087136/rabbitmq-direct-reply-to

The Asynchronous Client/Server Pattern
http://zguide.zeromq.org/page:all#advanced-request-reply
https://stackoverflow.com/questions/29830825/zmq-client-worker-communication-pattern

:::info

* Our code is still pretty simplistic and doesn't try to solve more complex (but important) problems, like:

* How should the client react if there are no servers running?
Should a client have some kind of timeout for the RPC?
* If the server malfunctions and raises an exception, should it be forwarded to the client?
* Protecting against invalid incoming messages (eg checking bounds, type) before processing.
* Handle error cases. How should the client react when the RPC server is down for a long time?
:::
http://www.rabbitmq.com/tutorials/tutorial-six-dotnet.html

All You Need to Know About Asynchronous Messaging and RabbitMQ
https://dzone.com/articles/all-you-need-to-know-about-asyncronous-messaging-u

Detecting Dead TCP Connections with Heartbeats and TCP Keepalives
https://www.rabbitmq.com/heartbeats.html

## Reference

* The IOTA API Reference, https://iota.readme.io/reference
* tangle-accelerator development, https://hackmd.io/s/HyqSNxKKQ
* IOTA C Client Library, https://github.com/iotaledger/entangled/tree/develop/cclient
* entangled/mobile/android/Interface.h，https://github.com/iotaledger/entangled/blob/develop/mobile/android/Interface.h
* IOTA Random Seed Generator，https://github.com/0x13a/iota-seed-generator
* 12/13, https://hackmd.io/s/H10c9WcyN#
* iota.lib.py/examples/send_transfer.py ，https://github.com/iotaledger/iota.lib.py/blob/master/examples/send_transfer.py#L32
* IOTA_SEED_RELATED_API，https://github.com/ajblane/IOTA_SEED_RELATED_API
* HOW TO INSTALL METASPLOIT FRAMEWORK UBUNTU 17.10, https://www.pentestgeek.com/tools/install-metasploit-framework-ubuntu
* Baseline Security Recommendations for IoT, https://www.enisa.europa.eu/publications/baseline-security-recommendations-for-iot
* 3 Top IoT “security” Architectures and How to Fix Them,
* The Security Architecture Wiki, https://www.peerlyst.com/posts/the-security-architecture-wiki-peerlyst?utm_source=facebook&utm_medium=social&utm_content=peerlyst_post&utm_campaign=peerlyst_shared_post
* Industry Best Practice for IoT Security (PDF), https://www.iot-architect.de/iot-security-industry-best-practice

* Catalog of 122 open-spec Linux hacker boards, http://linuxgizmos.com/catalog-of-122-open-spec-linux-hacker-boards/?fbclid=IwAR1YMTW4AtwVlfTbRAcWcDz46mHdKTo7afVO3KsbBxv9wLhqA29ynGcFcOg
* Creating a new Seed / Wallet, https://iotasupport.com/gui-newseed.shtml
* OP-TEE crypto implementation, https://github.com/OP-TEE/optee_os/blob/master/documentation/crypto.md
* Secure Key Services in OP-TEE, https://www.slideshare.net/linaroorg/hkg18402-build-secure-key-management-services-in-optee