Signature-based off-chain data/computing

$ bazel build //accelerator --cpu=aarch64 --crosstool_top=//tools/aarch64_compiler:toolchain --verbose_failures

INFO: Invocation ID: 8adbeb7f-2e1c-49ac-8230-e4708000ce86
ERROR: /home/ajubuntu/Desktop/tangle-accelerator/tools/aarch64_compiler/BUILD:3:1: in cc_toolchain_suite rule //tools/aarch64_compiler:toolchain: cc_toolchain_suite '//tools/aarch64_compiler:toolchain' does not contain a toolchain for cpu 'aarch64'
ERROR: Analysis of target '//accelerator:accelerator' failed; build aborted: Analysis of target '//tools/aarch64_compiler:toolchain' failed; build aborted
INFO: Elapsed time: 1.287s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded, 3 targets configured)
    Fetching @boost; fetching

Open trusted protocol - CA

#define DEFAULT_TAM_URI "http://localhost:54321/OTrP"

#define OTRP_URI L"http://localhost:54321/OTrP"

https://github.com/DLTcollab/general/issues/122


https://github.com/torvalds/linux/blob/master/Documentation/tee.txt
TEE subsys, linux/drivers/tee/tee_core.c, https://github.com/torvalds/linux/blob/master/drivers/tee/tee_core.c

發起交易

Secure data path issue

如果收集的資料至normal world，在做加密，會有一段時間有機會被惡意程式竊取。
解法:
要把抓資料的driver porting至optee的driver層，並提供optee API給TA使用。如果要做這步需要不少時間。

  • 安裝secure interrupt
  • serure optee drvier
  • 提供 optee API給 tasklet用

Send a transaction flow

Trustzone

關於Seed-Related TA-implemented IOTA API (iota_sign_address_gen/ iota_sign_signature_gen)加速的想法，目前未定案。

參考結案報告，需做Keccak運算，可能需要做不使用FPGA的加速(FPGA資源是做POW)，來增加產生交易速度。或者也可以做normal world的muti-thread設計來與ta合作取得address或者signature，參考:ta/concurrent

How to verify secure seed implementation

FPGA

參考FPGA-accelerated TrustZone-enabled IOTA Swarm Nodes，由於在新的FPGA板子，如果不是Cyclone V，需做以下工作:

  • 修改verilog，整合FPGA與CPU溝通的介面
  • 重新合成
  • 需修改linux driver，包含與FPGA溝通介面或者linux版本不同所需的修改。

Bazel

cc_library(
    name = "api",
    srcs = [
        "iota_client_core_api.c",
        "iota_client_extended_api.c",
    ],
    hdrs = [
        "iota_client_core_api.h",
        "iota_client_extended_api.h",
    ],
    visibility = ["//visibility:public"],
    deps = [
        ":service",
        "//cclient/http",
        "//common/helpers:checksum",
        "//common/helpers:digest",
        "//common/helpers:pow",
        "//common/helpers:sign",
        "//utils:time",
        "//utils/containers/hash:hash243_queue",
    ],
)
cc_library(
    name = "shared",
    hdrs = ["http.h"],
    visibility = ["//visibility:public"],
    deps = ["//cclient:service"],
)

cc_library(
    name = "http",
    srcs = [
        "http.c",
    ],
    visibility = ["//visibility:public"],
    deps = [
        ":shared",
        "//network:sockets",
        "@http_parser",
    ],
)
cc_binary(
    name = "libcclient.so",
    linkshared = True,
    deps = [":api"],
)

cc_library(
    name = "service",
    srcs = [
        "service.c",
    ],
    hdrs = ["service.h"],
    visibility = ["//visibility:public"],
    deps = [
        "//cclient/request:requests",
        "//cclient/response:responses",
        "//cclient/serialization:serializer_json",
    ],
)

CPU比較

FPGA與Truszone整合描述

  • 整合方案一: RPi 3透過區域網路方式連接FPGA board，但對外連接需要hub或者開啟無線網路。

  • 整合方案二:

  • rpi3與FPGA板子透過GPIO溝通或者其他介面。

LampaLab/iota_fpga


891*4.6 = 4098

Hikey 960, https://www.96boards.org/documentation/consumer/hikey/hikey960/getting-started/

查詢交易 (解交易)，尚未規畫完整

  • 需找出相關使用seed與PK的IOTA API
  • 撰寫TA
  • 提供secure API給normal world使用
  • 放交易queue設計

ta examples

taobao

https://world.taobao.com/dianpu/cat-35646333-810067291.htm


https://item.taobao.com/item.htm?spm=a21wu.12321156-tw.0.0.2558b750ADxZIS&id=557802312480

40000
https://www.mouser.tw/ProductDetail/Intel-Altera/5CSEMA4U23C6N?qs=TN6zvtaX%252baNEGVGZbjwaqA==

Intel Cyclone® V SE 5CSEBA6U23I7NDK
5382

pidiver

  • Raspberry-Pi-Mode
  • PiDiver doing USBDiving

https://microengineer.eu/2018/04/25/iota-pearl-diver-fpga/

https://microengineer.eu/2018/04/24/old-iota-pow-hardware-accelerator-fpga-for-altera-de1/

Cyclone 10 LP SPI Slave to Avalon Master Bridge Design Example
https://fpgacloud.intel.com/devstore/platform/17.0.0/Standard/cyclone-10-lp-spi-slave-to-avalon-master-bridge-design-example/

INTEL® CYCLONE® 10 LP FPGA APPLICATIONS
https://www.intel.com.tw/content/www/tw/zh/products/programmable/fpga/cyclone-10/lp/applications.html

Design Store
https://fpgacloud.intel.com/devstore/platform/?acds_version=any&family=cyclone-10-lp

How to enable SPI on HiKey 960
https://discuss.96boards.org/t/how-to-enable-spi-on-hikey-960/4750/7

Shadow-Box

https://github.com/kkamagui/shadow-box-for-arm

for general

hook

PoW remote server

https://hackmd.io/RkjqvkH8Scel53V7-XgHOA

issues

Trace optee 編譯流程

https://hackmd.io/n_wwCngqSI65f1SBUVR8ww

Trace tangle accelerator

https://hackmd.io/kSQfXM-OTQ-KdziaVS_msA?both

tangle accelerator and rabbitmq c client

https://hackmd.io/kSQfXM-OTQ-KdziaVS_msA?both

Cloud design pattern

Competing Consumers pattern
https://docs.microsoft.com/en-us/azure/architecture/patterns/competing-consumers

RabbitMQ Work Queues: Avoiding Data Inconsistency with Rebalanser, https://jack-vanlightly.com/blog/2018/7/25/rabbitmq-work-queues-avoiding-data-inconsistency-with-rebalanser

What is an idempotent operation?
https://stackoverflow.com/questions/1077412/what-is-an-idempotent-operation

  • f(f(x)) = f(x)
  • abs(abs(x)) = abs(x)

Asynchronous Messaging Primer
https://docs.microsoft.com/en-us/previous-versions/msp-n-p/dn589781(v=pandp.10)

Request/response messaging

Broadcast messaging

Cloud Design Patterns: Prescriptive Architecture Guidance for Cloud Applications
https://docs.microsoft.com/en-us/previous-versions/msp-n-p/dn568099(v%3Dpandp.10)

Microservices: Asynchronous Request Response Pattern
https://medium.com/@pulkitswarup/microservices-asynchronous-request-response-pattern-6d00ab78abb6

Asynchronous communication with message queue
https://codeblog.dotsandbrackets.com/asynchronous-communication-with-message-queue/

  • Callback queue
    • reply_to
    • correlation_id
  • RPC workers

RPC calls to multiple consumers
https://stackoverflow.com/questions/12407485/rpc-calls-to-multiple-consumers

Is the RabbitMQ RPC a kind of “Work queue” with response?
https://stackoverflow.com/questions/45695071/is-the-rabbitmq-rpc-a-kind-of-work-queue-with-response

RabbitMQ in Depth
https://manning-content.s3.amazonaws.com/download/b/5ceeb08-0e8d-4f94-8d06-0300996d9427/sample_ch01_Roy_Rabbit-MQ-in-Depth_September11.pdf

https://manning-content.s3.amazonaws.com/download/c/8a042e9-9293-43aa-bea1-b42ee0008c09/sample_ch02_Roy_Rabbit-MQ-in-Depth_September11.pdf

A Survey of Distributed Message Broker Queues
https://arxiv.org/pdf/1704.00411.pdf

RabbitMQ Essentials
http://aad.tpu.ru/1955/Java books/RabbitMQ Essentials.pdf

MESSAGE BROKERS AND RABBITMQ IN ACTION
http://aad.tpu.ru/1955/Java books/RabbitMQ Essentials.pdf

Getting Started with RabbitMQ and CloudAMQP
http://193.40.244.77/idu0080/wp-content/uploads/2016/02/Getting_Started_with_RabbitMQ_and_CloudAMQP-1.pdf

Asynchronous Joins Using RabbitMQ
https://www.compose.com/articles/asynchronous-joins-using-rabbitmq/

Decoupling Microservices using Message-based RPC IPC, with Spring, RabbitMQ, and AMPQ
https://programmaticponderings.com/2017/05/08/decoupling-microservices-using-message-based-rpc-ipc-with-spring-rabbitmq-and-ampq/

RabbitMQ Tutorials
http://www.rabbitmq.com/getstarted.html

rabbitmq direct reply-to 在springAMQP和python之间的使用
https://segmentfault.com/a/1190000010418978

1 理解传统的RPC模式运行流程
传统模式下 Client端向一个指定的队列里推送消息，并声明一个一次性排他队列，然后将发送消息头部的reply-to属性的值设置为队列的名字，correlation_id属性设置为一个随机生成的值用于消息鉴定然后发送消息。在发送后Client端监听声明的排他队列，当收到消息后比对correaltiion_id,正确则处理消息断开监听连接，然后此队列被系统自动回收。 在Server端收到消息后处理消息然后将消息返回，返回的消息的routing-key设置为reply-to的值，properties中设置correlation_id为收到的correlation_id值。这样就完成一次RPC交互模式。
要解决今天这个问题我们还要知道几个知识点：

1当消息发送到exchange后如果没有队列接收此消息，那么此消息就会丢失。
2 一次性的排他队列在Client不在监听此队列就会自动被rabbitmq删除

Real-world RPC with RabbitMQ and Node.JS
https://facundoolano.wordpress.com/2016/06/26/real-world-rpc-with-rabbitmq-and-node-js/

  • Direct reply-to uses a pseudo-queue instead
  • suggested not only using a single channel but registering a single consumer (i.e. a single callback function to handle all RPC responses). This meant introducing some structure to be able to route responses back to the promise that was expecting it.

Using direct reply-to for RPC calls #259
https://github.com/squaremo/amqp.node/issues/259

const REPLY_QUEUE = 'amq.rabbitmq.reply-to';
 
const createClient = (settings) => amqp.connect(settings.url, settings.socketOptions)
  .then((conn) => conn.createChannel())
  .then((channel) => {
    // create an event emitter where rpc responses will be published by correlationId
    channel.responseEmitter = new EventEmitter();
    channel.responseEmitter.setMaxListeners(0);
    channel.consume(REPLY_QUEUE,
      (msg) => channel.responseEmitter.emit(msg.properties.correlationId, msg.content),
      {noAck: true});
 
    return channel;
  });
 
const sendRPCMessage = (channel, message, rpcQueue) => new Promise((resolve) => {
  const correlationId = uuid.v4();
  // listen for the content emitted on the correlationId event
  channel.responseEmitter.once(correlationId, resolve);
  channel.sendToQueue(rpcQueue, new Buffer(message), { correlationId, replyTo: REPLY_QUEUE })
});
  • correlationId 接收時用來比對是否是送出的message

Creating a queue per remote method when using RabbitMQ?
https://stackoverflow.com/questions/31687652/creating-a-queue-per-remote-method-when-using-rabbitmq/31705146#31705146

  • exclusive queue
    • used by only one connection and the queue will be deleted when that connection close

RabbitMQ vs Kafka Part 2 - RabbitMQ Messaging Patterns
https://jack-vanlightly.com/blog/2017/12/5/rabbitmq-vs-kafka-part-2-rabbitmq-messaging-patterns-and-topologies

RabbitMQ vs Kafka Part 4 - Message Delivery Semantics and Guarantees
https://jack-vanlightly.com/blog/2017/12/15/rabbitmq-vs-kafka-part-4-message-delivery-semantics-and-guarantees

How to Make Your Messaging System Reliable and Keep Your Support Engineers Happy
https://jack-vanlightly.com/blog/2017/6/11/improving-reliability-and-incident-response-via-a-message-lifecycle

RabbitMQ: direct reply-to?
https://stackoverflow.com/questions/38087136/rabbitmq-direct-reply-to

The Asynchronous Client/Server Pattern
http://zguide.zeromq.org/page:all#advanced-request-reply
https://stackoverflow.com/questions/29830825/zmq-client-worker-communication-pattern

  • Our code is still pretty simplistic and doesn't try to solve more complex (but important) problems, like:

  • How should the client react if there are no servers running?
    Should a client have some kind of timeout for the RPC?

  • If the server malfunctions and raises an exception, should it be forwarded to the client?

  • Protecting against invalid incoming messages (eg checking bounds, type) before processing.

  • Handle error cases. How should the client react when the RPC server is down for a long time?

http://www.rabbitmq.com/tutorials/tutorial-six-dotnet.html

All You Need to Know About Asynchronous Messaging and RabbitMQ
https://dzone.com/articles/all-you-need-to-know-about-asyncronous-messaging-u

Detecting Dead TCP Connections with Heartbeats and TCP Keepalives
https://www.rabbitmq.com/heartbeats.html

Reference

Select a repo