# A Permanode Project

## ScyllaDB as permanode storage

- The Real-Time Big Data Database.
- NoSQL database.
- Replication across multiple nodes makes for reliable fault tolerance.
- [SycllaDB datasheet](https://1bpezptkft73xxds029zrs59-wpengine.netdna-ssl.com/wp-content/uploads/datasheet_why-scylladb.pdf)
- [Syclla documentation](https://docs.scylladb.com/)

ScyllaDB is a big data database that is able to store big chunk of data from nodes to clusters. In CAP theorem, ScyllaDB prefer availability and partition over consistency. Hence, consensus in distributed ledger should still in full node layer. Although, ScyllaDB doesn't guarantee consistency, it still provide options and configurations to make sure query has certain degree of consistency. This permanode solution is going to serve as side or second layer storage.

Usually, IRI provides standard HTTP APIs for ordinary query to interact with the Tangle. However, it also has a ZMQ publish service to let clients subscribe and obtain streams of information. We can subscribe `sn` which is a event of transaction that has recently been confirmed. The return data of `sn` is like following:

* Index 1: Index of the milestone that confirmed the transaction
* Index 2: Transaction hash
* Index 3: Address
* Index 4: Trunk transaction hash
* Index 5: Branch transaction hash
* Index 6: Bundle hash

With given metadata hashes we are able to store these confirmed transactions to ScyllaDB's keyspace which will also be declared as same name: the Tangle. In this keyspace, we are going to have four tables contained which would be demonstrate in below article. For now, we just want you to know these tables let clients like tangle-accelerator(TA) able to query transaction if it has sufficient informations.

As of clients, we assume they interact with permanodes mainly for searching transactions. So the most API being used would mostly likely to be `find_transactions` and `get_trytes`. If it is TA in this case, TA should be able to connect to a ScyllaDB node when receiving such requests.

### Data Model

* Bundle table
* use to store transaction data.
* signature is null if it's only '9's

"CREATE TABLE IF NOT EXISTS permanent.bundleTable ("
                    "bundle blob,"
                    "address blob,"
                    "transaction blob,"
                    "message blob,"
                    "value bigint,"
                    "timestamp bigint,"
                    "trunk blob,"
                    "branch blob,"
                    "PRIMARY KEY (bundle, address, transaction));"

* Edge table
* use edge to find address/bundle/transaction
* edge:
* address
* trunk
* branch
* I think it could further divide to 2 or 3 table to reduce storage space.

"CREATE TABLE IF NOT EXISTS permanent.edgeTable("
                    "edge blob,"
                    "bundle blob,"
                    "address blob,"
                    "transaction blob,"
                    "PRIMARY KEY (edge, bundle, address, transaction));"

### Operations and Table

#### oprations
* findTransactions with
* bundles => transations in this bundles
* address => transations in this address
* ~~tag => transations with this tag~~
* approvees => transactions directly approving approvees
* getTrytes

## Install Scylla C/C++ driver
- [github open source](https://github.com/datastax/cpp-driver)
- [Build from source](https://docs.datastax.com/en/developer/cpp-driver/2.13/topics/building/)
- [Install package](https://downloads.datastax.com/cpp-driver/)

### Install on Ubuntu 18.04
Depedancy is `libuv` but here we assume this is already installed.

# Default driver is for runtime. Suffix of dev means development files like shared libraries.
wget https://downloads.datastax.com/cpp-driver/ubuntu/18.04/cassandra/v2.14.1/cassandra-cpp-driver_2.14.1-1_amd64.deb
wget https://downloads.datastax.com/cpp-driver/ubuntu/18.04/cassandra/v2.14.1/cassandra-cpp-driver-dev_2.14.1-1_amd64.deb
sudo dpkg -i cassandra-cpp-driver_2.14.1-1_amd64.deb
sudo dpkg -i cassandra-cpp-driver-dev_2.14.1-1_amd64.deb

install libuv
wget https://downloads.datastax.com/cpp-driver/ubuntu/18.04/dependencies/libuv/v1.29.1/libuv1_1.29.1-1_amd64.deb
wget https://downloads.datastax.com/cpp-driver/ubuntu/18.04/dependencies/libuv/v1.29.1/libuv1-dev_1.29.1-1_amd64.deb

sudo dpkg -i libuv1_1.29.1-1_amd64.deb
sudo dpkg -i libuv1-dev_1.29.1-1_amd64.deb

To link the share library, use flag

$ -lcassandra 

echo "deb http://download.opensuse.org/repositories/network:/messaging:/zeromq:/release-stable/xUbuntu_18.04/ ./" >> /etc/apt/sources.list
wget https://download.opensuse.org/repositories/network:/messaging:/zeromq:/release-stable/xUbuntu_18.04/Release.key -O- | sudo apt-key add
sudo apt-get install libzmq3-dev

## Install and Run Scylla using Docker

- [Running a instance using Docker](https://university.scylladb.com/courses/scylla-essentials-overview/lessons/quick-wins-install-and-run-scylla/topic/install-and-start-scylladb/)
- [Basic CQL Operation](https://university.scylladb.com/courses/scylla-essentials-overview/lessons/quick-wins-install-and-run-scylla/topic/basic-cql-operations/)

## Run Scylla using source code
- [download](https://www.scylladb.com/download/open-source/#source)
- [building info](https://github.com/scylladb/scylla/blob/master/README.md)
- [installing gcc version > 8](https://linuxize.com/post/how-to-install-gcc-compiler-on-ubuntu-18-04/)
### Run Scylla Cluster

https://docs.scylladb.com/operating-scylla/admin/

[installing](https://docs.scylladb.com/getting-started/)

### Connect Scylla node with C/C++ driver
- Runuing a node using docker

- Check Table from cqlsh,

### add Scylla C/C++ driver into TA
- APIS
- insert_transaction_into_bundleTable
- select_transactions_with_bundle
- select_transactions_with_bundle

The first query is for finding all transactions within a bundle.
The second query is for finding all transactions within an address. But in Scylla, we must specify all the keys before the address as we create the bundle table with primary keys as the following order.
PRIMARY KEY (bundle, address, transaction)

With this constraint, we need to get all Bundle hashes to a given address from the Edge table first, then use Bundle hashes and address to get all transaction data from Bundle table.

- 

- generate bazel struct graph
- `xdot <(bazel query --nohost_deps --noimplicit_deps 'deps(//accelerator:accelerator)' --output graph) `