# ARWEAVE Dev Questionnaire
#### 0. General setup notes
https://docs.arweave.org/info/mining/mining-guide
`tune2fs -O large_dir /dev/sda` (disk when arweave data is located)
If you will not do it you will have problem with txs folder sync
ext4 is recommended
#### 1. What are the advised setup params for the node which should ignore chunks data but still contain the blockchain (blocks, transactions) data fully?
By default arweave node downloads blocks, txs and chunks
Blocks + txs 400-500 GB
Chunks 14 TB and growing
There are some recommended parameters for start arweave node:
* `sync_jobs 0` (do not download chunks)
* `max_disk_pool_buffer_mb 0 max_disk_pool_data_root_buffer_mb 0` (do not accept chunks from external peers with POST /chunk) (unless you already make NAT or firewall, see below)
* That should help use lowest possible storage
#### 2. What are the advised setup params to avoid data storage and network spam (to make a node as read-only as possible and optimized for it)?
You can disable 1984 port with firewall for external requests or launch arweave node inside NAT
Also there is built-in rate limiter 900 Req/min per IP
To increase default limit `requests_per_minute_limit (number)` e.g. `requests_per_minute_limit 1000000`
#### 3. Is there a way for direct node database sync with external database?
Current arweave storage implementation has blocks and txs folder, so you can read data directly from that.
>Note that some transactions are stored in rocksdb, so you can't read all transactions from txs folder
We will move all transactions to rocksdb later, so probably it's better to read all data with HTTP interface
Also pls note that fully syncronized node should have `<your IP>:1984` blocks > height
e.g. https://arweave.net/
```json
{
"network":"arweave.N.1",
"version":5,
"release":49,
"height":794387,
"current":"-bO8fHTgQFLv0JJWAm3Svpsnn8trzhWoY9m8ESSGrPqaNOD1viUXZzKDIMTK3u34",
"blocks":785905,
"peers":309,
"queue_length":0,
"node_state_latency":1
}
```
#### 4. What is the advised way for retrieving all transactions data?
Currently direct read from fs is possible, but use of HTTP interface is future-proof
https://docs.arweave.org/developers
https://docs.arweave.org/developers/server/http-api
https://github.com/ArweaveTeam/arweave-js#blocks
#### 5. If the advised way to retrieve transactions data is via HTTP API or GraphQL which one is preferable?
HTTP is much more low-level and has less overhead, but GraphQL is pretty ok
https://gql-guide.vercel.app/
#### 6. If using HTTP API or GraphQL, how can a node be optimized for maximum speed of data retrieval?
* Start arweave node with `requests_per_minute_limit 1000000`
* For continious block transfers
* Monitor /block/current endpoint. If you see unknown previous_block, request it until you reach some known blocks
Note `/block/hash/<hash>` uses indep_hash not hash field
* You should discard all transactions which are mentioned in "side chain" and apply transactions in "main chain" OR
do not include transactions with less than certain confirmation threshold (50 is super durable, for some urgent applications 3 confirmations is acceptable)
* For one-time import
* Method 1. Just read all file system contents of blocks and txs folders. Then sync all missing transactions from 1st block to last block to fill up your database with transactions in rocksdb.
* Method 2. Request all blocks with `/block/height/<1...>`
* In parallel request all txs `/tx/<tx_hash>` with some limitation in parallel requests (16 parallel requests should be ok)
#### 7. What is the chain confidence interval/finality period? What is the advised way to verify the presence of most recent transactions data in chain?
See above. 3-50 confirmations. Depends on your application.
Finality is probabilistic, same as bitcoin.
But 50 confirmations seems like kinda hard-coded finality
#### 8. What is the advised way to retrieve rewards data? Are there caveats while calculating address balance as a sum of all transactions sum and rewards sum?
Balance is sum of
* premine transactions (regular transactions in 1st block, should be easily parsed)
* mining rewards (is not regular transaction, you should see block for reward_addr)
* regular transfers
`/tx/<tx_hash>` is ok for premine transactions and regular transfers
There is no built-in code for getting block rewards. For https://explorer.ar.virdpool.com/ I use this patch to `ar_http_iface_middleware.erl`
```erlang
%% Return the block reward
process_request(get_block, [Type, ID, <<"reward">>], Req) ->
MaybeFilename = case Type of
<<"height">> ->
CurrentHeight = ar_node:get_height(),
case ID of
Height when Height < 0 ->
unavailable;
Height when Height > CurrentHeight ->
unavailable;
Height ->
BI = ar_node:get_block_index(),
Len = length(BI),
case Height > Len - 1 of
true ->
unavailable;
false ->
{H, _, _} = lists:nth(Len - Height, BI),
ar_storage:lookup_block_filename(H)
end
end;
<<"hash">> ->
ar_storage:lookup_block_filename(ID)
end,
case MaybeFilename of
unavailable ->
{404, #{}, <<"Block not found.">>, Req};
Filename ->
{ok, Binary} = file:read_file(Filename),
B = ar_serialize:json_struct_to_block(Binary),
case B#block.height of
0 ->
{200, #{}, "0", Req};
_ ->
TXs =
lists:map(
fun(Hash) ->
case hash_to_filename_decoded(tx, Hash) of
{error, invalid} ->
{error, 400, #{}, <<"TX Invalid hash.">>, Req};
{error, ID, unavailable} ->
case is_a_pending_tx(ID) of
true ->
{error, 202, #{}, <<"TX Pending">>, Req};
false ->
{error, 404, #{}, <<"TX Not Found.">>, Req}
end;
{Status, FilenameTx} ->
Result =
case Status of
ok ->
ar_storage:read_tx_file(FilenameTx);
migrated_v1 ->
ar_storage:read_migrated_v1_tx_file(FilenameTx)
end,
case Result of
{ok, TX} ->
{ok, TX};
{error, enoent} ->
{error, 404, #{}, <<>>, Req};
{error, data_unavailable} ->
{error, 404, #{}, <<>>, Req};
_ ->
{error, 500, #{}, <<>>, Req}
end
end
end,
B#block.txs
),
{Tx_list, Error_list} =
lists:foldl(
fun(TX, {Tx_list, Error_list}) ->
case TX of
{error, A0, A1, A2, A3} ->
{Tx_list, Error_list ++ [{A0, A1, A2, A3}]};
{ok, TX_ok} ->
{Tx_list ++ [TX_ok], Error_list}
end
end,
{[],[]},
TXs
),
case length(Error_list) > 0 of
true ->
lists:nth(1, Error_list);
false ->
case B#block.height of
0 ->
{200, #{}, "0", Req};
_ ->
PrevH = B#block.previous_block,
PrevH_filename = ar_storage:lookup_block_filename(PrevH),
{ok, Binary_prev} = file:read_file(PrevH_filename),
PrevB = ar_serialize:json_struct_to_block(Binary_prev),
Rate =
case PrevB#block.height >= ar_fork:height_2_5() of
true ->
PrevB#block.usd_to_ar_rate;
false ->
?USD_TO_AR_INITIAL_RATE
end,
{FinderReward, _RewardPool} =
ar_node_utils:get_miner_reward_and_endowment_pool({
PrevB#block.reward_pool,
Tx_list,
B#block.reward_addr,
B#block.weave_size,
B#block.height,
B#block.diff,
B#block.timestamp,
Rate
}),
{200, #{}, integer_to_binary(FinderReward), Req}
end
end
end
end;
```
#### 9. Is there stacking available in Arweave? Does it affect the address balance?
There is no proof of stake inside, so no staking.
There are profit sharing tokens (https://github.com/ArweaveTeam/SmartWeave/blob/master/CREATE-PST.md), but it's different
https://arweave.medium.com/profit-sharing-tokens-a-new-incentivization-mechanism-for-an-open-web-1f2532411d6e
Also read more below
#### 10. Can the tokens functioning inside the chain affect the address balance? If so, how?
You mean some equivalent of ERC20 (eth) on arweave, right?
There is smartweave as L2 smart contract system on top of arweave
https://github.com/ArweaveTeam/SmartWeave/blob/master/CONTRACT-GUIDE.md
There is PST on arweave
If you are interested in detailed guide how to track PSTs transfers/balances ask me for that
Sign in with Wallet
Connect another wallet